libav/decode.lisp
The decoder's own output, reused across every frame in the file.
One demuxer and one decoder on one stream of an open file.
The pump in decode-next-frame is the same whatever the stream carries; a
video adds pictures and their conversion, an audio-track adds samples. The
accessors keep the VIDEO- prefix they were born with: the picture path came
first, and every caller of it says video.
Lazily built, because its size depends on what the caller asks for.
The rgba staging buffer swscale writes into, kept across frames so a
player is not allocating megabytes per picture.
An open video file and its decoder.
Demuxing.
Decoding.
Converting.
Return the INDEXth AVStream of format-context.
Return the AVRational in slot as a Lisp rational, or NIL when undefined.
AVVulkanDeviceContext takes these arrays over. Allocate them with libavutil too: a Nix process can contain a second libc via the host SBCL, making CFFI's malloc incompatible with av_free.
Decode into AVVkFrames allocated on the renderer's existing VkDevice.
configuration is a plist containing :INSTANCE, :PHYSICAL-DEVICE, :DEVICE,
:GET-INSTANCE-PROC-ADDR, extension lists, and :QUEUE-FAMILIES. FFmpeg borrows
the Vulkan handles; the caller must therefore close the video before the
device.
The default C callback chooses FFmpeg's first offered format, which is Vulkan once HWDEVICE_CTX is attached. Keeping this callback in C also avoids re-entering SBCL from the middle of avcodec_receive_frame.
FFmpeg releases the extension arrays with its Vulkan context.
Ask FFmpeg to decode into reference-counted CVPixelBuffers.
Open pathname, find its best video stream, and start a decoder for it.
Returns a video. The caller owns it and must close-video it.
Only reached when something above signalled: release whatever of the half-built decoder we had taken ownership of.
Release video's decoder, demuxer, frame, and scaler. Idempotent.
Teardown computes too: swscale and the demuxer both do float arithmetic on the way out. An unmasked trap here would signal from inside whatever was releasing the video -- for luvcraft, the middle of closing the game -- and abandon everything that had not been released yet.
Evaluate body with variable bound to PATHNAME's decoder, closing it after.
Decode until video's frame holds the next picture. Return the frame or NIL.
NIL means the file is exhausted; video's frame is left holding the last
picture that was decoded.
Drain first: the codec may still be holding pictures from packets it was given earlier, and at end of file that is the only source left.
The codec wants more input. Read packets until one belongs to our stream, or the file ends and we flush with a null packet instead.
End of file, or a read error we treat as one. A null packet tells the codec to hand back everything it has buffered.
Seek video back to its start and reset the decoder.
Converting a decoded picture to rgba.
The scaler is cached against the size it was built for, since building one per frame would discard the filter tables it exists to precompute. A changed target size -- or a decoder that changed its mind about the source format mid-stream, which happens -- rebuilds it.
Return video's current picture as a height by width array of rgba words.
Each word is red in its low byte through alpha in its high byte, which is the
packing luvcraft's block atlas uses. array is filled and returned when given,
so a player can convert into the same array every frame.
alpha is written rather than taken from the conversion. swscale leaves the
alpha of an rgba target undefined when the source has no alpha of its own, and
in the block atlas that byte is not opacity at all -- it is the material's
surface height -- so the caller has to say what it means.
Copy count packed words out of foreign pixels into words, forcing opacity.
This is a per-picture inner loop -- a modest 512-wide screen is two hundred thousand words every time the film advances -- so it reads the foreign memory through a system area pointer rather than one CFFI call per pixel.
(frame)One demuxer and one decoder on one stream of an open file. The pump in DECODE-NEXT-FRAME is the same whatever the stream carries; a VIDEO adds pictures and their conversion, an AUDIO-TRACK adds samples. The accessors keep the VIDEO- prefix they were born with: the picture path came first, and every caller of it says…
An explicitly owned AVFrame.
(video)Decode until VIDEO's frame holds the next picture. Return the frame or NIL. NIL means the file is exhausted; VIDEO's frame is left holding the last picture that was decoded.
An open video file and its decoder.
An open file's sound and its decoder.
A MuPDF context: its allocator, its store, and its error stack.
Headings, paragraphs, figures and their IDs, mentions, marks.
(format-context index)(pointer type slot)Division of two represented quantities.
Test whether two compatible scalars are equal.
(strings)(codec-context configuration)Decode into AVVkFrames allocated on the renderer's existing VkDevice. CONFIGURATION is a plist containing :INSTANCE, :PHYSICAL-DEVICE, :DEVICE, :GET-INSTANCE-PROC-ADDR, extension lists, and :QUEUE-FAMILIES. FFmpeg borrows the Vulkan handles; the caller must therefore close the video before the device.
Logical disjunction of tests and raw truth values.
(code operation)(codec-context)(pathname &key (hardware :auto) hardware-configuration)Open PATHNAME, find its best video stream, and start a decoder for it. Returns a VIDEO. The caller owns it and must CLOSE-VIDEO it.
(video)(&optional directory)Load libavutil, libavcodec, and libavformat, and check their versions. DIRECTORY is useful for builds which have not been installed. When it is NIL, LUV_FFMPEG_LIBDIR is consulted before the platform soname search. Signals LIBAV-VERSION-MISMATCH when a loaded library disagrees with the headers this system was…
(&body body)Run BODY with the floating-point environment FFmpeg's own code expects. FFmpeg computes with floats that raise invalid-operation and divide-by-zero as a matter of course -- probing stream timing alone will do it -- and SBCL traps those by default, so an unmasked call dies inside avformat_find_stream_info rather than…
Logical conjunction of tests and raw truth values.
(video)Logical negation of one test or raw truth value.
(video)(frame)((variable pathname) &body body)(code)Subtraction or unary negation.
(video)(video)(video width height)(format)(video size)Test whether one compatible scalar is less than another.
(video pointer width height pitch)Convert VIDEO's current picture into POINTER as WIDTH by HEIGHT RGBA. PITCH is the destination's byte stride, which may exceed WIDTH times four when the caller is writing into a larger image.
(video pointer width height pitch)(video width height &key array (alpha 255))Return VIDEO's current picture as a HEIGHT by WIDTH array of RGBA words. Each word is red in its low byte through ALPHA in its high byte, which is the packing luvcraft's block atlas uses. ARRAY is filled and returned when given, so a player can convert into the same array every frame. ALPHA is written rather than…
Multiplication and scalar scaling.
(pixels words count opacity)Copy COUNT packed words out of foreign PIXELS into WORDS, forcing OPACITY. This is a per-picture inner loop -- a modest 512-wide screen is two hundred thousand words every time the film advances -- so it reads the foreign memory through a system area pointer rather than one CFFI call per pixel.
Opening a video file and pulling
rgbapictures out of it.The shape of FFmpeg's decode loop is not obvious from its names, so it is worth stating once. Demuxing and decoding are separate rates: one packet may produce no frames, or several. So the loop is a pump with two ends -- avcodec_send_packet feeds it, avcodec_receive_frame drains it -- and each end may answer EAGAIN meaning "I want the other end first". At the file's end the codec is flushed with a null packet, after which it keeps handing back the frames it had buffered until it finally says EOF.
This is the software path: the decoder writes planes into ordinary memory and swscale converts them to
rgba. It exists to get pictures onto a surface at all. The hardware path -- a VideoToolbox, VAAPI, or Vulkan frame whose data never touches the CPU -- reuses everything here except the last step, and is the reason the AVFrame binding knows whatframe-hardware-pmeans.