luv

Workshop wiki

audio.lisp

libav/audio.lisp

system luv/libav · 7 definitions · on GitHub

Opening a file's sound and pulling samples out of it.

An audio-track is the same pump as a video -- one demuxer, one decoder, decode-next-frame -- on the file's best audio stream, opened on its own AVFormatContext so a player can run picture and sound on different threads at different rates without either waiting for the other.

What comes out is deliberately plain: mono single-floats. A film on a wall is a point in the world; where it sits between the listener's ears is the player's business, not the file's, so channels are folded here and the sample rate is left as the file has it for the output device to resample. There is no swresample in this: interleaving and averaging a few thousand floats a frame is not worth a fifth library.

in-package#:luv.libav
defclassaudio-track
sample-rate:initarg:sample-rate:readeraudio-sample-rate
channel-count:initarg:channel-count:readeraudio-channel-count
duration:initarg:duration:initformnil:readeraudio-duration:documentation"The container's length in seconds, or NIL."
:documentation

An open file's sound and its decoder.

defunopen-audio
pathname

Open pathname's best audio stream and start a decoder for it.

Returns an audio-track, or NIL when the file has no sound. The caller owns it and must close-audio it.

let
truename
uiop:native-namestring
truenamepathname
format-contextnil
codec-contextnil
tracknil
unwind-protect
with-libav-native-environment
cffi:with-foreign-objects
context-cell:pointer
decoder-cell:pointer
setf
cffi:mem-refcontext-cell:pointer
cffi:null-pointer
check-code
%avformat-open-inputcontext-celltruename
cffi:null-pointer
cffi:null-pointer
'open-audio
setfformat-context
cffi:mem-refcontext-cell:pointer
check-code
%avformat-find-stream-infoformat-context
cffi:null-pointer
'open-audio
setf
cffi:mem-refdecoder-cell:pointer
cffi:null-pointer
let
index
%av-find-best-streamformat-context
cffi:foreign-enum-value'media-type:audio
-1-1decoder-cell0
when
minuspindex

A silent film is not an error.

return-fromopen-audionil
let*
decoder
cffi:mem-refdecoder-cell:pointer
stream
stream-pointerformat-contextindex
parameters
cffi:foreign-slot-valuestream'
:structav-stream
'codec-parameters
when
cffi:null-pointer-pdecoder
error"No decoder is available for the audio stream in ~A."pathname
setfcodec-context
%avcodec-alloc-context3decoder
when
cffi:null-pointer-pcodec-context
error"Could not allocate a decoder context for ~A."pathname
check-code
%avcodec-parameters-to-contextcodec-contextparameters
'open-audio
check-code
%avcodec-open2codec-contextdecoder
cffi:null-pointer
'open-audio
let
duration
cffi:foreign-slot-valueformat-context'
:structav-format-context
'duration
setftrack
make-instance'audio-track:pathnamepathname:format-contextformat-context:codec-contextcodec-context:stream-indexindex:sample-rate
cffi:foreign-slot-valuecodec-context'
:structav-codec-context
'sample-rate
:channel-count
cffi:foreign-slot-value
cffi:foreign-slot-pointercodec-context'
:structav-codec-context
'channel-layout
'
:structav-channel-layout
'channel-count

AV_TIME_BASE is a million.

:duration:frame:packet
%av-packet-alloc
setfformat-contextnilcodec-contextnil
track
progn
whencodec-context
cffi:with-foreign-object
cell:pointer
setf
cffi:mem-refcell:pointer
codec-context
%avcodec-free-contextcell
whenformat-context
cffi:with-foreign-object
cell:pointer
setf
cffi:mem-refcell:pointer
format-context
%avformat-close-inputcell
defunclose-audio
track

Release track's decoder, demuxer, and frame. Idempotent.

defunrewind-audio
track

Seek track back to its start and reset the decoder.

defundecode-next-audio-frame
track

Decode until track's frame holds the next run of samples, or NIL at the end of the file.

defunaudio-frame-sample-count
track

How many samples per channel track's current frame holds.

cffi:foreign-slot-value
frame-pointer
video-frametrack
'
:structav-frame
'sample-count
defunaudio-frame-mono-samples
track&optionalbuffer

Return track's current frame folded to mono single-floats, and how many.

buffer is reused when it is large enough; otherwise a fresh one is made. The count is the number of samples written, which is what matters, since the buffer may be longer. Every sample format a decoder hands out is read here directly -- planar or interleaved, 8/16/32-bit integer, float, or double -- because that is a small table and a resampler is a large dependency.

let*
frame
frame-pointer
video-frametrack
count
cffi:foreign-slot-valueframe'
:structav-frame
'sample-count
channels
cffi:foreign-slot-value
cffi:foreign-slot-pointerframe'
:structav-frame
'channel-layout
'
:structav-channel-layout
'channel-count
format
cffi:foreign-slot-valueframe'
:structav-frame
'format
planes
cffi:foreign-slot-valueframe'
:structav-frame
'extended-data
buffer
if
andbuffer
>=
lengthbuffer
count
buffer
make-array
maxcount1024
:element-type'single-float
scale
/1.0
max1channels
declare
type
simple-arraysingle-float
buffer
typefixnumcountchannels
flet
planar-p
keyword
memberkeyword'
:u8p:s16p:s32p:fltp:dblp
let*
keyword
cffi:foreign-enum-keyword'sample-formatformat:errorpnil
planar
planar-pkeyword
bytes
ecasekeyword
:u8:u8p
1
:s16:s16p
2
:s32:s32p:flt:fltp
4
:dbl:dblp
8
macrolet
fold
readerconvert

Sum the channels of each sample; planar data is one pointer per channel, interleaved is one pointer with the channels side by side.

`
dotimes
samplecount
let
sum0.0
declare
typesingle-floatsum
dotimes
channelchannels
let
pointer
ifplanar
cffi:inc-pointer
cffi:mem-arefplanes:pointerchannel
*samplebytes
cffi:inc-pointer
cffi:mem-arefplanes:pointer0
*
+
*samplechannels
channel
bytes
incfsum
,convert
cffi:mem-refpointer,reader
setf
arefbuffersample
ecasekeyword
:u8:u8p
fold:uint8
lambda
v
/
-v128
128.0
:s16:s16p
fold:int16
lambda
v
/v32768.0
:s32:s32p
fold:int32
lambda
v
/v2147483648.0
:flt:fltp
fold:float
lambda
v
v
:dbl:dblp
fold:double
lambda
v
coercev'single-float
valuesbuffercount