FFmpeg Streaming Consumer - RawVideo without Audio

#1
Part of the software I maintain uses CCG to obtain a channel stream over TCP using the following command...

ADD 1 STREAM tcp://localhost:80/raw -vf "format=bgra" -c:v rawvideo -c:a pcm_s16le -format rawvideo

I am well aware of other forum posts which state don't do this due to the large amount of data transferred however CCG sends to localhost only in this scenario so that isn't my issue.

My problem is that I need to extract raw video frames without the audio... however I can't omit the audio codec without Caspar complaining that it "Failed to find audio codec with id 0".

I don't need the audio and I'm not entirely sure how to remove the raw audio out of the byte data, even if I could figure this out the performance hit in doing so is more than I'm willing to deal with. I can easily calculate the byte size of each frame using the height, width, and format to determine stride and finally the frame size... however the audio data is throwing this off, I end up with video that constantly scrolls across the screen.

My goal is to obtain a zero latency stream to be transmitted over NDI using CCG 2.0.7. I can almost accomplish this by telling Caspar to send an mpegts stream instead and converting that to raw using a separate ffmpeg thread, unfortunately there is too much latency using this method.

It would be great if the ffmpeg -an command were supported so I could just strip the audio out completely however I can't get this to work in Caspar 2.0.7 or 2.1.

I'm still toying with grabbing the rawvideo (plus audio) from the streaming consumer, spawning a separate ffmpeg thread to strip the audio using -an before attempting to decode the raw frames... unfortunately I have not been successful with my ffmpeg command line yet, it just doesn't want to strip out the audio, I'm guessing something is wrong on the input side of the command.

Any suggestions for accomplishing this, maybe a channel configuration change or something else I'm not thinking of?

Re: FFmpeg Streaming Consumer - RawVideo without Audio

#4
Unfortunately that doesn't seem to have any impact either, I also made sure to turn off embedded audio and system audio just in case. FFmpeg in caspar is encoding an audio stream regardless.

I have a solution, not the best solution but it does work.

I tell caspar to create a stream using the following command.

ADD 1 STREAM tcp://localhost:80/raw -vf "format=bgra" -c:v rawvideo -format mpegts

So the only thing I changed here was the container format. I'm encapsulating rawvideo and audio streams directly into an mpegts container which causes ffmpeg to encode them into separate streams rather than placing it all in the rawvideo pixel data.

The downside is I still have to spawn another ffmpeg thread to grab my video stream however it is not transcoding, only stripping away the mpegts container and I verified it is still 0 latency. I used the command below...

ffmpeg.exe -f mpegts -i {my input pipe} -map 0:0 -copy_unknown -codec copy -f data {my output pipe}-

I'm piping in my mpegts stream from caspar and I'm excluding the audio stream [0:1] by only mapping the video stream [0:0] to the output. I have to use -copy_unknown otherwise ffmpeg won't include the stream because it doesn't understand the format (this is a newer command line option), and my container format is simply data for flat binary data.

Works like a charm, now I can successfully extract my rawvideo frames without audio and read them directly into NewTek NDI. At a future time I could extract the audio from this as well to add to NDI transport as separately however this is not one of my requirements currently.

If there are better ideas regarding this I would love to hear them, for now I think I can move forward though.


Thanks again for your prompt feedback!
Nick

Re: FFmpeg Streaming Consumer - RawVideo without Audio

#5
My idea would create a video stream with a silent audio. Why is that not working? And, by the way, system-audio is only for the audio out of the computer, that does not change anything inside the streaming output. The same is true for embedded audio in SDI.
Didi Kunz
CasparCG Client-Programmer, Template Maker & Live CG-Operator
Media Support, CH-5722 Gränichen, Switzerland http://mediasupport.ch/
Problems? Guide to posting Bug reports & Feature requests

Re: FFmpeg Streaming Consumer - RawVideo without Audio

#6
I assume because even though there is no volume there is still byte data pertaining to the audio stream in the raw frame. I'm not sure if this differs when using pcm versus mp3. I would expect that if mp3 is used the mp3 header / id3 tag would still be written to the stream. When dealing with the raw video frame it cannot be even a single byte off otherwise I can't decode it.

When you said mute were you referring to the mixer command to set the volume to zero? That is the only way I have ever muted the audio on a channel, is there another AMCP command?

Thanks for the heads regarding system audio and embedded audio, good to know.