Can't get to ingest WHIP from FFMPEG

[DWAM-gdl]

Hi

I'm trying to push WHIP to Mist from FFMPEG.
Mist version is 3.9 Free_aarch64
Encoder : RPI5 with Trixie 64 Bits
FFMPEG : 25/11/11 version for arm64

It works fine from OBS. I would like to use my own FFMPEG script for vMix output.

I have a basic FFMPEG test script. This script works perfectly with Broadcast Box ingest point [https://b.siobud.com/]

ffmpeg -hide_banner -re -i <my-input> -acodec libopus -ar 48000 -ac 2 \ -vcodec libx264 -profile:v high -preset ultrafast -tune zerolatency -threads 0 -bf 0 \ -authorization mytoken \ -f whip "http://192.168.0.72:8080/webrtc/whip"

I tried with and without the "-authorization" parameter, as described in the FFMPEG documentation : optional Bearer token for WHIP Authorization.

I can provide error messages from FFMPEG (quite long red lines) and Mist usually reports this

[11 Nov 2025, 19:18:10] MistOutWebRTC:whip (313298) [WARN] No SDP::MediaFormat found for encoding name RED. (../build/lib/sdp_media.cpp:582)
[11 Nov 2025, 19:18:10] MistOutWebRTC:whip (313298) [WARN] No SDP::MediaFormat found for encoding name ULPFEC. (../build/lib/sdp_media.cpp:582)
[11 Nov 2025, 19:18:10] MistOutWebRTC:whip (313298) [WARN] The selected profile-level-id was rewritten to 42e01f (../build/lib/sdp_media.cpp:1004)

and sometimes things like this too

[11 Nov 2025, 18:39:11] MistOutWebRTC:whip (312911) [WARN] Ending unfinished FU-A (../build/lib/rtp.cpp:1307)
[11 Nov 2025, 18:40:26] MistOutWebRTC:whip (312911) [WARN] Ending unfinished FU-A (../build/lib/rtp.cpp:1307)
[11 Nov 2025, 18:41:29] MistOutWebRTC:whip (312915) [WARN] Dropping invalid track 0: disappeared from metadata (../build/src/output/output.cpp:2252)
[11 Nov 2025, 18:41:29] MistOutWebRTC:whip (312915) [WARN] Dropping invalid track 1: disappeared from metadata (../build/src/output/output.cpp:2252)

I will also try from windows using NVENC and report if different.

Any idea what could be wrong ?
Let me know if you need more info or whatever

Best regards and thanks for the good work with Mist
G.

[Rokamun]

Heya, thanks for the report!

I must admit I haven't had the chance to test against ffmpeg. I don't think I've got versions yet that have this build in, so I might have to compile one myself for this... (I'll try and find some time for this soon to run some own checks.)

I can confirmthat we do not care about the authorization token. It's something you can use, but just the address to http://address:port/webrtc/streamname should be enough.

We do have some extra debug information available within WebRTC, sneakily hidden away:

There's a write packet log tickbox hidden away in the WebRTC optional parameters. If ticked the WeBRTC SDP offers are saved into /tmp/ as wrtc_http_PID. Chances are if you open that (with any text editor/viewer) you get way more useful information and we might want a copy of it! Feel free to mail it to info@mistserver.org . It does contain your address information used, so you might not want to put that publicly.

[DWAM-gdl]

Thanks a lot for the follow up !
Will test this asap and report

FYI you can easily find FFmpeg builds with WHIP support for all architectures. I used this one :
https://github.com/BtbN/FFmpeg-Builds/releases
FFmpeg has WHIP support for about 4 or 5 months ago already

Best regards

[DWAM-gdl]

I just tried with the packet log option. Here is the SDP offer and Mist answer (internal server so nothing to hide)

---

[894153605] SDP offer (them):
v=0
o=FFmpeg 4489045141692799359 2 IN IP4 127.0.0.1
s=FFmpegPublishSession
t=0 0
a=group:BUNDLE 0 1
a=extmap-allow-mixed
a=msid-semantic: WMS
m=audio 9 UDP/TLS/RTP/SAVPF 111
c=IN IP4 0.0.0.0
a=ice-ufrag:8956e956
a=ice-pwd:a5b76cc2514a21ca5602bdc4f7b655ff
a=fingerprint:sha-256 BA:49:1C:98:8F:9F:01:A4:B2:64:A5:26:E9:7C:14:5E:E2:F6:63:E0:F8:09:8D:44:BB:4F:B5:6E:00:8B:AB:15
a=setup:passive
a=mid:0
a=sendonly
a=msid:FFmpeg audio
a=rtcp-mux
a=rtpmap:111 opus/48000/2
a=ssrc:1349451338 cname:FFmpeg
a=ssrc:1349451338 msid:FFmpeg audio
m=video 9 UDP/TLS/RTP/SAVPF 106 105
c=IN IP4 0.0.0.0
a=ice-ufrag:8956e956
a=ice-pwd:a5b76cc2514a21ca5602bdc4f7b655ff
a=fingerprint:sha-256 BA:49:1C:98:8F:9F:01:A4:B2:64:A5:26:E9:7C:14:5E:E2:F6:63:E0:F8:09:8D:44:BB:4F:B5:6E:00:8B:AB:15
a=setup:passive
a=mid:1
a=sendonly
a=msid:FFmpeg video
a=rtcp-mux
a=rtcp-rsize
a=rtpmap:106 H264/90000
a=fmtp:106 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=420028
a=rtcp-fb106 nack
a=rtpmap:105 rtx/90000
a=fmtp:105 apt=106
a=ssrc-group:FID 1349451339 1349451340
a=ssrc:1349451339 cname:FFmpeg
a=ssrc:1349451339 msid:FFmpeg video

[894153606]Adding incoming main socket 3: 224.0.0.224:18203
[894153869] SDP answer (us):
v=0
o=- 1762965534403174 0 IN IP4 0.0.0.0
s=-
t=0 0
a=ice-lite
a=group:BUNDLE 0 1
m=audio 9 UDP/TLS/RTP/SAVPF 111
c=IN IP4 0.0.0.0
a=recvonly
a=rtcp:9
a=setup:passive
a=fingerprint:sha-256 81:F6:FB:5D:39:A7:4F:09:94:74:47:A8:B4:B1:EC:FA:98:B9:B8:92:94:67:83:4A:FA:DA:37:9A:EE:1D:B0:66
a=ice-ufrag:UH0lHuETNmbPAca5
a=ice-pwd:MbJyUYawdNkfwWua1GNi76t8vAVvEsAn
a=rtcp-mux
a=rtcp-rsize
a=rtpmap:111 opus/48000/2
a=rtcp-fb:111 nack
a=mid:0
a=fmtp:111 minptime=10;useinbandfec=1
a=candidate:1 1 udp 2130706431 192.168.0.72 18203 typ host
a=candidate:1 1 udp 2130706431 172.22.0.72 18203 typ host
a=candidate:1 1 udp 2130706431 fe80::35e0:6210:6409:94d3 18203 typ host
a=candidate:1 1 udp 2130706431 fe80::4c4f:3bff:fedb:9672 18203 typ host
a=end-of-candidates
m=video 9 UDP/TLS/RTP/SAVPF 106
c=IN IP4 0.0.0.0
a=recvonly
a=rtcp:9
a=setup:passive
a=fingerprint:sha-256 81:F6:FB:5D:39:A7:4F:09:94:74:47:A8:B4:B1:EC:FA:98:B9:B8:92:94:67:83:4A:FA:DA:37:9A:EE:1D:B0:66
a=ice-ufrag:UH0lHuETNmbPAca5
a=ice-pwd:MbJyUYawdNkfwWua1GNi76t8vAVvEsAn
a=rtcp-mux
a=rtcp-rsize
a=rtpmap:106 H264/90000
a=rtcp-fb:106 nack
a=rtcp-fb:106 goog-remb
a=mid:1
a=fmtp:106 profile-level-id=42e01f;level-asymmetry-allowed=1;packetization-mode=1
a=candidate:1 1 udp 2130706431 192.168.0.72 18203 typ host
a=candidate:1 1 udp 2130706431 172.22.0.72 18203 typ host
a=candidate:1 1 udp 2130706431 fe80::35e0:6210:6409:94d3 18203 typ host
a=candidate:1 1 udp 2130706431 fe80::4c4f:3bff:fedb:9672 18203 typ host
a=end-of-candidates

---

Yes I have 2 network interfaces, does not seem to cause problems with Mist, at least for other protocols...
Server is a Raspberry Pi 4 with Debian 13

Hope this helps !

[DWAM-gdl]

Here is what I see in the terminal when I execute my FFmpeg script...

Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/home/dwam/Videos/test.mp4':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
encoder : Lavf56.40.101
Duration: 00:04:12.91, start: 0.000000, bitrate: 4162 kb/s
Stream #0:00x1: Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709, progressive), 1920x1080 [SAR 1:1 DAR 16:9], 4030 kb/s, 29.97 fps, 29.97 tbr, 90k tbn (default)
Metadata:
handler_name : VideoHandler
vendor_id : [0][0][0][0]
Stream #0:10x2: Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 125 kb/s (default)
Metadata:
handler_name : SoundHandler
vendor_id : [0][0][0][0]
Stream mapping:
Stream #0:0 -> #0:0 (h264 (native) -> h264 (libx264))
Stream #0:1 -> #0:1 (aac (native) -> opus (libopus))
Press [q] to stop, [?] for help
[vist#0:0/h264 @ 0x555631ab3920] Resumed reading at pts 0.000 with rate 1.050 after a lag of 0.500s
[libopus @ 0x555631bb5be0] No bit rate set. Defaulting to 96000 bps.
[libx264 @ 0x555631ab53b0] using SAR=1/1
[libx264 @ 0x555631ab53b0] using cpu capabilities: ARMv8 NEON DotProd
[libx264 @ 0x555631ab53b0] profile Constrained Baseline, level 4.0, 4:2:0, 8-bit
[libx264 @ 0x555631ab53b0] 264 - core 165 - H.264/MPEG-4 AVC codec - Copyleft 2003-2025 - http://www.videolan.org/x264.html - options: cabac=0 ref=1 deblock=0:0:0 analyse=0:0 me=dia subme=0 psy=1 psy_rd=1.00:0.00 mixed_ref=0 me_range=16 chroma_me=1 trellis=0 8x8dct=0 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=0 threads=4 lookahead_threads=4 sliced_threads=1 slices=4 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=0 weightp=0 keyint=250 keyint_min=25 scenecut=0 intra_refresh=0 rc=crf mbtree=0 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=0
[http @ 0x7fff7c383260] Stream ends prematurely at 1535, should be 18446744073709551615
[WHIP muxer @ 0x555631ad5680] Failed to read response from url=http://192.168.0.72:8080/webrtc/whip, offer is v=0
o=FFmpeg 4489045141692799359 2 IN IP4 127.0.0.1
s=FFmpegPublishSession
t=0 0
a=group:BUNDLE 0 1
a=extmap-allow-mixed
a=msid-semantic: WMS
m=audio 9 UDP/TLS/RTP/SAVPF 111
c=IN IP4 0.0.0.0
a=ice-ufrag:8956e956
a=ice-pwd:a5b76cc2514a21ca5602bdc4f7b655ff
a=fingerprint:sha-256 BA:49:1C:98:8F:9F:01:A4:B2:64:A5:26:E9:7C:14:5E:E2:F6:63:E0:F8:09:8D:44:BB:4F:B5:6E:00:8B:AB:15
a=setup:passive
a=mid:0
a=sendonly
a=msid:FFmpeg audio
a=rtcp-mux
a=rtpmap:111 opus/48000/2
a=ssrc:1349451338 cname:FFmpeg
a=ssrc:1349451338 msid:FFmpeg audio
m=video 9 UDP/TLS/RTP/SAVPF 106 105
c=IN IP4 0.0.0.0
a=ice-ufrag:8956e956
a=ice-pwd:a5b76cc2514a21ca5602bdc4f7b655ff
a=fingerprint:sha-256 BA:49:1C:98:8F:9F:01:A4:B2:64:A5:26:E9:7C:14:5E:E2:F6:63:E0:F8:09:8D:44:BB:4F:B5:6E:00:8B:AB:15
a=setup:passive
a=mid:1
a=sendonly
a=msid:FFmpeg video
a=rtcp-mux
a=rtcp-rsize
a=rtpmap:106 H264/90000
a=fmtp:106 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=420028
a=rtcp-fb106 nack
a=rtpmap:105 rtx/90000
a=fmtp:105 apt=106
a=ssrc-group:FID 1349451339 1349451340
a=ssrc:1349451339 cname:FFmpeg
a=ssrc:1349451339 msid:FFmpeg video
, answer is (null)
[WHIP muxer @ 0x555631ad5680] Dispose resource http://192.168.0.72:8080/webrtc/XNrAzqg4KV ok
[out#0/whip @ 0x555631aa9520] Could not write header (incorrect codec parameters ?): Input/output error
[vf#0:0 @ 0x555631ab47f0] Error sending frames to consumers: Input/output error
[vf#0:0 @ 0x555631ab47f0] Task finished with error code: -5 (Input/output error)
[vf#0:0 @ 0x555631ab47f0] Terminating thread with return code -5 (Input/output error)
[vist#0:0/h264 @ 0x555631ab3920] Resumed reading at pts 0.133 with rate 1.050 after a lag of 32.790s
[out#0/whip @ 0x555631aa9520] Nothing was written into output file, because at least one of its streams received no packets.
frame= 0 fps=0.0 q=0.0 Lsize= 0KiB time=N/A bitrate=N/A speed=N/A elapsed=0:00:32.45
Conversion failed!

Maybe this could be helpful too ?

[DWAM-gdl]

Sorry don't understand why above message is bold and big

[DWAM-gdl]

As a follow-up I could do more testing with FFmpeg under Windows also with :
https://github.com/BtbN/FFmpeg-Builds/releases

I tried both nvenc and libx264 encoders combined with libopus for the audio. Nothing worked. Neither with Mist Server nor with Broadcast Box (which was unexpected for me).

I won't overload this thread with error messages but I can if you want me to. Just say if it can help.

However I have another information that could be helpful. Usually I prefer to use FFmpeg builds from gyan, here :
https://www.gyan.dev/ffmpeg/builds/
But his builds do not support WHIP yet, so I wrote him on his Github to ask why and if it was on schedule.
He replied : "The WHIP muxer does not currently work with the GNU TLS backend used in my builds. It's under development."

I suspect BtbN builds do not support WHIP correctly. I will investigate more on this side to make sure... It's all very new and there is almost no feedback or useful info to work with yet.

I hope you can fix Mist's WHIP ingest from FFmpeg Linux though as we know it can work, at least with Broadcast Box.

Feel free to ask if you need more info or me to try out things.

Best regards

[Rokamun]

Heya,

a=ssrc:1349451339 cname:FFmpeg
a=ssrc:1349451339 msid:FFmpeg video
, answer is (null)
[WHIP muxer @ 0x555631ad5680] Dispose resource http://192.168.0.72:8080/webrtc/XNrAzqg4KV ok
[out#0/whip @ 0x555631aa9520] Could not write header (incorrect codec parameters ?): Input/output error
[vf#0:0 @ 0x555631ab47f0] Error sending frames to consumers: Input/output error

The could not write header (incorrect codec parameters?) error is basically FFmpeg for error, usually unknown.

This is the important part of the ffmpeg log. This is saying that it didn't receive a respone, yet also received an OK. So it's a bit confusing, most likey it's wanting some kind of specific response and didn't get it. I'll try those BtbN builds and see what I can find.

It's a tad sad as the main FFmpeg builds in Arch Linux do not include WHIP yet either. Most likely the same issue. From what some quick looking around gotten us that it seems like it's an mbedtls vs openssl issue where mbedtls doesn't work with FFmpeg WHIP and openSSL does. Take this with a grain of salt though as we haven't done proper investigations here!

For Windows I'm not super surprised: Windows has a wildly different handling of sockets and it's a reason why we haven't gotten our own WebRTC working in Windows builds right now 😅

As a note: Encoding (nvenc vs libx264) should not matter at all as long as the H264 itself is valid. Obviously encoding settings could increase support, but it shouldn't be a deciding factor on whether it works.

[DWAM-gdl]

Thanks for the fast answer.
It's a bit over my head as I'm a user, not a developer but I feel like your assumptions are right in regards to TLS vs OpenSSL. The error messages I see all talk about DTLS failing (on Windows).

I got an answer from this guy from the BtbN project. It goes in the same direction as you regarding Windows FFmpeg builds WHIP support
You can read our conversation here : BtbN/FFmpeg-Builds#555
There might be precious info for you (I must confess I don't get it all in detail)

In case it might be useful, here is the conversation with Gyan, the other guy doing FFmpeg builds :
GyanD/codexffmpeg#204

I will obviously need to wait a bit more before using this WHIP protocol as I expected with vMix on Windows unless I go with OBS, but I still can experiment and learn WHIP and WHEP with Linux. This is an extremely interesting streaming protocol. What the guys from Broadcast Box have done with it is simply fantastic.

The fact that WHIP from OBS (windows) works OK with Mist Server is already a great achievement. I believe there is room for improvements (there are some errors on connecting sometimes and the negociation takes a lot of time, way more than with Broadcast Box for example, stopping the stream also takes surprisingly quite a lot time) but I had it running for hours and hours without a single problem, the stream from Mist was smooth and consistent and I wouldn't hesitate to use this workflow in production

Have you considered adding support for pulling WHEP feeds into Mist ? It could be a great addon imho... The ultra low latency is one very cool thing but the way WebRTC can simplify routing issues (port management on routers / NAT traversal, etc...) is another big argument...

That's it for now. Thanks again for your commitment and explanations. If you need a tester, count me in !

Bien à toi
Guillaume

[Rokamun]

Heya,

Just wanted to add that we started doing some debugging yesterday on the FFMPEG builds from BtbN and we've gotten further. The first issue was that the ffmpeg binaries required an content-length, which should be optional if you also close the connection in the same request so it wasn't included.
Now that it's added we've gotten a tad further, it's making connection, but bindings seem "wrong" as we're getting 100% packet loss now. That sounds much scarier than it is, it's most likely another setting we're missing. I don't know whether we can put some time into it today, but chances are we'll have this fixed in the coming week or the one after. (Depends a bit on our availability, I'll give you a poke if we have it working).

I'll check out the conversations, it might give a good hint.

We have quite a few things that could be improved on with WebRTC that we know of, the best one in my eyes would be implementing a full STUN server so we can actually holepunch firewalls. Currently a lot of the work is checking all interfaces/paths, deciding which one is working, generating an SSL certificate for the connection and then starting. So yeah...
(You can speed this up somewhat by setting the first to try path and if that works it'll get used immediately within the WebRTC configurations)

WHEP ingest is something we're adding in the future for sure, but I can't say exactly when. We don't have it too high in priority right now as it's the lesser likely scenario when we expect MistServer to be a server freely available and user sending towards it being behind firewalls/complex network configurations. When pushing WHIP this gets broken through, but the server pulling WHEP would mean it'll run into these issues.

[DWAM-gdl]

Hoe gat ?

Oh thank you so much for your concern !

Before I calmly analyse your reply I just wanted to inform you that someone gave a good tip on the BtbN thread I referred to.
He told me I could use a Windows build for FFmpeg with OpenSSL. So I tried and it worked almost right away with my scripts. Just had to increase the buffer_size for the muxer so now I can start improving my code using NVENC.

This new FFMPEG build with WHIP and OpenSSL is here : https://github.com/zhongfly/mpv-winbuild

So yes, it only worked with Broadcast Box. I tried Mist too but without surprize it failed, except I no longer see the DTLS error obviously.
Do you want me to reactivate the packet log feature and report ?
Not sure it would help but just tell me, that's the least I can do to help...

I will keep on working on WHIP. Found a new endpoint to test with last night : https://meshcast.io/

Dankjewel

[Thulinma]

Hey - just posting here to let you know we've not forgotten about this issue and we do plan to fix the compatibility with ffmpeg's WHIP implementation. We will post back here if/when there's progress, but please be aware it may take some time.