RTSP 461 Unsupported Transport Fix: SETUP Failed, UDP vs TCP, ffmpeg Camera Transport Error

Fix RTSP 461 Unsupported Transport and ffmpeg \\\"method SETUP failed: 461\\\". Covers UDP vs TCP interleaved, camera Transport headers, Frigate, Scrypted, and NVR integration transport errors.

rtsp 461 unsupported transport, method setup failed, ffmpeg rtsp, udp vs tcp, interleaved transport, camera troubleshooting, rtsp diagnostics

An ffmpeg or NVR session may complete OPTIONS and DESCRIBE, then stop at method SETUP failed: 461 Unsupported Transport. That sequence says the RTSP URL and authentication reached the server; the disputed part is whether the camera accepts the requested UDP ports, multicast mode, or RTP/AVP/TCP interleaved transport. Comparing the exact Transport request with the 461 response can prove a negotiation mismatch, while it cannot show undocumented transport modes that the firmware never advertises.

RTSP Inspector is useful here because the decisive evidence is the SETUP request and the Transport header.

What SETUP does

After DESCRIBE, the client knows the stream tracks from SDP. Then it sends SETUP for each track to negotiate media transport.

UDP example:

SETUP rtsp://camera/stream/trackID=1 RTSP/1.0
Transport: RTP/AVP;unicast;client_port=50000-50001

TCP interleaved example:

SETUP rtsp://camera/stream/trackID=1 RTSP/1.0
Transport: RTP/AVP/TCP;unicast;interleaved=0-1

If the camera does not support the requested mode, it may return:

RTSP/1.0 461 Unsupported Transport

UDP-only and TCP-only behavior

Many cameras support both UDP RTP and TCP interleaved RTP. Some do not. Some older cameras support only UDP. Some cloud relays or restreamers support only TCP interleaved. Some NVR paths behave differently from direct camera paths.

Symptoms:

  • UDP SETUP fails but TCP works.
  • TCP interleaved SETUP fails but UDP works.
  • Direct camera works, restream URL fails.
  • Main stream supports one mode, sub-stream supports another.
  • Client retries from UDP to TCP and still fails because the server rejects both requested formats.

The fix is not "always use TCP" or "always use UDP." The fix is to identify what the server actually accepts.

Transport header syntax matters

Some RTSP servers are strict. They may reject headers that are valid in theory but not accepted by firmware.

Compare:

Transport: RTP/AVP;unicast;client_port=50000-50001

and:

Transport: RTP/AVP/UDP;unicast;client_port=50000-50001

Some devices treat these differently. Some reject multicast. Some reject a client port range outside expected bounds. Some reject interleaved channel numbers not starting from zero.

RTSP Inspector should preserve the exact header string, not just a simplified "TCP" or "UDP" label.

Restreamers and proxy paths

When RTSP comes through a restreamer, NVR, or media bridge, the upstream and downstream transport capabilities may not match. The camera may support UDP, but the restreamer only exposes TCP. Or the restreamer may accept a SETUP request but fail when mapping media channels.

If a URL points to 127.0.0.1:8554 or another restreamer instead of the camera, diagnose that server's RTSP behavior, not only the camera's.

Debug checklist

Use this workflow:

  1. Capture DESCRIBE and SDP.
  2. Identify the track URL used in SETUP.
  3. Inspect the exact Transport header.
  4. Record the 461 Unsupported Transport response.
  5. Try UDP and TCP interleaved modes deliberately.
  6. Check whether the camera supports multicast, UDP unicast, or TCP interleaved.
  7. Compare main stream and sub-stream.
  8. Compare direct camera URL and NVR/restream URL.
  9. Avoid codec debugging until SETUP succeeds.
  10. Preserve the exact request/response pair for vendor support.

Final diagnosis

RTSP 461 Unsupported Transport means the server rejected the media transport requested during SETUP. The root cause is usually UDP/TCP mode mismatch, strict Transport header syntax, unsupported interleaved mode, unsupported UDP mode, restreamer behavior, or track-specific server limitations.

RTSP Inspector helps diagnose this at the correct layer: RTSP transport negotiation before RTP packets ever appear.

Treat 461 as a negotiated capability mismatch

A 461 response is useful because it happens before ordinary RTP delivery. The server saw a track URL and a Transport request, then declined that combination. Record the exact request/response pair before changing camera encoding, credentials, or firewall settings. Those later settings may matter to another failure, but they are not the first boundary in a 461 incident.

Evidence What it answers
DESCRIBE SDP Which tracks and control URLs the server advertised
SETUP request URI Which resource the client actually asked to configure
Requested Transport header UDP, multicast, or TCP-interleaved intent and parameters
461 response headers/body Whether the server advertises an accepted alternative
Client mode/config Why this transport was requested
Camera profile and firmware Which endpoint capability was being tested
Direct versus restream target Whether the responder is the camera or a proxy
Test time Lets support correlate logs to one request

A response cannot prove undocumented modes the firmware never reports. It can prove a mismatch between the requested parameters and the server behavior at that moment.

Compare one transport variable at a time

Do not flip several player flags and report that “TCP fixed it.” Keep URL, credentials, track, camera profile, and client version fixed while varying only the transport request. The resulting matrix tells an engineer whether the issue is generic SETUP, UDP-specific, TCP-interleaved-specific, or tied to a restream path.

Test Requested Transport Interpretation if it succeeds
UDP unicast RTP over client port pair Server accepts direct UDP for this track
TCP interleaved RTP inside RTSP TCP connection Server accepts interleaved channels for this track
Multicast if supported Multicast group transport Server permits that group-oriented mode
Same mode, direct camera Camera endpoint rather than proxy Separates camera capability from restream policy
Same mode, sub-stream Lower-profile track Finds track/profile-specific limitation
Same request with validated syntax Only header form changes Reveals strict firmware parsing if behavior differs

If UDP SETUP succeeds but no media arrives after PLAY, the 461 problem is solved and the next question is data-plane reachability. Use RTSP UDP/RTP firewall and NAT diagnostics rather than continuing to call it a transport-negotiation failure.

Read Transport parameters as a complete contract

The protocol token alone is not enough. Servers can be strict about client port pairs, multicast requests, unicast flags, interleaved channel numbers, and the exact profile form they accept. A camera may accept one syntactically valid header form and reject another due to limited firmware parsing or a product-specific policy. Preserve the literal header in the report.

Parameter Why it can trigger 461
RTP profile token Firmware may support only a narrow RTP/AVP or TCP form
unicast or multicast The profile may not permit the requested delivery model
client_port range Ports may be invalid, blocked by policy, or outside accepted bounds
interleaved channels The server may expect different channel allocation or TCP mode
mode or extension attributes Extra parameters can be unsupported by a strict server
Track control URL A video track and audio track can have different acceptance behavior
Aggregate versus track SETUP Server may require a specific resource form

The RTSP TCP interleaved channel mismatch guide is useful when SETUP accepts TCP but packets later appear on unexpected channels. That is a different failure from a server rejecting the Transport header with 461.

Restreamers create two independent transport boundaries

A URL aimed at an NVR, local bridge, or restreamer is not a direct camera test. The proxy may use one mode upstream and expose another downstream; it may also reject a client transport mode even if the camera itself supports it. Name the responder by address, port, product role, and path in every comparison.

Pattern Bounded conclusion
Direct camera accepts UDP; restreamer returns 461 Downstream proxy policy/capability is implicated
Direct camera rejects TCP; proxy accepts TCP Proxy may translate or serve a different downstream mode
Main stream works; sub-stream returns 461 Track/profile capability differs
Video SETUP works; audio track returns 461 Track-specific policy or transport differs
Both endpoints reject same mode Client request or shared device policy needs review
Both modes return 461 Inspect exact URI/header and endpoint documentation/support evidence

Do not claim that a restreamer “breaks the camera” from this alone. The trace identifies which RTSP server rejected which contract; that is the right owner for the next configuration or support question.

A support-ready 461 workflow

  1. Capture OPTIONS, DESCRIBE, and the full SDP.
  2. Select one advertised track and record its control URL.
  3. Preserve the exact SETUP request and 461 response.
  4. Repeat one controlled UDP or TCP-interleaved alternative.
  5. Keep the client and camera profile unchanged during that comparison.
  6. Repeat against direct camera and proxy only when both are authorized.
  7. If SETUP succeeds, move to PLAY/RTP evidence and stop calling the issue 461.
  8. Send the request matrix, not just an application error string, to the responsible owner.
Handoff item Why it prevents rework
Sanitized RTSP URL and responder identity Tells support which endpoint produced 461
SDP and selected track Connects SETUP to the actual advertised resource
Literal Transport header Preserves the disputed capability request
Status and server headers Records the server's response rather than a paraphrase
Transport matrix Shows what changed and what did not
Profile/client version/test time Makes reproduction possible
Next boundary Distinguishes rejected SETUP from later UDP/RTP delivery

RTSP Inspector is designed for that evidence chain, not for hiding it behind a black preview window. It can make the correct layer visible before a team changes unrelated codec settings.

FAQ: RTSP 461 Unsupported Transport

Does 461 mean the RTSP URL is wrong?

Usually not by itself. OPTIONS and DESCRIBE may already have succeeded. The immediate dispute is the SETUP resource and Transport contract. Verify the track URL from SDP, but focus the first test on the rejected request.

Should I always force TCP?

No. TCP interleaved is a valuable comparison but not a universal permanent setting. Camera support, NVR behavior, latency, capacity, firewall policy, and the deployment environment determine the right mode. Use the test to identify what the current server accepts.

Can an audio track cause 461 while video works?

Yes. Tracks can have separate control URLs and policy. Capture which SETUP failed and inspect the session structure with AAC audio and unknown SDP track diagnostics.

What if SETUP succeeds but the stream is still blank?

Then 461 is no longer the current failure. Capture PLAY and early RTP, then diagnose payload, network delivery, timestamp, keyframe, and codec readiness. Start with RTSP connects but no video.

How do you accept a correction for RTSP 461?

Preserve the failing SETUP request and response, resolved media control URL, requested Transport alternatives, unicast/multicast mode, UDP ports or interleaved channels, track type, session state, and server firmware/profile. Then change one transport field at a time. A broad client fallback that silently changes URL, profile, authentication, and transport cannot prove which element caused 461.

Controlled case Evidence Pass condition
Vendor-supported transport Exact SETUP/Transport pair Server accepts and returns a compatible selection
Unsupported lower transport Repeated 461 with documented reason Client reports a bounded compatibility error
Video and audio tracks Separate control URLs and responses Policy differences are visible per track
UDP versus interleaved TCP Same media profile and session policy Comparison changes only transport
Reconnect New RTSP session No stale Session/port/channel state is required

Inspect the server reply, not only the status code. If the server accepts a different transport than requested, verify that the client can actually receive and map it. If SETUP advances but PLAY or media fails, close the 461 diagnosis and move to the new boundary instead of continuing to tune transport headers.

Acceptance requires two fresh sessions, successful SETUP for every required track, PLAY, expected RTP/RTCP or interleaved channel traffic, and a first decodable frame. Keep the failed and corrected SETUP sequences in an RTSP Inspector report and use interleaved channel mismatch diagnostics if accepted channels are later mapped incorrectly.

Review Transport alternatives without silent fallback

If the client sends several Transport alternatives, preserve their order and the server’s exact selection. Some servers reject a combined header that contains an unsupported first alternative; others select a supported alternative. Test the documented single transport first, then the multi-alternative request, keeping the media track and session state fixed.

For UDP, record client_port, returned server_port, source/destination parameters, unicast or multicast, and RTCP policy. For interleaved TCP, record channel pairs and verify they do not collide across audio and video. For multicast, include group and TTL/source behavior where provided.

The client should surface a bounded compatibility message when no offered transport is supported, not loop SETUP indefinitely or proceed to PLAY with an unusable mapping. The server should remain available for a new supported request. Retain both rejection and accepted response so firmware and client teams can compare the same contract without reconstructing it from logs.

<!-- multilingual-related-reading:start -->

Related guides

Continue with the same-language pages below. They cover adjacent stages without changing the canonical owner of this topic:

<!-- multilingual-related-reading:end -->