H.264 FU-A RTP Fragmentation: Diagnosing Packet Loss, Reassembly Failure, and Broken RTSP Video

H.264 over RTP often breaks in ways that look like decoder bugs but are really packetization or packet-loss problems. The RTSP session works. RTP packets arrive. The payload type maps to H.264. Still, the video shows block corruption, freezes, black frames, "invalid NAL unit", "missing picture", "non-existing PPS", or "decode_slice_header error." Users search for "H264 FU-A RTP fragmentation", "RTSP H264 packet loss", "RTP NAL unit reassembly", "FU-A missing fragment", and "H264 over RTP decoder error" because the media payload is arriving but cannot be reconstructed cleanly.

RtspInspector is useful because H.264 RTP diagnostics require packet evidence: sequence numbers, timestamps, marker bits, NAL unit types, FU-A start/end bits, and packet loss around fragmented access units.

Why FU-A exists

An H.264 NAL unit can be larger than the network packet size. RTP payload format for H.264 supports fragmentation units. FU-A is a common fragmentation mode that splits one NAL unit across multiple RTP packets.

A fragmented NAL unit has:

• FU indicator
• FU header
• Start bit
• End bit
• Original NAL unit type
• Fragment payload bytes

The receiver must reassemble all fragments in order before giving the NAL unit to the decoder.

What a healthy FU-A sequence looks like

A simplified sequence:

RTP seq 1000: FU-A Start
RTP seq 1001: FU-A middle
RTP seq 1002: FU-A middle
RTP seq 1003: FU-A End, marker bit maybe set

All fragments share the same RTP timestamp because they belong to the same video time. If any fragment is missing, the reconstructed NAL unit is incomplete.

Missing middle fragment

If packet 1002 is lost:

1000 Start
1001 middle
1003 End

The receiver should treat the NAL unit as damaged. Passing incomplete data to the decoder can create corruption. Dropping the damaged access unit may produce a visible freeze or skipped frame, but it is often better than feeding broken NAL data.

Missing start or end fragment

Missing start fragment means the receiver may not know the original NAL type. Missing end fragment means the receiver never sees a complete NAL unit boundary.

Symptoms:

• Decoder waits for a complete frame.
• Video freezes until next IDR frame.
• Corruption lasts several frames.
• Error recovery depends on keyframe interval.

Long GOP settings make packet loss more visible because a lost reference frame can affect later frames.

MTU and fragmentation

Large NAL units cause more RTP fragmentation. Higher resolution, higher bitrate, and large keyframes increase fragment count. More fragments mean more chances for packet loss to damage a frame.

If corruption appears mostly on keyframes, inspect IDR frame fragmentation and packet loss around large bursts.

TCP interleaved vs UDP

UDP packet loss can drop individual FU-A fragments. TCP interleaved transport avoids UDP loss but can introduce latency and head-of-line blocking. If UDP shows missing RTP sequence numbers and TCP interleaved does not, the issue is likely media transport loss rather than H.264 encoder syntax.

Debug checklist

Use this workflow:

1. Confirm SDP maps payload type to H.264.
2. Track RTP sequence numbers.
3. Identify fragmented FU-A payloads.
4. Check FU-A start and end bits.
5. Verify all fragments in a NAL unit arrive.
6. Compare packet loss with visible corruption.
7. Inspect whether IDR frames are affected.
8. Check RTP timestamp and marker bit boundaries.
9. Compare UDP and TCP interleaved transport.
10. Consider lowering bitrate or keyframe size if network loss is confirmed.

Final diagnosis

H.264 FU-A problems are reassembly problems. A stream can carry RTP packets and still produce broken video if fragmented NAL units lose packets, arrive out of order, miss start/end fragments, or cross MTU-sensitive network paths.

RtspInspector helps expose FU-A packet structure and RTP continuity so H.264 corruption can be traced to packet evidence instead of being treated as a vague decoder failure.