RTSP Digest Authentication Debugging: 401 Unauthorized, Nonce, Realm, Basic vs Digest, and Camera Login Loops

RTSP authentication failures are some of the most common IP camera support cases. Users search for "RTSP 401 Unauthorized", "RTSP Digest authentication failed", "camera works in VLC but not NVR", "RTSP Basic vs Digest", "stale nonce", "wrong realm", and "IP camera login loop" because the symptom is simple but the cause is hidden inside the request and response headers.

RTSP Inspector is built for this exact class of problem. A player may only say "authentication failed". A protocol inspector can show the first unauthenticated DESCRIBE, the WWW-Authenticate challenge, the client Authorization response, the nonce value, the realm, the URI used in the digest calculation, and whether the camera rejects the second request.

Why RTSP authentication is confusing

Many cameras do not accept credentials on the first request. The normal Digest flow is:

1. Client sends DESCRIBE without authorization.
2. Camera returns 401 Unauthorized.
3. Camera includes WWW-Authenticate: Digest ....
4. Client recalculates the Digest response.
5. Client sends DESCRIBE again with Authorization: Digest ....
6. Camera either accepts the request or returns another 401.

The first 401 is not necessarily an error. The repeated 401 after the client sends Digest credentials is the important evidence.

Basic vs Digest camera settings

Some cameras expose a setting such as:

• Basic authentication
• Digest authentication
• Basic and Digest
• Digest only
• No authentication

If the client only supports Basic but the camera requires Digest, the stream fails. If the client sends Digest but the camera is configured for a vendor-specific variant, the stream may also fail.

Search terms that often describe this case:

• "RTSP Basic authentication camera"
• "RTSP Digest authentication camera"
• "VLC works but app gets 401"
• "NVR camera authentication failed"
• "ONVIF works but RTSP login fails"

The fix is not to guess the password again. First inspect which authentication scheme the camera actually advertised.

Realm mismatches

The Digest realm is part of the authentication calculation. If the client calculates the response with a different realm than the camera provided, authentication fails.

This can happen when:

• A proxy rewrites the challenge.
• Firmware changes the camera realm after upgrade.
• The client caches a previous challenge.
• Multiple cameras share one hostname through a reverse proxy.
• The app uses a saved profile from a different camera model.

RTSP Inspector should make the realm visible so the failure becomes concrete. The question is not "is the password wrong?" but "which exact realm and URI were used when the Digest response was generated?"

Nonce and stale nonce problems

The Digest nonce is a server-provided value. Some cameras expire it quickly. Some cameras reuse it for a session. Some cameras reject old nonces after reboot, firmware update, time drift, or too many failed attempts.

Useful evidence:

• Does the camera include stale=true?
• Does the client retry with a new nonce?
• Does the camera send a different nonce after every 401?
• Does authentication work once and then fail later?
• Does the same URL fail after the camera has been idle?

If the camera returns a fresh challenge but the client keeps sending the old nonce, the failure is client-side caching. If the camera returns repeated challenges with no useful progress, the issue may be camera firmware, lockout policy, or credential mismatch.

URI mismatch inside Digest authentication

Digest authentication includes the requested URI. A subtle mismatch can break login:

• Client connects to rtsp://192.168.1.10/stream1.
• Digest response is calculated for /stream1.
• Camera expects rtsp://192.168.1.10:554/stream1.
• Proxy forwards /live/stream1.
• Client retries with a normalized URL.

This is why raw request lines matter. The DESCRIBE URI, the Authorization header URI, and the final camera path should be compared.

Password is not the only cause

Support teams often reset passwords too early. Repeated 401 Unauthorized can also mean:

• Wrong authentication scheme.
• Digest realm mismatch.
• Stale nonce.
• URL path mismatch.
• Camera account has no RTSP permission.
• Account is locked after failed login attempts.
• Special characters in username or password are not URL encoded.
• Client stripped credentials from the redirected or retried URL.
• Camera requires ONVIF user creation before RTSP access.

The best article for SEO should say this clearly because many searches start with the assumption that the password is wrong.

Special characters in RTSP URLs

RTSP URLs often contain inline credentials:

rtsp://user:[email protected]:554/stream1

If the password contains @, :, /, ?, #, or %, the URL parser may split the string incorrectly. The protocol trace can show whether the client actually sent the intended username and whether the request path was damaged.

Better diagnostics separate:

• URL parsing.
• Authentication challenge.
• Digest calculation.
• Camera authorization decision.

What to capture

For a useful RTSP authentication report, collect:

• Full request method sequence: OPTIONS, DESCRIBE, SETUP, PLAY.
• First 401 Unauthorized response.
• WWW-Authenticate header.
• Authentication scheme.
• Realm.
• Nonce.
• Stale flag.
• Client Authorization header metadata.
• Request URI used for Digest.
• Second or third camera response.
• Timing between retries.

Do not publish passwords or full Digest response values in public support cases. For internal debugging, preserve enough header structure to prove the protocol path.

Diagnosis workflow

Use this process:

1. Confirm whether the first 401 is only a challenge.
2. Check whether the client retries with Authorization.
3. Compare Basic vs Digest.
4. Compare realm and nonce values.
5. Check whether stale=true appears.
6. Verify the URI used in the authorization header.
7. Check whether credentials contain reserved URL characters.
8. Confirm the account has RTSP permissions.
9. Test the same camera path after reboot or lockout window.
10. Save the trace for regression testing.

Final diagnosis

RTSP Digest authentication failures should be diagnosed from headers, not from player error text. The important evidence is the challenge, the retry, the nonce, the realm, the URI, and the final camera decision.

RTSP Inspector helps turn "RTSP 401 Unauthorized" into a specific finding: wrong scheme, stale nonce, realm mismatch, URL credential parsing, account permission, or camera lockout.