2026-06-02

HTTP/2 GOAWAY and RST_STREAM PCAP Analysis: Debugging Reset Streams, Proxy Limits, and gRPC Failures

How to diagnose HTTP/2 GOAWAY, RST_STREAM, gRPC unavailable errors, proxy stream limits, TLS ALPN negotiation, connection reuse, and packet capture evidence.

A gRPC UNAVAILABLE or stream reset may affect one HTTP/2 stream while neighboring requests continue over the same TCP connection; GOAWAY instead changes the connection's future. When decryption is available, map RST_STREAM, error codes, stream IDs, and last-stream ID to the failed request, then distinguish an HTTP/2 proxy reset from TCP or TLS termination. Without session keys, a PCAP still bounds connection timing and ALPN negotiation but cannot attribute encrypted HTTP/2 control frames.

PCAP Surgery is useful because HTTP/2 evidence must preserve TLS, ALPN, connection timing, stream resets, and TCP close behavior. If TLS is encrypted and keys are unavailable, packet captures still show timing, TCP resets, connection reuse, and sometimes decrypted HTTP/2 only in controlled environments.

HTTP/2 connection vs stream

HTTP/2 multiplexes multiple streams over one connection. A stream reset is not the same as a TCP connection reset.

RST_STREAM: one stream is cancelled or failed.
GOAWAY: endpoint is closing or draining the HTTP/2 connection.
TCP FIN/RST: the underlying connection closes or aborts.

Applications often collapse these into one error. Packet evidence and logs should separate them.

ALPN negotiation

HTTP/2 over TLS usually depends on ALPN. The TLS handshake negotiates h2 or another protocol. If ALPN does not negotiate HTTP/2, the client and server may fall back or fail.

Preserve:

ClientHello ALPN extension.
Server-selected ALPN where visible.
TLS alerts.
TCP resets during handshake.

If HTTP/2 never negotiated, do not debug RST_STREAM yet.

GOAWAY

GOAWAY tells the peer that no new streams should be created on that connection. It can be normal during graceful draining, deploys, proxy connection aging, or load balancer behavior. It becomes a problem when clients reuse draining connections incorrectly or when GOAWAY appears during active requests.

Important questions:

Who sent GOAWAY?
What was the last stream ID?
Did active streams fail?
Did client retry on a new connection?
Does GOAWAY happen at fixed connection age?

RST_STREAM

RST_STREAM terminates one HTTP/2 stream. Causes include:

Client cancellation.
Server rejecting a request.
Proxy timeout.
Flow-control issue.
Max stream limit.
gRPC deadline exceeded.
Backend reset translated by proxy.

The stream ID and timing matter. Without them, the packet story is incomplete.

TCP layer still matters

HTTP/2 sits on TCP. If the underlying connection has retransmissions, zero window, reset, MTU problems, or idle timeout, HTTP/2 errors may be secondary.

Correlate:

Stream reset time.
TCP retransmissions before reset.
FIN/RST sender.
Idle interval.
TLS close_notify if visible.

Checklist

Use this workflow:

Preserve DNS, TCP, and TLS handshake.
Confirm ALPN negotiated HTTP/2.
Identify whether failure is stream-level or connection-level.
Look for GOAWAY timing and sender.
Look for RST_STREAM timing and stream ID in decrypted traces or logs.
Correlate with proxy/load-balancer logs.
Check TCP retransmission, zero window, FIN, and RST.
Check whether client retries correctly.
Preserve packet timing when trimming.
Combine pcap evidence with HTTP/2 debug logs when encrypted.

Final diagnosis

HTTP/2 GOAWAY and RST_STREAM errors are not generic network failures. They are stream and connection control signals that must be correlated with ALPN, proxy behavior, gRPC deadlines, TCP health, and connection reuse.

PCAP Surgery helps preserve the timeline so HTTP/2 and gRPC failures can be reduced to the right layer: TLS negotiation, stream reset, connection drain, proxy timeout, or TCP transport failure.

Decode HTTP/2 control frames without mixing layers

When decryption is authorized and available, record frame type, stream ID, flags, sender, error code, and connection-relative time. RST_STREAM belongs to one stream. GOAWAY applies to connection admission and carries a last-stream ID. TCP RST aborts all streams on that TCP connection.

Signal	Scope	Key field	Client behavior to verify
RST_STREAM	One stream	Stream ID and error code	Retry only when method/policy permits
GOAWAY	Connection	Last-stream ID and error code	Stop new streams, retry eligible later IDs
TCP FIN	Underlying connection	Direction and close sequence	Drain/reconnect cleanly
TCP RST	All streams	Sender and prior state	Classify ambiguous in-flight calls
gRPC status/trailers	One RPC	Status and message	Surface correct application result

A GOAWAY with NO_ERROR can be normal graceful drain. Determine whether the failed stream ID is greater than the advertised last-stream ID, already processed, or still ambiguous. Client retry policy must account for idempotency; blindly replaying a non-idempotent RPC can duplicate work.

Common GOAWAY patterns

Fixed connection age

Proxies and servers may drain HTTP/2 connections after a configured age. Plot connection start to GOAWAY across several flows. If the duration is consistent, compare keepalive/max-age configuration and verify the client opens a replacement before exhausting capacity.

Deployment or backend drain

A rolling deploy may send GOAWAY while finishing accepted streams. Preserve active stream IDs, last-stream ID, client retries, and new connection timing. Correlate with deployment events rather than labeling graceful drain as packet loss.

Error GOAWAY

PROTOCOL_ERROR, ENHANCE_YOUR_CALM, FLOW_CONTROL_ERROR, or similar codes require frame-level and endpoint logs. Record preceding SETTINGS, WINDOW_UPDATE, header behavior, frame sizes, and rate/policy context. Avoid inferring the invalid frame from the code alone.

RST_STREAM attribution

Identify the sender and error code. CANCEL can reflect client deadline/cancellation, proxy timeout, or upstream cancellation translated across legs. REFUSED_STREAM may be retryable under HTTP/2 semantics. FLOW_CONTROL_ERROR points toward window accounting but still requires endpoint logs.

Pattern	Next evidence
Client RST_STREAM at deadline	Client deadline and cancellation logs
Proxy sends reset at fixed duration	Route/upstream timeout configuration
Server reset after request headers/data	Application or server HTTP/2 logs
Neighboring streams succeed	Stream-specific cause more likely
All streams fail with TCP RST	Transport/connection-level investigation

For a proxy, client and backend may use separate HTTP/2 or HTTP/1.1 connections. A downstream RST_STREAM may represent an upstream TCP reset or timeout. Correlate request IDs across legs.

What can be proven without TLS keys?

You can confirm DNS, TCP setup, visible ClientHello ALPN offer, connection lifetime, directional encrypted records, retransmissions, windows, FIN/RST, new connection attempts, and timing. You cannot claim a specific encrypted GOAWAY, RST_STREAM, stream ID, or gRPC status from packet sizes alone.

Use logs or an authorized controlled environment with session keys. State the boundary: “The encrypted connection closed 30 seconds after request data; proxy logs report GOAWAY” separates sources honestly.

Flow control versus timeout

HTTP/2 has stream- and connection-level flow control layered on TCP. When decrypted, inspect WINDOW_UPDATE frames and which stream stops. Also inspect TCP receive windows. A stream can stall while the TCP connection and other streams remain healthy.

Observation	Lead
One stream stops, others progress	Stream flow control or application stall
All streams stop, TCP window healthy	Connection-level HTTP/2/app/proxy stall
TCP Zero Window	Receiver TCP backpressure
Loss/retransmission burst	Transport delay may trigger deadlines

Use the TCP Zero Window guide and packet-loss workflow for transport evidence.

gRPC regression checklist

Record RPC method, idempotency, deadline, metadata size, request/response streaming mode, connection age, concurrent streams, proxy route, GOAWAY/RST code where available, last-stream ID, retry decision, and final application status. Test unary and streaming RPCs separately.

After a fix, verify active RPC completion during graceful drain, eligible retry on a new connection, no duplicate side effects, and stable behavior at maximum configured stream concurrency. A lower error count without correct retry semantics is not sufficient.

Actionable handoff

Provide client/proxy/backend topology, capture points, ALPN, flow tuple, connection age, stream ID and RPC correlation, last completed frame, control-frame sender/code, TCP health, client retry, and matching proxy/server logs. Keep one successful neighboring stream as a control.

Preserve handshake, SETTINGS where visible, relevant request stream, GOAWAY/RST_STREAM, neighboring streams needed for context, TCP termination, and replacement connection. Use capture scope help and the complete evidence workflow to create a reviewable derivative.

Is GOAWAY itself an error?

No. It is a connection-control mechanism and may represent graceful drain. Error code, last-stream ID, active-stream outcomes, and client response determine whether users see a failure.

Does `grpc UNAVAILABLE` prove the network failed?

No. It can result from proxy/backend unavailability, GOAWAY handling, connection reset, name resolution, or deadline/transport translation. Join stream, connection, proxy, and application evidence before assigning ownership.

Connection-drain timeline table

Build a timeline from connection establishment through GOAWAY and replacement. Include current concurrent streams, highest opened stream ID, last-stream ID, completion status of lower IDs, attempted higher IDs, and time until the new connection is ready.

Boundary	Question
Connection start	Is GOAWAY tied to max connection age?
Last accepted stream	Which requests are definitely in processing scope?
GOAWAY receipt	Did the client stop creating new streams?
Replacement connect	Was new capacity ready before demand resumed?
Retry	Was the RPC safe and actually retried once?
Completion	Did active lower-ID streams finish?

This table distinguishes a normal drain that the client mishandles from a server that terminates accepted work. It also reveals a connection-pool bug where every connection drains simultaneously and no replacement is opened.

How do max concurrent streams affect symptoms?

The peer advertises limits in SETTINGS. If a client exceeds or mismanages them, requests may queue locally or be refused. Record SETTINGS, active stream count, queue time in client logs, and whether opening another connection is allowed. Do not infer stream saturation from TCP packet count.

What should be tested during deploys?

Start long and short RPCs, initiate an authorized drain, and verify GOAWAY last-stream semantics, lower-ID completion, replacement connection, retry of eligible unprocessed calls, and no replay of non-idempotent operations. Repeat through every proxy layer that terminates HTTP/2.

What if the server uses HTTP/2 but the backend uses HTTP/1.1?

The proxy translates semantics. An upstream connection close or timeout can become downstream RST_STREAM or gRPC status. Capture/log both legs and treat the proxy as the attribution boundary.

Version the regression with proxy/server/client implementations because defaults for connection age, ping, keepalive, retries, and stream concurrency can change during upgrades.

Keep connection and stream identifiers in the evidence note even when payload must be removed. Stable case-local labels let reviewers correlate PCAP, proxy logs, client traces, and server spans without publishing customer identifiers.

For a final conclusion, say whether the control frame was decoded from authorized TLS keys, reported by endpoint logs, or merely inferred from application error. Those sources have different evidentiary strength and must not be merged.

Related guides

Continue with the same-language pages below. They cover adjacent stages without changing the canonical owner of this topic: