VPS Tutorials
AI API stops streaming: trace SSE, Nginx buffering and frontend reads
Compare backend, Nginx and public event arrival; separate SSE framing, buffering, frontend consumption and timeout behavior with four historical timing comparisons.

An AI response arrives all at once after deployment even though streaming is enabled. Send the same request directly to the backend, through Nginx and through the public hostname to find where events start accumulating. Changing models or adding memory is not a substitute for locating that boundary.
This guide concerns HTTP response streams, especially SSE, buffering and frontend consumption. For a bidirectional connection established with a 101 Upgrade handshake, use WebSocket troubleshooting.
| Observation | Check first | Next step |
|---|---|---|
| Direct backend also waits until completion | Application, SDK and model endpoint | Generation and flush times |
| Backend streams, Nginx batches | Buffering, compression and matching location | Endpoint proxy configuration |
| Origin streams, CDN delays | Edge cache, gateway conversion or response policies | Arrival times at each layer |
| curl streams, page updates at the end | response.text, parsing and UI updates | Frontend consumption |
| Stream opens, then closes after silence | Heartbeats and read timeouts | Last-event time and limits |
HTTP 200 does not demonstrate real-time delivery. Separate time to first token from later buffering: disabling a proxy buffer cannot generate a token that the model has not produced yet.
Use a test endpoint that actually emits small events at intervals. /events/test is only an example path and must be implemented. Compare the local backend and hostname:
curl -sS -N -i --max-time 10 \
-H 'Accept: text/event-stream' \
http://127.0.0.1:3000/events/test
curl -sS -N -i --max-time 10 \
-H 'Accept: text/event-stream' \
https://api.example.com/events/test
Replace port and domain. -N disables curl's own output buffering, not the server's. Preserve POST, authentication and request body for a real API. Do not put API keys in URLs or public logs. A timeout after ten seconds may simply be this example's observation limit.
Record event times as well as content. If backend logs show events every half-second but the client gets them together, investigate the intermediary.
The Nginx proxy guide explains that upstream reception and client delivery can follow different rhythms. A normal-page configuration may not suit events.
With streaming path /events/ and local backend on 3000, merge this into the existing HTTPS server:
location /events/ {
proxy_pass http://127.0.0.1:3000;
proxy_http_version 1.1;
proxy_set_header Connection "";
proxy_set_header Host $host;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header Accept-Encoding "";
proxy_buffering off;
proxy_cache off;
proxy_read_timeout 90s;
gzip off;
}
Keep TLS, authentication, rate limits and other routes. proxy_pass has no trailing slash, preserving /events/.... If the actual endpoint is /api/chat, configure it; a never-matched /events/ changes nothing.
The fragment addresses response buffering, proxy caching and compression separately. Clearing Accept-Encoding avoids requesting upstream compression. gzip off disables compression at this Nginx location; it does not decompress arbitrary upstream responses. Compare uncompressed responses first, then assess compression for your application.
proxy_request_buffering controls uploaded request bodies, not the returning answer. Disabling it is not the key SSE fix, nor does text streaming require larger upload limits.
The application can send X-Accel-Buffering: no in the upstream response to control buffering, unless Nginx is configured to ignore it. See proxy directives. Whether the browser sees this internal control header is not sufficient evidence: measure event arrival.
Check before loading:
sudo nginx -t
sudo systemctl reload nginx
Verify that the panel, container or deployment script uses this file. Check other proxy layers too.
SSE uses text/event-stream and blank lines between events, according to MDN:
data: {"text":"你好"}
data: {"text":",世界"}
The server must send and flush events promptly. Appending to a string and returning it at completion is not streaming even with the correct Content-Type. Framework middleware, compression or an SDK may aggregate first.
Network chunks and SSE events do not correspond one-to-one. A read can contain half an event or several. Retain incomplete fragments and parse the actual protocol; do not assume every chunk is JSON. Ordinary JSON, NDJSON and SSE need different parsers.
Enable streaming according to the model API's definition. A proxy cannot turn a completed ordinary JSON answer into a real token stream. Preserve real error status and type rather than disguising every error as a 200 event stream.
This pattern waits for completion:
const text = await response.text();
Response.text reads the response to its end before returning the text. Updating afterward hides incremental delivery.
Use a response-body reader or a suitable SSE client; see ReadableStream.getReader. Decode UTF-8 across chunk boundaries correctly, including multibyte Chinese text and accented characters. Update the UI after parsing complete events.
Native EventSource suits supported GET streams; it does not directly replace an AI API requiring POST bodies or custom Authorization headers. Keep authentication instead of moving secrets into query parameters. Handle browser cross-origin rejection separately with the CORS guide.
If the whole answer eventually arrives, investigate batching. If the connection closes and events disappear, investigate timeouts. Read timeouts usually measure silence between upstream reads rather than total request duration.
The example's 90 seconds must fit first-event generation, heartbeat and gateway limits. Where SSE permits it, the server can send a comment heartbeat such as : ping followed by a blank line. It is not a business event; do not insert it into ordinary JSON or another protocol.
Heartbeats cannot bypass every total-duration limit or make the model produce data sooner. Agree on error events, end markers and recovery after an already-started stream fails, so the UI does not remain stuck generating.
Compare backend, Nginx and public ingress for first and subsequent events. Then verify a real authenticated request and a controlled error, including correct UI completion.
The source article reports four local comparisons with Nginx 1.19.6 and the same mock backend, which emitted at 0, 0.5 and 1 seconds. All four responses had identical complete content:
| Path | Event 1 / 2 / 3 arrival, seconds from request start |
|---|---|
| Direct mock backend | 0.038 / 0.541 / 1.045 |
| Nginx buffering enabled | 1.006 / 1.006 / 1.006 |
| Nginx buffering disabled | 0.002 / 0.505 / 1.007 |
| Buffering enabled, upstream header disables it | 0.002 / 0.506 / 1.007 |
Under those conditions, enabled buffering grouped events; disabling it by configuration or upstream header preserved their rhythm. These are not model-speed or production-latency measurements and do not cover TLS, CDN, browser UI or long timeouts.