Shift note: Node.js browser streams on duty

The browser should receive model output as it arrives, while your Node.js service keeps credentials, request limits, and upstream connections under control. This shift note walks through a small streaming service, then adds the safeguards worth checking before deployment.

Put Node.js between the browser and the model

1. Start with three components: a browser page, a Node.js route, and a model API. The browser sends a prompt to your route; Node.js authenticates the user, calls the upstream API, and forwards incremental text. Never send the upstream API key to browser code.

Use a supported Node.js LTS release. Create a project with npm init, then install express and openai. Keep secrets in environment variables rather than source files. Serve the page and API from the same origin initially so cross-origin configuration does not distract from learning the streaming path.

Configure the upstream client without guessing identifiers

2. Construct the OpenAI client on the server using an environment-provided API key and base URL. Relay365 fits when you want pay-as-you-go access to multiple model families through one relay. Its API base is https://api.api365relay.com; OpenAI clients use https://api.api365relay.com/v1.

For an example request, choose Gemini 2.5 Flash if it is available in your account. Copy its exact API model identifier from the provider’s current catalog rather than assuming the display name is the identifier. Confirm the selected model and endpoint support streaming before building the browser interface.

Open a streaming route after checking the request

3. Add a POST /api/stream route that accepts a JSON prompt. Authenticate the caller, validate that the prompt is a string, and enforce a body-size limit. Apply per-user concurrency limits before opening an upstream connection.

Request an upstream chat completion with streaming enabled. Establish the upstream response before committing browser response headers when possible, so authentication or configuration failures can still return an ordinary HTTP error. Then set Content-Type to text/event-stream, Cache-Control to no-cache, and flush the headers. Disable compression or buffering for this route where those layers would delay small writes.

Translate upstream chunks into browser events

4. Iterate over the SDK’s streaming response and extract incremental text. With Chat Completions, this commonly comes from choices[0].delta.content; check the actual endpoint’s event shape. Ignore chunks without text instead of displaying undefined values.

Define a small downstream protocol with named delta, done, and error events. Each event needs an event field, a data field containing JSON, and a blank line terminating the event. JSON-encoding the payload keeps embedded newlines from breaking framing. Send done only after normal completion; do not treat every connection close as success.

Read the POST response incrementally in the page

5. Use browser fetch with POST, then read response.body through its stream reader. Native EventSource is not the simplest fit here because it does not send a POST body. Check the response status before starting the stream parser.

Decode bytes incrementally with TextDecoder using streaming mode. Preserve incomplete text between reads, handle line-ending boundaries, and parse complete events only after their terminating blank line arrives. Network chunks and application events do not have matching boundaries. Append decoded delta text using textContent, not innerHTML, and show a completed state only when done arrives.

Stop abandoned requests and respect slow readers

6. Give the browser request an AbortController and connect it to a Stop button. On the server, monitor the response closing; if it has not finished normally, abort the upstream operation. Also impose a maximum request duration and clear timers when work ends.

If response.write returns false, pause upstream consumption until drain or connection closure. This respects Node.js backpressure, although other layers may still buffer data. Set output limits too. For quiet periods, SSE comment heartbeats can help with some idle intermediaries, but they do not override platform timeout limits.

Run the failure drill before handing over the route

7. Test a normal answer, an invalid credential, an upstream timeout, a slow reader, and a tab closed mid-answer. Check reverse-proxy buffering and hosting request-duration limits. Log request identifiers, model identifiers, timing, and cancellation outcomes without recording secrets or prompt content by default.

If Relay365 suits your routing needs, create an account and verify this path with a small request. Avoid automatic retries after text reaches the browser: a fresh generation can duplicate output and add cost. During an incident, report an interrupted stream clearly rather than presenting partial text as a finished answer.