Command Palette

Search for a command to run...

Hectal
Back to the lesson: Topic 5.4 — Relaying a Stream to the Browser
LLM Engineering · Example 1 of 1

A buffering proxy turns a stream into a lump

Eight events are sent 100 ms apart. A pretend proxy forwards them when its buffer is full or the stream ends. Time is simulated.

Your server sits between the provider and the browser, so it must pass the stream along as it arrives. Buffering proxies, compression and wrong headers can silently turn a stream into one late lump.

Change the code and press Run (Ctrl+Enter). Predict what will happen first, then break it on purpose and read what changes. Your edits are saved and match the lesson page.

Ready for more? This phase has 5 hands-on exercises with automatic checks.

Practice questions

Write the code in the editor, run it, then open the model answer to compare.

01

Write flush_times(sent_times, buffer_events) that takes the times at which a server sent each event and a buffer size counted in events, and returns when each event arrives at the browser if the proxy forwards a batch only when it holds buffer_events events, or when the stream ends.

02

Write count_updates(times) that returns how many separate moments the screen can update (the number of distinct arrival times).

Explain it without notes

01

Why should the browser stream through your server instead of calling the model provider directly?

02

List three things that make a stream arrive in lumps after deployment, and how to fix each.

03

How do you make cancellation travel from the browser to the provider?

04

What can you not do when the provider fails midway, and what do you do instead?

A buffering proxy turns a stream into a lump
Sign in to run this example in your browser.

Expected output

no buffering       arrivals (ms): 500, 600, 700, 800, 900, 1000, 1100, 1200
                   first text after 500 ms, screen updates: 8
buffer 56 bytes    arrivals (ms): 800, 800, 800, 800, 1200, 1200, 1200, 1200
                   first text after 800 ms, screen updates: 2
buffer 4096 bytes  arrivals (ms): 1200, 1200, 1200, 1200, 1200, 1200, 1200, 1200
                   first text after 1200 ms, screen updates: 1