← Playground

Six waysin

Every demo of an AI product is a demo of the model. The input gets about four seconds — someone types a sentence, or says one, and then we all watch the answer arrive. But the channel is the part you design, and it decides what the model is even allowed to know. So there is no model on this page. Six ways to get something into one, and for each, the thing it actually receives.

Six channelsNothing leaves your browser

Each demo below captures for real — your microphone, your camera, your screen, your files. None of it is sent anywhere, because there is nowhere to send it: this is a static page with no server behind it and no key to call anything with. What each one shows instead is the payload. The exact thing a model would be handed, and underneath it, the part of what you did that never made the trip.

That second list is the one worth reading. Choosing an input method is mostly choosing what to throw away.

01

Typing

The oldest channel, and still the one that wins most of the time — because it is the only one where the person edits before anyone sees it. That editing is invisible downstream. Write something below, change your mind a few times, and watch what survives.

keystrokes 0 deleted 0 seconds 0
What the model gets
nothing yet
What it never sees
  • Every character you deleted.
  • How long you sat there.
  • Whether you were sure.
02

Voice

The fastest channel to start and the slowest to correct. Speaking a sentence takes a second; fixing one word in it takes longer than typing the whole thing would have. Note what arrives at the other end — not your voice, a string. Everything that made it speech is gone before the model is involved.

Built into the browser, no account and no key. Worth knowing: in Chrome this is server-based by default — the audio goes to Google to be transcribed. There is now an on-device flag, but the default is the network. Firefox doesn't implement it at all.

Press the button and say something.

idle
What the model gets
nothing yet
What it never sees
  • The audio itself.
  • Your tone, and the pause before the word you weren't sure about.
  • That you said it twice.
  • Which of two people in the room was talking.
03

Camera

Pointing at something is the most natural instruction a person can give and one of the hardest to receive. You know what you meant; the frame only knows what was in it. Take a shot and look at what actually goes over — one still image, at whatever size the encoder settled on.

Camera off.

idle
What the model gets
nothing yet
What it never sees
  • What you were pointing at — only what was in shot.
  • The second before and the second after.
  • Scale. A screw and a bolt look the same close up.
  • That the light was bad and you knew it.
04

Screen

The channel with the worst ratio of what you meant to what you sent. Sharing a screen to ask about one button hands over every other thing that was open at the time. It is the most useful input in this list and the one I would think hardest about shipping.

Not sharing.

idle
What the model gets
nothing yet
What it never sees
  • Which part of it you meant.
  • That the window behind was not meant to be in there.
  • What is above the fold of a page you scrolled past.
05

Files

The only channel where somebody else has already decided what the model sees. A document is not text until something converts it, and that converter — not you, and usually not the model either — chooses what a table is, whether the footnote belongs to this page, and what happens to the diagram. Drop something in and see how much of it survives.

Drop a file here, or choose one.

idle
What the model gets
nothing yet
What it never sees
  • The layout. Two columns become one, and nobody says which.
  • Which sheet, page or slide you actually cared about.
  • Anything the converter couldn't read, silently dropped.
06

Selection

Highlight-and-ask is the most precise input here and the one people misread most often. You pick a phrase; the product almost always sends more than the phrase, because a sentence with no context around it is not answerable. Select any part of the passage below to see both.

The second year we ran the payments project, someone added a field to the checkout form because a partner asked for it in a meeting nobody minuted. It stayed for four years. By the time anyone questioned it, three internal tools read from that field and two of them would have broken without it. The field was not the problem. The meeting was the problem, and there was no way to see it from the code.

nothing selected
What the model gets
nothing yet
What it never sees
  • Why you picked that bit.
  • Whether you meant the sentence or the word.
  • What you'd already read on the way to it.

What isn't here

Three channels are missing and a web page is the wrong place to fake them. Gesture needs a hand-tracking model in the page, which would be forty megabytes to demonstrate a point I can make in a sentence: it is fast to point and almost impossible to be precise, and no one has solved knowing when a gesture has started. Wearables put the microphone somewhere it hears everything, which changes the design problem from capture to consent.

And the one that will matter most: no input at all. An agent triggered by a schedule, a file appearing, a build failing. Nobody is at a keyboard. Everything on this page is about what a person hands over deliberately, and the interesting work of the next few years is about what happens when nobody hands over anything, and the thing acts on what it finds. I don't know what that interface looks like. Nobody has shown me one I believed yet.

Nothing here leaves your browser. There is no server behind this page and no key in it — a static site cannot hold one without publishing it. Camera and screen frames are drawn to a canvas in the page and thrown away when you press stop. The one exception is worth naming: browser speech recognition in Chrome sends your audio to Google to be transcribed, which is Chrome's doing rather than mine, and is exactly the kind of thing this page exists to make visible.