From the glasses to the swarm: a glance as a task

The boiler in the basement has stopped again. You crouch in front of it, look at the rating plate and say: find the part that keeps failing, check who has it, and order one. That glance holds the model number, the serial number and the error code on the display. It also holds the pile of post on the shelf next to the boiler and your neighbour’s bicycle.

Glasses with a camera make this kind of request natural. XREAL Aura, on pre-order since 7 October, pairs display glasses with a pocket puck rated at 48 TOPS, and Gemini on Android XR, as XREAL describes it, understands what you are looking at and helps with tasks in context. Seeing is solved well enough. The hard part is what happens next: turning a glance into a task that a swarm of agents can do without being handed your eyes.

The real cause: a glance is a bad brief

A glance is rich, ambiguous and over-shared at the same time. It contains the three facts the task needs, a dozen it does not, and no plan at all. The device that saw it sits on your face, with a battery sized for glasses and a puck sized for a pocket, and the task (search suppliers, compare, check compatibility, order) takes minutes and many steps. The work cannot stay where the seeing happened, and the seeing should not travel with the work.

How hand-offs from the glasses go wrong

The whole frame becomes the brief. Every sub-agent gets the picture, including the post on the shelf. A task that needed a model number now carries an address.

A reaction instead of a plan. One request, one long run, no to-do list. When the third supplier is out of stock, nothing knows what step it was on.

The work waits on your face. A long task runs on the device you are wearing, so it stops when you take the glasses off or the battery runs low.

The workers inherit your senses. Agents that can ask the camera for another look, or the microphone for another listen, have far more than the task needs.

Nobody checks the reading. One misread digit on the plate and the swarm orders, efficiently and correctly, the wrong part.

The approval lands in the wrong place. The question “order this one now?” appears as a wall of text in front of your eyes while you are on a ladder.

What KyberAgent does instead

KyberAgent is pre-launch and does not run on Android XR; this is the design, for any body with a camera. The pattern of a planner and sub-agents is described in a swarm, not a bot.

1. Extract, confirm, forget. On the body or in the pocket, KyberBot turns the glance into a few fields: model, serial, error code. It reads them back to you in one line, and the frame is dropped. Why an assistant that sees should remember so little is the subject of an assistant that sees what you see.

2. A plan with a to-do list. The planner writes the steps down (kyber agent plan, an illustrative command): identify the part, find suppliers, compare, check fit, ask, order. Each sub-agent gets a fresh context and one item, as in anatomy of a deep agent.

3. Each worker in a KyberCell. A sub-agent runs in its own cell: files and programs fixed when the cell is created, network closed by default and opened per program, and the payment credential added only at the authorised endpoint. No cell has a camera. KyberCell explained goes through it.

4. The swarm runs off your face. The work goes to a pocket deck, a home node or, when you allow it, KyberFusion. Where a swarm should run sets out the choice, and what stays local asks what a 48 TOPS puck changes about it.

5. The question comes back short. The approval returns to the body as one line and a yes or a no, at a moment you can give one. Spending money in your name is never a step the swarm takes alone.

6. Every step on the record. The cells log each call, each page and each connection, so that afterwards you can see why that supplier was chosen. That is every step on the record.

The same hand-off matters at work, where the person in the glasses is an employee and the swarm runs the company’s process; KYBERIOS looks at that in glasses on the shop floor. The body itself, glasses joined to a puck, is on KyberDeck in glasses with a puck. How background tasks already work on Google’s platform is in agents on Android XR.

A test you can run today

Take a photo of the rating plate on any appliance at home and give it to the assistant you use, with a request like “find a replacement filter for this and get it ready to order”. Stop before anything is paid for. Then check four things:

1. Did it read the model number back to you before searching?
2. Can you see its plan, or only its answer?
3. What did it ask you before the step that would spend money?
4. Is the photo still in its history a day later?

Each “no” is a part of the hand-off nobody designed. The rest of the design is on the homepage.