Read the room accurately

Distinguish registered tools, connected backends, and the capabilities of the current surface.

An agent should know what its environment exposes before claiming what it can perceive or do. The first tool for that is ar_inspect_surface.

{"name":"ar_inspect_surface","arguments":{}}

The result includes spatial tool names and the connected surface’s context. A connected body can be inspected from another conversation with the same Entity. It describes a live interface, not files inside the Entity’s execution workspace.

Three different kinds of evidence

Evidence	What it proves	What it does not prove
Tool definition	A callable interface is registered	Its optional backend is configured
Surface snapshot	What the connected session reports	Features in a different browser or headset session
Successful action and visible result	That action worked in this session	Every other action or future request will work

Desktop and immersive AR

Capability	Desktop browser	Immersive WebXR
Rendered shell and scene objects	Available in the spatial surface	Available in the spatial surface
Spatial tools	Use the current registered contract	Use the current registered contract
AR session	Not an immersive AR session	Requires device and browser support
Hand tracking	Do not assume	Depends on reported XR capabilities
Eye tracking	The shipped surface reports false	The shipped surface reports false
Spatial audio	The shipped surface advertises support	Playback still depends on session/browser behavior

The runtime’s advertised support and a user’s permission to start playback or capture are separate conditions.

Capture is not unrestricted vision

ar_request_capture requests a frame from the surface. The shipped capture helper captures the rendered WebGL canvas and resizes it. A successful canvas capture does not prove access to the user’s physical room or the headset’s passthrough camera feed.

Handle capture failure or missing visual input explicitly. Do not infer hand positions, gaze, or objects in a real room from desktop mode alone.

Optional services

For creations, use ar_world capabilities for the runtime contract and ar_world observe for actual object bounds, transforms, ownership, asset status, program errors, and performance. ar_blender capabilities separately checks Blender on the execution host. A successful Blender export and a ready Spatial asset are separate results.

The current world supports custom geometry, physics, instancing, scoped programs, controls, and Blender previews. It does not claim physical room mesh understanding or shared cloud world synchronization. See Creations for persistence and desktop/XR interaction boundaries.

Generated motion needs an external motion backend. The AR gateway reads NGRAM_AR_MOTION_PROVIDER_URL and optional token/timeout variables. The adapter calls /v1/motion and expects a compatible generated clip.

Browser pages can refuse embedding. Media playback can require a user gesture or be unavailable for a particular video. A registered ar_open_browser or ar_play_youtube tool cannot override those conditions.

A useful inspection habit

Inspect once when entering a session or when a capability changes. Use the returned evidence to act. Repeating a registry inventory is not progress on the user’s request; if an action fails, inspect that specific failure and report the limitation.