Build with eyes open

Give agents rendered feedback from Blender and the actual Spatial scene.

Agents can inspect what they build, change it, and compare the result. ar_blender render shows the working Blender scene. ar_request_capture shows the object in Spatial, with its actual material, scale, and surroundings.

Turn on view sharing

Click the camera button in the desktop top bar. Periwinkle with a white icon means agent view sharing is enabled. Click again to disable it and cancel a pending inspection.

Share view sends one current virtual view to the agent, even while ongoing permission is off. In XR, Vision on/off in the radial menu controls the same permission. Turning it on also shares one current view. The capture uses the headset eye pose when an immersive session is active.

View sharing includes rendered virtual objects. It does not include a webcam feed, physical room imagery, or headset passthrough. Blender renders are generated on the execution host and do not require permission to capture the user’s Spatial view.

Inspect Spatial

First use ar_world observe to get the actual entity ID. Then request the view you need:

{
  "name": "ar_request_capture",
  "arguments": {
    "options": {
      "target": "sword",
      "views": ["front", "right", "perspective"],
      "style": "studio",
      "isolate": true,
      "size": 1024
    }
  }
}

Replace sword with the observed ID. Omit options to capture the user’s current virtual view. Targeted views use an independent camera; the human’s position, orientation, and object placement remain intact.

Option	Meaning
target	Existing entity ID; its bounds determine framing
node	Unique authored mesh name within that target; frames a close-up
views	Up to four of front, back, left, right, top, bottom, perspective; aligned to Spatial world axes
position, lookAt	Explicit camera and aim positions as [x,y,z], Spatial Y-up metres
orbit	[azimuth,elevation] in degrees; elevation −90 through 90
distance	Camera distance in metres, .01 through 10,000
projection	perspective or orthographic
size	Integer 256 through 1536; maximum image dimension
isolate	Hide other objects for this inspection; requires a target
style	scene, studio, clay, or wireframe
includeColliders	Overlay the physics world’s collider outlines

Use scene to assess the actual app lighting. Studio supplies temporary neutral lighting and reflections. Clay reveals geometry without authored materials. Wireframe exposes topology. Diagnostic views help separate geometry problems from material or lighting problems.

The receipt includes scene revision, capture time, camera poses, image labels, and physicalCamera: false. An asset must finish loading before targeted inspection succeeds.

Inspect Blender

For an existing project, render without editing or publishing another model revision:

{
  "name": "ar_blender",
  "arguments": {
    "command": "render",
    "payload": {
      "project_id": "sculpture",
      "options": {"style":"studio","orbit":[35,20],"size":1024,"samples":32}
    }
  }
}

Blender options are objects (one to 64 object names), camera (an authored camera name), position, look_at, orbit, distance, projection, size, samples, and style. Coordinates use Blender Z-up. Styles are scene, studio, and clay; samples range from one to 256. Rendering uses CPU Cycles on the configured execution host.

For a 360-degree background, Blender also accepts projection: "equirectangular" with style: "scene" and an even width up to 4096. It returns a 2:1 JPEG and a configuration you can apply as the Spatial sky. See Environments. Ordinary perspective and orthographic Blender views remain limited to 1536 pixels; Spatial inspection cameras use the separate options above.

Inside an execute script, render_view(**options) returns a working image. The latest four renders in that script are attached to its tool result. The script still publishes normally on completion. Temporary review cameras, lights, and render settings are restored afterward.

Close the loop

1. Inspect the current geometry and available scene state.
2. Render the working Blender model from a useful angle.
3. Edit with bpy, then publish the next preview.
4. Inspect the loaded result in Spatial using scene lighting.
5. Compare diagnostic views if the model looks flat, dark, or incorrectly shaped.

Captures are on demand. Transport waiting does not trigger model polling. Agent-requested images return to the active tool round, rather than starting a second conversation turn. The Python harness retains the latest two visual tool results in the active turn and stores text receipts in durable history. Rendering stops when its tool call is cancelled; Spatial inspection also respects Stop and the permission toggle.

The inference model must support image inputs. Native OpenAI Responses and OpenAI-compatible Chat Completions transport paths both carry the image content. A successful export, loaded asset, or text receipt alone is not evidence that a model saw the pixels.