PICO MCP Extensions

PICO XR feature construction APIs for Unity MCP agents. Idempotent, non-destructive, ambiguity-aware.

Overview

This Unity package exposes PICO XR building blocks as MCP (Model Context Protocol) tools, enabling AI agents (e.g. Unity AI Assistant) to programmatically configure XR scenes for PICO devices. It is designed to be used together with the PICO Unity Integration SDK 6.0.x and aligns its dependency baseline with that SDK (XR Interaction Toolkit 3.x).

Package name: com.bytedance.pico.mcp-extensions
Version: 0.0.6
Unity: 6000.0+
Author: ByteDance PICO

Requirements

Dependency	Minimum Version
com.unity.xr.core-utils	2.3.0
com.unity.xr.interaction.toolkit	3.4.0
com.unity.inputsystem	1.18.0
Unity AI Assistant (Unity.AI.MCP.Editor)	2.x
com.bytedance.pico.xr	6.0.0

Features

Seven XR building blocks, each with Enable / Disable / Status semantics:

Block	Description
VST	Video See-Through (passthrough) - configures camera for transparent background and adds PXR_CameraEffectBlock
Controller	Mounts PICO controller visual models on Left/Right hand anchors
Locomotion	Enables XRI locomotion subtree with fine-grained presets (Move, Turn, Teleportation, GrabMove, Climb, Gravity, Jump)
Spatial Mesh	Configures PXR_SpatialMeshManager with auto-detected MeshPrefab (depends on VST)
Plane Detection	Configures PICO SensePack plane detection via a bundled PXR_PlaneDetectionManager driver (depends on VST; PICO-native runtime only)
Hand	Enables PICO hand tracking (virtual hands) plus an XRI hand-interactor rig so a pinch can drive grab
Grab	Object pick-up & drag; ensures the scene XRInteractionManager broker and can upgrade a target object into a grabbable

Additionally, a Package tool manages Unity packages and samples (install / remove / update / import samples / query resolvable version).

Dual runtime support (PICO-native + OpenXR)

The building blocks compile and configure correctly under both PICO XR runtimes:

• PICO-native runtime (ENABLE_PICO_XR_SDK)
• OpenXR runtime (ENABLE_PICO_OPENXR_SDK) — VST enables the PICO PassthroughFeature; Spatial Mesh forces MultiPass rendering and enables the PICOSpatialMesh feature; Hand enables the Unity XR Hands models plus the PICO hand-tracking / hand-interaction OpenXR features.

Plane Detection is PICO-native only. PICO ships no plane-detection OpenXR feature, so under the OpenXR runtime the plane provider is never created and the block is a no-op.

Architecture

Editor/
  PXR_MCP_Common.cs        # Shared helpers: XR Origin lifecycle, module visibility
  PXR_MCP_Features.cs      # Building block implementations (VST, Controller, Locomotion, SpatialMesh, Plane, Hand, Grab)
  PXR_MCP_PackageOps.cs    # Package Manager operations (add, remove, samples)
  Tools/
    PXR_MCP_Tools.cs       # MCP tool surface ([McpTool] entry points)
    PXR_MCP_Result.cs      # Uniform result envelope for LLM consumption

Layer 1 (Editor): Plain C# static methods for building-block operations.
Layer 2 (Tools): [McpTool]-annotated methods that wrap Layer 1 and return PXR_MCP_Result envelopes.

MCP Tools

Tool	Actions	Description
pico_xr_vst	Enable, Disable, Status	Manage Video See-Through
pico_xr_controller	Enable, Disable, Status	Manage PICO controller models
pico_xr_locomotion	Enable, Disable, Configure, Status	Manage locomotion with preset flags
pico_xr_spatial_mesh	Enable, Disable, Status	Manage spatial mesh (requires VST)
pico_xr_plane	Enable, Disable, Status	Manage PICO plane detection (requires VST; PICO-native runtime only)
pico_xr_hand	Enable, Disable, Status	Manage PICO hand tracking (virtual hands)
pico_xr_grab	Enable, Disable, Status, MakeGrabbable	Manage grab pick-up & drag; make_grabbable upgrades a target object
pico_xr_package	List, Info, Add, Remove, Update, ListSamples, ImportSample, Resolvable	Unity Package Manager operations; resolvable is a read-only query for the latest-compatible version
pico_xr_status	(none)	Aggregate snapshot of all blocks

Design Principles

• Idempotent: Re-running any operation with the same arguments is a safe no-op.
• Non-destructive: Never destroys or deactivates foreign (non-agent-owned) XR Origins.
• Module isolation: Enabling one block does not implicitly enable unrelated modules. Initial-create hides Controller and Locomotion so VST stays clean.
• No hardcoded versions: XRI paths and types are resolved dynamically via PackageInfo and reflection.
• Undo-safe: All scene modifications go through Unity's Undo system.

Installation

Add this package to your Unity project via the Package Manager:

1. Open Window > Package Manager
2. Click + > Add package from disk... (or add to Packages/manifest.json)
3. Ensure XRI Starter Assets sample is imported (required for XR Origin prefab)

License

Copyright (c) 2015-2022 PICO Technology Co., Ltd. All rights reserved. See LICENSE.md for details.