Repository navigation
feat: add Doom-style teleop control module (#1401) - #1554
christiefhyang wants to merge 32 commits into
Conversation
Greptile SummaryThis PR adds a new
Confidence Score: 3/5Not safe to merge as-is: a top-level pygame import in doom_teleop.py will prevent the entire Go2 and G1 blueprint files from loading in any environment without pygame, and stop() can skip the base-class teardown when called before start(). The unconditional top-level dimos/teleop/keyboard/doom_teleop.py (pygame import strategy and stop() lifecycle), dimos/robot/unitree/g1/blueprints/unitree_g1_blueprints.py (still has two broken imports from dimos.robot.unitree_webrtc.* flagged in prior rounds, plus the with_joystick naming issue) Important Files Changed
Sequence DiagramsequenceDiagram
participant User as User (keyboard/mouse)
participant DoomTeleop
participant pygame as pygame loop (bg thread)
participant CmdVel as /cmd_vel (Out[Twist])
participant GoalPose as /goal_pose (Out[PoseStamped])
participant CancelGoal as /cancel_goal (Out[Bool])
participant Odom as /odom (In[PoseStamped])
DoomTeleop->>pygame: start() → spawn thread
Odom-->>DoomTeleop: _on_odom() updates _current_pose
loop 50 Hz
User->>pygame: WSAD / mouse motion
pygame->>CmdVel: publish(Twist)
end
User->>pygame: Right-click
pygame->>GoalPose: publish(PoseStamped) [step forward]
User->>pygame: Middle-click
pygame->>GoalPose: publish(PoseStamped) [rotate in place]
User->>pygame: Space / focus lost
pygame->>CmdVel: publish(zero Twist)
pygame->>CancelGoal: "publish(Bool=True)"
DoomTeleop->>pygame: stop() → set stop event
pygame->>CmdVel: publish(zero Twist)
pygame->>CancelGoal: "publish(Bool=True)"
Reviews (3): Last reviewed commit: "Fix Doom teleop blueprint imports" | Re-trigger Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| foxglove_bridge(), | ||
| ) | ||
| .global_config(n_dask_workers=4, robot_model="unitree_g1") | ||
| .transports( |
There was a problem hiding this comment.
Wrong config key — n_dask_workers should be n_workers
GlobalConfig defines the field as n_workers (see dimos/core/global_config.py:38), and all existing blueprints in the repository use n_workers. The key n_dask_workers will be silently ignored (passed as **kwargs) or raise an error, meaning blueprints will fall back to the default of 2 workers instead of the intended count.
This affects every .global_config(n_dask_workers=...) call in both new blueprint files (G1 lines 92, 139, 144, 203; Go2 lines 101, 114, 170).
| .transports( | |
| .global_config(n_workers=4, robot_model="unitree_g1") |
| ("world/robot/camera", "camera_optical", GO2Connection.camera_info_static), | ||
| ], | ||
| ), | ||
| ).global_config(n_dask_workers=4, robot_model="unitree_go2") |
There was a problem hiding this comment.
Wrong config key — n_dask_workers should be n_workers
GlobalConfig.n_workers is the correct field (see dimos/core/global_config.py:38). n_dask_workers is not a recognized config key. Every existing blueprint in the repo uses n_workers. This affects lines 101, 114, and 170 in this file.
| ).global_config(n_dask_workers=4, robot_model="unitree_go2") | |
| ).global_config(n_workers=4, robot_model="unitree_go2") |
Refactor Unitree G1 blueprint configurations to simplify connections and improve structure.
This module restores public blueprint names for the Unitree Go2, mapping legacy names to current modules. Several imports and configurations have been updated or removed.
christiefhyang
left a comment
christiefhyang
left a comment
There was a problem hiding this comment.
Updated file path
| # ESC quits | ||
| self._stop_event.set() | ||
|
|
||
| keys = pygame.key.get_pressed() |
There was a problem hiding this comment.
@Dreamsorcerer I switched DoomTeleop over to use pygame.key.get_pressed() for WASD instead of maintaining our own key set, while still keeping the existing focus-loss handling. This should avoid the gotchas you mentioned around tracking pressed keys manually.
There was a problem hiding this comment.
Yes I will update this soon. Encountered some issues with controls related to middle mouse button and right click.
da7519f to
66c783e
Compare
This module provides pre-configured blueprints for various G1 robot setups, enhancing functionality and integration with additional components.
|
This update primarily fixes the interaction conflicts between “map click-to-goal” and “keyboard teleop” in the Command Center. The changes are focused on the actual web control pipeline, rather than the keyboard_teleop.py module itself. Scope of Fixes Issues Resolved Behavior Changes Main Modified Files Differentiates interactions by mouse button: |
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
| basic = autoconnect( | ||
| go2_connection(), | ||
| linux if platform.system() == "Linux" else mac, | ||
| websocket_vis(), |
There was a problem hiding this comment.
websocket_vis import — NameError at runtime
websocket_vis() is called on line 87 inside the basic blueprint, but websocket_vis is never imported in this file. Every attempt to instantiate any blueprint that includes basic (e.g. go2_with_doom, nav, agentic) will raise NameError: name 'websocket_vis' is not defined. The G1 blueprint file imports it correctly via from dimos.web.websocket_vis.websocket_vis_module import websocket_vis.
| _perception_and_memory = autoconnect( | ||
| spatial_memory(), | ||
| object_tracking(frame_id="camera_link"), | ||
| utilization(), |
There was a problem hiding this comment.
utilization import — NameError at runtime
utilization() is called on line 124 inside _perception_and_memory, but it is never imported in this file. Any blueprint that composes _perception_and_memory (i.e. standard, standard_sim, agentic, agentic_sim) will raise NameError: name 'utilization' is not defined at module load time. The Go2 blueprint file correctly imports it via from dimos.utils.monitoring import utilization.
❌ 1 Tests Failed:
View the top 1 failed test(s) by shortest run time
To view more test analytics, go to the Test Analytics Dashboard |
|
Addressed the current review comments:
Validation: Note: ruff was not available in the local Python environment. |
| import os | ||
| import threading | ||
|
|
||
| import pygame |
There was a problem hiding this comment.
import pygame breaks all non-doom blueprints when pygame is absent
Both unitree_go2_blueprints.py and unitree_g1_blueprints.py import doom_teleop at module level, so any process that loads either file — even to run a completely unrelated blueprint like standard, nav, or agentic — will raise ImportError if pygame is not installed. The established pattern in this codebase (see keyboard_teleop_module.py and b1/joystick_module.py) is a guarded try/except ImportError or a deferred per-method import, precisely to avoid this. Moving the import pygame inside _pygame_loop (or wrapping it in a try/except at module level and raising a friendly error in start()) would contain the dependency to callers that actually invoke the doom teleop loop.
|
Closed this old PR due to inactivity. Can be reopened later. |
3 participants
This PR adds a Doom-style teleoperation module that lets both Unitree Go2 and G1 be driven with a shared WSAD + mouse control scheme, modeled after first-person shooter (FPS) controls. The DoomTeleop module is robot-agnostic and only uses standard navigation interfaces: continuous velocity via /cmd_vel and optional discrete pose goals via /goal_pose and /cancel_goal.
The module is then integrated into the updated Unitree Go2 and G1 blueprint architecture so that both robots can use the same keyboard plus mouse teleop without changing any existing transport topics.
Motivation:
Implementation Details：
DoomTeleop Module:
Keyboard control (continuous velocity)
• Maps keyboard input to Twist messages:
• W / S: move forward / backward
• A / D: turn left / right
• Space: emergency stop (publishes a zero Twist and clears internal input state)
• Publishes cmd_vel: Out[Twist], which is transported on the existing /cmd_vel channel in the blueprints.
• When the window loses focus, the module clears its input state and sends a zero Twist so the robot does not keep moving on a stuck key.
Mouse control (yaw & optional pose)
• Horizontal mouse motion adjusts yaw by adding to angular.z, providing an FPS-like look and turn behavior on top of keyboard turning.
• Optional discrete pose helpers when navigation goal topics are wired:
• Right-click: sends a small forward step goal (PoseStamped) in front of the current pose.
• Middle-click: sends a small in-place rotation goal around the current position.
• Uses the existing navigation goal interfaces:
• goal_pose: Out[PoseStamped] mapped to /goal_pose
• cancel_goal: Out[Bool] mapped to /cancel_goal
• Space also cancels any active goal when cancel_goal is available.
Robot-agnostic design
• DoomTeleop does not contain any Unitree- or robot-specific code.
• It only publishes Twist, PoseStamped and Bool on the standard navigation interfaces, and can be reused with any robot that exposes /cmd_vel, /goal_pose and /cancel_goal in its transports.
Lifecycle and threading
• Uses a background thread plus pygame to handle keyboard and mouse events asynchronously so teleop input does not block the rest of the system.
• start and stop follow the existing Module lifecycle:
• start calls super().start and spawns the input thread.
• stop stops the thread, publishes a final zero Twist and cancels any active goal.
Go2 Blueprint Integration
• Adds a go2_with_doom blueprint that composes the existing Unitree Go2 base navigation stack with doom_teleop.
• Reuses the existing Go2 transports for:
• /cmd_vel for velocity commands
• /odom and navigation goal topics where available for optional pose goals
• No changes are made to existing Go2 blueprints; go2_with_doom is an additive configuration.
G1 Blueprint Integration
• Integrates doom_teleop into the updated modular G1 blueprint structure so it can be used alongside the existing joystick and keyboard teleop options.
• Wires DoomTeleop only through the established /cmd_vel and navigation goal interfaces (goal_pose, cancel_goal, odom), matching the existing G1 navigation stack.
• Existing G1 blueprints remain available and unchanged; Doom teleop is exposed as an additional configuration that can be selected when WSAD + mouse control is desired.
Testing
Simulation (MuJoCo)
Environment: Ubuntu with MuJoCo support.
Commands
Install dependencies:
Go2
G1
Verified
IRL Test Plan (to be run by maintainers)
Robot
Command
Go2
G1
Environment
Open indoor area with conservative speed limits on both the robot and the teleop module.
Checkpoints
Breaking Changes
None. This PR introduces a new module that integrates with existing interfaces without modifying or breaking any existing control paths.
Contributor License Agreement