Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions crates/tinycomputer-browser/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ it as a surface next to the desktop, and `tinycomputer` serves it over TinyBus.
| `src/linked/` | `AgentBrowser` (feature `agent-browser`): agent-browser's dispatcher linked in-process, one `DaemonState` per session, built without reading the host's `AGENT_BROWSER_*` environment |
| `src/convert/` | contract requests to agent-browser commands, as pure functions |
| `src/reply/` | agent-browser replies to typed results |
| `src/deep/` | controls a page keeps inside open shadow roots that the accessibility tree gives no ref: looked for after every snapshot (`deep.js`), listed with refs `d1`, `d2`, …, and pressed, typed into, or read where they show (`src/sessions/deep.rs`) |
| `src/surface/` | `BrowserSurface`: one session as a `tinycomputer-core` `Surface`; `sight/`, which reads the rendered page the way a person looks at it (the default, `docs/technical/specs/browser-sight.md`), and `tree.rs`, which parses snapshot text into a `Screen` when sight cannot reach the page or `Perception::Tree` is chosen |
| `src/error/` | `Error`: what a caller should do next, one published wire name per variant |
| `src/outputs/` | held screenshots and PDFs: bounded count, size and lifetime, chunked reads |
Expand Down
36 changes: 36 additions & 0 deletions crates/tinycomputer-browser/src/convert/interaction.rs
Original file line number Diff line number Diff line change
Expand Up @@ -164,6 +164,42 @@ fn wait_for(
}
}

/// The raw mouse event that moves the pointer to the window's point (`x`,
/// `y`), as a person's hand does: whatever is on top there sees it arrive.
#[must_use]
pub(crate) fn move_to(x: f64, y: f64) -> Value {
json!({
"action": "mouse",
"eventType": "mouseMoved",
"x": x,
"y": y,
"button": "none",
"clickCount": 0,
})
}

/// The raw mouse events of a person's press at the window's point (`x`,
/// `y`): the pointer moved there, then its left button pressed and
/// released. Whatever is on top there takes the press as trusted input.
#[must_use]
pub(crate) fn press_at(x: f64, y: f64) -> [Value; 3] {
let button = |event: &str| {
json!({
"action": "mouse",
"eventType": event,
"x": x,
"y": y,
"button": "left",
"clickCount": 1,
})
};
[
move_to(x, y),
button("mousePressed"),
button("mouseReleased"),
]
}

/// The engine selector for a ref or CSS target.
///
/// # Errors
Expand Down
2 changes: 1 addition & 1 deletion crates/tinycomputer-browser/src/convert/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ mod interaction;
mod page;
mod session;

pub(crate) use interaction::{action, selector};
pub(crate) use interaction::{action, move_to, press_at, selector};
pub(crate) use page::{evaluate, navigate, read, screenshot, snapshot};
pub(crate) use session::{launch, viewport};

Expand Down
543 changes: 543 additions & 0 deletions crates/tinycomputer-browser/src/deep/deep.js

Large diffs are not rendered by default.

Loading
Loading