Skip to content

Add Workday setup skills plan (decomposed skills + atomic flightcheck + evals)#162

Open
johnguy0 wants to merge 2 commits into
mainfrom
jonguy/workday-skills-plan
Open

Add Workday setup skills plan (decomposed skills + atomic flightcheck + evals)#162
johnguy0 wants to merge 2 commits into
mainfrom
jonguy/workday-skills-plan

Conversation

@johnguy0

Copy link
Copy Markdown
Collaborator

Summary

Adds planning docs only (no implementation code) under plans/workday-setup/ for a refactored, reusable set of skills (markdown playbooks an agent executes) that automatically configure Workday for the ESS Agent via Microsoft's simplified integration path. The plan is designed to be executed by a coding agent steered by a junior engineer, and reached multi-model rubber-duck consensus (no open questions) before this PR.

References:

What's in the plan

6 skills with an acyclic dependency chain:

  1. provision-power-platform-environment — PP env + Dataverse + Copilot Studio capacity (Power Platform Admin)
  2. install-ess — base ESS agent from AppSource (Environment Maker)
  3. provision-workday-entra-app — Entra app registration, user_impersonation scope, pre-authorize the Workday connector app, Graph delegated consent — fully automated via Microsoft Graph (Entra Admin; admin consent = Global Administrator)
  4. configure-workday-tenant — tenant-side configuration / attestation (Workday Admin — partly manual)
  5. install-workday-extension-pack — Workday topics, cloud flows, and connection references onto the agent (Environment Maker)
  6. create-new-topic — custom Workday scenarios beyond the OOTB set (Environment Maker + Workday SME)

Chain: 1->2; foundation ->3->4; {2,4}->5->6.

Cross-cutting docs:

  • master-checklist.md — trackable step-ID registry (S1.1...S6.2) + ID mint/reuse table.
  • flightcheck-single-checkpoint.md — adds --checkpoint <ID> so each setup step can be validated atomically (static registry + hydrate-then-filter contract; per-checkpoint client init).
  • evals.md — agent evals tied to the Workday functionality being enabled.
  • shared-building-blocks.md + command-wiring.md — reuse layer so skills don't duplicate logic.
  • README.md — primary index linking all sub-plans.

Key design rules baked in

  • Non-automatable steps are flagged MANUAL (e.g. InfoSec network attestation) and don't fail readiness.
  • Permission failures return an explicit error naming the missing role.
  • Flightcheck registry/drift is scoped to ESS+Workday setup-owned checkpoints only; other integrations stay validated via --scope.

Process / status

  • Multi-model rubber-duck campaign (gpt-5.5 + claude-opus-4.8) across rounds 2-9; Round-9: both reviewers READY, zero MAJORs.
  • Ready-to-review with no open questions. Docs-only change — no build/test impact.

Co-authored-by: Copilot 223556219+Copilot@users.noreply.github.com

johnguy0 and others added 2 commits June 23, 2026 17:55
Plan-only change under plans/workday-setup/: re-decomposes the /connect
workday monolith into 6 atomic, role-scoped skills for the simplified
Workday integration, plus shared building blocks, single-checkpoint
flightcheck, master checklist, command wiring, and per-skill evals.
Hardened via multi-model review. Also allowlist plans/ in .gitignore.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…uck consensus

Iterated rubber-duck passes (rounds 2-9, gpt-5.5 + claude-opus-4.8) until both
models returned READY with zero blocking findings. Key corrections, all
ground-truthed against solutions/ess-maker-skills/scripts/flightcheck source:

- Scope the flightcheck registry + single-checkpoint + drift test to the
  ESS+Workday setup-owned checkpoints (explicit owned-prefix allow-list;
  out-of-scope emitters such as SN-*, EXT-*, SAP-*, ENV-004-OR/UR/UC and the
  synthetic *-ERR sentinels are ignored, still validated via --scope).
- Correct the dynamic-family taxonomy: add WD-CONN-* (the generic per-connection
  enumerator emits on the simplified flavor); exact-first resolution keeps reused
  fixed WD-CONN-010/012/102 and new WD-CONN-AUTH-001 as literal entries while the
  family absorbs the dynamic detail rows. Removes the prior "exactly N families"
  / global "registry covers all emitted IDs" claims.
- Reuse pre-existing ENV-001 (environment exists) and ENV-002 (Dataverse
  provisioned) instead of re-minting; mint ENV-CAPACITY-001 for the Copilot
  Studio capacity check (dodges the ENV-005/006/007 numbering gap).
- Split WD-REST-001 (base-URL /api trim) and WD-REST-002 (/workers/me user
  context); WD-NET-001 is a MANUAL InfoSec allowlist attestation; WD-CONN-102
  cert-health has a terminal MANUAL state.
- Verified every minted checkpoint ID is absent from source (no collision) and
  every reused ID exists with the asserted meaning.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@johnguy0
johnguy0 requested a review from GrahamMcMynn June 24, 2026 00:00

@avneeshraiusit avneeshraiusit left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review — docs-only planning PR, recommend approve

Nicely structured plan. I spot-checked the load-bearing factual claims against solutions/ess-maker-skills/ and they hold up, which is the part that matters most for a plan others will execute:

  • Reused checkpoints exist and mean what the plan says: WD-PKG-001 (install-flavor fingerprint, simplified detection), WD-CONN-012 (connection-ref binding completeness), WD-CONN-102 (SAML signing-cert health), WD-CONN-010 (_check_entra_workday_federation_alignment). The repo's validation-matrix.md even labels WD-CONN-102 as "Entra-automated, Workday-manual comparison" — exactly the split S3.4/S4.4 rely on.
  • _check_flow_status emits WD-FLOW-{i+1:03d} (zero-padded) — matches the "reuse the existing emitter, don't re-mint" instruction.
  • Connector GUIDs are correct: 4e4707ca (Workday) is in connect/workday/step2.md; c26b24aa (ServiceNow) is in connect/azure/app-registration.md and servicenow-hrsd-itsm.md — confirming the "it's the correct ServiceNow connector, only the comment is wrong" framing.
  • Checkpoint-ID mapping is internally consistent across all 8 docs: every ID in each skill-N mint/reuse list matches its master-checklist row (S1.1–S6.2).

Minor findings (none blocking)

  1. One checkpoint backing two checklist rows slightly weakens the "every row → isolated --checkpoint" promise. WD-TENANT-001 backs both S4.2 and S4.3; WD-CONN-102 backs both S3.4 and S4.4. Running --checkpoint WD-TENANT-001 can't independently confirm S4.2 vs S4.3. Both are MANUAL/attest rows so impact is low, but a sentence acknowledging that attest rows may share a checkpoint and rely on the per-row acknowledgement (not the checkpoint) for done-ness would close the gap.

  2. Possible existing coverage for NameID. The source already has AUTH-006 (_run_saml_nameid_check) enumerating SAML NameID. skill-3 mints a new WD-ENTRA-NAMEID-001 for the set side (claimsMappingPolicy create+assign). Reasonable (set vs verify, and AUTH is outside the setup allow-list), but skill-3 should note explicitly whether AUTH-006 can serve the verify half rather than implying NameID is entirely greenfield.

  3. skill-6 → skill-4 runtime loop-back vs the "acyclic DAG" claim. skill-6 "loops back to configure-workday-tenant when scopes are missing," while skills are described as an acyclic chain. These don't actually conflict (the DAG validator is over the checkpoint prereq graph, not skill execution), but a one-line clarification that the loop-back is operational — not a static dependency — would prevent a reader from thinking the invariant is violated.

  4. Soft documentation mutual-reference: flightcheck-single-checkpoint defines registry scope by pointing at master-checklist's mint table, while master-checklist lists flightcheck-single-checkpoint under "Depends on." Resolved by the explicit ownership split + delivery order, so not a true cycle — just flagging it reads circular.

  5. Nit: README.md has no trailing newline.

Worth calling out as strengths

The MANUAL ≠ done rule, the network-reachability honesty (WD-NET-001 not presenting a local probe as a gate), the transitive prereq-closure requirement for client init, and the scoped drift-test design (owned-prefix allow-list, "never blanket-strip trailing -\d+") all pre-empt failure modes a naive implementation would hit.

@nehaoss

nehaoss commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

👋 Friendly reminder: This PR has been inactive for a while. Could the author or reviewers take a look and either push it forward, request changes, or close it if it's no longer needed? Let's keep our PR queue healthy! 🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants