Skip to content

Testing Center results report the routed topic as a truncated name ("p") for topics with the p_<plannerId>_ prefix that sf agent create itself generates #3635

Description

@jimnovakoff

CLI Version:
@salesforce/cli/2.149.9

Architecture:
win32-arm64

Node Version:
node-v24.14.1

Plugin Version:
@oclif/plugin-autocomplete 3.2.56 (core)
@oclif/plugin-commands 4.1.63 (core)
@oclif/plugin-help 6.2.58 (core)
@oclif/plugin-not-found 3.2.93 (core)
@oclif/plugin-plugins 5.4.87 (core)
@oclif/plugin-search 1.2.54 (core)
@oclif/plugin-update 4.7.59 (core)
@oclif/plugin-version 2.2.57 (core)
@oclif/plugin-warn-if-update-available 3.1.73 (core)
@oclif/plugin-which 3.2.61 (core)
@salesforce/cli 2.149.9 (core)
agent 2.0.5 (user)
apex 4.1.0 (core)
api 2.0.9 (core)
auth 5.0.6 (core)
code-analyzer 5.12.0 (user)
data 5.1.5 (core)
deploy-retrieve 4.1.2 (core)
info 4.0.9 (core)
limits 4.0.3 (core)
marketplace 2.0.5 (core)
org 6.0.9 (core)
packaging 3.0.5 (core)
schema 4.0.5 (core)
settings 3.0.5 (core)
sobject 2.0.5 (core)
telemetry 4.0.5 (core)
templates 57.0.9 (core)
trust 4.0.9 (core)
user 5.0.1 (core)

OS and Version:
Windows_NT 10.0.26200

Shell:
powershell

Summary

sf agent create names an agent's topics with an auto-generated prefix:
p_<plannerId>_<Topic_Name> (e.g. p_16xxV0000006xXX_Shipping_Details).
When an AiEvaluationDefinition test then runs against that agent, the run
results report the routed topic as the literal string p — in both
generatedData.topic and the topic assertion's actualValue. The name
appears to be truncated at the planner-id segment somewhere in the results
pipeline. Consequence: every topic assertion fails, no matter what
expected value the test author supplies, because nothing can equal p.

Built-in topics without the prefix (e.g. Off_Topic) are reported
correctly, which localizes the problem to the prefixed names.

This is server-side, not CLI parsing: @salesforce/agents
(lib/agentTester.js) spreads the API's generatedData payload through
verbatim (only invokedActions is HTML-decoded), so the p arrives in the
API response.

Steps to reproduce

  1. sf agent create --spec specs/agentSpec.yaml --name "Test Agent" ...
    (topics get p_<plannerId>_* developerNames — the default behavior)
  2. Create a Testing Center test whose cases set expectedTopic to a
    topic's full developerName; deploy and activate the agent.
  3. sf agent test run --api-name <test> -o <alias> --wait 15 --json

Actual result

Every case: topic_assertion: FAILURE with
expected: p_<plannerId>_<Topic_Name>, actual: p. The run's
generatedData.topic is also p.

Expected result

The routed topic is reported by its full developerName (or the docs state
which name form topic assertions compare against).

Workaround

Redeploy the topics under clean developerNames without the auto-generated
prefix (and update the planner bundle references); full names then
round-trip correctly and topic assertions pass. But note the colliding
defaults: the naming convention agent create generates is the one the
results pipeline cannot report.

Environment

  • sf CLI: <paste sf version --verbose>
  • Org: Agentforce Developer Edition (orgfarm-*.develop.my.salesforce.com)

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:afdxCLI issue within plugin-agent or agents libraryinvestigatingWe're actively investigating this issueowned by another teamThe Salesforce CLI team does not own this work but will pass on the information to the correct team.validatedVersion information for this issue has been validated

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions