Signing/verification canonicalization drops REQUIRED fields at their default value, breaks cross-SDK Agent Card signature verification

What happened

_canonicalize_agent_card (used by both create_agent_card_signer and create_signature_verifier in src/a2a/utils/signing.py) removes empty strings, empty lists, and empty dicts from the Agent Card before canonicalizing, with no exception for fields the spec marks REQUIRED. Section 8.4.1 of the A2A specification says the opposite: REQUIRED fields must stay in the canonical payload even when they hold their default value.

The result is that a card signed by an SDK that keeps a REQUIRED field at its default value in the signed payload (for example description: "" or skills: []) fails verification in a2a-python, because a2a-python recomputes a different canonical payload for the same card.

Where it happens

At main, commit 0d5473ca4fa6d40034a6a7c8d65bce5cd85d8167:

src/a2a/utils/signing.py, _canonicalize_agent_card (lines 198-208):

def _canonicalize_agent_card(agent_card: AgentCard) -> str:
    """Canonicalizes the Agent Card JSON according to RFC 8785 (JCS)."""
    card_dict = MessageToDict(
        agent_card,
    )
    # Remove signatures field if present
    card_dict.pop('signatures', None)

    # Recursively remove empty values
    cleaned_dict = _clean_empty(card_dict)
    return canonicalize(cleaned_dict)

Two things contribute:

MessageToDict(agent_card) is called without always_print_fields_with_no_presence=True. Proto3 fields that have no explicit presence tracking (this includes the REQUIRED fields description, skills, and AgentSkill.tags, none of which carry the optional keyword) are omitted from the dict entirely once they hold the default value, before _clean_empty ever runs.
_clean_empty (lines 167-195) then unconditionally strips any remaining empty string, list, or dict:

def _clean_empty(d: Any, depth: int = 0) -> Any:
    ...
    if isinstance(d, dict):
        cleaned_dict = {
            k: cleaned_v
            for k, v in d.items()
            if (cleaned_v := _clean_empty(v, depth + 1)) is not None
        }
        return cleaned_dict or None
    if isinstance(d, list):
        cleaned_list = [
            cleaned_v
            for v in d
            if (cleaned_v := _clean_empty(v, depth + 1)) is not None
        ]
        return cleaned_list or None
    if isinstance(d, str) and not d:
        return None
    return d

Neither step distinguishes a REQUIRED field at its default value from an optional or repeated field that happens to be empty.

Permalink:

a2a-python/src/a2a/utils/signing.py

Lines 167 to 208 in 0d5473c

    
           def _clean_empty(d: Any, depth: int = 0) -> Any: 
        
               """Recursively remove empty strings, lists and dicts from a dictionary. 
        
               Depth is bounded for the same reason canonicalization is: nesting reaches 
        
               this function from `AgentExtension.params`, and without the bound a deeply 
        
               nested card exhausts the interpreter stack here, before the canonicalizer 
        
               ever gets the chance to reject it. 
        
               """ 
        
               if depth > MAX_DEPTH: 
        
                   raise CanonicalizationError( 
        
                       f'nesting exceeds the maximum depth of {MAX_DEPTH}' 
        
                   ) 
        
               if isinstance(d, dict): 
        
                   cleaned_dict = { 
        
                       k: cleaned_v 
        
                       for k, v in d.items() 
        
                       if (cleaned_v := _clean_empty(v, depth + 1)) is not None 
        
                   } 
        
                   return cleaned_dict or None 
        
               if isinstance(d, list): 
        
                   cleaned_list = [ 
        
                       cleaned_v 
        
                       for v in d 
        
                       if (cleaned_v := _clean_empty(v, depth + 1)) is not None 
        
                   ] 
        
                   return cleaned_list or None 
        
               if isinstance(d, str) and not d: 
        
                   return None 
        
               return d 
        
           def _canonicalize_agent_card(agent_card: AgentCard) -> str: 
        
               """Canonicalizes the Agent Card JSON according to RFC 8785 (JCS).""" 
        
               card_dict = MessageToDict( 
        
                   agent_card, 
        
               ) 
        
               # Remove signatures field if present 
        
               card_dict.pop('signatures', None) 
        
               # Recursively remove empty values 
        
               cleaned_dict = _clean_empty(card_dict) 
        
               return canonicalize(cleaned_dict)

Spec requirement

A2A specification, docs/specification.md, section 8.4.1, "Canonicalization Requirements" (checked at commit 43e0c874d3baba68ed84b98678d7f2268438e69f):

Required fields: Fields marked with REQUIRED MUST always be present, even if the field value matches the default.

Default values: Fields with default values MUST be omitted unless the field is marked as REQUIRED or has the optional keyword.

The section's own worked example signs this fragment:

{
  "name": "Example Agent",
  "description": "",
  "capabilities": {
    "streaming": false,
    "pushNotifications": false,
    "extensions": []
  },
  "skills": []
}

and gives this as the canonical result:

{"capabilities":{"pushNotifications":false,"streaming":false},"description":"","name":"Example Agent","skills":[]}

description (REQUIRED, empty string) and skills (REQUIRED, empty array) both stay in the canonical payload. extensions (not REQUIRED, empty array) is omitted. Current _canonicalize_agent_card drops description and skills along with extensions, contradicting the first bullet above.

Minimal reproduction

Python only, no other SDK needed:

from a2a.types import AgentCapabilities, AgentCard, AgentInterface
from a2a.utils.signing import _canonicalize_agent_card

card = AgentCard(
    name="Example Agent",
    description="",
    version="1.0.0",
    supported_interfaces=[
        AgentInterface(
            url="https://example.com/a2a/v1",
            protocol_binding="JSONRPC",
            protocol_version="1.0",
        )
    ],
    capabilities=AgentCapabilities(streaming=False, push_notifications=False),
    default_input_modes=["text/plain"],
    default_output_modes=["text/plain"],
    skills=[],
)

print(_canonicalize_agent_card(card))

Expected per 8.4.1: description and skills are present in the output ("description":"", "skills":[]).
Actual: both are absent from the canonical payload.

Cross-SDK consequence

We ran this against a2a-go PR #441 (bb750f5c7913b967d8dcbe3d24537a8a84dd1a61), which keeps description and skills in the signed wire JSON when they are at their default value (its Go struct tags do not mark those fields omitempty). Five single-field cases, one Ed25519 key shared on both sides:

Case	Go signs, Python verifies	Python signs, Go verifies
`description: ""`	fails, `InvalidSignaturesError: No valid signature found`	passes
`skills: []`	fails, same error	passes
`capabilities.extensions: []` (not REQUIRED)	passes	passes
`skill.tags: []` (REQUIRED on `AgentSkill` per `a2a.proto`)	fails, same error	passes
control, no empty/default values	passes	passes

The a2a-go maintainer reached the same reading on #441: "python behaviour is not spec-compliant here".

Python signs a card and Go verifies it without trouble in every case, because a2a-go's own canonicalizer does not remove already-present fields. The failures only run in the direction where Go emits a spec-compliant payload (REQUIRED field present at default) and Python's canonicalizer removes what the signer actually signed.

Proposed fix direction

Build the canonical JSON according to the field-presence rules in section 8.4.1: preserve REQUIRED fields even at their default value, preserve explicitly set optional fields, and continue omitting non-required default-valued fields. One implementation could use MessageToDict(..., always_print_fields_with_no_presence=True) followed by descriptor-aware filtering based on field_behavior and field presence.

a2a-go PR fix: Sync jsonrpc and rest implementation of authenticated agent card #441 discussion, our comment on the Go golden test covering the same interop gap: fix: a2acrypto canonicalization and refactoring a2a-go#441 (comment)
agent_card_to_dict omits the REQUIRED empty skills field #1261 and [Bug]: AgentCard REQUIRED fields are never validated #1267 are related (same MessageToDict-drops-REQUIRED-empty-fields root behavior) but cover the card-serving and validation paths, not the signing/verification canonicalization function.

Independent data point from the Go side of this same defect family, because the fix here has a cross-SDK cost worth seeing before it lands.

I reproduced this on a third-party frozen card (the one from #445): gate_card_20260928.json, 9420 bytes, sha256 2df33ff1…, signed by the JS SDK with a production key, with the matching committed JWKS. Using a2a-python at 0d5473ca (the commit in your permalink), same card, no network needed — the JWKS is a file.

What _canonicalize_agent_card produces today on that card:

a2a-python @ 0d5473ca   6410 bytes   c5d5384a19f3a15c761ff93bf9fec892ec99615b6356d7675fda5b79286b59b1

That is byte-identical to what the JS SDK produces for the same card (canonicalizeAgentCard), and the JS-signed signature verifies against exactly those bytes. So the empty-removal + implicit-presence omission you are proposing to change is not incidental — it is the thing that makes Python and JS currently agree with each other.

The two steps you identify are both load-bearing for that. Splitting them out (same card, same commit):

MessageToDict(always_print=False) + _clean_empty  →  6410  c5d5384a…   (= today's output)
MessageToDict(always_print=True)  + no clean      →  6638  db15bd6c…   (rule-1 style)
MessageToDict(always_print=True)  + _clean_empty  →  6444  bf28623a…

So always_print_fields_with_no_presence=True alone moves the output 228 bytes away from the form the JS SDK signs. On this card those 228 bytes are exactly four leaves, and none of them is a REQUIRED field:

+ /capabilities/extensions/0/required = false
+ /capabilities/extensions/1/required = false
+ /supportedInterfaces/0/tenant = ""
+ /supportedInterfaces/1/tenant = ""

That is the part I would flag: rule 1 is about REQUIRED fields at their default value, but always_print_fields_with_no_presence prints every implicit-presence field, which overshoots the requirement and takes Python out of the JS agreement as a side effect. Scoping the change to actually-REQUIRED fields (so description: "" / skills: [] survive in the payload) keeps the rest of the form intact.

Note also that on this particular card the defect in #1278 does not manifest — description, name, version are non-empty and skills has 5 entries. That is presumably why JS and Python look aligned today, and it means the divergence only appears on cards where a REQUIRED field sits at its default: exactly the case your issue describes, and exactly the case that needs to be in whatever test lands.

Related threads, so the parts can be decided in one place rather than per SDK: #445 (Go: canonicalizing the served JSON as-is gives 6852 bytes and fails the JS signature; projecting through the message and dropping empty values gives 6410 and passes), A2A #2122 (§8.4.1 is under-determined about which form a new signature must use), and a2a-tck #245 (a rule-1 conformance corpus: R1-003/R1-004 isolate exactly the description: "" / skills: [] case — every REQUIRED field present except those two, so the competing resolutions differ in precisely them — plus a MUST-REJECT third reading). Whichever way the spec call goes, a single-SDK fix moves the break to that SDK's boundary instead of removing it — Python and JS agree today, Go does not.

Reproduction, if useful: a2a-python @ 0d5473ca, pip install -e ., load the card with ignore_unknown_fields=True (the card carries url, protocolVersion, preferredTransport, compensation, which this commit's proto does not have — url is rejected by a strict Parse), then call _canonicalize_agent_card and print len(utf8) + sha256. Same for the two variants. I did not run the JS SDK locally; the JS bytes there are the ones published in #445 and independently reconstructed in Go.

@kuangmi-bit

@kuangmi-bit thanks, this is useful. Your result rules out always_print_fields_with_no_presence=True as a fix by itself. On that JS-signed card it adds four non-REQUIRED leaves and changes bytes that Python and JS currently agree on.

It does not rule out the descriptor-aware approach from the issue. The remaining question is Rule 1 itself. a2aproject/A2A#2122 is still deciding how REQUIRED fields at their default value should be represented, and a2aproject/a2a-tck#245 records the competing outcomes for description: "" and skills: [].

So I'd keep #1278 open, but not change Python's canonical bytes until that rule is settled. Once it is, the Python fix can preserve exactly the fields the chosen rule requires without pulling in unrelated implicit-presence defaults.

Thanks for catching the overshoot.

@aeoess

Hi @aeoess and @kuangmi-bit,

Thank you both for the thorough analysis and for catching the subtle edge cases around protobuf presence and canonicalization.
Your diagnosis is correct. The current pipeline in src/a2a/utils/signing.py (MessageToDict + unconditional _clean_empty) operates without protobuf descriptor awareness, dropping empty values even when fields are designated REQUIRED per Section 8.4.1 of the A2A specification.
As @kuangmi-bit rightly demonstrated, setting always_print_fields_with_no_presence=True overshoots by retaining non-required default fields, which would break canonical byte compatibility with cards currently in circulation.
The proper fix requires inspecting the field descriptors on AgentCard to selectively preserve only REQUIRED fields when empty, while continuing to prune optional default fields.

I could assign this issue to either of you if you want to contribute for a fix.

@rohityan

Thanks @rohityan — I'd like to take it, with one caveat about overlap.

@aeoess wrote the descriptor-aware approach in the issue and your comment endorses it, so a competing PR from me would be the wrong shape. Two clean splits instead:

If aeoess implements — I'll take the verification half, which is the piece I can do least ambiguously: run their patch against the frozen third-party card from a2aproject/a2a-go#445 (9420 bytes, sha256 2df33ff1…, JS-signed, with the matching committed JWKS) and report (a) REQUIRED-only preservation adds exactly the REQUIRED leaves and nothing else, (b) cards with no empty REQUIRED field keep the bytes Python and JS currently agree on (6410 / c5d5384a…), and (c) each of the four non-REQUIRED leaves the blanket flag pulled in stays absent (6638 / db15bd6c… was that overshoot).

If aeoess would rather not — I'll implement the descriptor-aware canonicalization in src/a2a/utils/signing.py myself and hand them the same verification.

Either way, on the Rule 1 dependency: the descriptor-aware machinery (walk the AgentCard descriptors, keep pruning optional defaults, keep the REQUIRED set addressable) can land independently of a2aproject/A2A#2122; only the "preserve an empty REQUIRED field" branch depends on the rule the TSC picks there. I would rather build the rule-independent half first and gate the other behind #2122 than sit on it — say the word if you would prefer to wait for that decision.

One question so the assignment is unambiguous: which of the two halves do you want me on?

@rohityan

@rohityan yes, please assign it to me.

@kuangmi-bit thank you. I'll take the implementation and would really appreciate your verification against the frozen JS-signed card.

I'll keep the patch split so the a2aproject/A2A#2122 dependency is explicit:

descriptor-aware REQUIRED-field handling, while optional defaults keep being pruned and cards with no empty REQUIRED field keep today's canonical bytes.
the Rule 1 behavior that preserves an empty REQUIRED field, with the §8.4.1 worked example and empty description / skills cases.

I checked current main at fad0482: the REQUIRED annotation is available on the generated descriptors, while AgentInterface.tenant and AgentExtension.required are not REQUIRED, so the four leaves from the blanket always_print_fields_with_no_presence approach stay out.

Whether the second commit lands now or waits for the a2aproject/A2A#2122 decision is your call. I won't treat that issue as resolved by this patch.

@aeoess

@aeoess — sounds good: implementation is yours, I'll take the verification as an independent pass, run against the frozen fixture rather than through your test harness.

Fixture + anchors, taken from the #445 comment that published them (not from inside the package):

artifact	bytes	sha256
`gate_card_20260928.json` (gate 0.4.17, JS-signed, two ES256 signatures, kid `hs-2026-09`)	9420	`2df33ff120745a7f46b8afc1378d72b437b3f05b4db61c8d8b3655c581584455`
`gate_jwks_20260928.json`	256	`692fd49da681cc0d0bda9ad46963fa5f45957a59d10f84998e9b2e4061209d3c`

Reference form I compare against: the JS SDK bytes for that card, 6410 / c5d5384a19f3a15c761ff93bf9fec892ec99615b6356d7675fda5b79286b59b1.

What I will run on your branch (offline, no network; card and JWKS read from those committed files):

Rule, not field. On the frozen card the output must be 6410 / c5d5384a… byte-for-byte — the projection must not disturb a card that has no empty REQUIRED field.
Overshoot stays dead. The four leaves the blanket flag pulled in — /capabilities/extensions/{0,1}/required, /supportedInterfaces/{0,1}/tenant — must remain absent. (That form measured 6638 / db15bd6c… at 0d5473ca; it is the regression I will be watching for.)
The case the issue is about. On a variant where a REQUIRED field sits at its default (description: "", skills: [], taken from a2a-tck#245 R1-003/R1-004), the projection must include those leaves — and I will list the added paths, not just the byte delta, so a key-specific special case would be visible.
The rule reaches nested empties. On the same variant securityRequirements: [{}] must still collapse (empty values removed recursively, an element that becomes empty collapses), per the exchange with @ogasurfproject-jpg.
End-to-end. The bytes your canonicalizer produces for the frozen card must verify that card's signatures[1] with the committed JWKS (ES256). That is the contract that matters: a JS-signed card verifies in Python.

Report shape: an anchor→result table pinned by the input hashes above, the exact command, and any behavioural difference; happy to post it here or on the PR, whichever you prefer. Send the branch/commit when it is pushed and I will run it the same day. If it helps, verify the rule-independent half first (descriptor walk + optional-default pruning) while the #2122-gated branch stays separate, as you described.

Publisher confirmation for the fixture, since it is our card: both files are unchanged from what a2aproject/a2a-go#445 published, and they are committed at https://github.com/ogasurfproject-jpg/horizon-shield/tree/main/workers/a2a-card-sign/interop-go/fixtures (9420 / 2df33ff1… and 256 / 692fd49d…, matching the table above).

Please run against the files, not the live URL. The gate has moved to 0.4.20 since that snapshot, so https://gate.horizonshield.dev/.well-known/agent-card.json now serves different bytes and a different canonical form, even though it is signed with the same key (kid hs-2026-09). The fixture will not be re-signed or edited. If the key ever rotates, the old public key stays listed in https://gate.horizonshield.dev/.well-known/key-history.json, so signatures[1] on the frozen card remains checkable.

For the verification pass: the same frozen card plus six single-field cases are now in a public matrix where all three SDKs sign and verify each other, with per-SDK canonical hashes, so the before and after of this patch can be compared row by row. Today Python and JS agree on every case and Go disagrees with both as soon as any empty value is present; the section 8.4.1 worked example is reproduced by none of the three.
https://github.com/ogasurfproject-jpg/horizon-shield/tree/main/workers/a2a-card-sign/interop-matrix

fix(signing): keep REQUIRED fields at their default value in the cano…

@kuangmi-bit

@kuangmi-bit the branch is up as #1287, CI green. The head to verify is cdee28e53f2166ca04fc360868cae9fc2b7a3182. It has two commits.

3f907120 is the descriptor traversal only, with no canonical bytes change.
cdee28e5 is the Rule 1 behavior, kept separate for [Bug]: §8.4.1 canonicalization is under-determined - two SDKs produce different bytes for the same card, and neither reproduces the section's own worked example A2A#2122.

I ran the frozen card and the #1286 tests on it and put the numbers in the PR. I would still rather your pass stand as the independent one. Posting your table on the PR works for me.

Thank you all for your analysis and discussion. The inconsistencies you found are real.

There are really two problems tangled together here:

1) The SPEC 8.4.1 example itself was wrong

The card was invalid (skills: [] violates SPEC 5.7, and several REQUIRED fields were missing) and its canonical output couldn't be reproduced from its input.
That part is a clean, non-breaking fix - a2aproject/A2A#2318 and a2aproject/A2A#2319 are supposed to replace it with a valid, reproducible example.

2) Aligning the SDKs so empty/default REQUIRED fields verify cross-SDK

It's the harder one: it changes the signed bytes, which breaks existing signatures, so it can't ship outside a major release (It's a breaking change)

On the SDK side, #1279 is supposed to add a non-breaking warning when a card has missing or empty REQUIRED fields.

Thanks, the split makes sense, and the signer-side SHOULD I suggested on A2A#2122 points the same way as the #1279 warning.

Once #1279 adds the warning in signing._canonicalize_agent_card, I'll run it over the s1 and s2 cards in a2a-card-sign-v01 (a2a-tck#246) and post which cards warn. The expectation is that it fires on exactly the cards where the shipping SDKs produce different bytes and stays silent on the s0 controls.

test(signing): add cross-SDK signature vectors (#1286)

Navigation Menu

Repository navigation

Signing/verification canonicalization drops REQUIRED fields at their default value, breaks cross-SDK Agent Card signature verification #1278

What happened

Where it happens

Spec requirement

Minimal reproduction

Cross-SDK consequence

Proposed fix direction

Related

1) The SPEC 8.4.1 example itself was wrong

2) Aligning the SDKs so empty/default REQUIRED fields verify cross-SDK

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

	def _clean_empty(d: Any, depth: int = 0) -> Any:
	"""Recursively remove empty strings, lists and dicts from a dictionary.

	Depth is bounded for the same reason canonicalization is: nesting reaches
	this function from `AgentExtension.params`, and without the bound a deeply
	nested card exhausts the interpreter stack here, before the canonicalizer
	ever gets the chance to reject it.
	"""
	if depth > MAX_DEPTH:
	raise CanonicalizationError(
	f'nesting exceeds the maximum depth of {MAX_DEPTH}'
	)
	if isinstance(d, dict):
	cleaned_dict = {
	k: cleaned_v
	for k, v in d.items()
	if (cleaned_v := _clean_empty(v, depth + 1)) is not None
	}
	return cleaned_dict or None
	if isinstance(d, list):
	cleaned_list = [
	cleaned_v
	for v in d
	if (cleaned_v := _clean_empty(v, depth + 1)) is not None
	]
	return cleaned_list or None
	if isinstance(d, str) and not d:
	return None
	return d


	def _canonicalize_agent_card(agent_card: AgentCard) -> str:
	"""Canonicalizes the Agent Card JSON according to RFC 8785 (JCS)."""
	card_dict = MessageToDict(
	agent_card,
	)
	# Remove signatures field if present
	card_dict.pop('signatures', None)

	# Recursively remove empty values
	cleaned_dict = _clean_empty(card_dict)
	return canonicalize(cleaned_dict)

Signing/verification canonicalization drops REQUIRED fields at their default value, breaks cross-SDK Agent Card signature verification #1278

Description

What happened

Where it happens

Spec requirement

Minimal reproduction

Cross-SDK consequence

Proposed fix direction

Related

Activity

kuangmi-bit commented on Sep 29, 2026

aeoess commented on Sep 29, 2026

rohityan commented on Sep 30, 2026

kuangmi-bit commented on Sep 30, 2026

aeoess commented on Sep 30, 2026

kuangmi-bit commented on Sep 30, 2026

ogasurfproject-jpg commented on Sep 30, 2026

ogasurfproject-jpg commented on Sep 30, 2026

aeoess commented on Oct 1, 2026

Iwaniukooo11 commented on Oct 7, 2026

1) The SPEC 8.4.1 example itself was wrong

2) Aligning the SDKs so empty/default REQUIRED fields verify cross-SDK

ogasurfproject-jpg commented on Oct 7, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions