f5381 enforce trailerField==1 in DecodeRsaPssParams

[miyazakh]

asn: enforce trailerField == trailerFieldBC(1) in DecodeRsaPssParams

Summary

• Bug fix: Validate that the trailerField parameter in RSASSA-PSS-params
  equals trailerFieldBC(1) as mandated by RFC 8017 §A.2.3, in both the ASN
  template and non-template paths of DecodeRsaPssParams.
• Test: Extend test_wc_DecodeRsaPssParams with three trailerField-specific
  cases covering the valid value and two invalid values.

Problem

RFC 8017 §A.2.3 defines:

TrailerField ::= INTEGER { trailerFieldBC(1) }

and states:

The value of the trailerField field SHALL be trailerFieldBC(1).

DecodeRsaPssParams parsed the trailerField element syntactically but
never validated its integer value, so any value was silently accepted.

Non-template path (!WOLFSSL_ASN_TEMPLATE)

/* before */
ret = GetInteger16Bit(params, &idx, sz);
if (ret > 0) {      /* accepts 1, 2, 3, …, 65535 */
    ret = 0;
}
else if (ret != 0) {
    WOLFSSL_MSG("DecodeRsaPssParams: fail at trailer_value");
}
/* ret == 0 (value 0) also falls through silently */

Template path (WOLFSSL_ASN_TEMPLATE)

RSAPSSPARAMSASN_IDX_TRAILERINT appeared in the enum but
GetASN_Int16Bit was never called for it, so GetASN_Items validated
only the ASN.1 type tag while the integer value went completely unread.

Impact

• Any positive trailerField value (2, 3, …) and value 0 were accepted without
  error in both paths.
• PSS signature verification always assumes the standard 0xbc trailer
  regardless; a non-conformant trailerField in an AlgorithmIdentifier is
  therefore accepted by the parser while an RFC-compliant peer would reject it.
• This is a conformance / input-validation defect, not a signature-forgery
  vulnerability.

Fix

Both paths now reject trailerField != 1 with ASN_PARSE_E when
WOLFSSL_NO_ASN_STRICT is not defined (the default). When
WOLFSSL_NO_ASN_STRICT is defined the previous permissive behaviour is
preserved for backward compatibility.

Template path (wolfcrypt/src/asn.c)

#ifndef WOLFSSL_NO_ASN_STRICT
    word16 trailerVal = 1;
#endif
    ...
#ifndef WOLFSSL_NO_ASN_STRICT
    GetASN_Int16Bit(&dataASN[RSAPSSPARAMSASN_IDX_TRAILERINT], &trailerVal);
#endif
    ret = GetASN_Items(...);
    ...
#ifndef WOLFSSL_NO_ASN_STRICT
    /* RFC 8017 A.2.3: trailerField SHALL be trailerFieldBC(1). */
    if ((ret == 0) && (dataASN[RSAPSSPARAMSASN_IDX_TRAILERINT].tag != 0)) {
        if (trailerVal != 1) {
            WOLFSSL_MSG("DecodeRsaPssParams: trailerField must be 1");
            ret = ASN_PARSE_E;
        }
    }
#endif

Non-template path (wolfcrypt/src/asn.c)

ret = GetInteger16Bit(params, &idx, sz);
#ifndef WOLFSSL_NO_ASN_STRICT
    /* RFC 8017 A.2.3: trailerField SHALL be trailerFieldBC(1). */
    if (ret == 1) {
        ret = 0;
    }
    else {
        WOLFSSL_MSG("DecodeRsaPssParams: trailerField must be 1");
        if (ret >= 0)
            ret = ASN_PARSE_E;
    }
#else
    if (ret > 0) { ret = 0; }
    else if (ret != 0) { WOLFSSL_MSG("...fail at trailer_value"); }
#endif

Test

Three cases added to test_wc_DecodeRsaPssParams in
tests/api/test_asn.c. All use a minimal DER-encoded RSASSA-PSS-params
containing only the trailerField component:
SEQUENCE { [3] CONSTRUCTED { INTEGER <value> } }.

Test	DER (hex)	trailerField value	Guard	Expected
9	30 05 a3 03 02 01 01	1 (trailerFieldBC)	none — all modes	0 (success)
10	30 05 a3 03 02 01 02	2	!WOLFSSL_NO_ASN_STRICT	ASN_PARSE_E
11	30 05 a3 03 02 01 00	0	!WOLFSSL_NO_ASN_STRICT	ASN_PARSE_E

Tests 10 and 11 exercise both the template and non-template paths with
the same DER input; the path is selected at build time.

How to build and run

Template path (default), strict mode — all three tests active:

./configure --enable-rsapss --enable-all \
    CFLAGS="-DNO_WOLFSSL_CIPHER_SUITE_TEST"
git clean -dfx && make
./tests/unit.test --list | grep DecodeRsaPssParams
./tests/unit.test -<N>

Non-template path, strict mode:

./configure --enable-rsapss --enable-all --enable-asn=original \
    CFLAGS="-DNO_WOLFSSL_CIPHER_SUITE_TEST"
git clean -dfx && make

Non-strict mode — only Test 9 active, Tests 10/11 compiled out:

./configure --enable-rsapss --enable-all \
    CFLAGS="-DNO_WOLFSSL_CIPHER_SUITE_TEST -DWOLFSSL_NO_ASN_STRICT"
git clean -dfx && make

Files changed

File	Change
wolfcrypt/src/asn.c	Template path: capture and validate trailerField value; non-template path: enforce trailerField == 1; both guarded by !WOLFSSL_NO_ASN_STRICT
tests/api/test_asn.c	Add Tests 9–11 to test_wc_DecodeRsaPssParams

Checklist

• added tests
• updated/added doxygen
• updated appropriate READMEs
• Updated manual and documentation

[Copilot]

Pull request overview

This pull request tightens RFC 8017 conformance for RSASSA-PSS parameter decoding by enforcing trailerField == trailerFieldBC(1) (i.e., integer value 1) in DecodeRsaPssParams, while preserving legacy permissive behavior when WOLFSSL_NO_ASN_STRICT is defined. It also extends the unit tests to cover valid and invalid trailerField encodings.

Changes:

• Enforce trailerField value validation (== 1) in both ASN-template and non-template decoding paths (guarded by !WOLFSSL_NO_ASN_STRICT).
• Add unit tests covering trailerField values 1 (valid), 2 and 0 (invalid in strict mode).

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File	Description
wolfcrypt/src/asn.c	Adds strict-mode validation of trailerField value in both template and non-template decode paths.
tests/api/test_asn.c	Adds trailerField-focused DecodeRsaPssParams test vectors (strict and non-strict coverage via preprocessor guards).

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[github-actions]

MemBrowse Memory Report

No memory changes detected for:

• gcc-arm-cortex-m0plus
• gcc-arm-cortex-m3
• gcc-arm-cortex-m4
• gcc-arm-cortex-m4-baremetal
• gcc-arm-cortex-m4-crypto-only
• gcc-arm-cortex-m4-dtls13
• gcc-arm-cortex-m4-min-ecc
• gcc-arm-cortex-m4-openssl-compat
• gcc-arm-cortex-m4-pkcs7
• gcc-arm-cortex-m4-pq
• gcc-arm-cortex-m4-rsa-only
• gcc-arm-cortex-m4-sp-math
• gcc-arm-cortex-m4-tls12
• gcc-arm-cortex-m4-tls13
• gcc-arm-cortex-m7
• gcc-arm-cortex-m7-pq
• gcc-arm-cortex-m7-tls13
• linuxkm-pie
• linuxkm-standard
• stm32-sim-stm32h753