Skip to content

[python] Fix Python preprocess handling for numeric YAML scalars#11143

Closed
msyyc with Copilot wants to merge 8 commits into
mainfrom
copilot/fix-attributeerror-in-preprocess-description
Closed

[python] Fix Python preprocess handling for numeric YAML scalars#11143
msyyc with Copilot wants to merge 8 commits into
mainfrom
copilot/fix-attributeerror-in-preprocess-description

Conversation

Copilot AI commented Jul 2, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

The Python client preprocessor could crash when TypeSpec union enum values emitted numeric YAML scalars, e.g. date-like backtick labels. Numeric descriptions hit rstrip, and numeric enum value names hit snake-case conversion.

  • Preprocess normalization

    • Convert description scalars to strings before trimming/appending punctuation.
    • Convert type and enum value names to strings before reserved-word and casing logic.

  • Regression coverage

    • Added unit coverage for non-string descriptions.
    • Added unit coverage for numeric enum value names in both top-level and nested enum preprocessing.
union ApiVersion {
  string;
  `2020-01-01`: "2020-01-01";
}

This now preprocesses without raising AttributeError or TypeError from Python string helpers.

Copilot AI linked an issue Jul 2, 2026 that may be closed by this pull request
[http-client-python] AttributeError: 'int' object has no attribute 'rstrip' in preprocess update_description #11138
Open
@msyyc
fix: handle non-string python descriptions
e6cd892 
Co-authored-by: msyyc <70930885+msyyc@users.noreply.github.com>
@microsoft-github-policy-service microsoft-github-policy-service Bot added the emitter:client:python Issue for the Python client emitter: @typespec/http-client-python label Jul 2, 2026
Copilot AI and others added 2 commits July 2, 2026 03:37
@msyyc
docs: clarify python description preprocessing
81911f9 
Co-authored-by: msyyc <70930885+msyyc@users.noreply.github.com>
@msyyc
docs: specify scalar description handling
1960d38 
Co-authored-by: msyyc <70930885+msyyc@users.noreply.github.com>
Copilot AI changed the title [WIP] Fix AttributeError in preprocess update_description Fix Python preprocess handling for numeric YAML scalars Jul 2, 2026
Copilot finished work on behalf of msyyc July 2, 2026 03:41
Copilot AI requested a review from msyyc July 2, 2026 03:41
msyyc
msyyc reviewed Jul 2, 2026
View reviewed changes
Comment thread packages/http-client-python/tests/unit/test_name_converter.py Outdated
Copilot AI and others added 3 commits July 2, 2026 08:18
@msyyc
fix: preserve numeric enum separators
8681ab3 
Co-authored-by: msyyc <70930885+msyyc@users.noreply.github.com>
@msyyc
fix: clarify enum name fallback
9d09468 
Co-authored-by: msyyc <70930885+msyyc@users.noreply.github.com>
@msyyc
test: cover numeric enum value fallback
5feca87 
Co-authored-by: msyyc <70930885+msyyc@users.noreply.github.com>
Copilot AI requested a review from msyyc July 2, 2026 08:22
Copilot finished work on behalf of msyyc July 2, 2026 08:22
@msyyc msyyc marked this pull request as ready for review July 3, 2026 07:42
@pkg-pr-new

pkg-pr-new Bot commented Jul 3, 2026

Copy link
Copy Markdown

Open in StackBlitz

npm i https://pkg.pr.new/@typespec/http-client-python@11143

commit: 87c8a19

@msyyc msyyc changed the title Fix Python preprocess handling for numeric YAML scalars [python] Fix Python preprocess handling for numeric YAML scalars Jul 3, 2026
@github-actions

github-actions Bot commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

All changed packages have been documented.

  • ✅ @typespec/http-client-python
Show changes

@typespec/http-client-python - fix ✏️

Handle non-string YAML descriptions during Python client preprocessing.

@azure-sdk-automation

azure-sdk-automation Bot commented Jul 3, 2026

Copy link
Copy Markdown

You can try these changes here

🛝 Playground 🌐 Website 🛝 VSCode Extension

@msyyc msyyc mentioned this pull request Jul 3, 2026
Merged
@msyyc

msyyc commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

move to #11162

@msyyc msyyc closed this Jul 3, 2026
@msyyc msyyc deleted the copilot/fix-attributeerror-in-preprocess-description branch July 3, 2026 08:33
msyyc added a commit to msyyc/typespec that referenced this pull request Jul 3, 2026
@msyyc
Revert pygen defensive workarounds; keep emitter root-cause fix only
9c8ebe0 
The emitter force-quoting fix makes the non-string description/enum-name
handling from microsoft#11143 unnecessary, so revert those pygen changes and their
tests, leaving a focused emitter-only fix.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
live1206 pushed a commit to live1206/typespec that referenced this pull request Jul 7, 2026
@msyyc
[python] Fix enum member names corrupted by YAML 1.1/1.2 boundary (mi…
8f2b8b1 
…crosoft#11162)

fix microsoft#11138

## Problem

For a date-like TypeSpec label such as:

```typespec
union ApiVersion {
  string;
  `2020-01-01`: "2020-01-01";
}
```

the emitter's `enumName()` correctly computes the enum member name as
the string `2020_01_01`. However, the code model is serialized with
**js-yaml (YAML 1.2)**, which dumps this scalar **unquoted** (`name:
2020_01_01`). The Python generator then parses the YAML with **PyYAML
(YAML 1.1)**, where `2020_01_01` is a valid integer separator syntax, so
the name is read back as the integer `20200101` — corrupting the enum
member name.

This is the underlying root cause behind microsoft#11143. That PR worked around
the crash in pygen by reconstructing the name from the enum `value` via
regex, rather than fixing the corruption at the source.

## Fix

**Emitter (root cause)** — force-quote string scalars when serializing
the code model so every string round-trips faithfully through PyYAML:
- Added a shared `dumpCodeModelToYaml()` helper (`forceQuotes: true,
quotingType: '"'`) in `external-process.ts`, used by both the node and
pyodide serialization paths.

**pygen (defensive)** — simplified enum value name handling to a plain
`str()` coercion so the generator stays robust to any scalar, letting
the existing digit → `ENUM_` prefixing produce `ENUM_2020_01_01`.

## Tests

- New emitter unit test (`emitter/test/external-process.test.ts`)
asserting ambiguous scalars are quoted and round-trip as strings.
- Updated pygen unit tests covering both the happy path (string
`2021_01_01` → `ENUM_2021_01_01`) and the defensive fallback (int
`20210101` → `ENUM_20210101`).

Verified end-to-end: emitter dump → PyYAML load now yields the string
`'2020_01_01'` (previously `int 20200101`). Emitter vitest (4 passed)
and pygen pytest (16 passed) green.

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
tadelesh pushed a commit that referenced this pull request Jul 9, 2026
@msyyc
[python] Fix enum member names corrupted by YAML 1.1/1.2 boundary (#1…
06d3b1e 
…1162)

fix #11138

## Problem

For a date-like TypeSpec label such as:

```typespec
union ApiVersion {
  string;
  `2020-01-01`: "2020-01-01";
}
```

the emitter's `enumName()` correctly computes the enum member name as
the string `2020_01_01`. However, the code model is serialized with
**js-yaml (YAML 1.2)**, which dumps this scalar **unquoted** (`name:
2020_01_01`). The Python generator then parses the YAML with **PyYAML
(YAML 1.1)**, where `2020_01_01` is a valid integer separator syntax, so
the name is read back as the integer `20200101` — corrupting the enum
member name.

This is the underlying root cause behind #11143. That PR worked around
the crash in pygen by reconstructing the name from the enum `value` via
regex, rather than fixing the corruption at the source.

## Fix

**Emitter (root cause)** — force-quote string scalars when serializing
the code model so every string round-trips faithfully through PyYAML:
- Added a shared `dumpCodeModelToYaml()` helper (`forceQuotes: true,
quotingType: '"'`) in `external-process.ts`, used by both the node and
pyodide serialization paths.

**pygen (defensive)** — simplified enum value name handling to a plain
`str()` coercion so the generator stays robust to any scalar, letting
the existing digit → `ENUM_` prefixing produce `ENUM_2020_01_01`.

## Tests

- New emitter unit test (`emitter/test/external-process.test.ts`)
asserting ambiguous scalars are quoted and round-trip as strings.
- Updated pygen unit tests covering both the happy path (string
`2021_01_01` → `ENUM_2021_01_01`) and the defensive fallback (int
`20210101` → `ENUM_20210101`).

Verified end-to-end: emitter dump → PyYAML load now yields the string
`'2020_01_01'` (previously `int 20200101`). Emitter vitest (4 passed)
and pygen pytest (16 passed) green.

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@msyyc msyyc Awaiting requested review from msyyc msyyc is a code owner
@iscai-msft iscai-msft Awaiting requested review from iscai-msft iscai-msft is a code owner
@tadelesh tadelesh Awaiting requested review from tadelesh tadelesh is a code owner
@timotheeguerin timotheeguerin Awaiting requested review from timotheeguerin timotheeguerin is a code owner
@lmazuel lmazuel Awaiting requested review from lmazuel lmazuel is a code owner
@catalinaperalta catalinaperalta Awaiting requested review from catalinaperalta catalinaperalta is a code owner
@ChenxiJiang333 ChenxiJiang333 Awaiting requested review from ChenxiJiang333 ChenxiJiang333 is a code owner
@l0lawrence l0lawrence Awaiting requested review from l0lawrence l0lawrence is a code owner
@kashifkhan kashifkhan Awaiting requested review from kashifkhan kashifkhan is a code owner

Assignees

@msyyc msyyc

Copilot code review Copilot

Labels

emitter:client:python Issue for the Python client emitter: @typespec/http-client-python

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[http-client-python] AttributeError: 'int' object has no attribute 'rstrip' in preprocess update_description

2 participants

@msyyc