Skip to content

Commit 25d158f

Browse files
committed
Comprehensive string handling rewrite
Prevents OpenAPI Document content from escaping strings, docstrings, and f-strings in generated code by systematically rewriting all string handling. Solves GHSA-5293-mq8x-g3xj
1 parent ee9a8c4 commit 25d158f

393 files changed

Lines changed: 4464 additions & 2007 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
---
2+
default: security
3+
---
4+
5+
# Arbitrary code generation vulnerability
6+
7+
Prior to this release, malicious OpenAPI documents could cause openapi-python-client to generate arbitrary code, which
8+
would then be executed by consumers of the generated client.
9+
10+
If you generate code from OpenAPI documents you don't control, you should upgrade to this release as **soon as possible**
11+
and validate any previously-generated code.
12+
13+
See [the GitHub advisory](https://github.com/openapi-generators/openapi-python-client/security/advisories/GHSA-5293-mq8x-g3xj) for more details.
Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
---
2+
default: note
3+
---
4+
5+
# Breaking changes for all custom templates
6+
7+
**ALL** custom templates are expected to break with this version as a result of the security fix.
8+
9+
1. The `utils` global has been renamed to `strings`
10+
2. Most string values can no longer be rendered directly into templates,
11+
you must describe how the value is being used so it can be properly escaped using either a Python function or Jinja filter:
12+
1. `strings.snake_case()` / `| snakecase` (existing)
13+
2. `strings.kebab_case()` / `| kebabcase` (existing)
14+
3. `strings.pascal_case()` / `| pascalcase` (existing)
15+
4. `python_identifier()` (existing)
16+
5. `class_name()` (existing)
17+
6. `strings.safe_for_docstring()` / `| safe_for_docstring` for values which get injected into a `"""` docstring
18+
7. `strings.in_f_string_literal()` / `| in_f_string_literal` for values that go into `f""` f-strings
19+
8. `strings.in_double_quote_literal()` / `| in_double_quote_literal` for values that go into **non-f-string** `""` literals
20+
9. `.as_unembedded_code()` / `| as_unembedded_code` ONLY for `PythonCode` values—those that are intended to be Python code which is not embedded into any string/docstring. Examples include usages of `.python_code`, `.get_type_string()`, `.get_instance_type_string()`, `.get_type_strings_in_union()`. You *should not* assume these values are safe to put in docstrings, string literals, or f-string literals. Use the dedicated helpers for those.
21+
As always, you can check the diff of the built in templates for examples. You will also want to check generated output
22+
for "UntrustedString", which is how any string now requiring one of those functions will appear.
Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
---
2+
default: patch
3+
---
4+
5+
# Fixed invalid Python identifiers when resolving naming conflicts
6+
7+
When two property or parameter names conflicted after conversion to `snake_case` (e.g. `foo-bar` and `fooBar`), the conflict-resolution path preserved delimiters like `-`, `.`, and spaces in the generated Python identifiers, producing invalid code which failed generation. Conflicting names now keep their original casing but have any characters which are invalid in Python identifiers stripped (e.g. `foobar` and `fooBar`).
Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
---
2+
default: note
3+
---
4+
5+
# Many control characters now stripped from string literals
6+
7+
Out of an abundance of caution, most Unicode control characters are now stripped from string literals.
8+
If your API uses control characters as part of const values, enums, or JSON body property names you may have issues
9+
with this new version.
10+
11+
Most APIs are not expected to be affected by this change.
Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
---
2+
default: patch
3+
---
4+
5+
# Improve readability of error messages
6+
7+
Errors and warnings which include a snippet of your OpenAPI document now render that snippet as JSON, making them much easier to read.
Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
---
2+
default: patch
3+
---
4+
5+
# Stopped generating empty docstrings for models with no description

.github/workflows/fuzz.yml

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
name: Fuzz code generation
2+
3+
permissions:
4+
contents: read
5+
6+
on:
7+
push:
8+
branches: [ "main" ]
9+
pull_request:
10+
branches: [ "main" ]
11+
schedule:
12+
- cron: "23 8 * * *"
13+
workflow_dispatch:
14+
15+
jobs:
16+
fuzz:
17+
runs-on: ubuntu-latest
18+
timeout-minutes: 20
19+
env:
20+
OPENAPI_PYTHON_CLIENT_FUZZ_EXAMPLES: ${{ github.event_name == 'schedule' && '1000' || '100' }}
21+
steps:
22+
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
23+
- name: Set up Python
24+
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
25+
with:
26+
python-version: "3.14"
27+
cache: pip
28+
- name: Install PDM
29+
run: pip install pdm
30+
- name: Install dependencies
31+
run: pdm sync --clean
32+
- name: Fuzz generated Python
33+
run: pdm fuzz

.gitignore

Lines changed: 2 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -34,7 +34,5 @@ test-reports/
3434
htmlcov/
3535

3636
# Generated end to end test data
37-
my-test-api-client/
38-
custom-e2e/
39-
3-1-features-client
40-
tests/tmp
37+
tests/tmp
38+
end_to_end_tests/tmp/*

end_to_end_tests/__init__.py

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
1-
""" Generate a complete client and verify that it is correct """
1+
"""Generate a complete client and verify that it is correct"""
2+
23
import pytest
34

45
pytest.register_assert_rewrite("end_to_end_tests.end_to_end_test_helpers")

end_to_end_tests/__snapshots__/test_end_to_end.ambr

Lines changed: 9 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -34,12 +34,12 @@
3434
Warning(s) encountered while generating. Client was generated, but some pieces may be missing
3535
WARNING parsing PUT / within default. Endpoint will not be generated.
3636

37-
cannot parse parameter of endpoint put_: Invalid UUID value: 3
37+
cannot parse parameter of endpoint put: Invalid UUID value: 3
3838

3939

4040
WARNING parsing POST / within default. Endpoint will not be generated.
4141

42-
cannot parse parameter of endpoint post_: Invalid UUID value: notauuid
42+
cannot parse parameter of endpoint post: Invalid UUID value: notauuid
4343

4444

4545
If you believe this was a mistake or this tool is missing a feature you need, please open an issue at https://github.com/openapi-generators/openapi-python-client/issues/new/choose
@@ -69,7 +69,13 @@
6969

7070
Path parameter must be required
7171

72-
Parameter(name='optional', param_in=<ParameterLocation.PATH: 'path'>, description=None, required=False, deprecated=False, allowEmptyValue=False, style=None, explode=False, allowReserved=False, param_schema=Schema(title=None, multipleOf=None, maximum=None, exclusiveMaximum=None, minimum=None, exclusiveMinimum=None, maxLength=None, minLength=None, pattern=None, maxItems=None, minItems=None, uniqueItems=None, maxProperties=None, minProperties=None, required=None, enum=None, const=None, type=<DataType.STRING: 'string'>, allOf=[], oneOf=[], anyOf=[], schema_not=None, items=None, prefixItems=[], properties=None, additionalProperties=None, description=None, schema_format=None, default=None, nullable=False, discriminator=None, readOnly=None, writeOnly=None, xml=None, externalDocs=None, example=None, deprecated=None), example=None, examples=None, content=None)
72+
{
73+
"name": "optional",
74+
"in": "path",
75+
"schema": {
76+
"type": "string"
77+
}
78+
}
7379

7480
If you believe this was a mistake or this tool is missing a feature you need, please open an issue at https://github.com/openapi-generators/openapi-python-client/issues/new/choose
7581

0 commit comments

Comments
 (0)