Repository navigation
Global transform: false loads simple values as NULL #81
Description
With the global option transform: false (added in #72), simple values are loaded as NULL. The same file loads correctly with the default config.
Version: 0.14.0 and current main (f35cc58), xmlschema 4.3.2, DuckDB in memory. Also seen on PostgreSQL 18.
person.xsd:
<?xml version="1.0"?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema">
<xs:element name="people"><xs:complexType><xs:sequence>
<xs:element name="person" type="personType" maxOccurs="unbounded"/>
</xs:sequence></xs:complexType></xs:element>
<xs:complexType name="personType"><xs:sequence>
<xs:element name="name" type="xs:string"/>
<xs:element name="age" type="xs:int"/>
<xs:element name="address" type="addressType"/>
</xs:sequence></xs:complexType>
<xs:complexType name="addressType"><xs:sequence>
<xs:element name="city" type="xs:string"/>
</xs:sequence></xs:complexType>
</xs:schema>people.xml:
<?xml version="1.0"?>
<people><person><name>Ada</name><age>36</age><address><city>London</city></address></person></people>Script (prints each table's non-key columns):
import sys
import sqlalchemy
from xml2db import DataModel
for config in ({}, {"transform": False}):
engine = sqlalchemy.create_engine("duckdb:///:memory:")
model = DataModel(sys.argv[1], short_name="people", db_engine=engine, model_config=config)
model.parse_xml(sys.argv[2]).insert_into_target_tables()
with engine.connect() as conn:
tables = [t for t in sqlalchemy.inspect(engine).get_table_names() if not t.startswith("temp")]
rows = {t: conn.execute(sqlalchemy.text(f'select * from "{t}"')).mappings().all() for t in sorted(tables)}
print(config, {t: [{k: v for k, v in r.items() if not k.startswith(("pk_", "fk_", "xml2db"))} for r in rs] for t, rs in rows.items()})Output of python repro.py person.xsd people.xml:
{} {'people': [{}], 'people_person': [{}], 'person': [{'name': 'Ada', 'age': 36, 'address_city': 'London'}]}
{'transform': False} {'address': [{'city': None}], 'people': [{}], 'people_person': [{}], 'person': [{'name': None, 'age': None}]}
Expected with transform: false: person with name='Ada', age=36, and address with city='London'. The import reports success, so the loss is silent.
Setting transform: false per field instead (tables.person.fields.address.transform: false) works as expected.
A possible lead: in DataModelTransformedTable._get_field_transform, the global flag also turns off the default "join" for simple columns, not only elevate, and parsed simple values may then not be converted to scalars before insertion. I have not confirmed this.
🤖 Generated with Claude Code
Activity
Keep simple values when global transform is false (#86)