`ModelBuilder` missing `resource_requirements` and `inference_component_name` fields for `CustomOrchestrator` IC deployment

[lopezfelipe]

PySDK Version

• PySDK V2 (2.x)
• PySDK V3 (3.x)

Describe the bug

When deploying a CustomOrchestrator as an Inference Component — as documented in the Build and deploy AI inference workflows with new enhancements to the Amazon SageMaker Python SDK blog post — the ModelBuilder class internally reads self.resource_requirements and self.inference_component_name during build() to determine:

1. Whether to deploy the orchestrator as an IC or a standalone Endpoint (model_builder.py L4507)
2. The IC name and resource allocation to include in the deployable spec (model_builder.py L4534-4536)

However, neither resource_requirements nor inference_component_name are defined as dataclass fields on ModelBuilder. They cannot be passed through the constructor, despite being required for the documented CustomOrchestrator workflow.

The ModelBuilder dataclass fields are:

model, model_path, inference_spec, schema_builder, modelbuilder_list, role_arn,
sagemaker_session, image_uri, s3_model_data_url, source_code, env_vars, model_server,
model_metadata, log_level, content_type, accept_type, compute, network, instance_type,
mode, shared_libs, dependencies, image_config

The existing compute field is a Compute(ResourceConfig) class designed for training jobs (volume sizes, spot training, instance groups) — not inference component resource allocation.

To reproduce

Try the following script:

from sagemaker.serve.model_builder import ModelBuilder, SchemaBuilder
from sagemaker.serve.spec.inference_base import CustomOrchestrator
from sagemaker.core.inference_config import ResourceRequirements
from sagemaker.core.helper.session_helper import Session, get_execution_role

class MyOrchestrator(CustomOrchestrator):
    def __init__(self, endpoint_name, component_names):
        super().__init__()
        self.endpoint_name = endpoint_name
        self.component_names = component_names

    def handle(self, data, context=None):
        import json
        response = self.client.invoke_endpoint(
            EndpointName=self.endpoint_name,
            InferenceComponentName=self.component_names[0],
            Body=data if isinstance(data, (str, bytes)) else json.dumps(data),
            ContentType="application/json"
        )
        return json.loads(response["Body"].read())

role = get_execution_role()
sess = Session()

# This is the expected usage pattern per the reference notebook:
orchestrator = ModelBuilder(
    inference_spec=MyOrchestrator(
        endpoint_name="my-endpoint",
        component_names=["base-ic", "adapter-ic"],
    ),
    dependencies={"auto": False, "custom": ["cloudpickle"]},
    sagemaker_session=sess,
    role_arn=role,
    schema_builder=SchemaBuilder(sample_input="Test", sample_output={"generated_text": "test"}),
    resource_requirements=ResourceRequirements(
        requests={"memory": 4096, "num_accelerators": 1, "copies": 1, "num_cpus": 2}
    ),
    inference_component_name="my-orchestrator-ic",
)

Expected behavior

ModelBuilder accepts resource_requirements and inference_component_name as constructor parameters, consistent with the documented usage in the Llama3.1-Mistral inference workflow reference notebook (Cell 15):

orchestrator = ModelBuilder(
    inference_spec=PythonCustomInferenceEntryPoint(...),
    dependencies={"auto": False, "custom": ["cloudpickle", "graphene"]},
    sagemaker_session=Session(),
    role_arn=role,
    resource_requirements=ResourceRequirements(
        requests={"memory": 4096, "num_accelerators": 1, "copies": 1, "num_cpus": 2}
    ),
    inference_component_name=custom_orchestrator_name,
    schema_builder=SchemaBuilder(sample_input="Test", sample_output={"generated_text": "test"}),
    modelbuilder_list=[llama_model_builder, mistral_mb]
)

Screenshots or logs

╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ in <module>:4                                                                                    │
│                                                                                                  │
│    1 from sagemaker.serve.model_builder import ModelBuilder, SchemaBuilder                       │
│    2 from sagemaker.core.inference_config import ResourceRequirements                            │
│    3                                                                                             │
│ ❱  4 orchestrator = ModelBuilder(                                                                │
│    5 │   inference_spec=SequentialWorkflow(                                                      │
│    6 │   │   region_name=region,                                                                 │
│    7 │   │   endpoint_name=endpoint_name,                                                        │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
TypeError: ModelBuilder.__init__() got an unexpected keyword argument 'resource_requirements'

╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ in <module>:4                                                                                    │
│                                                                                                  │
│    1 from sagemaker.serve.model_builder import ModelBuilder, SchemaBuilder                       │
│    2 from sagemaker.core.inference_config import ResourceRequirements                            │
│    3                                                                                             │
│ ❱  4 orchestrator = ModelBuilder(                                                                │
│    5 │   inference_spec=SequentialWorkflow(                                                      │
│    6 │   │   region_name=region,                                                                 │
│    7 │   │   endpoint_name=endpoint_name,                                                        │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
TypeError: ModelBuilder.__init__() got an unexpected keyword argument 'inference_component_name'

Workaround

Setting attributes directly on the instance after construction:

```python
orchestrator = ModelBuilder(
    inference_spec=MyOrchestrator(...),
    dependencies={"auto": False, "custom": ["cloudpickle"]},
    sagemaker_session=sess,
    role_arn=role,
    schema_builder=SchemaBuilder(sample_input="Test", sample_output={"generated_text": "test"}),
)

# Workaround: set attributes post-construction
orchestrator.resource_requirements = ResourceRequirements(
    requests={"memory": 4096, "num_accelerators": 1, "copies": 1, "num_cpus": 2}
)
orchestrator.inference_component_name = "my-orchestrator-ic"

orchestrator.build()
```

This works because `build()` reads `self.resource_requirements` and `self.inference_component_name` via attribute access, but it is undocumented and inconsistent with the published reference sample.

**System information**
A description of your system. Please provide:
- **SageMaker Python SDK version**: `sagemaker-serve 1.20.0 (SDK v3)`
- **Framework name (eg. PyTorch) or algorithm (eg. KMeans)**: sagemaker-distribution-prod:3.2.0-cpu
- **Framework version**:
- **Python version**: 3.12
- **CPU or GPU**: GPU
- **Custom Docker image (Y/N)**: N

**Additional context**
Add any other context about the problem here.