feat(bridge): RoBERTa masked-LM adapter

[Canonik]

Description

Adds RobertaArchitectureAdapter so RobertaForMaskedLM checkpoints (roberta-base, roberta-large, distilroberta-base and the fine-tunes that keep the MLM head) load through TransformerBridge.boot_transformers. Before this the factory raised Unsupported architecture: RobertaForMaskedLM even though utilities/architectures.py already routed the family to the masked-LM loader.

The adapter subclasses BertArchitectureAdapter, calls super().__init__(cfg) and rebuilds component_mapping with RoBERTa's paths (roberta.* trunk, lm_head.* head), as discussed on the issue:

• token_type_embed stays mapped to roberta.embeddings.token_type_embeddings (a real [1, d_model] table at type_vocab_size = 1).
• The hook_mlp_in / hook_mlp_out redirects on BlockBridge are preserved, since RobertaLayer has BERT's sibling intermediate.dense / output.dense layout with no MLP module. The tiny CI test caches both aliases against the HF modules.
• The inherited prepare_model is a no-op here (no bert.pooler, no cls.* heads); the unit test pins that.
• n_ctx is reported as HF's max_position_embeddings minus the two rows RoBERTa reserves (position ids start at padding_idx + 1 = 2). I went with self.cfg.n_ctx -= 2 rather than pinning 512 so roberta-large (514 rows) and longer-table checkpoints get the same treatment; the n_ctx + 2 branch in W_pos fires either way and returns the [512, d_model] table aligned to content positions, while pos_embed.W_pos keeps the raw table. The one caveat is that build_bridge_from_module(tl_config=...) expects a config that still carries the table size, as its docstring says; a pre-adjusted one would be shifted twice. Happy to pin 512 if you prefer the literal version.
• No tokenizer flags are set on the adapter: <s> / </s> handling comes from each checkpoint's tokenizer at boot (detect_tokenizer_bos_eos sees bos = cls = 0); the parity test pins the detected flags.

Registered in the factory, supported_architectures/__init__.py, HF_SUPPORTED_ARCHITECTURES + CANONICAL_AUTHORS_BY_ARCH (FacebookAI, distilbert) and ARCHITECTURE_DESCRIPTIONS (the RoBERTa string already existed there, it just was not reachable).

Registry: entries for FacebookAI/roberta-base and distilbert/distilroberta-base, both run through verify_models: P1 = P2 = P3 = 100%, P4 not applicable to a masked LM (same as google-bert/bert-base-cased). verification_history.json carries the two matching records.

Tests:

• tests/unit/model_bridge/supported_architectures/test_roberta_adapter.py pins the mapping against BertArchitectureAdapter (same keys, bridge types, block layout, weight conversions), the RoBERTa HF paths, the context-length adjustment, the no-op prepare_model, untouched tokenizer flags and the factory routing.
• tests/integration/model_bridge/test_roberta_tiny.py runs in CI on a random-weight two-layer RobertaForMaskedLM: live wiring and tied decoder weight, W_pos / W_E_pos shapes, hook capture against the HF modules (including hook_mlp_in / hook_mlp_out and ln_final.hook_normalized), the position rows HF actually looks up (2.. for content, 1 for padding), padded and unpadded logit parity, and a block-output intervention.
• tests/integration/model_bridge/test_bridge_vs_hf_roberta_parity.py follows test_bridge_vs_hf_bert_parity.py on distilbert/distilroberta-base against an independent eager HF reference, including the padding-mask case. It is slow-marked (the checkpoint is not in the CI cache) with a row in tests/QUARANTINES.md; passes locally, 6/6.

Left padding: the bridge used to hand RoBERTa its own 0-based position ids (the gate in utilities/position_ids.py only recognised OPT-style mask-consuming embeddings), so left-padded rows landed on the two reserved rows while right padding, tensor inputs and verify_models were unaffected. Fixed in the last commit: a module exposing create_position_ids_from_input_ids now counts as owning its positions, with regression cases on the gate, the tiny model and the parity file. RoBERTa is the only registered architecture in that family, so nothing else changes behaviour.

One thing I would rather get your call on than change here: to_tokens strips the trailing </s> because the tokenizer appends it (tokenizer_appends_eos), a policy written for causal LMs, so bridge("text") feeds RoBERTa one token fewer than HF does, while BERT keeps [SEP]. Gating the strip on adapter.supports_generation would fix it but also changes encoder-decoder inputs.

Fixes #1870

Type of change

• New feature (non-breaking change which adds functionality)

Checklist:

• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes
• I have not rewritten tests relating to key interfaces which would affect backward compatibility

[jlarson4]

Looks good! My only request is that you trim down some of these comments. The adapter is 120 lines, but 40 of those lines are comments, making up 30% of the file.

[jlarson4]

Just one request (see above), I look forward to reviewing the rest of the commits!

[jlarson4]

Just one additional comment and then this should be ready to merge!

[jlarson4]

This subtracts once per adapter construction, so rebuilding from an existing bridge's cfg via build_bridge_from_module(tl_config=...) lands on 510 and W_pos doesn't drop the reserved rows. Deriving it from the loaded position table in prepare_model (ast.py sets n_ctx there) would make it more robust.