Skip to content

Preserve and validate redeclared token metadata - #825

Open
ydah wants to merge 1 commit into
masterfrom
fix/token-redeclaration-metadata
Open

ydah wants to merge 1 commit into
masterfrom
fix/token-redeclaration-metadata

Conversation

@ydah

@ydah ydah commented Oct 24, 2026

Copy link
Copy Markdown
Member

This change uses Bison 3.8.2 as the reference for token redeclaration behavior.

Declaring %left PLUS before %token <i> PLUS "+" previously discarded the alias and type. Preserve metadata across declarations, including repeated explicit token numbers.

Following Bison's behavior, reject repeated type declarations—even with the same type—and changes to an assigned token number. Warn about repeated token declarations and aliases. Keep the first alias and treat additional strings as anonymous tokens that can be named later.

context = Lrama::Context.new(states)

plus, alias_plus, minus, star = grammar.rules.last.rhs
expect(alias_plus).to be(plus)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From the viewpoint of human readability, I prefer the order to assertions follow the order of plus, alias_plus, minus, star so that

expect(plus.alias_name).to eq('"+"')
...
expect(alias_plus).to be(plus)
...
expect(minus).not_to be(plus)
...

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants