-
Notifications
You must be signed in to change notification settings - Fork 141
Open
Labels
A-tokenArea: tokenization, including definition of identifiers and wordsArea: tokenization, including definition of identifiers and wordsC-enhancementCategory: Raise on the bar on expectationsCategory: Raise on the bar on expectationsS-waiting-on-decisionStatus: Waiting on a go/no-go before implementingStatus: Waiting on a go/no-go before implementing
Description
#410 seems to fix only 2nd__
or __3rd
at the identifier tokenizer level.
Variable names such as level_2nd
are split as ["level", "2", "nd"]
by the SplitIdent
.
I think I can fix it by marking words whether they might be ordinal suffixes or not and then using a different dictionary according to that.
szepeviktor, yhay81, yu-iskw, Drew-Daniels and mmagician
Metadata
Metadata
Assignees
Labels
A-tokenArea: tokenization, including definition of identifiers and wordsArea: tokenization, including definition of identifiers and wordsC-enhancementCategory: Raise on the bar on expectationsCategory: Raise on the bar on expectationsS-waiting-on-decisionStatus: Waiting on a go/no-go before implementingStatus: Waiting on a go/no-go before implementing