For Haskell/Java, I implemented in 83681fa a translation from block comment terminator strings to regular expressions that works correctly for C-style comments /* ... */ and HTML-style comments . (The previous solution 799ab5f used a translation that sat there in BNFC.Lexing that could not deal with block comment correctly.)

However, it fails when the terminator string has non-trivial repetitions, e.g., ananas. Here the generated regular expression skips over the terminator e.g. in case of anananas.

Prg. Program ::= [Integer];
terminator Integer "";

comment "banana" "ananas";

Input:

0
banana anananas
1
banana ananas
2

Output:

[Abstract Syntax]
Prg [0,2]

[Linearized tree]
0 2

The correct lexing of anananas would, after scanning anana and seeing nas, jump to have seen anan and looking at as.

This is a well-known problem in recognizing a search word of length m in a text of length n in O(n+m) time, and solutions are implemented in the lexer generators we target. Thus, maybe we should not try to reinvent the wheel.
It seems that going via regular expressions for this problem is anyway expensive, using lexer states or start conditions (such as used for lexing comments with flex and JLex have) might be better also when targeting Alex.

Support block comment delimiters > 2 characters in Haskell backend #202

Description

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions