My old code that uses reference? and (WS | NEWLINE | LEADING_ASTERISK)* creates an explosion of possibilities.

I tried to guide the parser and separate the rule for reference and non-reference to explore one path at a time to eliminate some ambiguity and avoid a lot of backtracking. I don't know if this is good, as this becomes very verbose

Screenshot shows that this rule separation improves performance (compare it to #16458 (comment) and #16458 (comment))

@nrmancuso @rnveach @romani please share your opinion does this make sense?

I think it makes sense. It would be good to see how we look on memory usage while parsing some larger, nested html Javadocs. If we are hurting more for time than memory, I would suggest that we start extracting subrules to be reused where appropriate. This will help to make this grammar less buggy and ease maintenance.

We should optimize time, not a memory. With reasonable limits. Nowadays memory is cheeper.

Since these 2 are so similar, is there any benefit if we combine them so they aren't fully separate entries? I assume this won't really improve performance, so I am ok either way.

            | LINK_LITERAL (WS | NEWLINE | LEADING_ASTERISK)*
                ( (WS | NEWLINE | LEADING_ASTERISK)+ reference (WS | NEWLINE)* )?
                ( (WS | NEWLINE) description)?

At the least, maybe we should consider in the future making WS | NEWLINE | LEADING_ASTERISK and WS | NEWLINE its own group to somehow make this easier to read.

Since these 2 are so similar, is there any benefit if we combine them so they aren't fully separate entries?

I don't think so. I tried several options the one with 2 separate entries always leads to less parse time and IMO it is better in terms of readability.

At the least, maybe we should consider in the future making WS | NEWLINE | LEADING_ASTERISK and WS | NEWLINE its own group to somehow make this easier to read.

Yes, There should be a clean way to rewrite all this rules to improve the readability of this parse.