Addresses #1865 and #2293

What:

The proposal would add whitespace-delimited metadata to the ends of dialog lines, separated from the dialog line itself by carat ^:
this is a line of dialog^attribute1=value attribute2=value

Why:

To quote Jarbas in chat,

if i'm not missing anything we are discussing 3 features enabled by this metadata scheme

• gender value for grammatical correctness
• atitude value, weights for this would be global (i really wanna say increase sarcasm by 20%)
• individual line weights (after atitude selection)

How:

MustacheDialogRenderer's existing logic would be extended to parse the file for lines matching the desired attributes. I've looked at a few different ways of doing this, and I think recursive, broader searches, dropping the "lowest value" attribute each time, are sadly the easiest route.

In this case, that would likely mean that Mycroft would first look for a line matching the device's grammatical gender and its attitude, plucking random lines until it runs out. After that, it would look for a line matching the device's grammatical gender and default/no attitude. Failing that, it would revert to the existing logic and pull random lines with no regard for metadata.

I worry about the computational consequences if, and only if, several such attributes are added and dialog files start cropping up with hundreds of lines. I stress tested the existing renderer when recent_phrases was added, and it's lightning for any reasonable values of max_recent_phrases. Still, this system does run the risk of parsing entire dialog files n + 1 times for n attributes, and there's further multiplicative complexity within the loop. Right now, we're talking about kilobytes, so who cares, but... yeah...

Regarding "individual line weights," that was in reference to easter eggs that get obnoxious if they fire too often. There'd be a few ways to do it, but I think the most straightforward would be to shove them in a dict that works like recent_phrases, phrases being the keys, and their values representing a countdown until the phrase gets "unlocked." Each time a line is rendered, the values in that dict would decrement, and any keys with the value 0 would be removed from the dict, allowing the parser to choose them for rendering again.

In that case, 'weight' might be the wrong term. The attribute would instead reflect a "cooldown" before the phrase is spoken again. Also, I should point out that, because the dict I describe would be per dialog renderer, the cooldown would not guarantee that any other phrases from the same .dialog file are spoken in between instances of the weighted line. It would only guarantee a minimal number of interactions with Mycroft before the line is spoken a second time.

It also lives in memory, so it remains possible that a user hears your easter egg on their last skill invocation before shutdown, and again on their first invocation after startup.

I propose, and will likely prototype during the break, adding the metadata immediately, and simultaneously adding attitude weights to mycroft.conf, along with an intent for tweaking those values verbally. Increase sarcasm by 20%, indeed! That's at least three of us now who are tickled by that reference 👨‍🚀

Lingua Franca has a part to play in handling grammatical gender, so I think that should probably be added to core at the time of the switch. That way, Lingua Franca can report whether the configured language is gendered, what (if any) default gender it uses, and any other information which might be relevant to the problem.