Currently, there is a mix between configs that are used when the metric is instantiated and configs that are later passed to compute. This makes life harder in several ways as pointed out in #137 and #138 and also makes the combine function from #150 harder to use. The two main points:

• It is hard to know in advance what configs are available (e.g. useful for eval on the hub)
• It is hard to configure metrics in advance that are used in a wrapped object. One always needs to pass the kwargs downstream (affects evaluator and combine).

I think we could solve this by moving all kwargs to the load. If a user wants to run a metric with different configs they can just load several instances of the metric which is cheap. In addition we could wrap the configs in something like a dataclass that specifies the types of configs and options when only a limited number of options are available (e.g. F1 can only be use binary or multilabel).

What do you think? @lewtun @lhoestq @sashavor