In the datasets library, there is a handy get_dataset_config_names() function (docs) that returns all the available config names for a given dataset.

It would be useful to have a similar function in evaluate, since metrics like glue and bleurt have multiple config names, and programatic access to these options is handy.

However, a quick look at the evaluate codebase shows this isn't quite as easy to achieve as in datasets. The main limitation seems to be the absence of a builder config for metrics / evaluation modules. This is used in datasets here to quickly inspect the set of possible config names.

cc @TristanThrush this would allow us to avoid needing to import every metric explicitly in the evaluation service UI (when we check for supported metrics).