We have a few hundred audio classification models from transformers on the Hub and it would be nice if one could easily evaluate them for accuracy, F1 score etc with an evaluator.

The pipeline in question is audio-classification and the relevant docs are here.

A good model/dataset pair to test is this one for keyword spotting (ks):

• superb/wav2vec2-base-superb-ks: https://huggingface.co/superb/wav2vec2-base-superb-ks
• The ks config of superb: https://huggingface.co/datasets/superb

Together with #324, this evaluator would bring coverage to all audio transformers models on the Hub 🔥