when calculating the slot metric, the parameter "labels_ignore" of the function
def eval_preds(pred_intents=None, lab_intents=None, pred_slots=None, lab_slots=None,
eval_metrics='all', labels_ignore='Other', labels_merge=None, pad='Other',
slot_level_combination=True)

is set as "Other". This result in the case, eg. label: what is the weather [datetime: today] prediction: what [datetime: is the weather today] treated as a correct prediction.
Whether this is by design or a mistake? If this is a mistake, could someone please update the compute_metrics code used for online evaluation and the baseline metric values in the competition webpage and the leaderboard?