Feature Mixing randomly selects a subset of N feature dimensions from each modality and swaps them to obtain new features, which are then concatenated to form the multimodal outlier features.
featuremixing_example_1D_2D.py
Follow the instructions in MultiOOD to prepare the dataset and setup the environments.
cd OOD_detection/HMDB-rgb-flow/
Train the Far-OOD model using A2D and Feature Mixing for HMDB:
python train_video_flow.py --dataset 'HMDB' --lr 0.0001 --seed 0 --bsz 16 --num_workers 10 --use_single_pred --use_a2d --a2d_max_hellinger --a2d_ratio 0.1 --use_featuremixing --max_ood_hellinger --a2d_ratio_ood 1.0 --ood_entropy_ratio 1.0 --nepochs 50 --appen '' --save_best --save_checkpoint --datapath '/path/to/HMDB51/'
Save the evaluation files for HMDB:
python test_video_flow.py --bsz 16 --num_workers 2 --dataset 'HMDB' --appen 'a2d_fm_best_' --resumef '/path/to/model_best.pt'
Save the evaluation files for UCF:
python test_video_flow.py --bsz 16 --num_workers 2 --far_ood --dataset 'HMDB' --ood_dataset 'UCF' --appen 'a2d_fm_best_' --resumef '/path/to/model_best.pt'
Save the evaluation files for HAC:
python test_video_flow.py --bsz 16 --num_workers 2 --far_ood --dataset 'HMDB' --ood_dataset 'HAC' --appen 'a2d_fm_best_' --resumef '/path/to/model_best.pt'
Save the evaluation files for Kinetics:
python test_video_flow.py --bsz 16 --num_workers 2 --far_ood --dataset 'HMDB' --ood_dataset 'Kinetics' --appen 'a2d_fm_best_' --resumef '/path/to/model_best.pt'
Save the evaluation files for EPIC:
cd OOD_detection/EPIC-rgb-flow/
python test_video_flow_epic.py --bsz 16 --num_workers 2 --far_ood --dataset 'HMDB' --ood_dataset 'EPIC' --appen 'a2d_fm_best_' --resumef '/path/to/model_best.pt'
Evaluation for UCF (change --ood_dataset to UCF, EPIC, HAC, or Kinetics):
python eval_video_flow_far_ood.py --postprocessor ebo --appen 'a2d_fm_best_' --dataset 'HMDB' --ood_dataset 'UCF' --path 'HMDB-rgb-flow/'
The code and dataset for OOD Segmentation will be available soon.
If you find our work useful in your research please consider citing our paper:
@article{liu2025fm,
title={Extremely Simple Multimodal Outlier Synthesis for Out-of-Distribution Detection and Segmentation},
author={Liu, Moru and Dong, Hao and Kelly, Jessica and Fink, Olga and Trapp, Mario},
journal={arXiv preprint arXiv:2505.16985},
year={2025}
}
MultiOOD: Scaling Out-of-Distribution Detection for Multiple Modalities
DPU: Dynamic Prototype Updating for Multimodal Out-of-Distribution Detection
NNG-Mix: Improving Semi-supervised Anomaly Detection with Pseudo-anomaly Generation
Survey: Advances in Multimodal Adaptation and Generalization: From Traditional Approaches to Foundation Models
Many thanks to the open-source projects SimMMDG and MultiOOD.