[CVPR 2025] FALCON: Fairness Learning via Contrastive Attention Approach to Continual Semantic Scene Understanding

FALCON: Fairness Learning via Contrastive Attention Approach to Continual Semantic Scene Understanding
Thanh-Dat Truong, Utsav Prabhu, Bhiksha Raj, Jackson Cothren and Khoa Luu
University of Arkansas, Computer Vision and Image Understanding Lab, CVIU

Abstract

Continual Learning in semantic scene segmentation aims to continually learn new unseen classes in dynamic environments while maintaining previously learned knowledge. Prior studies focused on modeling the catastrophic forgetting and background shift challenges in continual learning. However, fairness, another major challenge that causes unfair predictions leading to low performance among major and minor classes, still needs to be well addressed. In addition, prior methods have yet to model the unknown classes well, thus resulting in producing non-discriminative features among unknown classes. This paper presents a novel Fairness Learning via Contrastive Attention Approach to continual learning in semantic scene understanding. In particular, we first introduce a new Fairness Contrastive Clustering loss to address the problems of catastrophic forgetting and fairness. Then, we propose an attention-based visual grammar approach to effectively model the background shift problem and unknown classes, producing better feature representations for different unknown classes. Through our experiments, our proposed approach achieves State-of-the-Art (SOTA) performance on different continual learning benchmarks, i.e., ADE20K, Cityscapes, and Pascal VOC. It promotes the fairness of the continual semantic segmentation model.

Dataset

Two scripts are available to download ADE20K and Pascal-VOC 2012, please see in the data folder.

Testing

To run testing, use the scripts in the scripts folder. For example, to test the performance of FALCON on the 100-50 setting of ADE20K, do

bash scripts/ade/FALCON_ade_100-50_segformer.sh

Acknowledgements

This codebase is borrowed from PLOP, RCIL, and FairCL

Citation

If you find this code useful for your research, please consider citing:

@inproceedings{truong2023falcon,
    title={FALCON: Fairness Learning via Contrastive Attention Approach to Continual Semantic Scene Understanding},
    author={Truong, Thanh-Dat and Prabhu, Utsav and Raj, Bhiksha and Cothren, Jackson and Luu, Khoa},
    booktitle={The IEEE/CVF Conference on Computer Vision and Pattern Recognition},
    year={2025}
}