Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add New Data Augmentation Technique: OA-Mix #11916

Open
wants to merge 7 commits into
base: main
Choose a base branch
from

Conversation

dazory
Copy link

@dazory dazory commented Aug 21, 2024

oamix_examples

Motivation

This pull request introduces OA-Mix, a key component of the OA-DG (Object-Aware Domain Generalization) framework, designed to improve domain generalization in object detection. The goal is to enhance the model's robustness against domain shifts by generating diverse multi-domain data without compromising object annotations.

OA-Mix was first introduced in the paper titled "Object-Aware Domain Generalization for Object Detection," which was presented as an oral session at AAAI 2024. The method has shown state-of-the-art performance on domain generalization benchmarks for object detection.

Modification

This PR implements the following:

  • Implemented the OA-Mix class with configurable parameters.
  • Integration of OA-Mix into the existing data pipeline for seamless use during model training.
  • Added test functions to validate the effectiveness and stability of the OA-Mix method.
  • Updated the level_to_mag() function to prevent the level from being divided into only two steps when min_mag=0.0 and max_mag=0.1 in translation transformations by changing round(*, 1) to round(*, 2).

BC-breaking (Optional)

The proposed modification is backward-compatible. OA-Mix is introduced as an optional augmentation technique, and the update to level_to_mag() only improves the precision of existing transformations, without breaking previous usage.

If you want to use OAMix, simply modify the train_pipeline in the configuration file as shown below:

train_pipeline = [
    dict(type='LoadImageFromFile', backend_args=None),
    dict(type='LoadAnnotations', with_bbox=True),
    dict(
        type='RandomResize',
        scale=[(2048, 800), (2048, 1024)],
        keep_ratio=True),
    dict(type='RandomFlip', prob=0.5),
    dict(type='OAMix')
    dict(type='PackDetInputs')
]

Use cases (Optional)

OA-Mix is particularly useful for:

  • Scenarios where models need to generalize to unseen domains, such as varying weather conditions or image corruptions in real-world environments.
  • Object detection tasks requiring high fidelity in localization and classification under domain shifts, especially when only a single-source domain is available for training.
  • Documentation and example usage have been updated to demonstrate how OA-Mix can be integrated into different object detection models.

Below is a performance comparison between a baseline object detection model and the same model with OA-Mix applied:

Model Dataset Backbone mAP mPC Gauss. Shot Impulse Defocus Glass Motion Zoom Snow Frost Fog Bright Contrast Elastic Pixel JPEG
Faster R-CNN Cityscapes-C R-50-FPN 42.2 15.4 0.5 1.1 1.1 17.2 16.5 18.3 2.1 2.2 12.3 29.8 32.0 24.1 40.1 18.7 15.1
Faster R-CNN + OA-Mix Cityscapes-C R-50-FPN 42.7 20.8 7.2 9.6 7.7 22.8 18.8 21.9 5.4 5.2 23.6 37.3 38.7 31.9 40.2 22.2 20.2

The model was evaluated using the robust detection benchmark, which can be run using the test_robustness.py script provided by mmdetection.

Checklist

  • Pre-commit tools have been run, and all issues have been resolved.
  • Comprehensive unit tests have been added to ensure the correctness and reproducibility of the OA-Mix implementation.
  • The modification has been tested with MMDet, confirming compatibility and improved performance.
  • Documentation has been updated accordingly, including docstrings and example tutorials demonstrating how to use OA-Mix.

@CLAassistant
Copy link

CLAassistant commented Aug 21, 2024

CLA assistant check
All committers have signed the CLA.

@dazory
Copy link
Author

dazory commented Aug 23, 2024

Hi @jbwang1997, I just wanted to kindly remind you to review this pull request when you get a chance. Your feedback would be greatly appreciated. 😊 Thanks in advance!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants