diff --git a/docs/zh_cn/advanced_guides/add_datasets.md b/docs/zh_cn/advanced_guides/add_datasets.md
index 4ea14934ed2..22fbf3462fc 100644
--- a/docs/zh_cn/advanced_guides/add_datasets.md
+++ b/docs/zh_cn/advanced_guides/add_datasets.md
@@ -1,4 +1,62 @@
-# 新增自定义数据集（待更新）
+# 新增自定义数据集
+
+## 新增自定义数据集
+
+在这里，我们展示如何构建一个新的数据集。
+
+1. 创建一个新文件 `mmseg/datasets/example.py`
+
+ ```python
+ from mmseg.registry import DATASETS
+ from .basesegdataset import BaseSegDataset
+
+
+ @DATASETS.register_module()
+ class ExampleDataset(BaseSegDataset):
+
+ METAINFO = dict(
+ classes=('xxx', 'xxx', ...),
+ palette=[[x, x, x], [x, x, x], ...])
+
+ def __init__(self, aeg1, arg2):
+ pass
+ ```
+
+2. 在 `mmseg/datasets/__init__.py` 中导入模块
+
+ ```python
+ from .example import ExampleDataset
+ ```
+
+3. 通过创建一个新的数据集配置文件 `configs/_base_/datasets/example_dataset.py` 来使用它
+
+ ```python
+ dataset_type = 'ExampleDataset'
+ data_root = 'data/example/'
+ ...
+ ```
+
+4. 在 `mmseg/utils/class_names.py` 中补充数据集元信息
+
+ ```python
+ def example_classes():
+ return [
+ 'xxx', 'xxx',
+ ...
+ ]
+
+ def example_palette():
+ return [
+ [x, x, x], [x, x, x],
+ ...
+ ]
+ dataset_aliases ={
+ 'example': ['example', ...],
+ ...
+ }
+ ```
+
+**注意：** 如果新数据集不满足 mmseg 的要求，则需要在 `tools/dataset_converters/` 中准备一个数据集预处理脚本
## 通过重新组织数据来定制数据集
@@ -26,30 +84,17 @@
一个训练对将由 img_dir/ann_dir 里同样首缀的文件组成。
-如果给定 `split` 参数，只有部分在 img_dir/ann_dir 里的文件会被加载。
-我们可以对被包括在 split 文本里的文件指定前缀。
+有些数据集不会发布测试集或测试集的标注，如果没有测试集的标注，我们就无法在本地进行评估模型，因此我们在配置文件中将验证集设置为默认测试集。
-除此以外，一个 split 文本如下所示：
-
-```none
-xxx
-zzz
-```
+关于如何构建自己的数据集或实现新的数据集类，请参阅[数据集指南](./datasets.md)以获取更多详细信息。
-只有
-
-`data/my_dataset/img_dir/train/xxx{img_suffix}`,
-`data/my_dataset/img_dir/train/zzz{img_suffix}`,
-`data/my_dataset/ann_dir/train/xxx{seg_map_suffix}`,
-`data/my_dataset/ann_dir/train/zzz{seg_map_suffix}` 将被加载。
-
-注意：标注是跟图像同样的形状 (H, W)，其中的像素值的范围是 `[0, num_classes - 1]`。
+**注意：** 标注是跟图像同样的形状 (H, W)，其中的像素值的范围是 `[0, num_classes - 1]`。
您也可以使用 [pillow](https://pillow.readthedocs.io/en/stable/handbook/concepts.html#palette) 的 `'P'` 模式去创建包含颜色的标注。
## 通过混合数据去定制数据集
MMSegmentation 同样支持混合数据集去训练。
-当前它支持拼接 (concat), 重复 (repeat) 和多图混合 (multi-image mix)数据集。
+当前它支持拼接 (concat), 重复 (repeat) 和多图混合 (multi-image mix) 数据集。
### 重复数据集
@@ -58,79 +103,29 @@ MMSegmentation 同样支持混合数据集去训练。
```python
dataset_A_train = dict(
- type='RepeatDataset',
- times=N,
- dataset=dict( # 这是 Dataset_A 数据集的原始配置
- type='Dataset_A',
- ...
- pipeline=train_pipeline
- )
+ type='RepeatDataset',
+ times=N,
+ dataset=dict( # 这是 Dataset_A 数据集的原始配置
+ type='Dataset_A',
+ ...
+ pipeline=train_pipeline
)
+)
```
### 拼接数据集
-有2种方式去拼接数据集。
-
-1. 如果您想拼接的数据集是同样的类型，但有不同的标注文件，
- 您可以按如下操作去拼接数据集的配置文件：
-
- 1. 您也许可以拼接两个标注文件夹 `ann_dir`
-
- ```python
- dataset_A_train = dict(
- type='Dataset_A',
- img_dir = 'img_dir',
- ann_dir = ['anno_dir_1', 'anno_dir_2'],
- pipeline=train_pipeline
- )
- ```
-
- 2. 您也可以去拼接两个 `split` 文件列表
-
- ```python
- dataset_A_train = dict(
- type='Dataset_A',
- img_dir = 'img_dir',
- ann_dir = 'anno_dir',
- split = ['split_1.txt', 'split_2.txt'],
- pipeline=train_pipeline
- )
- ```
+如果要拼接不同的数据集，可以按如下方式连接数据集配置。
- 3. 您也可以同时拼接 `ann_dir` 文件夹和 `split` 文件列表
-
- ```python
- dataset_A_train = dict(
- type='Dataset_A',
- img_dir = 'img_dir',
- ann_dir = ['anno_dir_1', 'anno_dir_2'],
- split = ['split_1.txt', 'split_2.txt'],
- pipeline=train_pipeline
- )
- ```
-
- 在这样的情况下， `ann_dir_1` 和 `ann_dir_2` 分别对应于 `split_1.txt` 和 `split_2.txt`
-
-2. 如果您想拼接不同的数据集，您可以如下去拼接数据集的配置文件：
-
- ```python
- dataset_A_train = dict()
- dataset_B_train = dict()
-
- data = dict(
- imgs_per_gpu=2,
- workers_per_gpu=2,
- train = [
- dataset_A_train,
- dataset_B_train
- ],
- val = dataset_A_val,
- test = dataset_A_test
- )
- ```
+```python
+dataset_A_train = dict()
+dataset_B_train = dict()
+concatenate_dataset = dict(
+ type='ConcatDataset',
+ datasets=[dataset_A_train, dataset_B_train])
+```
-一个更复杂的例子如下：分别重复 `Dataset_A` 和 `Dataset_B` N 次和 M 次，然后再去拼接重复后的数据集
+下面是一个更复杂的示例，它分别重复 `Dataset_A` 和 `Dataset_B` N 次和 M 次，然后连接重复的数据集。
```python
dataset_A_train = dict(
@@ -159,41 +154,36 @@ dataset_B_train = dict(
pipeline=train_pipeline
)
)
-data = dict(
- imgs_per_gpu=2,
- workers_per_gpu=2,
- train = [
- dataset_A_train,
- dataset_B_train
- ],
- val = dataset_A_val,
- test = dataset_A_test
-)
+train_dataloader = dict(
+ dataset=dict(
+ type='ConcatDataset',
+ datasets=[dataset_A_train, dataset_B_train]))
+
+val_dataloader = dict(dataset=dataset_A_val)
+test_dataloader = dict(dataset=dataset_A_test)
```
+您可以参考 mmengine 的基础数据集[教程](https://mmengine.readthedocs.io/zh_CN/latest/advanced_tutorials/basedataset.html)以了解更多详细信息
+
### 多图混合集
-我们使用 `MultiImageMixDataset` 作为包装(wrapper)去混合多个数据集的图片。
-`MultiImageMixDataset`可以被类似mosaic和mixup的多图混合数据増广使用。
+我们使用 `MultiImageMixDataset` 作为包装（wrapper）去混合多个数据集的图片。
+`MultiImageMixDataset`可以被类似 mosaic 和 mixup 的多图混合数据増广使用。
-`MultiImageMixDataset`与`Mosaic`数据増广一起使用的例子：
+`MultiImageMixDataset` 与 `Mosaic` 数据増广一起使用的例子：
```python
train_pipeline = [
dict(type='RandomMosaic', prob=1),
dict(type='Resize', img_scale=(1024, 512), keep_ratio=True),
dict(type='RandomFlip', prob=0.5),
- dict(type='Normalize', **img_norm_cfg),
- dict(type='DefaultFormatBundle'),
- dict(type='Collect', keys=['img', 'gt_semantic_seg']),
+ dict(type='PackSegInputs')
]
train_dataset = dict(
type='MultiImageMixDataset',
dataset=dict(
- classes=classes,
- palette=palette,
type=dataset_type,
reduce_zero_label=False,
img_dir=data_root + "images/train",
diff --git a/docs/zh_cn/advanced_guides/add_metrics.md b/docs/zh_cn/advanced_guides/add_metrics.md
index 3a371e357e8..0637b447284 100644
--- a/docs/zh_cn/advanced_guides/add_metrics.md
+++ b/docs/zh_cn/advanced_guides/add_metrics.md
@@ -1 +1,81 @@
-# 新增评测指标 (待更新)
+# 新增评测指标
+
+## 使用 MMSegmentation 的源代码进行开发
+
+在这里，我们用 `CustomMetric` 作为例子来展示如何开发一个新的评测指标。
+
+1. 创建一个新文件 `mmseg/evaluation/metrics/custom_metric.py`。
+
+ ```python
+ from typing import List, Sequence
+
+ from mmengine.evaluator import BaseMetric
+
+ from mmseg.registry import METRICS
+
+
+ @METRICS.register_module()
+ class CustomMetric(BaseMetric):
+
+ def __init__(self, arg1, arg2):
+ """
+ The metric first processes each batch of data_samples and predictions,
+ and appends the processed results to the results list. Then it
+ collects all results together from all ranks if distributed training
+ is used. Finally, it computes the metrics of the entire dataset.
+ """
+
+ def process(self, data_batch: dict, data_samples: Sequence[dict]) -> None:
+ pass
+
+ def compute_metrics(self, results: list) -> dict:
+ pass
+
+ def evaluate(self, size: int) -> dict:
+ pass
+ ```
+
+ 在上面的示例中，`CustomMetric` 是 `BaseMetric` 的子类。它有三个方法：`process`，`compute_metrics` 和 `evaluate`。
+
+ - `process()` 处理一批数据样本和预测。处理后的结果需要显示地传给 `self.results` ，将在处理所有数据样本后用于计算指标。更多细节请参考 [MMEngine 文档](https://github.com/open-mmlab/mmengine/blob/main/docs/zh_cn/design/evaluation.md)
+
+ - `compute_metrics()` 用于从处理后的结果中计算指标。
+
+ - `evaluate()` 是一个接口，用于计算指标并返回结果。它将由 `ValLoop` 或 `TestLoop` 在 `Runner` 中调用。在大多数情况下，您不需要重写此方法，但如果您想做一些额外的工作，可以重写它。
+
+ **注意：** 您可以在[这里](https://github.com/open-mmlab/mmengine/blob/main/mmengine/runner/loops.py#L366) 找到 `Runner` 调用 `evaluate()` 方法的过程。`Runner` 是训练和测试过程的执行器，您可以在[训练引擎文档](./engine.md)中找到有关它的详细信息。
+
+2. 在 `mmseg/evaluation/metrics/__init__.py` 中导入新的指标。
+
+ ```python
+ from .custom_metric import CustomMetric
+ __all__ = ['CustomMetric', ...]
+ ```
+
+3. 在配置文件中设置新的评测指标
+
+ ```python
+ val_evaluator = dict(type='CustomMetric', arg1=xxx, arg2=xxx)
+ test_evaluator = dict(type='CustomMetric', arg1=xxx, arg2=xxx)
+ ```
+
+## 使用发布版本的 MMSegmentation 进行开发
+
+上面的示例展示了如何使用 MMSegmentation 的源代码开发新指标。如果您想使用 MMSegmentation 的发布版本开发新指标，可以按照以下步骤操作。
+
+1. 创建一个新文件 `/Path/to/metrics/custom_metric.py`，实现 `process`，`compute_metrics` 和 `evaluate` 方法，`evaluate` 方法是可选的。
+
+2. 在代码或配置文件中导入新的指标。
+
+ ```python
+ from path.to.metrics import CustomMetric
+ ```
+
+ 或者
+
+ ```python
+ custom_imports = dict(imports=['/Path/to/metrics'], allow_failed_imports=False)
+
+ val_evaluator = dict(type='CustomMetric', arg1=xxx, arg2=xxx)
+ test_evaluator = dict(type='CustomMetric', arg1=xxx, arg2=xxx)
+ ```
diff --git a/docs/zh_cn/advanced_guides/add_models.md b/docs/zh_cn/advanced_guides/add_models.md
index 2f0a5af0d18..e05c07c8bac 100644
--- a/docs/zh_cn/advanced_guides/add_models.md
+++ b/docs/zh_cn/advanced_guides/add_models.md
@@ -166,7 +166,7 @@ loss_decode=dict(type='MyLoss', loss_weight=1.0))
### 添加新的数据预处理器（data preprocessor）
-在 MMSegmentation 1.x 版本中，我们使用 [SegDataPreProcessor](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/mmseg/models/data_preprocessor.py#L13) 将数据复制到目标设备，并将数据预处理为默认的模型输入格式。这里我们将展示如何开发一个新的数据预处理器。
+在 MMSegmentation 1.x 版本中，我们使用 [SegDataPreProcessor](https://github.com/open-mmlab/mmsegmentation/blob/main/mmseg/models/data_preprocessor.py#L13) 将数据复制到目标设备，并将数据预处理为默认的模型输入格式。这里我们将展示如何开发一个新的数据预处理器。
1. 创建一个新文件 `mmseg/models/my_datapreprocessor.py`。
diff --git a/docs/zh_cn/advanced_guides/contribute_dataset.md b/docs/zh_cn/advanced_guides/contribute_dataset.md
new file mode 100644
index 00000000000..4222de32a6c
--- /dev/null
+++ b/docs/zh_cn/advanced_guides/contribute_dataset.md
@@ -0,0 +1,461 @@
+# 在 mmsegmentation projects 中贡献一个标准格式的数据集
+
+- 在开始您的贡献流程前，请先阅读[《OpenMMLab 贡献代码指南》](https://mmcv.readthedocs.io/zh_CN/latest/community/contributing.html)，以详细的了解 OpenMMLab 代码库的代码贡献流程。
+- 该教程以 [Gaofen Image Dataset (GID)](https://www.sciencedirect.com/science/article/pii/S0034425719303414) 高分 2 号卫星所拍摄的遥感图像语义分割数据集作为样例，来演示在 mmsegmentation 中的数据集贡献流程。
+
+## 步骤 1： 配置 mmsegmentation 开发所需必要环境
+
+- 开发所必需的环境安装请参考[中文快速入门指南](https://github.com/open-mmlab/mmsegmentation/blob/main/docs/zh_cn/get_started.md)或[英文 get_started](https://github.com/open-mmlab/mmsegmentation/blob/main/docs/en/get_started.md)。
+
+- 如果您已安装了最新版的 pytorch、mmcv、mmengine，那么您可以跳过步骤 1 至[步骤 2](<#[步骤-2](#%E6%AD%A5%E9%AA%A4-2%E4%BB%A3%E7%A0%81%E8%B4%A1%E7%8C%AE%E5%89%8D%E7%9A%84%E5%87%86%E5%A4%87%E5%B7%A5%E4%BD%9C)>)。
+
+- **注：** 在此处无需安装 mmsegmentation，只需安装开发 mmsegmentation 所必需的 pytorch、mmcv、mmengine 等即可。
+
+**新建虚拟环境（如已有合适的开发环境，可跳过）**
+
+- 从[官方网站](https://docs.conda.io/en/latest/miniconda.html)下载并安装 Miniconda
+- 创建一个 conda 环境，并激活
+
+```shell
+conda create --name openmmlab python=3.8 -y
+conda activate openmmlab
+```
+
+**安装 pytorch （如环境下已安装 pytorch，可跳过）**
+
+- 参考 [official instructions](https://pytorch.org/get-started/locally/) 安装 **PyTorch**
+
+**使用 mim 安装 mmcv、mmengine**
+
+- 使用 [MIM](https://github.com/open-mmlab/mim) 安装 [MMCV](https://github.com/open-mmlab/mmcv)
+
+```shell
+pip install -U openmim
+mim install mmengine
+mim install "mmcv>=2.0.0"
+```
+
+## 步骤 2：代码贡献前的准备工作
+
+### 2.1 Fork mmsegmentation 仓库
+
+- 通过浏览器打开[mmsegmentation 官方仓库](https://github.com/open-mmlab/mmsegmentation/tree/main)。
+- 登录您的 GitHub 账户，以下步骤均需在 GitHub 登录的情况下进行。
+- Fork mmsegmentation 仓库
+ 
+- Fork 之后，mmsegmentation 仓库将会出现在您的个人仓库中。
+
+### 2.2 在您的代码编写软件中 git clone mmsegmentation
+
+这里以 VSCODE 为例
+
+- 打开 VSCODE，新建终端窗口并激活您在[步骤 1 ](#%E6%AD%A5%E9%AA%A4-1-%E9%85%8D%E7%BD%AE-mmsegmentation-%E5%BC%80%E5%8F%91%E6%89%80%E9%9C%80%E5%BF%85%E8%A6%81%E7%8E%AF%E5%A2%83)中所安装的虚拟环境。
+- 在您 GitHub 的个人仓库中找到您 Fork 的 mmsegmentation 仓库，复制其链接。
+ 
+- 在终端中执行命令
+ ```bash
+ git clone {您所复制的个人仓库的链接}
+ ```
+ 
+ **注：** 如提示以下信息，请在 GitHub 中添加 [SSH 秘钥](https://docs.github.com/en/authentication/connecting-to-github-with-ssh/generating-a-new-ssh-key-and-adding-it-to-the-ssh-agent)
+ 
+- 进入 mmsegmentation 目录（之后的操作均在 mmsegmentation 目录下）。
+ ```bash
+ cd mmsegmentation
+ ```
+- 在终端中执行以下命令，添加官方仓库为上游仓库。
+ ```bash
+ git remote add upstream git@github.com:open-mmlab/mmsegmentation.git
+ ```
+- 使用以下命令检查 remote 是否添加成功。
+ ```bash
+ git remote -v
+ ```
+ 
+
+### 2.3 切换目录至 mmsegmentation 并从源码安装mmsegmentation
+
+在`mmsegmentation`目录下执行`pip install -v -e .`，通过源码构建方式安装 mmsegmentaion 库。
+安装完成后，您将能看到如下图所示的文件树。
+
+
+### 2.4 切换分支为 dev-1.x
+
+正如您在[ mmsegmentation 官网](https://github.com/open-mmlab/mmsegmentation/tree/main)所见，该仓库有许多分支，默认分支`main`为稳定的发行版本，以及用于贡献者进行开发的`dev-1.x`分支。`dev-1.x`分支是贡献者们用来提交创意和 PR 的分支，`dev-1.x`分支的内容会被周期性的合入到`main`分支。
+
+
+回到 VSCODE 中，在终端执行命令
+
+```bash
+git checkout dev-1.x
+```
+
+### 2.5 创新属于自己的新分支
+
+在基于`dev-1.x`分支下，使用如下命令，创建属于您自己的分支。
+
+```bash
+# git checkout -b 您的GitHubID/您的分支想要实现的功能的名字
+# git checkout -b AI-Tianlong/support_GID_dataset
+git checkout -b {您的GitHubID/您的分支想要实现的功能的名字}
+```
+
+### 2.6 配置 pre-commit
+
+OpenMMLab 仓库对代码质量有着较高的要求，所有提交的 PR 必须要通过代码格式检查。pre-commit 详细配置参阅[配置 pre-commit](https://mmcv.readthedocs.io/zh_CN/latest/community/contributing.html#pre-commit)。
+
+## 步骤 3：在`mmsegmentation/projects`下贡献您的代码
+
+**先对 GID 数据集进行分析**
+
+这里以贡献高分 2 号遥感图像语义分割数据集 GID 为例，GID 数据集是由我国自主研发的高分 2 号卫星所拍摄的光学遥感图像所创建，经图像预处理后共提供了 150 张 6800x7200 像素的 RGB 三通道遥感图像。并提供了两种不同类别数的数据标注，一种是包含 5 类有效物体的 RGB 标签，另一种是包含 15 类有效物体的 RGB 标签。本教程将针对 5 类标签进行数据集贡献流程讲解。
+
+GID 的 5 类有效标签分别为：0-背景-\[0,0,0\](mask 标签值-标签名称-RGB 标签值)、1-建筑-\[255,0,0\]、2-农田-\[0,255,0\]、3-森林-\[0,0,255\]、4-草地-\[255,255,0\]、5-水-\[0,0,255\]。在语义分割任务中，标签是与原图尺寸一致的单通道图像，标签图像中的像素值为真实样本图像中对应像素所包含的物体的类别。GID 数据集提供的是具有 RGB 三通道的彩色标签，为了模型的训练需要将 RGB 标签转换为 mask 标签。并且由于图像尺寸为 6800x7200 像素，对于神经网络的训练来有些过大，所以将每张图像裁切成了没有重叠的 512x512 的图像以便进行训练。
+
+
+### 3.1 在`mmsegmentation/projects`下创建新的项目文件夹
+
+在`mmsegmentation/projects`下创建文件夹`gid_dataset`
+
+
+### 3.2 贡献您的数据集代码
+
+为了最终能将您在 projects 中贡献的代码更加顺畅的移入核心库中（对代码要求质量更高），非常建议按照核心库的目录来编辑您的数据集文件。
+关于数据集有 4 个必要的文件：
+
+- **1** `mmseg/datasets/gid.py` 定义了数据集的尾缀、CLASSES、PALETTE、reduce_zero_label等
+- **2** `configs/_base_/gid.py` GID 数据集的配置文件，定义了数据集的`dataset_type`（数据集类型，`mmseg/datasets/gid.py`中注册的数据集的类名）、`data_root`(数据集所在的根目录，建议将数据集通过软连接的方式将数据集放至`mmsegmentation/data`)、`train_pipline`(训练的数据流)、`test_pipline`(测试和验证时的数据流)、`img_rations`(多尺度预测时的多尺度配置)、`tta_pipeline`（多尺度预测）、`train_dataloader`(训练集的数据加载器)、`val_dataloader`(验证集的数据加载器)、`test_dataloader`(测试集的数据加载器)、`val_evaluator`(验证集的评估器)、`test_evaluator`(测试集的评估器)。
+- **3** 使用了 GID 数据集的模型训练配置文件
+ 这个是可选的，但是强烈建议您添加。在核心库中，所贡献的数据集需要和参考文献中所提出的结果精度对齐，为了后期将您贡献的代码合并入核心库。如您的算力充足，最好能提供对应的模型配置文件在您贡献的数据集上所验证的结果以及相应的权重文件，并撰写较为详细的README.md文档。[示例参考结果](https://github.com/open-mmlab/mmsegmentation/tree/main/configs/deeplabv3plus#mapillary-vistas-v12)
+ 
+- **4** 使用如下命令格式： 撰写`docs/zh_cn/user_guides/2_dataset_prepare.md`来添加您的数据集介绍，包括但不限于数据集的下载方式，数据集目录结构、数据集生成等一些必要性的文字性描述和运行命令。以更好地帮助用户能更快的实现数据集的准备工作。
+
+### 3.3 贡献`tools/dataset_converters/gid.py`
+
+由于 GID 数据集是由未经过切分的 6800x7200 图像所构成的数据集，并且没有划分训练集、验证集与测试集。以及其标签为 RGB 彩色标签，需要将标签转换为单通道的 mask label。为了方便训练，首先将 GID 数据集进行裁切和标签转换，并进行数据集划分，构建为 mmsegmentation 所支持的格式。
+
+```python
+# tools/dataset_converters/gid.py
+import argparse
+import glob
+import math
+import os
+import os.path as osp
+from PIL import Image
+
+import mmcv
+import numpy as np
+from mmengine.utils import ProgressBar, mkdir_or_exist
+
+def parse_args():
+ parser = argparse.ArgumentParser(
+ description='Convert GID dataset to mmsegmentation format')
+ parser.add_argument('dataset_img_path', help='GID images folder path')
+ parser.add_argument('dataset_label_path', help='GID labels folder path')
+ parser.add_argument('--tmp_dir', help='path of the temporary directory')
+ parser.add_argument('-o', '--out_dir', help='output path', default='data/gid')
+ parser.add_argument(
+ '--clip_size',
+ type=int,
+ help='clipped size of image after preparation',
+ default=256)
+ parser.add_argument(
+ '--stride_size',
+ type=int,
+ help='stride of clipping original images',
+ default=256)
+ args = parser.parse_args()
+ return args
+
+GID_COLORMAP = dict(
+ Background=(0, 0, 0), #0-背景-黑色
+ Building=(255, 0, 0), #1-建筑-红色
+ Farmland=(0, 255, 0), #2-农田-绿色
+ Forest=(0, 0, 255), #3-森林-蓝色
+ Meadow=(255, 255, 0),#4-草地-黄色
+ Water=(0, 0, 255)#5-水-蓝色
+)
+palette = list(GID_COLORMAP.values())
+classes = list(GID_COLORMAP.keys())
+
+#############用列表来存一个 RGB 和一个类别的对应################
+def colormap2label(palette):
+ colormap2label_list = np.zeros(256**3, dtype = np.longlong)
+ for i, colormap in enumerate(palette):
+ colormap2label_list[(colormap[0] * 256 + colormap[1])*256+colormap[2]] = i
+ return colormap2label_list
+
+#############给定那个列表，和vis_png然后生成masks_png################
+def label_indices(RGB_label, colormap2label_list):
+ RGB_label = RGB_label.astype('int32')
+ idx = (RGB_label[:, :, 0] * 256 + RGB_label[:, :, 1]) * 256 + RGB_label[:, :, 2]
+ # print(idx.shape)
+ return colormap2label_list[idx]
+
+def RGB2mask(RGB_label, colormap2label_list):
+ # RGB_label = np.array(Image.open(RGB_label).convert('RGB')) #打开RGB_png
+ mask_label = label_indices(RGB_label, colormap2label_list) # .numpy()
+ return mask_label
+
+colormap2label_list = colormap2label(palette)
+
+def clip_big_image(image_path, clip_save_dir, args, to_label=False):
+ """
+ Original image of GID dataset is very large, thus pre-processing
+ of them is adopted. Given fixed clip size and stride size to generate
+ clipped image, the intersection of width and height is determined.
+ For example, given one 6800 x 7200 original image, the clip size is
+ 256 and stride size is 256, thus it would generate 29 x 27 = 783 images
+ whose size are all 256 x 256.
+
+ """
+
+ image = mmcv.imread(image_path, channel_order='rgb')
+ # image = mmcv.bgr2gray(image)
+
+ h, w, c = image.shape
+ clip_size = args.clip_size
+ stride_size = args.stride_size
+
+ num_rows = math.ceil((h - clip_size) / stride_size) if math.ceil(
+ (h - clip_size) /
+ stride_size) * stride_size + clip_size >= h else math.ceil(
+ (h - clip_size) / stride_size) + 1
+ num_cols = math.ceil((w - clip_size) / stride_size) if math.ceil(
+ (w - clip_size) /
+ stride_size) * stride_size + clip_size >= w else math.ceil(
+ (w - clip_size) / stride_size) + 1
+
+ x, y = np.meshgrid(np.arange(num_cols + 1), np.arange(num_rows + 1))
+ xmin = x * clip_size
+ ymin = y * clip_size
+
+ xmin = xmin.ravel()
+ ymin = ymin.ravel()
+ xmin_offset = np.where(xmin + clip_size > w, w - xmin - clip_size,
+ np.zeros_like(xmin))
+ ymin_offset = np.where(ymin + clip_size > h, h - ymin - clip_size,
+ np.zeros_like(ymin))
+ boxes = np.stack([
+ xmin + xmin_offset, ymin + ymin_offset,
+ np.minimum(xmin + clip_size, w),
+ np.minimum(ymin + clip_size, h)
+ ], axis=1)
+
+ if to_label:
+ image = RGB2mask(image, colormap2label_list) #这里得改一下
+
+ for count, box in enumerate(boxes):
+ start_x, start_y, end_x, end_y = box
+ clipped_image = image[start_y:end_y,
+ start_x:end_x] if to_label else image[
+ start_y:end_y, start_x:end_x, :]
+ img_name = osp.basename(image_path).replace('.tif', '')
+ img_name = img_name.replace('_label', '')
+ if count % 3 == 0:
+ mmcv.imwrite(
+ clipped_image.astype(np.uint8),
+ osp.join(
+ clip_save_dir.replace('train', 'val'),
+ f'{img_name}_{start_x}_{start_y}_{end_x}_{end_y}.png'))
+ else:
+ mmcv.imwrite(
+ clipped_image.astype(np.uint8),
+ osp.join(
+ clip_save_dir,
+ f'{img_name}_{start_x}_{start_y}_{end_x}_{end_y}.png'))
+ count += 1
+
+def main():
+ args = parse_args()
+
+ """
+ According to this paper: https://ieeexplore.ieee.org/document/9343296/
+ select 15 images contained in GID, , which cover the whole six
+ categories, to generate train set and validation set.
+
+ According to Paper: https://ieeexplore.ieee.org/document/9343296/
+
+ """
+
+ if args.out_dir is None:
+ out_dir = osp.join('data', 'gid')
+ else:
+ out_dir = args.out_dir
+
+ print('Making directories...')
+ mkdir_or_exist(osp.join(out_dir, 'img_dir', 'train'))
+ mkdir_or_exist(osp.join(out_dir, 'img_dir', 'val'))
+ mkdir_or_exist(osp.join(out_dir, 'ann_dir', 'train'))
+ mkdir_or_exist(osp.join(out_dir, 'ann_dir', 'val'))
+
+ src_path_list = glob.glob(os.path.join(args.dataset_img_path, '*.tif'))
+ print(f'Find {len(src_path_list)} pictures')
+
+ prog_bar = ProgressBar(len(src_path_list))
+
+ dst_img_dir = osp.join(out_dir, 'img_dir', 'train')
+ dst_label_dir = osp.join(out_dir, 'ann_dir', 'train')
+
+ for i, img_path in enumerate(src_path_list):
+ label_path = osp.join(args.dataset_label_path, osp.basename(img_path.replace('.tif', '_label.tif')))
+
+ clip_big_image(img_path, dst_img_dir, args, to_label=False)
+ clip_big_image(label_path, dst_label_dir, args, to_label=True)
+ prog_bar.update()
+
+ print('Done!')
+
+if __name__ == '__main__':
+ main()
+```
+
+### 3.4 贡献`mmseg/datasets/gid.py`
+
+可参考[`projects/mapillary_dataset/mmseg/datasets/mapillary.py`](https://github.com/open-mmlab/mmsegmentation/blob/main/projects/mapillary_dataset/mmseg/datasets/mapillary.py)并在此基础上修改相应变量以适配您的数据集。
+
+```python
+# mmseg/datasets/gid.py
+# Copyright (c) OpenMMLab. All rights reserved.
+from mmseg.datasets.basesegdataset import BaseSegDataset
+from mmseg.registry import DATASETS
+
+# 注册数据集类
+@DATASETS.register_module()
+class GID_Dataset(BaseSegDataset):
+ """Gaofen Image Dataset (GID)
+
+ Dataset paper link:
+ https://www.sciencedirect.com/science/article/pii/S0034425719303414
+ https://x-ytong.github.io/project/GID.html
+
+ GID 6 classes: background(others), built-up, farmland, forest, meadow, water
+
+ In This example, select 10 images from GID dataset as training set,
+ and select 5 images as validation set.
+ The selected images are listed as follows:
+
+ GF2_PMS1__L1A0000647767-MSS1
+ GF2_PMS1__L1A0001064454-MSS1
+ GF2_PMS1__L1A0001348919-MSS1
+ GF2_PMS1__L1A0001680851-MSS1
+ GF2_PMS1__L1A0001680853-MSS1
+ GF2_PMS1__L1A0001680857-MSS1
+ GF2_PMS1__L1A0001757429-MSS1
+ GF2_PMS2__L1A0000607681-MSS2
+ GF2_PMS2__L1A0000635115-MSS2
+ GF2_PMS2__L1A0000658637-MSS2
+ GF2_PMS2__L1A0001206072-MSS2
+ GF2_PMS2__L1A0001471436-MSS2
+ GF2_PMS2__L1A0001642620-MSS2
+ GF2_PMS2__L1A0001787089-MSS2
+ GF2_PMS2__L1A0001838560-MSS2
+
+ The ``img_suffix`` is fixed to '.tif' and ``seg_map_suffix`` is
+ fixed to '.tif' for GID.
+ """
+ METAINFO = dict(
+ classes=('Others', 'Built-up', 'Farmland', 'Forest',
+ 'Meadow', 'Water'),
+
+ palette=[[0, 0, 0], [255, 0, 0], [0, 255, 0], [0, 255, 255],
+ [255, 255, 0], [0, 0, 255]])
+
+ def __init__(self,
+ img_suffix='.png',
+ seg_map_suffix='.png',
+ reduce_zero_label=None,
+ **kwargs) -> None:
+ super().__init__(
+ img_suffix=img_suffix,
+ seg_map_suffix=seg_map_suffix,
+ reduce_zero_label=reduce_zero_label,
+ **kwargs)
+```
+
+### 3.5 贡献使用 GID 的训练 config file
+
+```python
+_base_ = [
+ '../../../configs/_base_/models/deeplabv3plus_r50-d8.py',
+ './_base_/datasets/gid.py',
+ '../../../configs/_base_/default_runtime.py',
+ '../../../configs/_base_/schedules/schedule_240k.py'
+]
+custom_imports = dict(
+ imports=['projects.gid_dataset.mmseg.datasets.gid'])
+
+crop_size = (256, 256)
+data_preprocessor = dict(size=crop_size)
+model = dict(
+ data_preprocessor=data_preprocessor,
+ pretrained='open-mmlab://resnet101_v1c',
+ backbone=dict(depth=101),
+ decode_head=dict(num_classes=6),
+ auxiliary_head=dict(num_classes=6))
+
+```
+
+### 3.6 撰写`docs/zh_cn/user_guides/2_dataset_prepare.md`
+
+**Gaofen Image Dataset (GID)**
+
+- GID 数据集可在[此处](https://x-ytong.github.io/project/Five-Billion-Pixels.html)进行下载。
+- GID 数据集包含 150 张 6800x7200 的大尺寸图像，标签为 RGB 标签。
+- 此处选择 15 张图像生成训练集和验证集，该 15 张图像包含了所有六类信息。所选的图像名称如下：
+
+```None
+ GF2_PMS1__L1A0000647767-MSS1
+ GF2_PMS1__L1A0001064454-MSS1
+ GF2_PMS1__L1A0001348919-MSS1
+ GF2_PMS1__L1A0001680851-MSS1
+ GF2_PMS1__L1A0001680853-MSS1
+ GF2_PMS1__L1A0001680857-MSS1
+ GF2_PMS1__L1A0001757429-MSS1
+ GF2_PMS2__L1A0000607681-MSS2
+ GF2_PMS2__L1A0000635115-MSS2
+ GF2_PMS2__L1A0000658637-MSS2
+ GF2_PMS2__L1A0001206072-MSS2
+ GF2_PMS2__L1A0001471436-MSS2
+ GF2_PMS2__L1A0001642620-MSS2
+ GF2_PMS2__L1A0001787089-MSS2
+ GF2_PMS2__L1A0001838560-MSS2
+```
+
+执行以下命令进行裁切及标签的转换，需要修改为您所存储 15 张图像及标签的路径。
+
+```
+python projects/gid_dataset/tools/dataset_converters/gid.py [15 张图像的路径] [15 张标签的路径]
+```
+
+完成裁切后的 GID 数据结构如下：
+
+```none
+mmsegmentation
+├── mmseg
+├── tools
+├── configs
+├── data
+│ ├── gid
+│ │ ├── ann_dir
+| │ │ │ ├── train
+| │ │ │ ├── val
+│ │ ├── img_dir
+| │ │ │ ├── train
+| │ │ │ ├── val
+
+```
+
+### 3.7 贡献的代码及文档通过`pre-commit`检查
+
+使用命令
+
+```bash
+git add .
+git commit -m "添加描述"
+git push
+```
+
+### 3.8 在 GitHub 中向 mmsegmentation 提交 PR
+
+具体步骤可见[《OpenMMLab 贡献代码指南》](https://mmcv.readthedocs.io/zh_CN/latest/community/contributing.html)
diff --git a/docs/zh_cn/advanced_guides/data_flow.md b/docs/zh_cn/advanced_guides/data_flow.md
index 0716d36d1b4..20dbe07e75d 100644
--- a/docs/zh_cn/advanced_guides/data_flow.md
+++ b/docs/zh_cn/advanced_guides/data_flow.md
@@ -16,7 +16,7 @@ val_cfg = dict(type='ValLoop')
test_cfg = dict(type='TestLoop')
```
-在上图中，红色线表示 [train_step](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/docs/en/advanced_guides/models.md#train_step) ***（[中文链接待更新](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/docs/zh_cn/advanced_guides/models.md#train_step)）*** ，在每次训练迭代中，数据加载器（dataloader）从存储中加载图像并传输到数据预处理器（data preprocessor），数据预处理器会将图像放到特定的设备上，并将数据堆叠到批处理中，之后模型接受批处理数据作为输入，最后将模型的输出发送给优化器（optimizer）。蓝色线表示 [val_step](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/docs/en/advanced_guides/models.md#val_step) 和 [test_step](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/docs/en/advanced_guides/models.md#test_step) ***（[中文链接待更新](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/docs/zh_cn/advanced_guides/models.md#test_step)）*** 。这两个过程的数据流除了模型输出与 `train_step` 不同外，其余均和 `train_step` 类似。由于在评估时模型参数会被冻结，因此模型的输出将被传递给 [Evaluator](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/docs/en/advanced_guides/evaluation.md#ioumetric) ***（[中文链接待更新](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/docs/zh_cn/advanced_guides/evaluation.md#ioumetric)）***
+在上图中，红色线表示 [train_step](./models.md#train_step)，在每次训练迭代中，数据加载器（dataloader）从存储中加载图像并传输到数据预处理器（data preprocessor），数据预处理器会将图像放到特定的设备上，并将数据堆叠到批处理中，之后模型接受批处理数据作为输入，最后将模型的输出发送给优化器（optimizer）。蓝色线表示 [val_step](./models.md#val_step) 和 [test_step](./models.md#test_step)。这两个过程的数据流除了模型输出与 `train_step` 不同外，其余均和 `train_step` 类似。由于在评估时模型参数会被冻结，因此模型的输出将被传递给 [Evaluator](./evaluation.md#ioumetric)。
来计算指标。
## MMSegmentation 中的数据流约定
@@ -28,7 +28,7 @@ test_cfg = dict(type='TestLoop')
数据加载器（DataLoader）是 MMEngine 的训练和测试流程中的一个重要组件。
从概念上讲，它源于 [PyTorch](https://pytorch.org/) 并保持一致。DataLoader 从文件系统加载数据，原始数据通过数据准备流程后被发送给数据预处理器。
-MMSegmentation 在 [PackSegInputs](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/mmseg/datasets/transforms/formatting.py#L12) 中定义了默认数据格式， 它是 `train_pipeline` 和 `test_pipeline` 的最后一个组件。有关数据转换 `pipeline` 的更多信息，请参阅[数据转换文档](https://mmsegmentation.readthedocs.io/en/dev-1.x/advanced_guides/transforms.html)。 ***（[中文链接待更新](https://mmsegmentation.readthedocs.io/zh_CN/dev-1.x/advanced_guides/transforms.html)）***
+MMSegmentation 在 [PackSegInputs](https://github.com/open-mmlab/mmsegmentation/blob/main/mmseg/datasets/transforms/formatting.py#L12) 中定义了默认数据格式， 它是 `train_pipeline` 和 `test_pipeline` 的最后一个组件。有关数据转换 `pipeline` 的更多信息，请参阅[数据转换文档](./transforms.md)。
在没有任何修改的情况下，PackSegInputs 的返回值通常是一个包含 `inputs` 和 `data_samples` 的 `dict`。以下伪代码展示了 mmseg 中数据加载器输出的数据类型，它是从数据集中获取的一批数据样本，数据加载器将它们打包成一个字典列表。`inputs` 是输入进模型的张量列表，`data_samples` 包含了输入图像的 meta information 和相应的 ground truth。
@@ -39,11 +39,11 @@ dict(
)
```
-**注意：** [SegDataSample](https://github.com/open-mmlab/mmsegmentation/blob/1.x/mmseg/structures/seg_data_sample.py) 是 MMSegmentation 的数据结构接口，用于连接不同组件。`SegDataSample` 实现了抽象数据元素 `mmengine.structures.BaseDataElement`，更多信息请在 [MMEngine](https://github.com/open-mmlab/mmengine) 中参阅 [SegDataSample 文档](https://mmsegmentation.readthedocs.io/zh_CN/1.x/advanced_guides/structures.html)和[数据元素文档](https://mmengine.readthedocs.io/zh_CN/latest/advanced_tutorials/data_element.html)。
+**注意：** [SegDataSample](https://github.com/open-mmlab/mmsegmentation/blob/1.x/mmseg/structures/seg_data_sample.py) 是 MMSegmentation 的数据结构接口，用于连接不同组件。`SegDataSample` 实现了抽象数据元素 `mmengine.structures.BaseDataElement`，更多信息请在 [MMEngine](https://github.com/open-mmlab/mmengine) 中参阅 [SegDataSample 文档](./structures.md)和[数据元素文档](https://mmengine.readthedocs.io/zh_CN/latest/advanced_tutorials/data_element.html)。
### 数据预处理器到模型
-虽然在[上面的图](##数据流概述)中分开绘制了数据预处理器和模型，但数据预处理器是模型的一部分，因此可以在[模型教程](https://mmsegmentation.readthedocs.io/en/dev-1.x/advanced_guides/models.html)中找到数据预处理器章节。 ***（[中文链接待更新](https://mmsegmentation.readthedocs.io/zh_CN/dev-1.x/advanced_guides/models.html)）***
+虽然在[上面的图](##数据流概述)中分开绘制了数据预处理器和模型，但数据预处理器是模型的一部分，因此可以在[模型教程](./models.md)中找到数据预处理器章节。
数据预处理器的返回值是一个包含 `inputs` 和 `data_samples` 的字典，其中 `inputs` 是批处理图像的 4D 张量，`data_samples` 中添加了一些用于数据预处理的额外元信息。当传递给网络时，字典将被解包为两个值。 以下伪代码展示了数据预处理器的返回值和模型的输入值。
@@ -61,22 +61,22 @@ class Network(BaseSegmentor):
pass
```
-**注意：** 模型的前向传播有 3 种模式，由输入参数 mode 控制，更多信息请参阅[模型教程](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/docs/en/advanced_guides/models.md)。 ***（[中文链接待更新](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/docs/zh_cn/advanced_guides/models.md)）***
+**注意：** 模型的前向传播有 3 种模式，由输入参数 mode 控制，更多信息请参阅[模型教程](./models.md)。
### 模型输出
-如[模型教程](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/docs/en/advanced_guides/models.md#forward) ***（[中文链接待更新](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/docs/zh_cn/advanced_guides/models.md#forward)）*** 所提到的 3 种前向传播具有 3 种输出。
+如[模型教程](./models.md#forward) ***（[中文链接待更新](./models.md#forward)）*** 所提到的 3 种前向传播具有 3 种输出。
`train_step` 和 `test_step`（或 `val_step`）分别对应于 `'loss'` 和 `'predict'`。
-在 `test_step` 或 `val_step` 中，推理结果会被传递给 `Evaluator` 。您可以参阅[评估文档](https://mmsegmentation.readthedocs.io/en/dev-1.x/advanced_guides/evaluation.html) ***（[中文链接待更新](https://mmsegmentation.readthedocs.io/zh_CN/dev-1.x/advanced_guides/evaluation.html)）*** 来获取更多关于 `Evaluator` 的信息。
+在 `test_step` 或 `val_step` 中，推理结果会被传递给 `Evaluator` 。您可以参阅[评估文档](./evaluation.md)来获取更多关于 `Evaluator` 的信息。
-在推理后，MMSegmentation 中的 [BaseSegmentor](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/mmseg/models/segmentors/base.py#L15) 会对推理结果进行简单的后处理以打包推理结果。神经网络生成的分割 logits，经过 `argmax` 操作后的分割 mask 和 ground truth（如果存在）将被打包到类似 `SegDataSample` 的实例。 [postprocess_result](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/mmseg/models/segmentors/base.py#L132) 的返回值是一个 **`SegDataSample`的`List`**。下图显示了这些 `SegDataSample` 实例的关键属性。
+在推理后，MMSegmentation 中的 [BaseSegmentor](https://github.com/open-mmlab/mmsegmentation/blob/main/mmseg/models/segmentors/base.py#L15) 会对推理结果进行简单的后处理以打包推理结果。神经网络生成的分割 logits，经过 `argmax` 操作后的分割 mask 和 ground truth（如果存在）将被打包到类似 `SegDataSample` 的实例。 [postprocess_result](https://github.com/open-mmlab/mmsegmentation/blob/main/mmseg/models/segmentors/base.py#L132) 的返回值是一个 **`SegDataSample`的`List`**。下图显示了这些 `SegDataSample` 实例的关键属性。
-与数据预处理器一致，损失函数也是模型的一部分，它是[解码头](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/mmseg/models/decode_heads/decode_head.py#L142)的属性之一。
+与数据预处理器一致，损失函数也是模型的一部分，它是[解码头](https://github.com/open-mmlab/mmsegmentation/blob/main/mmseg/models/decode_heads/decode_head.py#L142)的属性之一。
-在 MMSegmentation 中，`decode_head` 的 [loss_by_feat](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/mmseg/models/decode_heads/decode_head.py#L291) 方法是用于计算损失的统一接口。
+在 MMSegmentation 中，`decode_head` 的 [loss_by_feat](https://github.com/open-mmlab/mmsegmentation/blob/main/mmseg/models/decode_heads/decode_head.py#L291) 方法是用于计算损失的统一接口。
参数：
@@ -87,4 +87,4 @@ class Network(BaseSegmentor):
- dict\[str, Tensor\]：一个损失组件的字典
-**注意：** `train_step` 将损失传递进 OptimWrapper 以更新模型中的权重，更多信息请参阅 [train_step](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/docs/en/advanced_guides/models.md#train_step)。 ***（[中文链接待更新](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/docs/zh_cn/advanced_guides/models.md#train_step)）***
+**注意：** `train_step` 将损失传递进 OptimWrapper 以更新模型中的权重，更多信息请参阅 [train_step](./models.md#train_step)。
diff --git a/docs/zh_cn/advanced_guides/datasets.md b/docs/zh_cn/advanced_guides/datasets.md
index 546e97f70d6..b45f2d22bb6 100644
--- a/docs/zh_cn/advanced_guides/datasets.md
+++ b/docs/zh_cn/advanced_guides/datasets.md
@@ -1,10 +1,10 @@
# 数据集
-在 MMSegmentation 算法库中, 所有 Dataset 类的功能有两个: 加载[预处理](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/docs/en/user_guides/2_dataset_prepare.md) 之后的数据集的信息, 和将数据送入[数据集变换流水线](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/mmseg/datasets/basesegdataset.py#L141) 中, 进行[数据变换操作](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/docs/zh_cn/advanced_guides/transforms.md). 加载的数据集信息包括两类: 元信息 (meta information), 数据集本身的信息, 例如数据集总共的类别, 和它们对应调色盘信息: 数据信息 (data information) 是指每组数据中图片和对应标签的路径. 下文中介绍了 MMSegmentation 1.x 中数据集的常用接口, 和 mmseg 数据集基类中数据信息加载与修改数据集类别的逻辑, 以及数据集与数据变换流水线 (pipeline) 的关系.
+在 MMSegmentation 算法库中, 所有 Dataset 类的功能有两个: 加载[预处理](../user_guides/2_dataset_prepare.md) 之后的数据集的信息, 和将数据送入[数据集变换流水线](https://github.com/open-mmlab/mmsegmentation/blob/main/mmseg/datasets/basesegdataset.py#L141) 中, 进行[数据变换操作](./transforms.md). 加载的数据集信息包括两类: 元信息 (meta information), 数据集本身的信息, 例如数据集总共的类别, 和它们对应调色盘信息: 数据信息 (data information) 是指每组数据中图片和对应标签的路径. 下文中介绍了 MMSegmentation 1.x 中数据集的常用接口, 和 mmseg 数据集基类中数据信息加载与修改数据集类别的逻辑, 以及数据集与数据变换流水线 (pipeline) 的关系.
## 常用接口
-以 Cityscapes 为例, 介绍数据集常用接口. 如需运行以下示例, 请在当前工作目录下的 `data` 目录下载并[预处理](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/docs/en/user_guides/2_dataset_prepare.md#cityscapes) Cityscapes 数据集.
+以 Cityscapes 为例, 介绍数据集常用接口. 如需运行以下示例, 请在当前工作目录下的 `data` 目录下载并[预处理](../user_guides/2_dataset_prepare.md#cityscapes) Cityscapes 数据集.
实例化 Cityscapes 训练数据集:
@@ -96,7 +96,7 @@ print(dataset.metainfo)
'reduce_zero_label': False}
```
-数据集 `__getitem__` 方法的返回值, 是经过数据增强的样本数据的输出, 同样也是一个字典, 包括两个字段, `'inputs'` 字段是当前样本经过数据增强操作的图像, 类型为 torch.Tensor, `'data_samples'` 字段存放的数据类型是 MMSegmentation 1.x 新添加的数据结构 [`Segdatasample`](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/docs/zh_cn/advanced_guides/structures.md), 其中`gt_sem_seg` 字段是经过数据增强的标签数据.
+数据集 `__getitem__` 方法的返回值, 是经过数据增强的样本数据的输出, 同样也是一个字典, 包括两个字段, `'inputs'` 字段是当前样本经过数据增强操作的图像, 类型为 torch.Tensor, `'data_samples'` 字段存放的数据类型是 MMSegmentation 1.x 新添加的数据结构 [`Segdatasample`](./structures.md), 其中`gt_sem_seg` 字段是经过数据增强的标签数据.
```python
print(dataset[0])
@@ -166,13 +166,13 @@ print(dataset[0])
## BaseSegDataset
-由于 MMSegmentation 中的所有数据集的基本功能均包括(1) 加载[数据集预处理](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/docs/zh_cn/user_guides/2_dataset_prepare.md) 之后的数据信息和 (2) 将数据送入数据变换流水线中进行数据变换, 因此在 MMSegmentation 中将其中的共同接口抽象成 [`BaseSegDataset`](https://mmsegmentation.readthedocs.io/en/dev-1.x/api.html?highlight=BaseSegDataset#mmseg.datasets.BaseSegDataset)，它继承自 [MMEngine 的 `BaseDataset`](https://github.com/open-mmlab/mmengine/blob/main/docs/en/advanced_tutorials/basedataset.md), 遵循 OpenMMLab 数据集初始化统一流程, 支持高效的内部数据存储格式, 支持数据集拼接、数据集重复采样等功能.
+由于 MMSegmentation 中的所有数据集的基本功能均包括(1) 加载[数据集预处理](../user_guides/2_dataset_prepare.md) 之后的数据信息和 (2) 将数据送入数据变换流水线中进行数据变换, 因此在 MMSegmentation 中将其中的共同接口抽象成 [`BaseSegDataset`](https://mmsegmentation.readthedocs.io/zh_CN/latest/api.html?highlight=BaseSegDataset#mmseg.datasets.BaseSegDataset)，它继承自 [MMEngine 的 `BaseDataset`](https://github.com/open-mmlab/mmengine/blob/main/docs/en/advanced_tutorials/basedataset.md), 遵循 OpenMMLab 数据集初始化统一流程, 支持高效的内部数据存储格式, 支持数据集拼接、数据集重复采样等功能.
在 MMSegmentation BaseSegDataset 中重新定义了**数据信息加载方法**（`load_data_list`）和并新增了 `get_label_map` 方法用来**修改数据集的类别信息**.
### 数据信息加载
-数据信息加载的内容是样本数据的图片路径和标签路径, 具体实现在 MMSegmentation 的 BaseSegDataset 的 [`load_data_list`](https://github.com/open-mmlab/mmsegmentation/blob/163277bfe0fa8fefb63ee5137917fafada1b301c/mmseg/datasets/basesegdataset.py#L231) 中.
-主要有两种获取图片和标签的路径方法, 如果当数据集目录按以下目录结构组织, [`load_data_list`](https://github.com/open-mmlab/mmsegmentation/blob/163277bfe0fa8fefb63ee5137917fafada1b301c/mmseg/datasets/basesegdataset.py#L231)) 会根据数据路径和后缀来解析.
+数据信息加载的内容是样本数据的图片路径和标签路径, 具体实现在 MMSegmentation 的 BaseSegDataset 的 [`load_data_list`](https://github.com/open-mmlab/mmsegmentation/blob/main/mmseg/datasets/basesegdataset.py#L231) 中.
+主要有两种获取图片和标签的路径方法, 如果当数据集目录按以下目录结构组织, [`load_data_list`](https://github.com/open-mmlab/mmsegmentation/blob/main/mmseg/datasets/basesegdataset.py#L231)) 会根据数据路径和后缀来解析.
```
├── data
@@ -322,7 +322,7 @@ print(dataset.metainfo)
'reduce_zero_label': False}
```
-可以看到, 数据集元信息的类别和默认 Cityscapes 不同. 并且, 定义了标签重映射的字段 `label_map` 用来修改每个分割掩膜上的像素的类别索引, 分割标签类别会根据 `label_map`, 将类别重映射, [具体实现](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/mmseg/datasets/basesegdataset.py#L151):
+可以看到, 数据集元信息的类别和默认 Cityscapes 不同. 并且, 定义了标签重映射的字段 `label_map` 用来修改每个分割掩膜上的像素的类别索引, 分割标签类别会根据 `label_map`, 将类别重映射, [具体实现](https://github.com/open-mmlab/mmsegmentation/blob/main/mmseg/datasets/basesegdataset.py#L151):
```python
gt_semantic_seg_copy = gt_semantic_seg.copy()
diff --git a/docs/zh_cn/advanced_guides/engine.md b/docs/zh_cn/advanced_guides/engine.md
index a5746fcec89..79b4c8d2296 100644
--- a/docs/zh_cn/advanced_guides/engine.md
+++ b/docs/zh_cn/advanced_guides/engine.md
@@ -61,21 +61,21 @@ OpenMMLab 将模型训练和测试过程抽象为 `Runner`, 插入钩子可以
- 默认钩子 (default hooks)
-它们实现了训练时所必需的功能, 在配置文件中用 `default_hooks` 定义传给 `Runner`, `Runner` 通过 [`register_default_hooks`](https://github.com/open-mmlab/mmengine/blob/090104df21acd05a8aadae5a0d743a7da3314f6f/mmengine/runner/runner.py#L1780) 方法注册.
+它们实现了训练时所必需的功能, 在配置文件中用 `default_hooks` 定义传给 `Runner`, `Runner` 通过 [`register_default_hooks`](https://github.com/open-mmlab/mmengine/blob/main/mmengine/runner/runner.py#L1780) 方法注册.
钩子有对应的优先级, 优先级越高, 越早被执行器调用. 如果优先级一样, 被调用的顺序和钩子注册的顺序一致.
不建议用户修改默认钩子的优先级, 可以参考 [mmengine hooks 文档](https://github.com/open-mmlab/mmengine/blob/main/docs/zh_cn/tutorials/hook.md) 了解钩子优先级的定义.
下面是 MMSegmentation 中所用到的默认钩子：
-| 钩子 | 功能 | 优先级 |
-| :-----------------------------------------------------------------------------------------------------------------------: | :---------------------------------------------------------------------------------------------: | :---------------: |
-| [IterTimerHook](https://github.com/open-mmlab/mmengine/blob/main/mmengine/hooks/iter_timer_hook.py) | 记录 iteration 花费的时间. | NORMAL (50) |
-| [LoggerHook](https://github.com/open-mmlab/mmengine/blob/main/mmengine/hooks/logger_hook.py) | 从 `Runner` 里不同的组件中收集日志记录, 并将其输出到终端, JSON 文件, tensorboard, wandb 等下游. | BELOW_NORMAL (60) |
-| [ParamSchedulerHook](https://github.com/open-mmlab/mmengine/blob/main/mmengine/hooks/param_scheduler_hook.py) | 更新优化器里面的一些超参数, 例如学习率的动量. | LOW (70) |
-| [CheckpointHook](https://github.com/open-mmlab/mmengine/blob/main/mmengine/hooks/checkpoint_hook.py) | 规律性地保存 checkpoint 文件. | VERY_LOW (90) |
-| [DistSamplerSeedHook](https://github.com/open-mmlab/mmengine/blob/main/mmengine/hooks/sampler_seed_hook.py) | 确保分布式采样器 shuffle 是打开的. | NORMAL (50) |
-| [SegVisualizationHook](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/mmseg/visualization/local_visualizer.py) | 可视化验证和测试过程里的预测结果. | NORMAL (50) |
+| 钩子 | 功能 | 优先级 |
+| :--------------------------------------------------------------------------------------------------------------------: | :---------------------------------------------------------------------------------------------: | :---------------: |
+| [IterTimerHook](https://github.com/open-mmlab/mmengine/blob/main/mmengine/hooks/iter_timer_hook.py) | 记录 iteration 花费的时间. | NORMAL (50) |
+| [LoggerHook](https://github.com/open-mmlab/mmengine/blob/main/mmengine/hooks/logger_hook.py) | 从 `Runner` 里不同的组件中收集日志记录, 并将其输出到终端, JSON 文件, tensorboard, wandb 等下游. | BELOW_NORMAL (60) |
+| [ParamSchedulerHook](https://github.com/open-mmlab/mmengine/blob/main/mmengine/hooks/param_scheduler_hook.py) | 更新优化器里面的一些超参数, 例如学习率的动量. | LOW (70) |
+| [CheckpointHook](https://github.com/open-mmlab/mmengine/blob/main/mmengine/hooks/checkpoint_hook.py) | 规律性地保存 checkpoint 文件. | VERY_LOW (90) |
+| [DistSamplerSeedHook](https://github.com/open-mmlab/mmengine/blob/main/mmengine/hooks/sampler_seed_hook.py) | 确保分布式采样器 shuffle 是打开的. | NORMAL (50) |
+| [SegVisualizationHook](https://github.com/open-mmlab/mmsegmentation/blob/main/mmseg/visualization/local_visualizer.py) | 可视化验证和测试过程里的预测结果. | NORMAL (50) |
-MMSegmentation 会在 [`defualt_hooks`](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/configs/_base_/schedules/schedule_160k.py#L19-L25) 里面注册一些训练所必需功能的钩子::
+MMSegmentation 会在 [`defualt_hooks`](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/_base_/schedules/schedule_160k.py#L19-L25) 里面注册一些训练所必需功能的钩子::
```python
default_hooks = dict(
@@ -94,6 +94,7 @@ default_hooks = dict(
以 `default_hooks` 里面的 `logger` 和 `checkpoint` 为例, 我们来介绍如何修改 `default_hooks` 中默认的钩子.
(1) 模型保存配置
+
`default_hooks` 使用 `checkpoint` 字段来初始化[模型保存钩子 (CheckpointHook)](https://github.com/open-mmlab/mmengine/blob/main/mmengine/hooks/checkpoint_hook.py#L19).
```python
@@ -104,6 +105,7 @@ checkpoint = dict(type='CheckpointHook', interval=1)
更多相关参数的细节可以参考[这里](https://mmengine.readthedocs.io/zh_CN/latest/api/generated/mmengine.hooks.CheckpointHook.html#checkpointhook).
(2) 日志配置
+
`日志钩子 (LoggerHook)` 被用来收集 `执行器 (Runner)` 里面不同组件的日志信息然后写入终端, JSON 文件, tensorboard 和 wandb 等地方.
```python
@@ -126,7 +128,7 @@ visualizer = dict(
- 自定义钩子 (custom hooks)
-自定义钩子在配置通过 `custom_hooks` 定义, `Runner` 通过 [`register_custom_hooks`](https://github.com/open-mmlab/mmengine/blob/090104df21acd05a8aadae5a0d743a7da3314f6f/mmengine/runner/runner.py#L1852) 方法注册.
+自定义钩子在配置通过 `custom_hooks` 定义, `Runner` 通过 [`register_custom_hooks`](https://github.com/open-mmlab/mmengine/blob/main/mmengine/runner/runner.py#L1820) 方法注册.
自定义钩子优先级需要在配置文件里设置, 如果没有设置, 则会被默认设置为 `NORMAL`. 下面是部分 MMEngine 中实现的自定义钩子:
| 钩子 | 用法 |
@@ -145,7 +147,7 @@ custom_hooks = [
### SegVisualizationHook
-MMSegmentation 实现了 [`SegVisualizationHook`](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/mmseg/engine/hooks/visualization_hook.py#L17), 用来在验证和测试时可视化预测结果.
+MMSegmentation 实现了 [`SegVisualizationHook`](https://github.com/open-mmlab/mmsegmentation/blob/main/mmseg/engine/hooks/visualization_hook.py#L17), 用来在验证和测试时可视化预测结果.
`SegVisualizationHook` 重写了基类 `Hook` 中的 `_after_iter` 方法, 在验证或测试时, 根据指定的迭代次数间隔调用 `visualizer` 的 `add_datasample` 方法绘制语义分割结果, 具体实现如下:
```python
@@ -181,7 +183,7 @@ class SegVisualizationHook(Hook):
```
-关于可视化更多的细节可以查看[这里](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/docs/en/user_guides/visualization.md).
+关于可视化更多的细节可以查看[这里](../user_guides/visualization.md).
## 优化器
@@ -234,7 +236,7 @@ optim_wrapper = dict(type='AmpOptimWrapper', optimizer=optimizer)
在模型训练中, 如果想在优化器里为不同参数分别设置优化策略, 例如设置不同的学习率、权重衰减等超参数, 可以通过设置配置文件里 `optim_wrapper` 中的 `paramwise_cfg` 来实现.
-下面的配置文件以 [ViT `optim_wrapper`](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/configs/vit/vit_vit-b16-ln_mln_upernet_8xb2-160k_ade20k-512x512.py#L15-L27) 为例介绍 `paramwise_cfg` 参数使用.
+下面的配置文件以 [ViT `optim_wrapper`](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/vit/vit_vit-b16-ln_mln_upernet_8xb2-160k_ade20k-512x512.py#L15-L27) 为例介绍 `paramwise_cfg` 参数使用.
训练时将 `pos_embed`, `mask_token`, `norm` 模块的 weight decay 参数的系数设置成 0.
即: 在训练时, 这些模块的 weight decay 将被变为 `weight_decay * decay_mult`=0.
@@ -257,9 +259,9 @@ optim_wrapper = dict(
### 优化器封装构造器
-默认的优化器封装构造器 [`DefaultOptimWrapperConstructor`](https://github.com/open-mmlab/mmengine/blob/376251961da47ea8254ab808ae5c51e1430f18dc/mmengine/optim/optimizer/default_constructor.py#L19) 根据输入的 `optim_wrapper` 和 `optim_wrapper` 中定义的 `paramwise_cfg` 来构建训练中使用的优化器. 当 [`DefaultOptimWrapperConstructor`](https://github.com/open-mmlab/mmengine/blob/376251961da47ea8254ab808ae5c51e1430f18dc/mmengine/optim/optimizer/default_constructor.py#L19) 功能不能满足需求时, 可以自定义优化器封装构造器来实现超参数的配置.
+默认的优化器封装构造器 [`DefaultOptimWrapperConstructor`](https://github.com/open-mmlab/mmengine/blob/main/mmengine/optim/optimizer/default_constructor.py#L19) 根据输入的 `optim_wrapper` 和 `optim_wrapper` 中定义的 `paramwise_cfg` 来构建训练中使用的优化器. 当 [`DefaultOptimWrapperConstructor`](https://github.com/open-mmlab/mmengine/blob/main/mmengine/optim/optimizer/default_constructor.py#L19) 功能不能满足需求时, 可以自定义优化器封装构造器来实现超参数的配置.
-MMSegmentation 中的实现了 [`LearningRateDecayOptimizerConstructor`](https://github.com/open-mmlab/mmsegmentation/blob/b21df463d47447f33c28d9a4f46136ad64d34a40/mmseg/engine/optimizers/layer_decay_optimizer_constructor.py#L104), 可以对以 ConvNeXt, BEiT 和 MAE 为骨干网络的模型训练时, 骨干网络的模型参数的学习率按照定义的衰减比例（`decay_rate`）逐层递减, 在配置文件中的配置如下:
+MMSegmentation 中的实现了 [`LearningRateDecayOptimizerConstructor`](https://github.com/open-mmlab/mmsegmentation/blob/main/mmseg/engine/optimizers/layer_decay_optimizer_constructor.py#L104), 可以对以 ConvNeXt, BEiT 和 MAE 为骨干网络的模型训练时, 骨干网络的模型参数的学习率按照定义的衰减比例（`decay_rate`）逐层递减, 在配置文件中的配置如下:
```python
optim_wrapper = dict(
diff --git a/docs/zh_cn/advanced_guides/models.md b/docs/zh_cn/advanced_guides/models.md
index 408a57863c4..6eb22517a46 100644
--- a/docs/zh_cn/advanced_guides/models.md
+++ b/docs/zh_cn/advanced_guides/models.md
@@ -30,17 +30,17 @@
## 基本接口
-MMSegmentation 封装 `BaseModel` 并实现了 [BaseSegmenter](https://github.com/open-mmlab/mmsegmentation/blob/1.x/mmseg/models/segmentors/base.py#L15) 类，主要提供 `forward`、`train_step`、`val_step` 和 `test_step` 接口。接下来将详细介绍这些接口。
+MMSegmentation 封装 `BaseModel` 并实现了 [BaseSegmentor](https://github.com/open-mmlab/mmsegmentation/blob/1.x/mmseg/models/segmentors/base.py#L15) 类，主要提供 `forward`、`train_step`、`val_step` 和 `test_step` 接口。接下来将详细介绍这些接口。
### forward
- 
+ 
编码器解码器数据流
- 
+ 
级联编码器解码器数据流
@@ -115,7 +115,7 @@ MMSegmentation 封装 `BaseModel` 并实现了 [BaseSegmenter](https://github.co
-Dict\[str, `torch.Tensor`\]：用于记录日志的张量的`字典`。
- 
+ 
train_step 数据流
@@ -132,7 +132,7 @@ MMSegmentation 封装 `BaseModel` 并实现了 [BaseSegmenter](https://github.co
- `list` - 给定数据的预测结果。
- 
+ 
test_step/val_step 数据流
diff --git a/docs/zh_cn/advanced_guides/training_tricks.md b/docs/zh_cn/advanced_guides/training_tricks.md
index a33c0ea9cfd..e5b8e4dae1e 100644
--- a/docs/zh_cn/advanced_guides/training_tricks.md
+++ b/docs/zh_cn/advanced_guides/training_tricks.md
@@ -1,4 +1,4 @@
-# 训练技巧（待更新）
+# 训练技巧
MMSegmentation 支持如下训练技巧：
@@ -9,17 +9,17 @@ MMSegmentation 支持如下训练技巧：
在 MMSegmentation 里面，您也可以在配置文件里添加如下行来让解码头组件的学习率是主干组件的10倍。
```python
-optimizer=dict(
+optim_wrapper=dict(
paramwise_cfg = dict(
custom_keys={
'head': dict(lr_mult=10.)}))
```
-通过这种修改，任何被分组到 `'head'` 的参数的学习率都将乘以10。您也可以参照 [MMCV 文档](https://mmcv.readthedocs.io/en/latest/api.html#mmcv.runner.DefaultOptimizerConstructor) 获取更详细的信息。
+通过这种修改，任何被分组到 `'head'` 的参数的学习率都将乘以10。您也可以参照 [MMEngine 文档](https://mmengine.readthedocs.io/zh_CN/latest/tutorials/optim_wrapper.html#id6) 获取更详细的信息。
## 在线难样本挖掘 (Online Hard Example Mining, OHEM)
-对于训练时采样，我们在 [这里](https://github.com/open-mmlab/mmsegmentation/tree/master/mmseg/core/seg/sampler) 做了像素采样器。
+MMSegmentation 中实现了像素采样器，训练时可以对特定像素进行采样，例如 OHEM(Online Hard Example Mining)，可以解决样本不平衡问题，
如下例子是使用 PSPNet 训练并采用 OHEM 策略的配置：
```python
@@ -58,38 +58,17 @@ model=dict(
```python
_base_ = './fcn_unet_s5-d16_64x64_40k_drive.py'
model = dict(
- decode_head=dict(loss_decode=[dict(type='CrossEntropyLoss', loss_name='loss_ce', loss_weight=1.0),
- dict(type='DiceLoss', loss_name='loss_dice', loss_weight=3.0)]),
- auxiliary_head=dict(loss_decode=[dict(type='CrossEntropyLoss', loss_name='loss_ce',loss_weight=1.0),
- dict(type='DiceLoss', loss_name='loss_dice', loss_weight=3.0)]),
- )
+ decode_head=dict(loss_decode=[
+ dict(type='CrossEntropyLoss', loss_name='loss_ce', loss_weight=1.0),
+ dict(type='DiceLoss', loss_name='loss_dice', loss_weight=3.0)
+ ]),
+ auxiliary_head=dict(loss_decode=[
+ dict(type='CrossEntropyLoss', loss_name='loss_ce', loss_weight=1.0),
+ dict(type='DiceLoss', loss_name='loss_dice', loss_weight=3.0)
+ ]),
+)
```
通过这种方式，确定训练过程中损失函数的权重 `loss_weight` 和在训练日志里的名字 `loss_name`。
-注意： `loss_name` 的名字必须带有 `loss_` 前缀，这样它才能被包括在反传的图里。
-
-## 在损失函数中忽略特定的 label 类别
-
-默认设置 `avg_non_ignore=False`， 即每个像素都用来计算损失函数。尽管其中的一些像素属于需要被忽略的类别。
-
-对于训练时损失函数的计算，我们目前支持使用 `avg_non_ignore` 和 `ignore_index` 来忽略 label 特定的类别。 这样损失函数将只在非忽略类别像素中求平均值，会获得更好的表现。这里是[相关 PR](https://github.com/open-mmlab/mmsegmentation/pull/1409)。以 `unet` 使用 `Cityscapes` 数据集训练为例，
-在计算损失函数时，忽略 label 为0的背景，并且仅在不被忽略的像素上计算均值。配置文件写为:
-
-```python
-_base_ = './fcn_unet_s5-d16_4x4_512x1024_160k_cityscapes.py'
-model = dict(
- decode_head=dict(
- ignore_index=0,
- loss_decode=dict(
- type='CrossEntropyLoss', use_sigmoid=False, loss_weight=1.0, avg_non_ignore=True),
- auxiliary_head=dict(
- ignore_index=0,
- loss_decode=dict(
- type='CrossEntropyLoss', use_sigmoid=False, loss_weight=1.0, avg_non_ignore=True)),
- ))
-```
-
-通过这种方式，确定训练过程中损失函数的权重 `loss_weight` 和在训练日志里的名字 `loss_name`。
-
-注意： `loss_name` 的名字必须带有 `loss_` 前缀，这样它才能被包括在反传的图里。
+注意： `loss_name` 的名字必须带有 `loss_` 前缀，这样它才能被包括在计算图里。
diff --git a/docs/zh_cn/get_started.md b/docs/zh_cn/get_started.md
index 38e93e9cb4e..ca375f3d9d6 100644
--- a/docs/zh_cn/get_started.md
+++ b/docs/zh_cn/get_started.md
@@ -4,7 +4,7 @@
本教程中，我们将会演示如何使用 PyTorch 准备环境。
-MMSegmentation 可以在 Linux, Windows 和 macOS 系统上运行，并且需要安装 Python 3.6+, CUDA 9.2+ 和 PyTorch 1.5+
+MMSegmentation 可以在 Linux, Windows 和 macOS 系统上运行，并且需要安装 Python 3.7+, CUDA 10.2+ 和 PyTorch 1.8+
**注意:**
如果您已经安装了 PyTorch, 可以跳过该部分，直接到[下一小节](##安装)。否则，您可以按照以下步骤操作。
@@ -43,7 +43,7 @@ conda install pytorch torchvision cpuonly -c pytorch
```shell
pip install -U openmim
mim install mmengine
-mim install "mmcv>=2.0.0rc1"
+mim install "mmcv>=2.0.0"
```
**步骤 1.** 安装 MMSegmentation
@@ -51,7 +51,7 @@ mim install "mmcv>=2.0.0rc1"
情况 a: 如果您想立刻开发和运行 mmsegmentation，您可通过源码安装：
```shell
-git clone -b dev-1.x https://github.com/open-mmlab/mmsegmentation.git
+git clone -b main https://github.com/open-mmlab/mmsegmentation.git
cd mmsegmentation
pip install -v -e .
# '-v' 表示详细模式，更多的输出
@@ -62,7 +62,7 @@ pip install -v -e .
情况 b: 如果您把 mmsegmentation 作为依赖库或者第三方库，可以通过 pip 安装：
```shell
-pip install "mmsegmentation>=1.0.0rc0"
+pip install "mmsegmentation>=1.0.0"
```
### 验证是否安装成功
@@ -87,8 +87,7 @@ python demo/image_demo.py demo/demo.png configs/pspnet/pspnet_r50-d8_4xb2-40k_ci
您将在当前文件夹中看到一个新图像 `result.jpg`，其中所有目标都覆盖了分割 mask
-选项 (b). 如果您通过 pip 安装 mmsegmentation, 打开您的 python
-解释器，复制粘贴以下代码：
+选项 (b). 如果您通过 pip 安装 mmsegmentation, 打开您的 python 解释器，复制粘贴以下代码：
```python
from mmseg.apis import inference_model, init_model, show_result_pyplot
@@ -111,8 +110,8 @@ show_result_pyplot(model, img, result, show=True, out_file='result.jpg', opacity
# 在一段视频上测试并可视化分割结果
video = mmcv.VideoReader('video.mp4')
for frame in video:
- result = inference_segmentor(model, frame)
- show_result_pyplot(model, result, wait_time=1)
+ result = inference_model(model, frame)
+ show_result_pyplot(model, frame, result, wait_time=1)
```
您可以修改上面的代码来测试单个图像或视频，这两个选项都可以验证安装是否成功。
@@ -137,15 +136,15 @@ MMCV 包含 C++ 和 CUDA 扩展，因此与 PyTorch 的依赖方式比较复杂
为了使用 pip 而不是 MIM 安装 MMCV, 请参考 [MMCV 安装指南](https://mmcv.readthedocs.io/en/latest/get_started/installation.html). 这需要手动指定一个基于 PyTorch 版本及其 CUDA 版本的 find-url.
-例如，以下命令可为 PyTorch 1.10.x and CUDA 11.3 安装 mmcv==2.0.0rc1
+例如，以下命令可为 PyTorch 1.10.x and CUDA 11.3 安装 mmcv==2.0.0
```shell
-pip install mmcv==2.0.0rc1 -f https://download.openmmlab.com/mmcv/dist/cu113/torch1.10/index.html
+pip install mmcv==2.0.0 -f https://download.openmmlab.com/mmcv/dist/cu113/torch1.10/index.html
```
#### 在仅有 CPU 的平台安装
-MMSegmentation 可以在仅有 CPU 的版本上运行。在 CPU 模式，您可以训练（需要 MMCV-Lite 版本 >= 2.0.0rc0），测试和推理模型。
+MMSegmentation 可以在仅有 CPU 的版本上运行。在 CPU 模式，您可以训练（需要 MMCV 版本 >= 2.0.0），测试和推理模型。
#### 在 Google Colab 上安装
@@ -156,7 +155,7 @@ MMSegmentation 可以在仅有 CPU 的版本上运行。在 CPU 模式，您可
```shell
!pip3 install openmim
!mim install mmengine
-!mim install "mmcv>=2.0.0rc1"
+!mim install "mmcv>=2.0.0"
```
**Step 2.** 通过源码安装 MMSegmentation
@@ -164,7 +163,7 @@ MMSegmentation 可以在仅有 CPU 的版本上运行。在 CPU 模式，您可
```shell
!git clone https://github.com/open-mmlab/mmsegmentation.git
%cd mmsegmentation
-!git checkout dev-1.x
+!git checkout main
!pip install -e .
```
@@ -173,7 +172,7 @@ MMSegmentation 可以在仅有 CPU 的版本上运行。在 CPU 模式，您可
```python
import mmseg
print(mmseg.__version__)
-# 示例输出: 1.0.0rc0
+# 示例输出: 1.0.0
```
**注意:**
@@ -195,6 +194,16 @@ docker build -t mmsegmentation docker/
docker run --gpus all --shm-size=8g -it -v {DATA_DIR}:/mmsegmentation/data mmsegmentation
```
+### 可选依赖
+
+#### 安装 GDAL
+
+[GDAL](https://gdal.org/) 是一个用于栅格和矢量地理空间数据格式的转换库。安装 GDAL 可以读取复杂格式和极大的遥感图像。
+
+```shell
+conda install GDAL
+```
+
## 问题解答
如果您在安装过程中遇到了其他问题，请第一时间查阅 [FAQ](notes/faq.md) 文件。如果没有找到答案，您也可以在 GitHub 上提出 [issue](https://github.com/open-mmlab/mmsegmentation/issues/new/choose)
diff --git a/docs/zh_cn/imgs/qq_group_qrcode.jpg b/docs/zh_cn/imgs/qq_group_qrcode.jpg
deleted file mode 100644
index 417347449fe..00000000000
Binary files a/docs/zh_cn/imgs/qq_group_qrcode.jpg and /dev/null differ
diff --git a/docs/zh_cn/imgs/seggroup_qrcode.jpg b/docs/zh_cn/imgs/seggroup_qrcode.jpg
deleted file mode 100644
index 9684582ee1c..00000000000
Binary files a/docs/zh_cn/imgs/seggroup_qrcode.jpg and /dev/null differ
diff --git a/docs/zh_cn/index.rst b/docs/zh_cn/index.rst
index e66c178689d..ce5e49977dc 100644
--- a/docs/zh_cn/index.rst
+++ b/docs/zh_cn/index.rst
@@ -23,7 +23,7 @@
:maxdepth: 1
:caption: 迁移指引
- migration.md
+ migration/index.rst
.. toctree::
:caption: 接口文档（英文）
diff --git a/docs/zh_cn/migration/interface.md b/docs/zh_cn/migration/interface.md
index cd16d2cbc68..42f91bf50ac 100644
--- a/docs/zh_cn/migration/interface.md
+++ b/docs/zh_cn/migration/interface.md
@@ -2,7 +2,7 @@
## 引言
-本指南介绍了 MMSegmentation 0.x 和 MMSegmentation1.x 在行为和 API 方面的基本区别，以及这些如何都与您的迁移过程相关。
+本指南介绍了 MMSegmentation 0.x 和 MMSegmentation1.x 在表现和 API 方面的基本区别，以及这些与迁移过程的关系。
## 新的依赖
@@ -12,11 +12,11 @@ MMSegmentation 1.x 依赖于一些新的软件包，您可以准备一个新的
1. [MMEngine](https://github.com/open-mmlab/mmengine)：MMEngine 是 OpenMMLab 2.0 架构的核心，我们将许多与计算机视觉无关的内容从 MMCV 拆分到 MMEngine 中。
-2. [MMCV](https://github.com/open-mmlab/mmcv)：OpenMMLab 的计算机视觉包。这不是一个新的依赖，但您需要将其升级到 **2.0.0rc1** 以上的版本。
+2. [MMCV](https://github.com/open-mmlab/mmcv)：OpenMMLab 的计算机视觉包。这不是一个新的依赖，但您需要将其升级到 **2.0.0** 或以上的版本。
-3. [MMClassification](https://github.com/open-mmlab/mmclassification)（可选）：OpenMMLab 的图像分类工具箱和基准。这不是一个新的依赖，但您需要将其升级到 **1.0.0rc0** 以上的版本。
+3. [MMClassification](https://github.com/open-mmlab/mmclassification)（可选）：OpenMMLab 的图像分类工具箱和基准。这不是一个新的依赖，但您需要将其升级到 **1.0.0rc6** 版本。
-4. [MMDetection](https://github.com/open-mmlab/mmdetection)(可选): OpenMMLab 的目标检测工具箱和基准。这不是一个新的依赖，但您需要将其升级到 **3.0.0rc0** 以上的版本。
+4. [MMDetection](https://github.com/open-mmlab/mmdetection)(可选): OpenMMLab 的目标检测工具箱和基准。这不是一个新的依赖，但您需要将其升级到 **3.0.0** 或以上的版本。
## 启动训练
@@ -46,7 +46,7 @@ OpenMMLab 2.0 的主要改进是发布了 MMEngine，它为启动训练任务的
--resume='auto' |
-| 培训练期间是否不评估检查点 |
+训练期间是否不评估检查点 |
--no-validate |
--cfg-options val_cfg=None val_dataloader=None val_evaluator=None |
@@ -102,11 +102,11 @@ OpenMMLab 2.0 的主要改进是发布了 MMEngine，它为启动训练任务的
- `mean`（Sequence，可选）：R、G、B 通道的像素平均值。默认为 None。
-- `std`（Sequence，可选）：R、G、B通道的像素标准差。默认为 None。
+- `std`（Sequence，可选）：R、G、B 通道的像素标准差。默认为 None。
- `size`（Sequence，可选）：固定的填充大小。
-- `size_divisor`（int，可选）：填充大小的除法因子。
+- `size_divisor`（int，可选）：填充图像可以被当前值整除。
- `seg_pad_val`（float，可选）：分割图的填充值。默认值：255。
@@ -154,14 +154,14 @@ train_dataloader = dict(
batch_size=4,
num_workers=4,
dataset=dict(...),
- sampler=dict(type='DefaultSampler', shuffle=True) # necessary
+ sampler=dict(type='DefaultSampler', shuffle=True) # 必须
)
val_dataloader = dict(
batch_size=4,
num_workers=4,
dataset=dict(...),
- sampler=dict(type='DefaultSampler', shuffle=False) # necessary
+ sampler=dict(type='DefaultSampler', shuffle=False) # 必须
)
test_dataloader = val_dataloader
@@ -417,10 +417,10 @@ runner = dict(type='IterBasedRunner', max_iters=20000)
```python
-# The `val_interval` is the original `evaluation.interval`.
+# `val_interval` 是旧版本的 `evaluation.interval`。
train_cfg = dict(type='IterBasedTrainLoop', max_iters=20000, val_interval=2000)
-val_cfg = dict(type='ValLoop') # Use the default validation loop.
-test_cfg = dict(type='TestLoop') # Use the default test loop.
+val_cfg = dict(type='ValLoop') # 使用默认的验证循环。
+test_cfg = dict(type='TestLoop') # 使用默认的测试循环。
```
|
@@ -438,22 +438,22 @@ test_cfg = dict(type='TestLoop') # Use the default test loop.
```python
default_hooks = dict(
- # record the time of every iterations.
+ # 记录每次迭代的时间。
timer=dict(type='IterTimerHook'),
- # print log every 50 iterations.
+ # 每50次迭代打印一次日志。
logger=dict(type='LoggerHook', interval=50, log_metric_by_epoch=False),
- # enable the parameter scheduler.
+ # 启用参数调度程序。
param_scheduler=dict(type='ParamSchedulerHook'),
- # save checkpoint every 2000 iterations.
+ # 每2000次迭代保存一次检查点。
checkpoint=dict(type='CheckpointHook', by_epoch=False, interval=2000),
- # set sampler seed in distributed environment.
+ # 在分布式环境中设置采样器种子。
sampler_seed=dict(type='DistSamplerSeedHook'),
- # validation results visualization.
+ # 验证结果可视化。
visualization=dict(type='SegVisualizationHook'))
```
@@ -505,13 +505,13 @@ visualizer = dict(
```python
env_cfg = dict(
- # whether to enable cudnn benchmark
+ # 是否启用 cudnn_benchmark
cudnn_benchmark=False,
- # set multi process parameters
+ # 设置多进程参数
mp_cfg=dict(mp_start_method='fork', opencv_num_threads=0),
- # set distributed parameters
+ # 设置分布式参数
dist_cfg=dict(backend='nccl'),
)
```
diff --git a/docs/zh_cn/migration/package.md b/docs/zh_cn/migration/package.md
index d8d2245bed0..19e5f18c9c4 100644
--- a/docs/zh_cn/migration/package.md
+++ b/docs/zh_cn/migration/package.md
@@ -1,6 +1,6 @@
-#包结构更改
+# 包结构更改
-本节包含您对 MMSeg 0.x 和 1.x 之间的变化感到好奇的内容。
+本节包含您对 MMSeg 0.x 和 1.x 之间的变化可能感到好奇的内容。
@@ -49,7 +49,7 @@
## `mmseg.ops`
-`ops` 包包含 `encoding` 和 `wrappers`，它们被移到了 `mmseg.models.utils` 中。
+`ops` 包含 `encoding` 和 `wrappers`，它们被移到了 `mmseg.models.utils` 中。
## 增加的包
@@ -110,4 +110,4 @@ OpenMMLab 2.0 将 `BaseDataset` 定义为数据集的函数和接口，MMSegment
### `mmseg.models`
-`models` 没有太大变化，只是从以前的 `mmseg.ops` 中添加了 `encoding` 和 `wrappers`
+`models` 没有太大变化，只是从以前的 `mmseg.ops` 添加了 `encoding` 和 `wrappers`
diff --git a/docs/zh_cn/notes/faq.md b/docs/zh_cn/notes/faq.md
index 09fde025fde..bf3b3417802 100644
--- a/docs/zh_cn/notes/faq.md
+++ b/docs/zh_cn/notes/faq.md
@@ -1,8 +1,122 @@
-# 常见问题解答（FAQ）（待更新）
+# 常见问题解答（FAQ）
-我们在这里列出了使用时的一些常见问题及其相应的解决方案。 如果您发现有一些问题被遗漏，请随时提 PR 丰富这个列表。 如果您无法在此获得帮助，请使用 [issue模板](https://github.com/open-mmlab/mmsegmentation/blob/master/.github/ISSUE_TEMPLATE/error-report.md/)创建问题，但是请在模板中填写所有必填信息，这有助于我们更快定位问题。
+我们在这里列出了使用时的一些常见问题及其相应的解决方案。 如果您发现有一些问题被遗漏，请随时提 PR 丰富这个列表。 如果您无法在此获得帮助，请使用 [issue 模板](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/.github/ISSUE_TEMPLATE/error-report.md/)创建问题，但是请在模板中填写所有必填信息，这有助于我们更快定位问题。
+
+## 安装
+
+兼容的 MMSegmentation 和 MMCV 版本如下。请安装正确版本的 MMCV 以避免安装问题。
+
+| MMSegmentation version | MMCV version | MMEngine version | MMClassification (optional) version | MMDetection (optional) version |
+| :--------------------: | :----------------------------: | :---------------: | :---------------------------------: | :----------------------------: |
+| dev-1.x branch | mmcv >= 2.0.0 | MMEngine >= 0.7.4 | mmpretrain>=1.0.0rc7 | mmdet >= 3.0.0 |
+| main branch | mmcv >= 2.0.0 | MMEngine >= 0.7.4 | mmpretrain>=1.0.0rc7 | mmdet >= 3.0.0 |
+| 1.1.2 | mmcv >= 2.0.0 | MMEngine >= 0.7.4 | mmpretrain>=1.0.0rc7 | mmdet >= 3.0.0 |
+| 1.1.1 | mmcv >= 2.0.0 | MMEngine >= 0.7.4 | mmpretrain>=1.0.0rc7 | mmdet >= 3.0.0 |
+| 1.1.0 | mmcv >= 2.0.0 | MMEngine >= 0.7.4 | mmpretrain>=1.0.0rc7 | mmdet >= 3.0.0 |
+| 1.0.0 | mmcv >= 2.0.0rc4 | MMEngine >= 0.7.1 | mmcls==1.0.0rc6 | mmdet >= 3.0.0 |
+| 1.0.0rc6 | mmcv >= 2.0.0rc4 | MMEngine >= 0.5.0 | mmcls>=1.0.0rc0 | mmdet >= 3.0.0rc6 |
+| 1.0.0rc5 | mmcv >= 2.0.0rc4 | MMEngine >= 0.2.0 | mmcls>=1.0.0rc0 | mmdet>=3.0.0rc6 |
+| 1.0.0rc4 | mmcv == 2.0.0rc3 | MMEngine >= 0.1.0 | mmcls>=1.0.0rc0 | mmdet>=3.0.0rc4, \<=3.0.0rc5 |
+| 1.0.0rc3 | mmcv == 2.0.0rc3 | MMEngine >= 0.1.0 | mmcls>=1.0.0rc0 | mmdet>=3.0.0rc4, \<=3.0.0rc5 |
+| 1.0.0rc2 | mmcv == 2.0.0rc3 | MMEngine >= 0.1.0 | mmcls>=1.0.0rc0 | mmdet>=3.0.0rc4, \<=3.0.0rc5 |
+| 1.0.0rc1 | mmcv >= 2.0.0rc1, \<=2.0.0rc3> | MMEngine >= 0.1.0 | mmcls>=1.0.0rc0 | Not required |
+| 1.0.0rc0 | mmcv >= 2.0.0rc1, \<=2.0.0rc3> | MMEngine >= 0.1.0 | mmcls>=1.0.0rc0 | Not required |
+
+如果您已经安装了版本不合适的 mmcv，请先运行`pip uninstall mmcv`卸载已安装的 mmcv，如您先前安装的为 mmcv-full（存在于 OpenMMLab 1.x），请运行`pip uninstall mmcv-full`进行卸载。
+
+- 如出现 "No module named 'mmcv'"
+ 1. 使用`pip uninstall mmcv`卸载环境中现有的 mmcv
+ 2. 按照[安装说明](../get_started.md)安装对应的 mmcv
## 如何获知模型训练时需要的显卡数量
-- 看模型的config文件的命名。可以参考[学习配置文件](https://github.com/open-mmlab/mmsegmentation/blob/master/docs/zh_cn/tutorials/config.md)中的`配置文件命名风格`部分。比如，对于名字为`segformer_mit-b0_8x1_1024x1024_160k_cityscapes.py`的config文件，`8x1`代表训练其对应的模型需要的卡数为8，每张卡中的batch size为1。
-- 看模型的log文件。点开该模型的log文件，并在其中搜索`nGPU`，在`nGPU`后的数字个数即训练时所需的卡数。比如，在log文件中搜索`nGPU`得到`nGPU 0,1,2,3,4,5,6,7`的记录，则说明训练该模型需要使用八张卡。
+- 看模型的 config 文件命名。可以参考[了解配置文件](../user_guides/1_config.md)中的`配置文件命名风格`部分。比如，对于名字为`segformer_mit-b0_8xb1-160k_cityscapes-1024x1024.py`的 config 文件，`8xb1`代表训练其对应的模型需要的卡数为 8，每张卡中的 batch size 为 1。
+- 看模型的 log 文件。点开该模型的 log 文件，并在其中搜索`nGPU`，在`nGPU`后的数字个数即训练时所需的卡数。比如，在 log 文件中搜索`nGPU`得到`nGPU 0,1,2,3,4,5,6,7`的记录，则说明训练该模型需要使用八张卡。
+
+## auxiliary head 是什么
+
+简单来说，这是一个提高准确率的深度监督技术。在训练阶段，`decode_head`用于输出语义分割的结果，`auxiliary_head` 只是增加了一个辅助损失，其产生的分割结果对你的模型结果没有影响，仅在在训练中起作用。您可以阅读这篇[论文](https://arxiv.org/pdf/1612.01105.pdf)了解更多信息。
+
+## 运行测试脚本时如何输出绘制分割掩膜的图像
+
+在测试脚本中，我们提供了`--out`参数来控制是否输出保存预测的分割掩膜图像。您可以运行以下命令输出测试结果：
+
+```shell
+python tools/test.py ${CONFIG_FILE} ${CHECKPOINT_FILE} --out ${OUTPUT_DIR}
+```
+
+更多用例细节可查阅[文档](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/docs/zh_cn/user_guides/4_train_test.md#%E6%B5%8B%E8%AF%95%E5%B9%B6%E4%BF%9D%E5%AD%98%E5%88%86%E5%89%B2%E7%BB%93%E6%9E%9C)，[PR #2712](https://github.com/open-mmlab/mmsegmentation/pull/2712) 以及[迁移文档](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/docs/zh_cn/migration/interface.md#%E6%B5%8B%E8%AF%95%E5%90%AF%E5%8A%A8)了解相关说明。
+
+## 如何处理二值分割任务?
+
+MMSegmentation 使用 `num_classes` 和 `out_channels` 来控制模型最后一层 `self.conv_seg` 的输出。更多细节可以参考 [这里](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/mmseg/models/decode_heads/decode_head.py)。
+
+`num_classes` 应该和数据集本身类别个数一致，当是二值分割时，数据集只有前景和背景两类，所以 `num_classes` 为 2. `out_channels` 控制模型最后一层的输出的通道数，通常和 `num_classes` 相等，但当二值分割时候，可以有两种处理方法, 分别是：
+
+- 设置 `out_channels=2`，在训练时以 Cross Entropy Loss 作为损失函数，在推理时使用 `F.softmax()` 归一化 logits 值，然后通过 `argmax()` 得到每个像素的预测结果。
+
+- 设置 `out_channels=1`，在训练时以 Binary Cross Entropy Loss 作为损失函数，在推理时使用 `F.sigmoid()` 和 `threshold` 得到预测结果，`threshold` 默认为 0.3。
+
+对于实现上述两种计算二值分割的方法，需要在 `decode_head` 和 `auxiliary_head` 的配置里修改。下面是对样例 [pspnet_unet_s5-d16.py](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/configs/_base_/models/pspnet_unet_s5-d16.py) 做出的对应修改。
+
+- (1) `num_classes=2`, `out_channels=2` 并在 `CrossEntropyLoss` 里面设置 `use_sigmoid=False`。
+
+```python
+decode_head=dict(
+ type='PSPHead',
+ in_channels=64,
+ in_index=4,
+ num_classes=2,
+ out_channels=2,
+ loss_decode=dict(
+ type='CrossEntropyLoss', use_sigmoid=False, loss_weight=1.0)),
+auxiliary_head=dict(
+ type='FCNHead',
+ in_channels=128,
+ in_index=3,
+ num_classes=2,
+ out_channels=2,
+ loss_decode=dict(
+ type='CrossEntropyLoss', use_sigmoid=False, loss_weight=0.4)),
+```
+
+- (2) `num_classes=2`, `out_channels=1` 并在 `CrossEntropyLoss` 里面设置 `use_sigmoid=True`.
+
+```python
+decode_head=dict(
+ type='PSPHead',
+ in_channels=64,
+ in_index=4,
+ num_classes=2,
+ out_channels=1,
+ loss_decode=dict(
+ type='CrossEntropyLoss', use_sigmoid=True, loss_weight=1.0)),
+auxiliary_head=dict(
+ type='FCNHead',
+ in_channels=128,
+ in_index=3,
+ num_classes=2,
+ out_channels=1,
+ loss_decode=dict(
+ type='CrossEntropyLoss', use_sigmoid=True, loss_weight=0.4)),
+```
+
+## `reduce_zero_label` 的作用
+
+数据集中 `reduce_zero_label` 参数类型为布尔类型，默认为 False，它的功能是为了忽略数据集 label 0。具体做法是将 label 0 改为 255，其余 label 相应编号减 1，同时 decode head 里将 255 设为 ignore index，即不参与 loss 计算。
+以下是 `reduce_zero_label` 具体实现逻辑:
+
+```python
+if self.reduce_zero_label:
+ # avoid using underflow conversion
+ gt_semantic_seg[gt_semantic_seg == 0] = 255
+ gt_semantic_seg = gt_semantic_seg - 1
+ gt_semantic_seg[gt_semantic_seg == 254] = 255
+```
+
+关于您的数据集是否需要使用 reduce_zero_label，有以下两类情况：
+
+- 例如在 [Potsdam](https://github.com/open-mmlab/mmsegmentation/blob/1.x/docs/en/user_guides/2_dataset_prepare.md#isprs-potsdam) 数据集上，有 0-不透水面、1-建筑、2-低矮植被、3-树、4-汽车、5-杂乱，六类。但该数据集提供了两种 RGB 标签，一种为图像边缘处有黑色像素的标签，另一种是没有黑色边缘的标签。对于有黑色边缘的标签，在 [dataset_converters.py](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/tools/dataset_converters/potsdam.py)中，其将黑色边缘转换为 label 0，其余标签分别为 1-不透水面、2-建筑、3-低矮植被、4-树、5-汽车、6-杂乱，那么此时，就应该在数据集 [potsdam.py](https://github.com/open-mmlab/mmsegmentation/blob/ff95416c3b5ce8d62b9289f743531398efce534f/mmseg/datasets/potsdam.py#L23) 中将`reduce_zero_label=True`。如果使用的是没有黑色边缘的标签，那么 mask label 中只有 0-5，此时就应该使`reduce_zero_label=False`。需要结合您的实际情况来使用。
+- 例如在第 0 类为 background 类别的数据集上，如果您最终是需要将背景和您的其余类别分开时，是不需要使用`reduce_zero_label`的，此时在数据集中应该将其设置为`reduce_zero_label=False`
+
+**注意:** 使用 `reduce_zero_label` 请确认数据集原始类别个数，如果只有两类，需要关闭 `reduce_zero_label` 即设置 `reduce_zero_label=False`。
diff --git a/docs/zh_cn/overview.md b/docs/zh_cn/overview.md
index 7dce105a813..ed147956d08 100644
--- a/docs/zh_cn/overview.md
+++ b/docs/zh_cn/overview.md
@@ -42,7 +42,7 @@ MMSeg 主要包含了 apis, structures, datasets, models, engine, evaluation 和
以下是详细步骤，将带您一步步学习如何使用 MMSegmentation :
-1. 有关安装说明，请参阅 [开始你的第一步](getting_started.md)。
+1. 有关安装说明，请参阅 [开始你的第一步](get_started.md)。
2. 对于初学者来说，MMSegmentation 是开始语义分割之旅的最好选择，因为这里实现了许多 SOTA 模型以及经典的模型 [model](model_zoo.md) 。另外，将各类组件和高级 API 結合使用，可以更便捷的执行分割任务。关于 MMSegmentation 的基本用法，请参考下面的教程：
@@ -62,8 +62,8 @@ MMSeg 主要包含了 apis, structures, datasets, models, engine, evaluation 和
4. MMSegmentation 也为用户自定义和一些前沿的研究提供了教程，请参考下面的教程来建立你自己的分割项目：
- [添加新的模型](advanced_guides/add_models.md)
- - [添加新的数据集](advanced_guides/add_dataset.md)
- - [添加新的 transform](advanced_guides/add_transform.md)
+ - [添加新的数据集](advanced_guides/add_datasets.md)
+ - [添加新的 transform](advanced_guides/add_transforms.md)
- [自定义 runtime](advanced_guides/customize_runtime.md)
5. 如果您更熟悉 MMSegmentation v0.x , 以下是 MMSegmentation v0.x 迁移到 v1.x 的文档
diff --git a/docs/zh_cn/user_guides/2_dataset_prepare.md b/docs/zh_cn/user_guides/2_dataset_prepare.md
index c9c3606977d..5532624bef4 100644
--- a/docs/zh_cn/user_guides/2_dataset_prepare.md
+++ b/docs/zh_cn/user_guides/2_dataset_prepare.md
@@ -1,3 +1,750 @@
-## 准备数据集（待更新）
+# 教程2：准备数据集
-中文版文档支持中，请先阅读[英文版本](../../en/user_guides/2_dataset_prepare.md)
+我们建议将数据集根目录符号链接到 `$MMSEGMENTATION/data`。
+如果您的目录结构不同，您可能需要更改配置文件中相应的路径。
+对于中国境内的用户，我们也推荐通过开源数据平台 [OpenDataLab](https://opendatalab.com/) 来下载dsdl标准数据，以获得更好的下载和使用体验，这里有一个下载dsdl数据集并进行训练的案例[DSDLReadme](../../../configs/dsdl/README.md)，欢迎尝试。
+
+```none
+mmsegmentation
+├── mmseg
+├── tools
+├── configs
+├── data
+│ ├── cityscapes
+│ │ ├── leftImg8bit
+│ │ │ ├── train
+│ │ │ ├── val
+│ │ ├── gtFine
+│ │ │ ├── train
+│ │ │ ├── val
+│ ├── VOCdevkit
+│ │ ├── VOC2012
+│ │ │ ├── JPEGImages
+│ │ │ ├── SegmentationClass
+│ │ │ ├── ImageSets
+│ │ │ │ ├── Segmentation
+│ │ ├── VOC2010
+│ │ │ ├── JPEGImages
+│ │ │ ├── SegmentationClassContext
+│ │ │ ├── ImageSets
+│ │ │ │ ├── SegmentationContext
+│ │ │ │ │ ├── train.txt
+│ │ │ │ │ ├── val.txt
+│ │ │ ├── trainval_merged.json
+│ │ ├── VOCaug
+│ │ │ ├── dataset
+│ │ │ │ ├── cls
+│ ├── ade
+│ │ ├── ADEChallengeData2016
+│ │ │ ├── annotations
+│ │ │ │ ├── training
+│ │ │ │ ├── validation
+│ │ │ ├── images
+│ │ │ │ ├── training
+│ │ │ │ ├── validation
+│ ├── coco_stuff10k
+│ │ ├── images
+│ │ │ ├── train2014
+│ │ │ ├── test2014
+│ │ ├── annotations
+│ │ │ ├── train2014
+│ │ │ ├── test2014
+│ │ ├── imagesLists
+│ │ │ ├── train.txt
+│ │ │ ├── test.txt
+│ │ │ ├── all.txt
+│ ├── coco_stuff164k
+│ │ ├── images
+│ │ │ ├── train2017
+│ │ │ ├── val2017
+│ │ ├── annotations
+│ │ │ ├── train2017
+│ │ │ ├── val2017
+│ ├── CHASE_DB1
+│ │ ├── images
+│ │ │ ├── training
+│ │ │ ├── validation
+│ │ ├── annotations
+│ │ │ ├── training
+│ │ │ ├── validation
+│ ├── DRIVE
+│ │ ├── images
+│ │ │ ├── training
+│ │ │ ├── validation
+│ │ ├── annotations
+│ │ │ ├── training
+│ │ │ ├── validation
+│ ├── HRF
+│ │ ├── images
+│ │ │ ├── training
+│ │ │ ├── validation
+│ │ ├── annotations
+│ │ │ ├── training
+│ │ │ ├── validation
+│ ├── STARE
+│ │ ├── images
+│ │ │ ├── training
+│ │ │ ├── validation
+│ │ ├── annotations
+│ │ │ ├── training
+│ │ │ ├── validation
+| ├── dark_zurich
+| │ ├── gps
+| │ │ ├── val
+| │ │ └── val_ref
+| │ ├── gt
+| │ │ └── val
+| │ ├── LICENSE.txt
+| │ ├── lists_file_names
+| │ │ ├── val_filenames.txt
+| │ │ └── val_ref_filenames.txt
+| │ ├── README.md
+| │ └── rgb_anon
+| │ | ├── val
+| │ | └── val_ref
+| ├── NighttimeDrivingTest
+| | ├── gtCoarse_daytime_trainvaltest
+| | │ └── test
+| | │ └── night
+| | └── leftImg8bit
+| | | └── test
+| | | └── night
+│ ├── loveDA
+│ │ ├── img_dir
+│ │ │ ├── train
+│ │ │ ├── val
+│ │ │ ├── test
+│ │ ├── ann_dir
+│ │ │ ├── train
+│ │ │ ├── val
+│ ├── potsdam
+│ │ ├── img_dir
+│ │ │ ├── train
+│ │ │ ├── val
+│ │ ├── ann_dir
+│ │ │ ├── train
+│ │ │ ├── val
+│ ├── vaihingen
+│ │ ├── img_dir
+│ │ │ ├── train
+│ │ │ ├── val
+│ │ ├── ann_dir
+│ │ │ ├── train
+│ │ │ ├── val
+│ ├── iSAID
+│ │ ├── img_dir
+│ │ │ ├── train
+│ │ │ ├── val
+│ │ │ ├── test
+│ │ ├── ann_dir
+│ │ │ ├── train
+│ │ │ ├── val
+│ ├── synapse
+│ │ ├── img_dir
+│ │ │ ├── train
+│ │ │ ├── val
+│ │ ├── ann_dir
+│ │ │ ├── train
+│ │ │ ├── val
+│ ├── REFUGE
+│ │ ├── images
+│ │ │ ├── training
+│ │ │ ├── validation
+│ │ │ ├── test
+│ │ ├── annotations
+│ │ │ ├── training
+│ │ │ ├── validation
+│ │ │ ├── test
+│ ├── mapillary
+│ │ ├── training
+│ │ │ ├── images
+│ │ │ ├── v1.2
+| │ │ │ ├── instances
+| │ │ │ ├── labels
+| │ │ │ └── panoptic
+│ │ │ ├── v2.0
+| │ │ │ ├── instances
+| │ │ │ ├── labels
+| │ │ │ ├── panoptic
+| │ │ │ └── polygons
+│ │ ├── validation
+│ │ │ ├── images
+| │ │ ├── v1.2
+| │ │ │ ├── instances
+| │ │ │ ├── labels
+| │ │ │ └── panoptic
+│ │ │ ├── v2.0
+| │ │ │ ├── instances
+| │ │ │ ├── labels
+| │ │ │ ├── panoptic
+| │ │ │ └── polygons
+│ ├── bdd100k
+│ │ ├── images
+│ │ │ └── 10k
+| │ │ │ ├── test
+| │ │ │ ├── train
+| │ │ │ └── val
+│ │ └── labels
+│ │ │ └── sem_seg
+| │ │ │ ├── colormaps
+| │ │ │ │ ├──train
+| │ │ │ │ └──val
+| │ │ │ ├── masks
+| │ │ │ │ ├──train
+| │ │ │ │ └──val
+| │ │ │ ├── polygons
+| │ │ │ │ ├──sem_seg_train.json
+| │ │ │ │ └──sem_seg_val.json
+| │ │ │ └── rles
+| │ │ │ │ ├──sem_seg_train.json
+| │ │ │ │ └──sem_seg_val.json
+│ ├── nyu
+│ │ ├── images
+│ │ │ ├── train
+│ │ │ ├── test
+│ │ ├── annotations
+│ │ │ ├── train
+│ │ │ ├── test
+```
+
+## 用 MIM 下载数据集
+
+通过使用 [OpenXLab](https://openxlab.org.cn/datasets)，您可以直接下载开源数据集。通过平台的搜索功能，您可以快速轻松地找到他们正在寻找的数据集。使用平台上的格式化数据集，您可以高效地跨数据集执行任务。
+
+如果您使用 MIM 下载，请确保版本大于 v0.3.8。您可以使用以下命令进行更新、安装、登录和数据集下载：
+
+```shell
+# upgrade your MIM
+pip install -U openmim
+
+# install OpenXLab CLI tools
+pip install -U openxlab
+# log in OpenXLab
+openxlab login
+
+# download ADE20K by MIM
+mim download mmsegmentation --dataset ade20k
+```
+
+## Cityscapes
+
+Cityscapes [官方网站](https://www.cityscapes-dataset.com/)可以下载 Cityscapes 数据集，按照官网要求注册并登陆后，数据可以在[这里](https://www.cityscapes-dataset.com/downloads/)找到。
+
+按照惯例，`**labelTrainIds.png` 用于 cityscapes 训练。
+我们提供了一个基于 [cityscapesscripts](https://github.com/mcordts/cityscapesScripts) 的[脚本](https://github.com/open-mmlab/mmsegmentation/blob/1.x/tools/dataset_converters/cityscapes.py)用于生成 `**labelTrainIds.png`。
+
+```shell
+# --nproc 表示 8 个转换进程，也可以省略。
+python tools/dataset_converters/cityscapes.py data/cityscapes --nproc 8
+```
+
+## Pascal VOC
+
+Pascal VOC 2012 可从[此处](http://host.robots.ox.ac.uk/pascal/VOC/voc2012/VOCtrainval_11-May-2012.tar)下载。
+此外，Pascal VOC 数据集的最新工作通常利用额外的增强数据，可以在[这里](http://www.eecs.berkeley.edu/Research/Projects/CS/vision/grouping/semantic_contours/benchmark.tgz)找到。
+
+如果您想使用增强的 VOC 数据集，请运行以下命令将增强数据的标注转换为正确的格式。
+
+```shell
+# --nproc 表示 8 个转换进程，也可以省略。
+python tools/dataset_converters/voc_aug.py data/VOCdevkit data/VOCdevkit/VOCaug --nproc 8
+```
+
+请参考[拼接数据集文档](../advanced_guides/add_datasets.md#拼接数据集)及 [voc_aug 配置示例](../../../configs/_base_/datasets/pascal_voc12_aug.py)以详细了解如何将它们拼接并合并训练。
+
+## ADE20K
+
+ADE20K 的训练和验证集可以从这个[链接](http://data.csail.mit.edu/places/ADEchallenge/ADEChallengeData2016.zip)下载。
+如果需要下载测试数据集，可以在[官网](http://host.robots.ox.ac.uk/)注册后，下载[测试集](http://host.robots.ox.ac.uk:8080/eval/downloads/VOC2010test.tar)。
+
+## Pascal Context
+
+Pascal Context 的训练和验证集可以从[此处](http://host.robots.ox.ac.uk/pascal/VOC/voc2010/VOCtrainval_03-May-2010.tar)下载。注册后，您也可以从[此处](http://host.robots.ox.ac.uk:8080/eval/downloads/VOC2010test.tar)下载测试集。
+
+从原始数据集中抽出部分数据作为验证集，您可以从[此处](https://codalabuser.blob.core.windows.net/public/trainval_merged.json)下载 trainval_merged.json 文件。
+
+请先安装 [Detail](https://github.com/zhanghang1989/detail-api) 工具然后运行以下命令将标注转换为正确的格式。
+
+```shell
+python tools/dataset_converters/pascal_context.py data/VOCdevkit data/VOCdevkit/VOC2010/trainval_merged.json
+```
+
+## COCO Stuff 10k
+
+数据可以通过 wget 在[这里](http://calvin.inf.ed.ac.uk/wp-content/uploads/data/cocostuffdataset/cocostuff-10k-v1.1.zip)下载。
+
+对于 COCO Stuff 10k 数据集，请运行以下命令下载并转换数据集。
+
+```shell
+# 下载
+mkdir coco_stuff10k && cd coco_stuff10k
+wget http://calvin.inf.ed.ac.uk/wp-content/uploads/data/cocostuffdataset/cocostuff-10k-v1.1.zip
+
+# 解压
+unzip cocostuff-10k-v1.1.zip
+
+# --nproc 表示 8 个转换进程，也可以省略。
+python tools/dataset_converters/coco_stuff10k.py /path/to/coco_stuff10k --nproc 8
+```
+
+按照惯例，`/path/to/coco_stuff164k/annotations/*2014/*_labelTrainIds.png` 中的 mask 标注用于 COCO Stuff 10k 的训练和测试。
+
+## COCO Stuff 164k
+
+对于 COCO Stuff 164k 数据集，请运行以下命令下载并转换增强的数据集。
+
+```shell
+# 下载
+mkdir coco_stuff164k && cd coco_stuff164k
+wget http://images.cocodataset.org/zips/train2017.zip
+wget http://images.cocodataset.org/zips/val2017.zip
+wget http://calvin.inf.ed.ac.uk/wp-content/uploads/data/cocostuffdataset/stuffthingmaps_trainval2017.zip
+
+# 解压
+unzip train2017.zip -d images/
+unzip val2017.zip -d images/
+unzip stuffthingmaps_trainval2017.zip -d annotations/
+
+# --nproc 表示 8 个转换进程，也可以省略。
+python tools/dataset_converters/coco_stuff164k.py /path/to/coco_stuff164k --nproc 8
+```
+
+按照惯例，`/path/to/coco_stuff164k/annotations/*2017/*_labelTrainIds.png` 中的 mask 标注用于 COCO Stuff 164k 的训练和测试。
+
+此数据集的详细信息可在[此处](https://github.com/nightrome/cocostuff#downloads)找到。
+
+## CHASE DB1
+
+CHASE DB1 的训练和验证集可以从[此处](https://staffnet.kingston.ac.uk/~ku15565/CHASE_DB1/assets/CHASEDB1.zip)下载。
+
+请运行以下命令，准备 CHASE DB1 数据集：
+
+```shell
+python tools/dataset_converters/chase_db1.py /path/to/CHASEDB1.zip
+```
+
+该脚本将自动调整数据集目录结构，使其满足 MMSegmentation 数据集加载要求。
+
+## DRIVE
+
+按照[官网](https://drive.grand-challenge.org/)要求，注册并登陆后，便可以下载 DRIVE 的训练和验证数据集。
+
+要将 DRIVE 数据集转换为 MMSegmentation 的格式，请运行以下命令：
+
+```shell
+python tools/dataset_converters/drive.py /path/to/training.zip /path/to/test.zip
+```
+
+该脚本将自动调整数据集目录结构，使其满足 MMSegmentation 数据集加载要求。
+
+## HRF
+
+请下载 [health.zip](https://www5.cs.fau.de/fileadmin/research/datasets/fundus-images/healthy.zip)、[glaucoma.zip](https://www5.cs.fau.de/fileadmin/research/datasets/fundus-images/glaucoma.zip)、[diabetic_retinopathy.zip](https://www5.cs.fau.de/fileadmin/research/datasets/fundus-images/diabetic_retinopathy.zip)、[healthy_manualsegm.zip](https://www5.cs.fau.de/fileadmin/research/datasets/fundus-images/healthy_manualsegm.zip)、[glaucoma_manualsegm.zip](https://www5.cs.fau.de/fileadmin/research/datasets/fundus-images/glaucoma_manualsegm.zip) 和 [diabetic_retinopathy_manualsegm.zip](https://www5.cs.fau.de/fileadmin/research/datasets/fundus-images/diabetic_retinopathy_manualsegm.zip)，无需解压，可以直接运行以下命令，准备 HRF 数据集：
+
+```shell
+python tools/dataset_converters/hrf.py /path/to/healthy.zip /path/to/healthy_manualsegm.zip /path/to/glaucoma.zip /path/to/glaucoma_manualsegm.zip /path/to/diabetic_retinopathy.zip /path/to/diabetic_retinopathy_manualsegm.zip
+```
+
+该脚本将自动调整数据集目录结构，使其满足 MMSegmentation 数据集加载要求。
+
+## STARE
+
+请下载 [stare images.tar](http://cecas.clemson.edu/~ahoover/stare/probing/stare-images.tar)、[labels-ah.tar](http://cecas.clemson.edu/~ahoover/stare/probing/labels-ah.tar) 和 [labels-vk.tar](http://cecas.clemson.edu/~ahoover/stare/probing/labels-vk.tar)，无需解压，可以直接运行以下命令，准备 STARE 数据集：
+
+```shell
+python tools/dataset_converters/stare.py /path/to/stare-images.tar /path/to/labels-ah.tar /path/to/labels-vk.tar
+```
+
+该脚本将自动调整数据集目录结构，使其满足 MMSegmentation 数据集加载要求。
+
+## Dark Zurich
+
+由于我们只支持在此数据集上的模型测试，因此您只需要下载并解压[验证数据集](https://data.vision.ee.ethz.ch/csakarid/shared/GCMA_UIoU/Dark_Zurich_val_anon.zip)。
+
+## Nighttime Driving
+
+由于我们只支持在此数据集上的模型测试，因此您只需要下载并解压[验证数据集](http://data.vision.ee.ethz.ch/daid/NighttimeDriving/NighttimeDrivingTest.zip)。
+
+## LoveDA
+
+数据可以从[此处](https://drive.google.com/drive/folders/1ibYV0qwn4yuuh068Rnc-w4tPi0U0c-ti?usp=sharing)下载 LaveDA 数据集。
+
+或者可以从 [zenodo](https://zenodo.org/record/5706578#.YZvN7SYRXdF) 下载。下载后，无需解压，直接运行以下命令：
+
+```shell
+# 下载 Train.zip
+wget https://zenodo.org/record/5706578/files/Train.zip
+# 下载 Val.zip
+wget https://zenodo.org/record/5706578/files/Val.zip
+# 下载 Test.zip
+wget https://zenodo.org/record/5706578/files/Test.zip
+```
+
+请对于 LoveDA 数据集，请运行以下命令调整数据集目录。
+
+```shell
+python tools/dataset_converters/loveda.py /path/to/loveDA
+```
+
+可将模型对 LoveDA 的测试集的预测结果上传至到数据集[测试服务器](https://codalab.lisn.upsaclay.fr/competitions/421)，查看评测结果。
+
+有关 LoveDA 的更多详细信息，可查看[此处](https://github.com/Junjue-Wang/LoveDA).
+
+## ISPRS Potsdam
+
+[Potsdam](https://www.isprs.org/education/benchmarks/UrbanSemLab/2d-sem-label-potsdam.aspx) 城市语义分割数据集用于 2D 语义分割竞赛 —— Potsdam。
+
+数据集可以在竞赛[主页](https://www.isprs.org/education/benchmarks/UrbanSemLab/default.aspx)上请求获得。
+这里也提供了[BaiduNetdisk](https://pan.baidu.com/s/1K-cLVZnd1X7d8c26FQ-nGg?pwd=mseg)，提取码：mseg、 [Google Drive](https://drive.google.com/drive/folders/1w3EJuyUGet6_qmLwGAWZ9vw5ogeG0zLz?usp=sharing)以及[OpenDataLab](https://opendatalab.com/ISPRS_Potsdam/download)。
+实验中需要下载 '2_Ortho_RGB.zip' 和 '5_Labels_all_noBoundary.zip'。
+
+对于 Potsdam 数据集，请运行以下命令调整数据集目录。
+
+```shell
+python tools/dataset_converters/potsdam.py /path/to/potsdam
+```
+
+在我们的默认设置中，将生成 3456 张图像用于训练和 2016 张图像用于验证。
+
+## ISPRS Vaihingen
+
+[Vaihingen](https://www.isprs.org/education/benchmarks/UrbanSemLab/2d-sem-label-vaihingen.aspx) 城市语义分割数据集用于 2D 语义分割竞赛 —— Vaihingen。
+
+数据集可以在竞赛[主页](https://www.isprs.org/education/benchmarks/UrbanSemLab/default.aspx)上请求获得。
+这里也提供了[BaiduNetdisk](https://pan.baidu.com/s/109D3WLrLafsuYtLeerLiiA?pwd=mseg)，提取码：mseg 、 [Google Drive](https://drive.google.com/drive/folders/1w3NhvLVA2myVZqOn2pbiDXngNC7NTP_t?usp=sharing)。
+实验中需要下载 'ISPRS_semantic_labeling_Vaihingen.zip' 和 'ISPRS_semantic_labeling_Vaihingen_ground_truth_eroded_COMPLETE.zip'。
+
+对于 Vaihingen 数据集，请运行以下命令调整数据集目录。
+
+```shell
+python tools/dataset_converters/vaihingen.py /path/to/vaihingen
+```
+
+在我们的默认设置（`clip_size`=512, `stride_size`=256）中，将生成 344 张图像用于训练和 398 张图像用于验证。
+
+## iSAID
+
+iSAID 数据集可从 [DOTA-v1.0](https://captain-whu.github.io/DOTA/dataset.html) 下载训练/验证/测试数据集的图像数据，
+
+并从 [iSAID](https://captain-whu.github.io/iSAID/dataset.html)下载训练/验证数据集的标注数据。
+
+该数据集是航空图像实例分割和语义分割任务的大规模数据集。
+
+下载 iSAID 数据集后，您可能需要按照以下结构进行数据集准备。
+
+```none
+├── data
+│ ├── iSAID
+│ │ ├── train
+│ │ │ ├── images
+│ │ │ │ ├── part1.zip
+│ │ │ │ ├── part2.zip
+│ │ │ │ ├── part3.zip
+│ │ │ ├── Semantic_masks
+│ │ │ │ ├── images.zip
+│ │ ├── val
+│ │ │ ├── images
+│ │ │ │ ├── part1.zip
+│ │ │ ├── Semantic_masks
+│ │ │ │ ├── images.zip
+│ │ ├── test
+│ │ │ ├── images
+│ │ │ │ ├── part1.zip
+│ │ │ │ ├── part2.zip
+```
+
+```shell
+python tools/dataset_converters/isaid.py /path/to/iSAID
+```
+
+在我们的默认设置（`patch_width`=896, `patch_height`=896, `overlap_area`=384）中，将生成 33978 张图像用于训练和 11644 张图像用于验证。
+
+## LIP(Look Into Person) dataset
+
+该数据集可以从[此页面](https://lip.sysuhcp.com/overview.php)下载。
+
+请运行以下命令来解压数据集。
+
+```shell
+unzip LIP.zip
+cd LIP
+unzip TrainVal_images.zip
+unzip TrainVal_parsing_annotations.zip
+cd TrainVal_parsing_annotations
+unzip TrainVal_parsing_annotations.zip
+mv train_segmentations ../
+mv val_segmentations ../
+cd ..
+```
+
+LIP 数据集的内容包括：
+
+```none
+├── data
+│ ├── LIP
+│ │ ├── train_images
+│ │ │ ├── 1000_1234574.jpg
+│ │ │ ├── ...
+│ │ ├── train_segmentations
+│ │ │ ├── 1000_1234574.png
+│ │ │ ├── ...
+│ │ ├── val_images
+│ │ │ ├── 100034_483681.jpg
+│ │ │ ├── ...
+│ │ ├── val_segmentations
+│ │ │ ├── 100034_483681.png
+│ │ │ ├── ...
+```
+
+## Synapse dataset
+
+此数据集可以从[此页面](https://www.synapse.org/#!Synapse:syn3193805/wiki/)下载。
+
+遵循 [TransUNet](https://arxiv.org/abs/2102.04306) 的数据准备设定，将原始训练集（30 次扫描）拆分为新的训练集（18 次扫描）和验证集（12 次扫描）。请运行以下命令来准备数据集。
+
+```shell
+unzip RawData.zip
+cd ./RawData/Training
+```
+
+然后创建 `train.txt` 和 `val.txt` 以拆分数据集。
+
+根据 TransUnet，以下是数据集的划分。
+
+train.txt
+
+```none
+img0005.nii.gz
+img0006.nii.gz
+img0007.nii.gz
+img0009.nii.gz
+img0010.nii.gz
+img0021.nii.gz
+img0023.nii.gz
+img0024.nii.gz
+img0026.nii.gz
+img0027.nii.gz
+img0028.nii.gz
+img0030.nii.gz
+img0031.nii.gz
+img0033.nii.gz
+img0034.nii.gz
+img0037.nii.gz
+img0039.nii.gz
+img0040.nii.gz
+```
+
+val.txt
+
+```none
+img0008.nii.gz
+img0022.nii.gz
+img0038.nii.gz
+img0036.nii.gz
+img0032.nii.gz
+img0002.nii.gz
+img0029.nii.gz
+img0003.nii.gz
+img0001.nii.gz
+img0004.nii.gz
+img0025.nii.gz
+img0035.nii.gz
+```
+
+synapse 数据集的内容包括：
+
+```none
+├── Training
+│ ├── img
+│ │ ├── img0001.nii.gz
+│ │ ├── img0002.nii.gz
+│ │ ├── ...
+│ ├── label
+│ │ ├── label0001.nii.gz
+│ │ ├── label0002.nii.gz
+│ │ ├── ...
+│ ├── train.txt
+│ ├── val.txt
+```
+
+然后，使用此命令转换 synapse 数据集。
+
+```shell
+python tools/dataset_converters/synapse.py --dataset-path /path/to/synapse
+```
+
+注意，MMSegmentation 的默认评估指标（例如 mean dice value）是在 2D 切片图像上计算的，这与 [TransUNet](https://arxiv.org/abs/2102.04306) 等一些论文中的 3D 扫描结果是不同的。
+
+## REFUGE
+
+在 [REFUGE Challenge](https://refuge.grand-challenge.org) 官网上注册并下载 [REFUGE 数据集](https://refuge.grand-challenge.org/REFUGE2Download)。
+
+然后，解压 `REFUGE2.zip`，原始数据集的内容包括：
+
+```none
+├── REFUGE2
+│ ├── REFUGE2
+│ │ ├── Annotation-Training400.zip
+│ │ ├── REFUGE-Test400.zip
+│ │ ├── REFUGE-Test-GT.zip
+│ │ ├── REFUGE-Training400.zip
+│ │ ├── REFUGE-Validation400.zip
+│ │ ├── REFUGE-Validation400-GT.zip
+│ ├── __MACOSX
+```
+
+请运行以下命令转换 REFUGE 数据集：
+
+```shell
+python tools/convert_datasets/refuge.py --raw_data_root=/path/to/refuge/REFUGE2/REFUGE2
+```
+
+脚本会将目录结构转换如下：
+
+```none
+│ ├── REFUGE
+│ │ ├── images
+│ │ │ ├── training
+│ │ │ ├── validation
+│ │ │ ├── test
+│ │ ├── annotations
+│ │ │ ├── training
+│ │ │ ├── validation
+│ │ │ ├── test
+```
+
+包含 400 张用于训练的图像、400 张用于验证的图像和 400 张用于测试的图像，这与 REFUGE 2018 数据集相同。
+
+## Mapillary Vistas Datasets
+
+- Mapillary Vistas [官方网站](https://www.mapillary.com/dataset/vistas) 可以下载 Mapillary Vistas 数据集，按照官网要求注册并登陆后，数据可以在[这里](https://www.mapillary.com/dataset/vistas)找到。
+
+- Mapillary Vistas 数据集使用 8-bit with color-palette 来存储标签。不需要进行转换操作。
+
+- 假设您已将数据集 zip 文件放在 `mmsegmentation/data/mapillary` 中
+
+- 请运行以下命令来解压数据集。
+
+ ```bash
+ cd data/mapillary
+ unzip An-ZjB1Zm61yAZG0ozTymz8I8NqI4x0MrYrh26dq7kPgfu8vf9ImrdaOAVOFYbJ2pNAgUnVGBmbue9lTgdBOb5BbKXIpFs0fpYWqACbrQDChAA2fdX0zS9PcHu7fY8c-FOvyBVxPNYNFQuM.zip
+ ```
+
+- 解压后，您将获得类似于此结构的 Mapillary Vistas 数据集。语义分割 mask 标签在 `labels` 文件夹中。
+
+ ```none
+ mmsegmentation
+ ├── mmseg
+ ├── tools
+ ├── configs
+ ├── data
+ │ ├── mapillary
+ │ │ ├── training
+ │ │ │ ├── images
+ │ │ │ ├── v1.2
+ | │ │ │ ├── instances
+ | │ │ │ ├── labels
+ | │ │ │ └── panoptic
+ │ │ │ ├── v2.0
+ | │ │ │ ├── instances
+ | │ │ │ ├── labels
+ | │ │ │ ├── panoptic
+ | │ │ │ └── polygons
+ │ │ ├── validation
+ │ │ │ ├── images
+ | │ │ ├── v1.2
+ | │ │ │ ├── instances
+ | │ │ │ ├── labels
+ | │ │ │ └── panoptic
+ │ │ │ ├── v2.0
+ | │ │ │ ├── instances
+ | │ │ │ ├── labels
+ | │ │ │ ├── panoptic
+ | │ │ │ └── polygons
+ ```
+
+- 您可以在配置中使用 `MapillaryDataset_v1` 和 `Mapillary Dataset_v2` 设置数据集版本。
+ 在此处 [V1.2](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/_base_/datasets/mapillary_v1.py) 和 [V2.0](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/_base_/datasets/mapillary_v2.py) 查看 Mapillary Vistas 数据集配置文件
+
+## LEVIR-CD
+
+[LEVIR-CD](https://justchenhao.github.io/LEVIR/) 大规模遥感建筑变化检测数据集。
+
+数据集可以在[主页](https://justchenhao.github.io/LEVIR/)上请求获得。
+
+数据集的补充版本可以在[主页](https://github.com/S2Looking/Dataset)上请求获得。
+
+请下载数据集的补充版本，然后解压 `LEVIR-CD+.zip`，数据集的内容包括：
+
+```none
+│ ├── LEVIR-CD+
+│ │ ├── train
+│ │ │ ├── A
+│ │ │ ├── B
+│ │ │ ├── label
+│ │ ├── test
+│ │ │ ├── A
+│ │ │ ├── B
+│ │ │ ├── label
+```
+
+对于 LEVIR-CD 数据集，请运行以下命令无重叠裁剪影像：
+
+```shell
+python tools/dataset_converters/levircd.py --dataset-path /path/to/LEVIR-CD+ --out_dir /path/to/LEVIR-CD
+```
+
+裁剪后的影像大小为256x256，与原论文保持一致。
+
+## BDD100K
+
+- 可以从[官方网站](https://bdd-data.berkeley.edu/) 下载 BDD100K数据集（语义分割任务主要是10K数据集），按照官网要求注册并登陆后，数据可以在[这里](https://bdd-data.berkeley.edu/portal.html#download)找到。
+
+- 图像数据对应的名称是是`10K Images`, 语义分割标注对应的名称是`Segmentation`
+
+- 下载后，可以使用以下代码进行解压
+
+ ```bash
+ unzip ~/bdd100k_images_10k.zip -d ~/mmsegmentation/data/
+ unzip ~/bdd100k_sem_seg_labels_trainval.zip -d ~/mmsegmentation/data/
+ ```
+
+就可以得到以下文件结构了：
+
+```none
+mmsegmentation
+├── mmseg
+├── tools
+├── configs
+├── data
+│ ├── bdd100k
+│ │ ├── images
+│ │ │ └── 10k
+| │ │ │ ├── test
+| │ │ │ ├── train
+| │ │ │ └── val
+│ │ └── labels
+│ │ │ └── sem_seg
+| │ │ │ ├── colormaps
+| │ │ │ │ ├──train
+| │ │ │ │ └──val
+| │ │ │ ├── masks
+| │ │ │ │ ├──train
+| │ │ │ │ └──val
+| │ │ │ ├── polygons
+| │ │ │ │ ├──sem_seg_train.json
+| │ │ │ │ └──sem_seg_val.json
+| │ │ │ └── rles
+| │ │ │ │ ├──sem_seg_train.json
+| │ │ │ │ └──sem_seg_val.json
+```
+
+## NYU
+
+- 您可以从 [这个链接](https://drive.google.com/file/d/1wC-io-14RCIL4XTUrQLk6lBqU2AexLVp/view?usp=share_link) 下载 NYU 数据集
+
+- 下载完成后，您可以使用 [tools/dataset_converters/nyu.py](/tools/dataset_converters/nyu.py) 脚本来解压和组织数据到所需的格式
+
+ ```bash
+ python tools/dataset_converters/nyu.py nyu.zip
+ ```
diff --git a/docs/zh_cn/user_guides/3_inference.md b/docs/zh_cn/user_guides/3_inference.md
index d2fe60076f8..0afcb4b05d6 100644
--- a/docs/zh_cn/user_guides/3_inference.md
+++ b/docs/zh_cn/user_guides/3_inference.md
@@ -1,3 +1,244 @@
-## 使用预训练模型推理（待更新）
+# 教程3：使用预训练模型推理
-中文版文档支持中，请先阅读[英文版本](../../en/user_guides/3_inference.md)
+MMSegmentation 在 [Model Zoo](../Model_Zoo.md) 中为语义分割提供了预训练的模型，并支持多个标准数据集，包括 Cityscapes、ADE20K 等。
+本说明将展示如何使用现有模型对给定图像进行推理。
+关于如何在标准数据集上测试现有模型，请参阅本[指南](./4_train_test.md)
+
+MMSegmentation 为用户提供了数个接口，以便轻松使用预训练的模型进行推理。
+
+- [教程3：使用预训练模型推理](#教程3使用预训练模型推理)
+ - [推理器](#推理器)
+ - [基本使用](#基本使用)
+ - [初始化](#初始化)
+ - [可视化预测结果](#可视化预测结果)
+ - [模型列表](#模型列表)
+ - [推理 API](#推理-api)
+ - [mmseg.apis.init_model](#mmsegapisinit_model)
+ - [mmseg.apis.inference_model](#mmsegapisinference_model)
+ - [mmseg.apis.show_result_pyplot](#mmsegapisshow_result_pyplot)
+
+## 推理器
+
+在 MMSegmentation 中，我们提供了最**方便的**方式 `MMSegInferencer` 来使用模型。您只需 3 行代码就可以获得图像的分割掩膜。
+
+### 基本使用
+
+以下示例展示了如何使用 `MMSegInferencer` 对单个图像执行推理。
+
+```
+>>> from mmseg.apis import MMSegInferencer
+>>> # 将模型加载到内存中
+>>> inferencer = MMSegInferencer(model='deeplabv3plus_r18-d8_4xb2-80k_cityscapes-512x1024')
+>>> # 推理
+>>> inferencer('demo/demo.png', show=True)
+```
+
+可视化结果应如下所示：
+
+
+
+
Abstract
+
+We address the problem of estimating a high quality dense depth map from a single RGB input image. We start out with a baseline encoder-decoder convolutional neural network architecture and pose the question of how the global processing of information can help improve overall depth estimation. To this end, we propose a transformer-based architecture block that divides the depth range into bins whose center value is estimated adaptively per image. The final depth values are estimated as linear combinations of the bin centers. We call our new building block AdaBins. Our results show a decisive improvement over the state-of-the-art on several popular depth datasets across all metrics.We also validate the effectiveness of the proposed block with an ablation study and provide the code and corresponding pre-trained weights of the new state-of-the-art model.
+
+Our main contributions are the following:
+
+- We propose an architecture building block that performs global processing of the scene’s information.We propose to divide the predicted depth range into bins where the bin widths change per image. The final depth estimation is a linear combination of the bin center values.
+- We show a decisive improvement for supervised single image depth estimation across all metrics for the two most popular datasets, NYU and KITTI.
+- We analyze our findings and investigate different modifications on the proposed AdaBins block and study their effect on the accuracy of the depth estimation.
+
+
+
+
Performance
+
+### NYU and KITTI
+
+| Model | Encoder | Training epoch | Batchsize | Train Resolution | δ1 | δ2 | δ3 | REL | RMS | RMS log | params(M) | Links |
+| ------------- | --------------- | -------------- | --------- | ---------------- | ----- | ----- | ----- | ----- | ----- | ------- | --------- | ----------------------------------------------------------------------------------------------------------------------- |
+| AdaBins_nyu | EfficientNet-B5 | 25 | 16 | 416x544 | 0.903 | 0.984 | 0.997 | 0.103 | 0.364 | 0.044 | 78 | [model](https://download.openmmlab.com/mmsegmentation/v0.5/adabins/adabins_efficient_b5_nyu_third-party-f68d6bd3.pth) |
+| AdaBins_kitti | EfficientNet-B5 | 25 | 16 | 352x764 | 0.964 | 0.995 | 0.999 | 0.058 | 2.360 | 0.088 | 78 | [model](https://download.openmmlab.com/mmsegmentation/v0.5/adabins/adabins_efficient-b5_kitty_third-party-a1aa6f36.pth) |
+
+## Citation
+
+```bibtex
+@article{10.1109/cvpr46437.2021.00400,
+ author = {Bhat, S. A. and Alhashim, I. and Wonka, P.},
+ title = {Adabins: depth estimation using adaptive bins},
+ journal = {2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
+ year = {2021},
+ doi = {10.1109/cvpr46437.2021.00400}
+}
+```
diff --git a/projects/Adabins/backbones/__init__.py b/projects/Adabins/backbones/__init__.py
new file mode 100644
index 00000000000..04ae180be5f
--- /dev/null
+++ b/projects/Adabins/backbones/__init__.py
@@ -0,0 +1,4 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+from .adabins_backbone import AdabinsBackbone
+
+__all__ = ['AdabinsBackbone']
diff --git a/projects/Adabins/backbones/adabins_backbone.py b/projects/Adabins/backbones/adabins_backbone.py
new file mode 100644
index 00000000000..07d73809e39
--- /dev/null
+++ b/projects/Adabins/backbones/adabins_backbone.py
@@ -0,0 +1,141 @@
+import timm
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+from mmcv.cnn import ConvModule, build_conv_layer
+from mmengine.model import BaseModule
+
+from mmseg.registry import MODELS
+
+
+class UpSampleBN(nn.Module):
+ """ UpSample module
+ Args:
+ skip_input (int): the input feature
+ output_features (int): the output feature
+ norm_cfg (dict, optional): Config dict for normalization layer.
+ Default: dict(type='BN', requires_grad=True).
+ act_cfg (dict, optional): The activation layer of AAM:
+ Aggregate Attention Module.
+ """
+
+ def __init__(self,
+ skip_input,
+ output_features,
+ norm_cfg=dict(type='BN'),
+ act_cfg=dict(type='LeakyReLU')):
+ super().__init__()
+
+ self._net = nn.Sequential(
+ ConvModule(
+ in_channels=skip_input,
+ out_channels=output_features,
+ kernel_size=3,
+ stride=1,
+ padding=1,
+ bias=True,
+ norm_cfg=norm_cfg,
+ act_cfg=act_cfg,
+ ),
+ ConvModule(
+ in_channels=output_features,
+ out_channels=output_features,
+ kernel_size=3,
+ stride=1,
+ padding=1,
+ bias=True,
+ norm_cfg=norm_cfg,
+ act_cfg=act_cfg,
+ ))
+
+ def forward(self, x, concat_with):
+ up_x = F.interpolate(
+ x,
+ size=[concat_with.size(2),
+ concat_with.size(3)],
+ mode='bilinear',
+ align_corners=True)
+ f = torch.cat([up_x, concat_with], dim=1)
+ return self._net(f)
+
+
+class Encoder(nn.Module):
+ """ the efficientnet_b5 model
+ Args:
+ basemodel_name (str): the name of base model
+ """
+
+ def __init__(self, basemodel_name):
+ super().__init__()
+ self.original_model = timm.create_model(
+ basemodel_name, pretrained=True)
+ # Remove last layer
+ self.original_model.global_pool = nn.Identity()
+ self.original_model.classifier = nn.Identity()
+
+ def forward(self, x):
+ features = [x]
+ for k, v in self.original_model._modules.items():
+ if k == 'blocks':
+ for ki, vi in v._modules.items():
+ features.append(vi(features[-1]))
+ else:
+ features.append(v(features[-1]))
+ return features
+
+
+@MODELS.register_module()
+class AdabinsBackbone(BaseModule):
+ """ the backbone of the adabins
+ Args:
+ basemodel_name (str):the name of base model
+ num_features (int): the middle feature
+ num_classes (int): the classes number
+ bottleneck_features (int): the bottleneck features
+ conv_cfg (dict): Config dict for convolution layer.
+ """
+
+ def __init__(self,
+ basemodel_name,
+ num_features=2048,
+ num_classes=128,
+ bottleneck_features=2048,
+ conv_cfg=dict(type='Conv')):
+ super().__init__()
+ self.encoder = Encoder(basemodel_name)
+ features = int(num_features)
+ self.conv2 = build_conv_layer(
+ conv_cfg,
+ bottleneck_features,
+ features,
+ kernel_size=1,
+ stride=1,
+ padding=1)
+ self.up1 = UpSampleBN(
+ skip_input=features // 1 + 112 + 64, output_features=features // 2)
+ self.up2 = UpSampleBN(
+ skip_input=features // 2 + 40 + 24, output_features=features // 4)
+ self.up3 = UpSampleBN(
+ skip_input=features // 4 + 24 + 16, output_features=features // 8)
+ self.up4 = UpSampleBN(
+ skip_input=features // 8 + 16 + 8, output_features=features // 16)
+
+ self.conv3 = build_conv_layer(
+ conv_cfg,
+ features // 16,
+ num_classes,
+ kernel_size=3,
+ stride=1,
+ padding=1)
+
+ def forward(self, x):
+ features = self.encoder(x)
+ x_block0, x_block1, x_block2, x_block3, x_block4 = features[
+ 3], features[4], features[5], features[7], features[10]
+ x_d0 = self.conv2(x_block4)
+ x_d1 = self.up1(x_d0, x_block3)
+ x_d2 = self.up2(x_d1, x_block2)
+ x_d3 = self.up3(x_d2, x_block1)
+ x_d4 = self.up4(x_d3, x_block0)
+ out = self.conv3(x_d4)
+ return out
diff --git a/projects/Adabins/configs/_base_/datasets/nyu.py b/projects/Adabins/configs/_base_/datasets/nyu.py
new file mode 100644
index 00000000000..1b49ec7e8de
--- /dev/null
+++ b/projects/Adabins/configs/_base_/datasets/nyu.py
@@ -0,0 +1,32 @@
+dataset_type = 'NYUDataset'
+data_root = 'data/nyu'
+
+test_pipeline = [
+ dict(dict(type='LoadImageFromFile', to_float32=True)),
+ dict(dict(type='LoadDepthAnnotation', depth_rescale_factor=1e-3)),
+ dict(
+ type='PackSegInputs',
+ meta_keys=('img_path', 'depth_map_path', 'ori_shape', 'img_shape',
+ 'pad_shape', 'scale_factor', 'flip', 'flip_direction',
+ 'category_id'))
+]
+
+val_dataloader = dict(
+ batch_size=1,
+ num_workers=4,
+ persistent_workers=True,
+ sampler=dict(type='DefaultSampler', shuffle=False),
+ dataset=dict(
+ type=dataset_type,
+ data_root=data_root,
+ test_mode=True,
+ data_prefix=dict(
+ img_path='images/test', depth_map_path='annotations/test'),
+ pipeline=test_pipeline))
+test_dataloader = val_dataloader
+
+val_evaluator = dict(
+ type='DepthMetric', max_depth_eval=10.0, crop_type='nyu_crop')
+test_evaluator = val_evaluator
+val_cfg = dict(type='ValLoop')
+test_cfg = dict(type='TestLoop')
diff --git a/projects/Adabins/configs/_base_/default_runtime.py b/projects/Adabins/configs/_base_/default_runtime.py
new file mode 100644
index 00000000000..272b4d24679
--- /dev/null
+++ b/projects/Adabins/configs/_base_/default_runtime.py
@@ -0,0 +1,15 @@
+default_scope = 'mmseg'
+env_cfg = dict(
+ cudnn_benchmark=True,
+ mp_cfg=dict(mp_start_method='fork', opencv_num_threads=0),
+ dist_cfg=dict(backend='nccl'),
+)
+vis_backends = [dict(type='LocalVisBackend')]
+visualizer = dict(
+ type='SegLocalVisualizer', vis_backends=vis_backends, name='visualizer')
+log_processor = dict(by_epoch=False)
+log_level = 'INFO'
+load_from = None
+resume = False
+
+tta_model = dict(type='SegTTAModel')
diff --git a/projects/Adabins/configs/_base_/models/Adabins.py b/projects/Adabins/configs/_base_/models/Adabins.py
new file mode 100644
index 00000000000..35cbd8c5777
--- /dev/null
+++ b/projects/Adabins/configs/_base_/models/Adabins.py
@@ -0,0 +1,35 @@
+# model settings
+norm_cfg = dict(type='SyncBN', requires_grad=True)
+data_preprocessor = dict(
+ type='SegDataPreProcessor',
+ mean=[123.675, 116.28, 103.53],
+ std=[58.395, 57.12, 57.375],
+ bgr_to_rgb=True,
+ pad_val=0,
+ seg_pad_val=255)
+model = dict(
+ type='DepthEstimator',
+ data_preprocessor=data_preprocessor,
+ # pretrained='open-mmlab://resnet50_v1c',
+ backbone=dict(
+ type='AdabinsBackbone',
+ basemodel_name='tf_efficientnet_b5_ap',
+ num_features=2048,
+ num_classes=128,
+ bottleneck_features=2048,
+ ),
+ decode_head=dict(
+ type='AdabinsHead',
+ in_channels=128,
+ n_query_channels=128,
+ patch_size=16,
+ embedding_dim=128,
+ num_heads=4,
+ n_bins=256,
+ min_val=0.001,
+ max_val=10,
+ norm='linear'),
+
+ # model training and testing settings
+ train_cfg=dict(),
+ test_cfg=dict(mode='whole'))
diff --git a/projects/Adabins/configs/adabins/adabins_efficient_b5_4x16_25e_NYU_416x544.py b/projects/Adabins/configs/adabins/adabins_efficient_b5_4x16_25e_NYU_416x544.py
new file mode 100644
index 00000000000..5c00ea152bf
--- /dev/null
+++ b/projects/Adabins/configs/adabins/adabins_efficient_b5_4x16_25e_NYU_416x544.py
@@ -0,0 +1,15 @@
+_base_ = [
+ '../_base_/models/Adabins.py', '../_base_/datasets/nyu.py',
+ '../_base_/default_runtime.py'
+]
+custom_imports = dict(
+ imports=['projects.Adabins.backbones', 'projects.Adabins.decode_head'],
+ allow_failed_imports=False)
+crop_size = (416, 544)
+data_preprocessor = dict(size=crop_size)
+norm_cfg = dict(type='SyncBN', requires_grad=True)
+model = dict(
+ data_preprocessor=data_preprocessor,
+ backbone=dict(),
+ decode_head=dict(),
+)
diff --git a/projects/Adabins/configs/adabins/adabins_efficient_b5_4x16_25e_kitti_352x704.py b/projects/Adabins/configs/adabins/adabins_efficient_b5_4x16_25e_kitti_352x704.py
new file mode 100644
index 00000000000..330cdf41a5b
--- /dev/null
+++ b/projects/Adabins/configs/adabins/adabins_efficient_b5_4x16_25e_kitti_352x704.py
@@ -0,0 +1,12 @@
+_base_ = ['../_base_/models/Adabins.py']
+custom_imports = dict(
+ imports=['projects.Adabins.backbones', 'projects.Adabins.decode_head'],
+ allow_failed_imports=False)
+crop_size = (352, 704)
+data_preprocessor = dict(size=crop_size)
+norm_cfg = dict(type='SyncBN', requires_grad=True)
+model = dict(
+ data_preprocessor=data_preprocessor,
+ backbone=dict(),
+ decode_head=dict(min_val=0.001, max_val=80),
+)
diff --git a/projects/Adabins/decode_head/__init__.py b/projects/Adabins/decode_head/__init__.py
new file mode 100644
index 00000000000..c7d62df12bb
--- /dev/null
+++ b/projects/Adabins/decode_head/__init__.py
@@ -0,0 +1,4 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+from .adabins_head import AdabinsHead
+
+__all__ = ['AdabinsHead']
diff --git a/projects/Adabins/decode_head/adabins_head.py b/projects/Adabins/decode_head/adabins_head.py
new file mode 100644
index 00000000000..ee043172ab9
--- /dev/null
+++ b/projects/Adabins/decode_head/adabins_head.py
@@ -0,0 +1,179 @@
+from typing import List, Tuple
+
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+from mmcv.cnn import build_conv_layer
+from torch import Tensor
+
+from mmseg.registry import MODELS
+
+
+class PatchTransformerEncoder(nn.Module):
+ """the Patch Transformer Encoder.
+
+ Args:
+ in_channels (int): the channels of input
+ patch_size (int): the path size
+ embedding_dim (int): The feature dimension.
+ num_heads (int): the number of encoder head
+ conv_cfg (dict): Config dict for convolution layer.
+ """
+
+ def __init__(self,
+ in_channels,
+ patch_size=10,
+ embedding_dim=128,
+ num_heads=4,
+ conv_cfg=dict(type='Conv')):
+ super().__init__()
+ encoder_layers = nn.TransformerEncoderLayer(
+ embedding_dim, num_heads, dim_feedforward=1024)
+ self.transformer_encoder = nn.TransformerEncoder(
+ encoder_layers, num_layers=4) # takes shape S,N,E
+
+ self.embedding_convPxP = build_conv_layer(
+ conv_cfg,
+ in_channels,
+ embedding_dim,
+ kernel_size=patch_size,
+ stride=patch_size)
+ self.positional_encodings = nn.Parameter(
+ torch.rand(500, embedding_dim), requires_grad=True)
+
+ def forward(self, x):
+ embeddings = self.embedding_convPxP(x).flatten(
+ 2) # .shape = n,c,s = n, embedding_dim, s
+ embeddings = embeddings + self.positional_encodings[:embeddings.shape[
+ 2], :].T.unsqueeze(0)
+
+ # change to S,N,E format required by transformer
+ embeddings = embeddings.permute(2, 0, 1)
+ x = self.transformer_encoder(embeddings) # .shape = S, N, E
+ return x
+
+
+class PixelWiseDotProduct(nn.Module):
+ """the pixel wise dot product."""
+
+ def __init__(self):
+ super().__init__()
+
+ def forward(self, x, K):
+ n, c, h, w = x.size()
+ _, cout, ck = K.size()
+ assert c == ck, 'Number of channels in x and Embedding dimension ' \
+ '(at dim 2) of K matrix must match'
+ y = torch.matmul(
+ x.view(n, c, h * w).permute(0, 2, 1),
+ K.permute(0, 2, 1)) # .shape = n, hw, cout
+ return y.permute(0, 2, 1).view(n, cout, h, w)
+
+
+@MODELS.register_module()
+class AdabinsHead(nn.Module):
+ """the head of the adabins,include mViT.
+
+ Args:
+ in_channels (int):the channels of the input
+ n_query_channels (int):the channels of the query
+ patch_size (int): the patch size
+ embedding_dim (int):The feature dimension.
+ num_heads (int):the number of head
+ n_bins (int):the number of bins
+ min_val (float): the min width of bin
+ max_val (float): the max width of bin
+ conv_cfg (dict): Config dict for convolution layer.
+ norm (str): the activate method
+ align_corners (bool, optional): Geometrically, we consider the pixels
+ of the input and output as squares rather than points.
+ """
+
+ def __init__(self,
+ in_channels,
+ n_query_channels=128,
+ patch_size=16,
+ embedding_dim=128,
+ num_heads=4,
+ n_bins=100,
+ min_val=0.1,
+ max_val=10,
+ conv_cfg=dict(type='Conv'),
+ norm='linear',
+ align_corners=False,
+ threshold=0):
+ super().__init__()
+ self.out_channels = n_bins
+ self.align_corners = align_corners
+ self.norm = norm
+ self.num_classes = n_bins
+ self.min_val = min_val
+ self.max_val = max_val
+ self.n_query_channels = n_query_channels
+ self.patch_transformer = PatchTransformerEncoder(
+ in_channels, patch_size, embedding_dim, num_heads)
+ self.dot_product_layer = PixelWiseDotProduct()
+ self.threshold = threshold
+ self.conv3x3 = build_conv_layer(
+ conv_cfg,
+ in_channels,
+ embedding_dim,
+ kernel_size=3,
+ stride=1,
+ padding=1)
+ self.regressor = nn.Sequential(
+ nn.Linear(embedding_dim, 256), nn.LeakyReLU(), nn.Linear(256, 256),
+ nn.LeakyReLU(), nn.Linear(256, n_bins))
+ self.conv_out = nn.Sequential(
+ build_conv_layer(conv_cfg, in_channels, n_bins, kernel_size=1),
+ nn.Softmax(dim=1))
+
+ def forward(self, x):
+ # n, c, h, w = x.size()
+ tgt = self.patch_transformer(x.clone()) # .shape = S, N, E
+
+ x = self.conv3x3(x)
+
+ regression_head, queries = tgt[0,
+ ...], tgt[1:self.n_query_channels + 1,
+ ...]
+
+ # Change from S, N, E to N, S, E
+ queries = queries.permute(1, 0, 2)
+ range_attention_maps = self.dot_product_layer(
+ x, queries) # .shape = n, n_query_channels, h, w
+
+ y = self.regressor(regression_head) # .shape = N, dim_out
+ if self.norm == 'linear':
+ y = torch.relu(y)
+ eps = 0.1
+ y = y + eps
+ elif self.norm == 'softmax':
+ return torch.softmax(y, dim=1), range_attention_maps
+ else:
+ y = torch.sigmoid(y)
+ bin_widths_normed = y / y.sum(dim=1, keepdim=True)
+ out = self.conv_out(range_attention_maps)
+
+ bin_widths = (self.max_val -
+ self.min_val) * bin_widths_normed # .shape = N, dim_out
+ bin_widths = F.pad(
+ bin_widths, (1, 0), mode='constant', value=self.min_val)
+ bin_edges = torch.cumsum(bin_widths, dim=1)
+
+ centers = 0.5 * (bin_edges[:, :-1] + bin_edges[:, 1:])
+ n, dim_out = centers.size()
+ centers = centers.view(n, dim_out, 1, 1)
+
+ pred = torch.sum(out * centers, dim=1, keepdim=True)
+ return bin_edges, pred
+
+ def predict(self, inputs: Tuple[Tensor], batch_img_metas: List[dict],
+ test_cfg, **kwargs) -> Tensor:
+ """Forward function for testing, only ``pam_cam`` is used."""
+ pred = self.forward(inputs)[-1]
+ final = torch.clamp(pred, self.min_val, self.max_val)
+
+ final[torch.isinf(final)] = self.max_val
+ final[torch.isnan(final)] = self.min_val
+ return final
diff --git a/projects/CAT-Seg/README.md b/projects/CAT-Seg/README.md
new file mode 100644
index 00000000000..890e461ce4c
--- /dev/null
+++ b/projects/CAT-Seg/README.md
@@ -0,0 +1,92 @@
+# CAT-Seg
+
+> [CAT-Seg: Cost Aggregation for Open-Vocabulary Semantic Segmentation](https://arxiv.org/abs/2303.11797)
+
+## Introduction
+
+
+
+Official Repo
+
+Code Snippet
+
+## Abstract
+
+
+
+Existing works on open-vocabulary semantic segmentation have utilized large-scale vision-language models, such as CLIP, to leverage their exceptional open-vocabulary recognition capabilities. However, the problem of transferring these capabilities learned from image-level supervision to the pixel-level task of segmentation and addressing arbitrary unseen categories at inference makes this task challenging. To address these issues, we aim to attentively relate objects within an image to given categories by leveraging relational information among class categories and visual semantics through aggregation, while also adapting the CLIP representations to the pixel-level task. However, we observe that direct optimization of the CLIP embeddings can harm its open-vocabulary capabilities. In this regard, we propose an alternative approach to optimize the imagetext similarity map, i.e. the cost map, using a novel cost aggregation-based method. Our framework, namely CATSeg, achieves state-of-the-art performance across all benchmarks. We provide extensive ablation studies to validate our choices. [Project page](https://ku-cvlab.github.io/CAT-Seg).
+
+
+
+
+
+CAT-Seg model structure
+
+
+
diff --git a/projects/isnet/README.md b/projects/isnet/README.md
index 3a3172a9d99..0a79ad6a4fa 100644
--- a/projects/isnet/README.md
+++ b/projects/isnet/README.md
@@ -96,11 +96,11 @@ A project does not necessarily have to be finished in a single PR, but it's esse
- [ ] Type hints and docstrings
-
+
- [ ] Unit tests
-
+
- [ ] Code polishing
@@ -108,10 +108,10 @@ A project does not necessarily have to be finished in a single PR, but it's esse
- [ ] Metafile.yml
-
+
- [ ] Move your modules into the core package following the codebase's file hierarchy structure.
-
+
- [ ] Refactor your modules into the core package following the codebase's file hierarchy structure.
diff --git a/projects/mapillary_dataset/README.md b/projects/mapillary_dataset/README.md
index cdc61d53a93..44a1e33ef93 100644
--- a/projects/mapillary_dataset/README.md
+++ b/projects/mapillary_dataset/README.md
@@ -10,7 +10,7 @@ This project implements **`Mapillary Vistas Dataset`**
### Dataset preparing
-Preparing `Mapillary Vistas Dataset` dataset following [Mapillary Vistas Dataset Preparing Guide](https://github.com/open-mmlab/mmsegmentation/tree/dev-1.x/projects/mapillary_dataset/docs/en/user_guides/2_dataset_prepare.md)
+Preparing `Mapillary Vistas Dataset` dataset following [Mapillary Vistas Dataset Preparing Guide](https://github.com/open-mmlab/mmsegmentation/tree/main/projects/mapillary_dataset/docs/en/user_guides/2_dataset_prepare.md)
```none
mmsegmentation
diff --git a/projects/mapillary_dataset/docs/en/user_guides/2_dataset_prepare.md b/projects/mapillary_dataset/docs/en/user_guides/2_dataset_prepare.md
index fa074543300..c5cbc0f9b80 100644
--- a/projects/mapillary_dataset/docs/en/user_guides/2_dataset_prepare.md
+++ b/projects/mapillary_dataset/docs/en/user_guides/2_dataset_prepare.md
@@ -47,7 +47,7 @@
```
- You could set Datasets version with `MapillaryDataset_v1` and `MapillaryDataset_v2` in your configs.
- View the Mapillary Vistas Datasets config file here [V1.2](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/configs/_base_/datasets/mapillary_v1.py) and [V2.0](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/configs/_base_/datasets/mapillary_v2.py)
+ View the Mapillary Vistas Datasets config file here [V1.2](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/_base_/datasets/mapillary_v1.py) and [V2.0](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/_base_/datasets/mapillary_v2.py)
- **View datasets labels index and palette**
diff --git a/projects/medical/2d_image/ct/cranium/README.md b/projects/medical/2d_image/ct/cranium/README.md
new file mode 100644
index 00000000000..d3fa64ea406
--- /dev/null
+++ b/projects/medical/2d_image/ct/cranium/README.md
@@ -0,0 +1,142 @@
+# Brain CT Images with Intracranial Hemorrhage Masks (Cranium)
+
+## Description
+
+This project supports **`Brain CT Images with Intracranial Hemorrhage Masks (Cranium)`**, which can be downloaded from [here](https://www.kaggle.com/datasets/vbookshelf/computed-tomography-ct-images).
+
+### Dataset Overview
+
+This dataset consists of head CT (Computed Thomography) images in jpg format. There are 2500 brain window images and 2500 bone window images, for 82 patients. There are approximately 30 image slices per patient. 318 images have associated intracranial image masks. Also included are csv files containing hemorrhage diagnosis data and patient data.
+This is version 1.0.0 of this dataset. A full description of this dataset as well as updated versions can be found here:
+https://physionet.org/content/ct-ich/1.0.0/
+
+### Statistic Information
+
+| Dataset Name | Anatomical Region | Task Type | Modality | Num. Classes | Train/Val/Test Images | Train/Val/Test Labeled | Release Date | License |
+| ----------------------------------------------------------------------------------- | ----------------- | ------------ | -------- | ------------ | --------------------- | ---------------------- | ------------ | --------------------------------------------------------- |
+| [Cranium](https://www.kaggle.com/datasets/vbookshelf/computed-tomography-ct-images) | head_and_neck | segmentation | ct | 2 | 2501/-/- | yes/-/- | 2020 | [CC-BY 4.0](https://creativecommons.org/licenses/by/4.0/) |
+
+| Class Name | Num. Train | Pct. Train | Num. Val | Pct. Val | Num. Test | Pct. Test |
+| :--------: | :--------: | :--------: | :------: | :------: | :-------: | :-------: |
+| background | 2501 | 99.93 | - | - | - | - |
+| hemorrhage | 318 | 0.07 | - | - | - | - |
+
+Note:
+
+- `Pct` means percentage of pixels in this category in all pixels.
+
+### Visualization
+
+
+
+## Dataset Citation
+
+```
+@article{hssayeni2020computed,
+ title={Computed tomography images for intracranial hemorrhage detection and segmentation},
+ author={Hssayeni, Murtadha and Croock, MS and Salman, AD and Al-khafaji, HF and Yahya, ZA and Ghoraani, B},
+ journal={Intracranial Hemorrhage Segmentation Using A Deep Convolutional Model. Data},
+ volume={5},
+ number={1},
+ pages={179},
+ year={2020}
+}
+```
+
+### Prerequisites
+
+- Python v3.8
+- PyTorch v1.10.0
+- pillow(PIL) v9.3.0 9.3.0
+- scikit-learn(sklearn) v1.2.0 1.2.0
+- [MIM](https://github.com/open-mmlab/mim) v0.3.4
+- [MMCV](https://github.com/open-mmlab/mmcv) v2.0.0rc4
+- [MMEngine](https://github.com/open-mmlab/mmengine) v0.2.0 or higher
+- [MMSegmentation](https://github.com/open-mmlab/mmsegmentation) v1.0.0rc5
+
+All the commands below rely on the correct configuration of `PYTHONPATH`, which should point to the project's directory so that Python can locate the module files. In `cranium/` root directory, run the following line to add the current directory to `PYTHONPATH`:
+
+```shell
+export PYTHONPATH=`pwd`:$PYTHONPATH
+```
+
+### Dataset Preparing
+
+- download dataset from [here](https://www.kaggle.com/datasets/vbookshelf/computed-tomography-ct-images) and decompress data to path `'data/'`.
+- run script `"python tools/prepare_dataset.py"` to format data and change folder structure as below.
+- run script `"python ../../tools/split_seg_dataset.py"` to split dataset and generate `train.txt`, `val.txt` and `test.txt`. If the label of official validation set and test set cannot be obtained, we generate `train.txt` and `val.txt` from the training set randomly.
+
+```none
+ mmsegmentation
+ ├── mmseg
+ ├── projects
+ │ ├── medical
+ │ │ ├── 2d_image
+ │ │ │ ├── ct
+ │ │ │ │ ├── cranium
+ │ │ │ │ │ ├── configs
+ │ │ │ │ │ ├── datasets
+ │ │ │ │ │ ├── tools
+ │ │ │ │ │ ├── data
+ │ │ │ │ │ │ ├── train.txt
+ │ │ │ │ │ │ ├── val.txt
+ │ │ │ │ │ │ ├── images
+ │ │ │ │ │ │ │ ├── train
+ │ │ │ │ | │ │ │ ├── xxx.png
+ │ │ │ │ | │ │ │ ├── ...
+ │ │ │ │ | │ │ │ └── xxx.png
+ │ │ │ │ │ │ ├── masks
+ │ │ │ │ │ │ │ ├── train
+ │ │ │ │ | │ │ │ ├── xxx.png
+ │ │ │ │ | │ │ │ ├── ...
+ │ │ │ │ | │ │ │ └── xxx.png
+```
+
+### Divided Dataset Information
+
+***Note: The table information below is divided by ourselves.***
+
+| Class Name | Num. Train | Pct. Train | Num. Val | Pct. Val | Num. Test | Pct. Test |
+| :--------: | :--------: | :--------: | :------: | :------: | :-------: | :-------: |
+| background | 2000 | 99.93 | 501 | 99.92 | - | - |
+| hemorrhage | 260 | 0.07 | 260 | 0.08 | - | - |
+
+### Training commands
+
+To train models on a single server with one GPU. (default)
+
+```shell
+mim train mmseg ./configs/${CONFIG_FILE}
+```
+
+### Testing commands
+
+To test models on a single server with one GPU. (default)
+
+```shell
+mim test mmseg ./configs/${CONFIG_FILE} --checkpoint ${CHECKPOINT_PATH}
+```
+
+## Checklist
+
+- [x] Milestone 1: PR-ready, and acceptable to be one of the `projects/`.
+
+ - [x] Finish the code
+ - [x] Basic docstrings & proper citation
+ - [ ] Test-time correctness
+ - [x] A full README
+
+- [ ] Milestone 2: Indicates a successful model implementation.
+
+ - [ ] Training-time correctness
+
+- [ ] Milestone 3: Good to be a part of our core package!
+
+ - [ ] Type hints and docstrings
+ - [ ] Unit tests
+ - [ ] Code polishing
+ - [ ] Metafile.yml
+
+- [ ] Move your modules into the core package following the codebase's file hierarchy structure.
+
+- [ ] Refactor your modules into the core package following the codebase's file hierarchy structure.
diff --git a/projects/medical/2d_image/ct/cranium/configs/cranium_512x512.py b/projects/medical/2d_image/ct/cranium/configs/cranium_512x512.py
new file mode 100644
index 00000000000..d9b44362a5c
--- /dev/null
+++ b/projects/medical/2d_image/ct/cranium/configs/cranium_512x512.py
@@ -0,0 +1,42 @@
+dataset_type = 'CraniumDataset'
+data_root = 'data/'
+img_scale = (512, 512)
+train_pipeline = [
+ dict(type='LoadImageFromFile'),
+ dict(type='LoadAnnotations'),
+ dict(type='Resize', scale=img_scale, keep_ratio=False),
+ dict(type='RandomFlip', prob=0.5),
+ dict(type='PhotoMetricDistortion'),
+ dict(type='PackSegInputs')
+]
+test_pipeline = [
+ dict(type='LoadImageFromFile'),
+ dict(type='Resize', scale=img_scale, keep_ratio=False),
+ dict(type='LoadAnnotations'),
+ dict(type='PackSegInputs')
+]
+train_dataloader = dict(
+ batch_size=16,
+ num_workers=4,
+ persistent_workers=True,
+ sampler=dict(type='InfiniteSampler', shuffle=True),
+ dataset=dict(
+ type=dataset_type,
+ data_root=data_root,
+ ann_file='train.txt',
+ data_prefix=dict(img_path='images/', seg_map_path='masks/'),
+ pipeline=train_pipeline))
+val_dataloader = dict(
+ batch_size=1,
+ num_workers=4,
+ persistent_workers=True,
+ sampler=dict(type='DefaultSampler', shuffle=False),
+ dataset=dict(
+ type=dataset_type,
+ data_root=data_root,
+ ann_file='val.txt',
+ data_prefix=dict(img_path='images/', seg_map_path='masks/'),
+ pipeline=test_pipeline))
+test_dataloader = val_dataloader
+val_evaluator = dict(type='IoUMetric', iou_metrics=['mIoU', 'mDice'])
+test_evaluator = dict(type='IoUMetric', iou_metrics=['mIoU', 'mDice'])
diff --git a/projects/medical/2d_image/ct/cranium/configs/fcn-unet-s5-d16_unet-{use-sigmoid}_1xb16-0.01-20k_cranium-512x512.py b/projects/medical/2d_image/ct/cranium/configs/fcn-unet-s5-d16_unet-{use-sigmoid}_1xb16-0.01-20k_cranium-512x512.py
new file mode 100644
index 00000000000..ac013a215ae
--- /dev/null
+++ b/projects/medical/2d_image/ct/cranium/configs/fcn-unet-s5-d16_unet-{use-sigmoid}_1xb16-0.01-20k_cranium-512x512.py
@@ -0,0 +1,18 @@
+_base_ = [
+ 'mmseg::_base_/models/fcn_unet_s5-d16.py', './cranium_512x512.py',
+ 'mmseg::_base_/default_runtime.py',
+ 'mmseg::_base_/schedules/schedule_20k.py'
+]
+custom_imports = dict(imports='datasets.cranium_dataset')
+img_scale = (512, 512)
+data_preprocessor = dict(size=img_scale)
+optimizer = dict(lr=0.01)
+optim_wrapper = dict(optimizer=optimizer)
+model = dict(
+ data_preprocessor=data_preprocessor,
+ decode_head=dict(
+ num_classes=2, loss_decode=dict(use_sigmoid=True), out_channels=1),
+ auxiliary_head=None,
+ test_cfg=dict(mode='whole', _delete_=True))
+vis_backends = None
+visualizer = dict(vis_backends=vis_backends)
diff --git a/projects/medical/2d_image/ct/cranium/configs/fcn-unet-s5-d16_unet_1xb16-0.0001-20k_cranium-512x512.py b/projects/medical/2d_image/ct/cranium/configs/fcn-unet-s5-d16_unet_1xb16-0.0001-20k_cranium-512x512.py
new file mode 100644
index 00000000000..c71110a21f7
--- /dev/null
+++ b/projects/medical/2d_image/ct/cranium/configs/fcn-unet-s5-d16_unet_1xb16-0.0001-20k_cranium-512x512.py
@@ -0,0 +1,17 @@
+_base_ = [
+ 'mmseg::_base_/models/fcn_unet_s5-d16.py', './cranium_512x512.py',
+ 'mmseg::_base_/default_runtime.py',
+ 'mmseg::_base_/schedules/schedule_20k.py'
+]
+custom_imports = dict(imports='datasets.cranium_dataset')
+img_scale = (512, 512)
+data_preprocessor = dict(size=img_scale)
+optimizer = dict(lr=0.0001)
+optim_wrapper = dict(optimizer=optimizer)
+model = dict(
+ data_preprocessor=data_preprocessor,
+ decode_head=dict(num_classes=2),
+ auxiliary_head=None,
+ test_cfg=dict(mode='whole', _delete_=True))
+vis_backends = None
+visualizer = dict(vis_backends=vis_backends)
diff --git a/projects/medical/2d_image/ct/cranium/configs/fcn-unet-s5-d16_unet_1xb16-0.001-20k_cranium-512x512.py b/projects/medical/2d_image/ct/cranium/configs/fcn-unet-s5-d16_unet_1xb16-0.001-20k_cranium-512x512.py
new file mode 100644
index 00000000000..abbdac285b2
--- /dev/null
+++ b/projects/medical/2d_image/ct/cranium/configs/fcn-unet-s5-d16_unet_1xb16-0.001-20k_cranium-512x512.py
@@ -0,0 +1,17 @@
+_base_ = [
+ 'mmseg::_base_/models/fcn_unet_s5-d16.py', './cranium_512x512.py',
+ 'mmseg::_base_/default_runtime.py',
+ 'mmseg::_base_/schedules/schedule_20k.py'
+]
+custom_imports = dict(imports='datasets.cranium_dataset')
+img_scale = (512, 512)
+data_preprocessor = dict(size=img_scale)
+optimizer = dict(lr=0.001)
+optim_wrapper = dict(optimizer=optimizer)
+model = dict(
+ data_preprocessor=data_preprocessor,
+ decode_head=dict(num_classes=2),
+ auxiliary_head=None,
+ test_cfg=dict(mode='whole', _delete_=True))
+vis_backends = None
+visualizer = dict(vis_backends=vis_backends)
diff --git a/projects/medical/2d_image/ct/cranium/configs/fcn-unet-s5-d16_unet_1xb16-0.01-20k_cranium-512x512.py b/projects/medical/2d_image/ct/cranium/configs/fcn-unet-s5-d16_unet_1xb16-0.01-20k_cranium-512x512.py
new file mode 100644
index 00000000000..418595268f9
--- /dev/null
+++ b/projects/medical/2d_image/ct/cranium/configs/fcn-unet-s5-d16_unet_1xb16-0.01-20k_cranium-512x512.py
@@ -0,0 +1,17 @@
+_base_ = [
+ 'mmseg::_base_/models/fcn_unet_s5-d16.py', './cranium_512x512.py',
+ 'mmseg::_base_/default_runtime.py',
+ 'mmseg::_base_/schedules/schedule_20k.py'
+]
+custom_imports = dict(imports='datasets.cranium_dataset')
+img_scale = (512, 512)
+data_preprocessor = dict(size=img_scale)
+optimizer = dict(lr=0.01)
+optim_wrapper = dict(optimizer=optimizer)
+model = dict(
+ data_preprocessor=data_preprocessor,
+ decode_head=dict(num_classes=2),
+ auxiliary_head=None,
+ test_cfg=dict(mode='whole', _delete_=True))
+vis_backends = None
+visualizer = dict(vis_backends=vis_backends)
diff --git a/projects/medical/2d_image/ct/cranium/datasets/cranium_dataset.py b/projects/medical/2d_image/ct/cranium/datasets/cranium_dataset.py
new file mode 100644
index 00000000000..d65f1cbfc68
--- /dev/null
+++ b/projects/medical/2d_image/ct/cranium/datasets/cranium_dataset.py
@@ -0,0 +1,31 @@
+from mmseg.datasets import BaseSegDataset
+from mmseg.registry import DATASETS
+
+
+@DATASETS.register_module()
+class CraniumDataset(BaseSegDataset):
+ """CraniumDataset dataset.
+
+ In segmentation map annotation for CraniumDataset,
+ 0 stands for background, which is included in 2 categories.
+ ``reduce_zero_label`` is fixed to False. The ``img_suffix``
+ is fixed to '.png' and ``seg_map_suffix`` is fixed to '.png'.
+
+ Args:
+ img_suffix (str): Suffix of images. Default: '.png'
+ seg_map_suffix (str): Suffix of segmentation maps. Default: '.png'
+ reduce_zero_label (bool): Whether to mark label zero as ignored.
+ Default to False.
+ """
+ METAINFO = dict(classes=('background', 'hemorrhage'))
+
+ def __init__(self,
+ img_suffix='.png',
+ seg_map_suffix='.png',
+ reduce_zero_label=False,
+ **kwargs) -> None:
+ super().__init__(
+ img_suffix=img_suffix,
+ seg_map_suffix=seg_map_suffix,
+ reduce_zero_label=reduce_zero_label,
+ **kwargs)
diff --git a/projects/medical/2d_image/ct/cranium/tools/prepare_dataset.py b/projects/medical/2d_image/ct/cranium/tools/prepare_dataset.py
new file mode 100644
index 00000000000..1aa4e435614
--- /dev/null
+++ b/projects/medical/2d_image/ct/cranium/tools/prepare_dataset.py
@@ -0,0 +1,66 @@
+import os
+
+import numpy as np
+from PIL import Image
+
+root_path = 'data/'
+img_suffix = '.png'
+seg_map_suffix = '.png'
+save_img_suffix = '.png'
+save_seg_map_suffix = '.png'
+tgt_img_dir = os.path.join(root_path, 'images/train/')
+tgt_mask_dir = os.path.join(root_path, 'masks/train/')
+os.system('mkdir -p ' + tgt_img_dir)
+os.system('mkdir -p ' + tgt_mask_dir)
+
+
+def read_single_array_from_pil(path):
+ return np.asarray(Image.open(path))
+
+
+def save_png_from_array(arr, save_path, mode=None):
+ Image.fromarray(arr, mode=mode).save(save_path)
+
+
+def convert_label(img, convert_dict):
+ arr = np.zeros_like(img, dtype=np.uint8)
+ for c, i in convert_dict.items():
+ arr[img == c] = i
+ return arr
+
+
+patients_dir = os.path.join(
+ root_path, 'Cranium/computed-tomography-images-for-' +
+ 'intracranial-hemorrhage-detection-and-segmentation-1.0.0' +
+ '/Patients_CT')
+
+patients = sorted(os.listdir(patients_dir))
+for p in patients:
+ data_dir = os.path.join(patients_dir, p, 'brain')
+ file_names = os.listdir(data_dir)
+ img_w_mask_names = [
+ _.replace('_HGE_Seg', '') for _ in file_names if 'Seg' in _
+ ]
+ img_wo_mask_names = [
+ _ for _ in file_names if _ not in img_w_mask_names and 'Seg' not in _
+ ]
+
+ for file_name in file_names:
+ path = os.path.join(data_dir, file_name)
+ img = read_single_array_from_pil(path)
+ tgt_name = file_name.replace('.jpg', img_suffix)
+ tgt_name = p + '_' + tgt_name
+ if 'Seg' in file_name: # is a mask
+ tgt_name = tgt_name.replace('_HGE_Seg', '')
+ mask_path = os.path.join(tgt_mask_dir, tgt_name)
+ mask = convert_label(img, convert_dict={0: 0, 255: 1})
+ save_png_from_array(mask, mask_path)
+ else:
+ img_path = os.path.join(tgt_img_dir, tgt_name)
+ pil = Image.fromarray(img).convert('RGB')
+ pil.save(img_path)
+
+ if file_name in img_wo_mask_names:
+ mask = np.zeros_like(img, dtype=np.uint8)
+ mask_path = os.path.join(tgt_mask_dir, tgt_name)
+ save_png_from_array(mask, mask_path)
diff --git a/projects/medical/2d_image/dermoscopy/isic2016_task1/README.md b/projects/medical/2d_image/dermoscopy/isic2016_task1/README.md
new file mode 100644
index 00000000000..6e44e415ed6
--- /dev/null
+++ b/projects/medical/2d_image/dermoscopy/isic2016_task1/README.md
@@ -0,0 +1,149 @@
+# ISIC-2016 Task1
+
+## Description
+
+This project support **`ISIC-2016 Task1 `**, and the dataset used in this project can be downloaded from [here](https://challenge.isic-archive.com/data/#2016).
+
+### Dataset Overview
+
+The overarching goal of the challenge is to develop image analysis tools to enable the automated diagnosis of melanoma from dermoscopic images.
+
+This challenge provides training data (~900 images) for participants to engage in all 3 components of lesion image analysis. A separate test dataset (~350 images) will be provided for participants to generate and submit automated results.
+
+### Original Statistic Information
+
+| Dataset name | Anatomical region | Task type | Modality | Num. Classes | Train/Val/Test Images | Train/Val/Test Labeled | Release Date | License |
+| ---------------------------------------------------------------- | ----------------- | ------------ | ---------- | ------------ | --------------------- | ---------------------- | ------------ | ---------------------------------------------------------------------- |
+| [ISIC-2016 Task1](https://challenge.isic-archive.com/data/#2016) | full body | segmentation | dermoscopy | 2 | 900/-/379- | yes/-/yes | 2016 | [CC-0](https://creativecommons.org/share-your-work/public-domain/cc0/) |
+
+| Class Name | Num. Train | Pct. Train | Num. Val | Pct. Val | Num. Test | Pct. Test |
+| :---------: | :--------: | :--------: | :------: | :------: | :-------: | :-------: |
+| background | 900 | 82.08 | - | - | 379 | 81.98 |
+| skin lesion | 900 | 17.92 | - | - | 379 | 18.02 |
+
+Note:
+
+- `Pct` means percentage of pixels in this category in all pixels.
+
+### Visualization
+
+
+
+### Prerequisites
+
+- Python 3.8
+- PyTorch 1.10.0
+- pillow(PIL) 9.3.0
+- scikit-learn(sklearn) 1.2.0
+- [MIM](https://github.com/open-mmlab/mim) v0.3.4
+- [MMCV](https://github.com/open-mmlab/mmcv) v2.0.0rc4
+- [MMEngine](https://github.com/open-mmlab/mmengine) v0.2.0 or higher
+- [MMSegmentation](https://github.com/open-mmlab/mmsegmentation) v1.0.0rc5
+
+All the commands below rely on the correct configuration of PYTHONPATH, which should point to the project's directory so that Python can locate the module files. In isic2016_task1/ root directory, run the following line to add the current directory to PYTHONPATH:
+
+```shell
+export PYTHONPATH=`pwd`:$PYTHONPATH
+```
+
+### Dataset preparing
+
+- download dataset from [here](https://challenge.isic-archive.com/data/#2016) and decompression data to path 'data/'.
+- run script `"python tools/prepare_dataset.py"` to split dataset and change folder structure as below.
+- run script `"python ../../tools/split_seg_dataset.py"` to split dataset and generate `train.txt` and `test.txt`. If the label of official validation set and test set can't be obtained, we generate `train.txt` and `val.txt` from the training set randomly.
+
+```none
+ mmsegmentation
+ ├── mmseg
+ ├── projects
+ │ ├── medical
+ │ │ ├── 2d_image
+ │ │ │ ├── dermoscopy
+ │ │ │ │ ├── isic2016_task1
+ │ │ │ │ │ ├── configs
+ │ │ │ │ │ ├── datasets
+ │ │ │ │ │ ├── tools
+ │ │ │ │ │ ├── data
+ │ │ │ │ │ │ ├── train.txt
+ │ │ │ │ │ │ ├── test.txt
+ │ │ │ │ │ │ ├── images
+ │ │ │ │ │ │ │ ├── train
+ │ │ │ │ | │ │ │ ├── xxx.png
+ │ │ │ │ | │ │ │ ├── ...
+ │ │ │ │ | │ │ │ └── xxx.png
+ │ │ │ │ │ │ │ ├── test
+ │ │ │ │ | │ │ │ ├── yyy.png
+ │ │ │ │ | │ │ │ ├── ...
+ │ │ │ │ | │ │ │ └── yyy.png
+ │ │ │ │ │ │ ├── masks
+ │ │ │ │ │ │ │ ├── train
+ │ │ │ │ | │ │ │ ├── xxx.png
+ │ │ │ │ | │ │ │ ├── ...
+ │ │ │ │ | │ │ │ └── xxx.png
+ │ │ │ │ │ │ │ ├── test
+ │ │ │ │ | │ │ │ ├── yyy.png
+ │ │ │ │ | │ │ │ ├── ...
+ │ │ │ │ | │ │ │ └── yyy.png
+```
+
+### Training commands
+
+```shell
+mim train mmseg ./configs/${CONFIG_PATH}
+```
+
+To train on multiple GPUs, e.g. 8 GPUs, run the following command:
+
+```shell
+mim train mmseg ./configs/${CONFIG_PATH} --launcher pytorch --gpus 8
+```
+
+### Testing commands
+
+```shell
+mim test mmseg ./configs/${CONFIG_PATH} --checkpoint ${CHECKPOINT_PATH}
+```
+
+
+
+## Results
+
+### ISIC-2016 Task1
+
+| Method | Backbone | Crop Size | lr | mIoU | mDice | config |
+| :-------------: | :------: | :-------: | :----: | :--: | :---: | :---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------: |
+| fcn_unet_s5-d16 | unet | 512x512 | 0.01 | - | - | [config](https://github.com/open-mmlab/mmsegmentation/tree/dev-1.x/projects/medical/2d_image/dermoscopy/isic2016_task1/configs/fcn-unet-s5-d16_unet_1xb16-0.01-20k_isic2016-task1-512x512.py) |
+| fcn_unet_s5-d16 | unet | 512x512 | 0.001 | - | - | [config](https://github.com/open-mmlab/mmsegmentation/tree/dev-1.x/projects/medical/2d_image/dermoscopy/isic2016_task1/configs/fcn-unet-s5-d16_unet_1xb16-0.001-20k_isic2016-task1-512x512.py) |
+| fcn_unet_s5-d16 | unet | 512x512 | 0.0001 | - | - | [config](https://github.com/open-mmlab/mmsegmentation/tree/dev-1.x/projects/medical/2d_image/dermoscopy/isic2016_task1/configs/fcn-unet-s5-d16_unet_1xb16-0.0001-20k_isic2016-task1-512x512.py) |
+
+## Checklist
+
+- [x] Milestone 1: PR-ready, and acceptable to be one of the `projects/`.
+
+ - [x] Finish the code
+
+ - [x] Basic docstrings & proper citation
+
+ - [x] Test-time correctness
+
+ - [x] A full README
+
+- [x] Milestone 2: Indicates a successful model implementation.
+
+ - [x] Training-time correctness
+
+- [ ] Milestone 3: Good to be a part of our core package!
+
+ - [ ] Type hints and docstrings
+
+ - [ ] Unit tests
+
+ - [ ] Code polishing
+
+ - [ ] Metafile.yml
+
+- [ ] Move your modules into the core package following the codebase's file hierarchy structure.
+
+- [ ] Refactor your modules into the core package following the codebase's file hierarchy structure.
diff --git a/projects/medical/2d_image/dermoscopy/isic2016_task1/configs/fcn-unet-s5-d16_unet_1xb16-0.0001-20k_isic2016-task1-512x512.py b/projects/medical/2d_image/dermoscopy/isic2016_task1/configs/fcn-unet-s5-d16_unet_1xb16-0.0001-20k_isic2016-task1-512x512.py
new file mode 100644
index 00000000000..5638de4d560
--- /dev/null
+++ b/projects/medical/2d_image/dermoscopy/isic2016_task1/configs/fcn-unet-s5-d16_unet_1xb16-0.0001-20k_isic2016-task1-512x512.py
@@ -0,0 +1,17 @@
+_base_ = [
+ 'mmseg::_base_/models/fcn_unet_s5-d16.py', './isic2016-task1_512x512.py',
+ 'mmseg::_base_/default_runtime.py',
+ 'mmseg::_base_/schedules/schedule_20k.py'
+]
+custom_imports = dict(imports='datasets.isic2016-task1_dataset')
+img_scale = (512, 512)
+data_preprocessor = dict(size=img_scale)
+optimizer = dict(lr=0.0001)
+optim_wrapper = dict(optimizer=optimizer)
+model = dict(
+ data_preprocessor=data_preprocessor,
+ decode_head=dict(num_classes=2),
+ auxiliary_head=None,
+ test_cfg=dict(mode='whole', _delete_=True))
+vis_backends = None
+visualizer = dict(vis_backends=vis_backends)
diff --git a/projects/medical/2d_image/dermoscopy/isic2016_task1/configs/fcn-unet-s5-d16_unet_1xb16-0.001-20k_isic2016-task1-512x512.py b/projects/medical/2d_image/dermoscopy/isic2016_task1/configs/fcn-unet-s5-d16_unet_1xb16-0.001-20k_isic2016-task1-512x512.py
new file mode 100644
index 00000000000..bf17faa5380
--- /dev/null
+++ b/projects/medical/2d_image/dermoscopy/isic2016_task1/configs/fcn-unet-s5-d16_unet_1xb16-0.001-20k_isic2016-task1-512x512.py
@@ -0,0 +1,17 @@
+_base_ = [
+ 'mmseg::_base_/models/fcn_unet_s5-d16.py', './isic2016-task1_512x512.py',
+ 'mmseg::_base_/default_runtime.py',
+ 'mmseg::_base_/schedules/schedule_20k.py'
+]
+custom_imports = dict(imports='datasets.isic2016-task1_dataset')
+img_scale = (512, 512)
+data_preprocessor = dict(size=img_scale)
+optimizer = dict(lr=0.001)
+optim_wrapper = dict(optimizer=optimizer)
+model = dict(
+ data_preprocessor=data_preprocessor,
+ decode_head=dict(num_classes=2),
+ auxiliary_head=None,
+ test_cfg=dict(mode='whole', _delete_=True))
+vis_backends = None
+visualizer = dict(vis_backends=vis_backends)
diff --git a/projects/medical/2d_image/dermoscopy/isic2016_task1/configs/fcn-unet-s5-d16_unet_1xb16-0.01-20k_isic2016-task1-512x512.py b/projects/medical/2d_image/dermoscopy/isic2016_task1/configs/fcn-unet-s5-d16_unet_1xb16-0.01-20k_isic2016-task1-512x512.py
new file mode 100644
index 00000000000..f7bfcf6158f
--- /dev/null
+++ b/projects/medical/2d_image/dermoscopy/isic2016_task1/configs/fcn-unet-s5-d16_unet_1xb16-0.01-20k_isic2016-task1-512x512.py
@@ -0,0 +1,17 @@
+_base_ = [
+ 'mmseg::_base_/models/fcn_unet_s5-d16.py', './isic2016-task1_512x512.py',
+ 'mmseg::_base_/default_runtime.py',
+ 'mmseg::_base_/schedules/schedule_20k.py'
+]
+custom_imports = dict(imports='datasets.isic2016-task1_dataset')
+img_scale = (512, 512)
+data_preprocessor = dict(size=img_scale)
+optimizer = dict(lr=0.01)
+optim_wrapper = dict(optimizer=optimizer)
+model = dict(
+ data_preprocessor=data_preprocessor,
+ decode_head=dict(num_classes=2),
+ auxiliary_head=None,
+ test_cfg=dict(mode='whole', _delete_=True))
+vis_backends = None
+visualizer = dict(vis_backends=vis_backends)
diff --git a/projects/medical/2d_image/dermoscopy/isic2016_task1/configs/isic2016-task1_512x512.py b/projects/medical/2d_image/dermoscopy/isic2016_task1/configs/isic2016-task1_512x512.py
new file mode 100644
index 00000000000..029f5d4d7ec
--- /dev/null
+++ b/projects/medical/2d_image/dermoscopy/isic2016_task1/configs/isic2016-task1_512x512.py
@@ -0,0 +1,42 @@
+dataset_type = 'ISIC2017Task1'
+data_root = 'data/'
+img_scale = (512, 512)
+train_pipeline = [
+ dict(type='LoadImageFromFile'),
+ dict(type='LoadAnnotations'),
+ dict(type='Resize', scale=img_scale, keep_ratio=False),
+ dict(type='RandomFlip', prob=0.5),
+ dict(type='PhotoMetricDistortion'),
+ dict(type='PackSegInputs')
+]
+test_pipeline = [
+ dict(type='LoadImageFromFile'),
+ dict(type='Resize', scale=img_scale, keep_ratio=False),
+ dict(type='LoadAnnotations'),
+ dict(type='PackSegInputs')
+]
+train_dataloader = dict(
+ batch_size=16,
+ num_workers=4,
+ persistent_workers=True,
+ sampler=dict(type='InfiniteSampler', shuffle=True),
+ dataset=dict(
+ type=dataset_type,
+ data_root=data_root,
+ ann_file='train.txt',
+ data_prefix=dict(img_path='images/', seg_map_path='masks/'),
+ pipeline=train_pipeline))
+val_dataloader = dict(
+ batch_size=1,
+ num_workers=4,
+ persistent_workers=True,
+ sampler=dict(type='DefaultSampler', shuffle=False),
+ dataset=dict(
+ type=dataset_type,
+ data_root=data_root,
+ ann_file='test.txt',
+ data_prefix=dict(img_path='images/', seg_map_path='masks/'),
+ pipeline=test_pipeline))
+test_dataloader = val_dataloader
+val_evaluator = dict(type='IoUMetric', iou_metrics=['mIoU', 'mDice'])
+test_evaluator = dict(type='IoUMetric', iou_metrics=['mIoU', 'mDice'])
diff --git a/projects/medical/2d_image/dermoscopy/isic2016_task1/datasets/isic2016-task1_dataset.py b/projects/medical/2d_image/dermoscopy/isic2016_task1/datasets/isic2016-task1_dataset.py
new file mode 100644
index 00000000000..8f11bdd0ba9
--- /dev/null
+++ b/projects/medical/2d_image/dermoscopy/isic2016_task1/datasets/isic2016-task1_dataset.py
@@ -0,0 +1,30 @@
+from mmseg.datasets import BaseSegDataset
+from mmseg.registry import DATASETS
+
+
+@DATASETS.register_module()
+class ISIC2017Task1(BaseSegDataset):
+ """ISIC2017Task1 dataset.
+
+ In segmentation map annotation for ISIC2017Task1,
+ ``reduce_zero_label`` is fixed to False. The ``img_suffix``
+ is fixed to '.png' and ``seg_map_suffix`` is fixed to '.png'.
+
+ Args:
+ img_suffix (str): Suffix of images. Default: '.png'
+ seg_map_suffix (str): Suffix of segmentation maps. Default: '.png'
+ reduce_zero_label (bool): Whether to mark label zero as ignored.
+ Default to False.
+ """
+ METAINFO = dict(classes=('normal', 'skin lesion'))
+
+ def __init__(self,
+ img_suffix='.png',
+ seg_map_suffix='.png',
+ reduce_zero_label=False,
+ **kwargs) -> None:
+ super().__init__(
+ img_suffix=img_suffix,
+ seg_map_suffix=seg_map_suffix,
+ reduce_zero_label=reduce_zero_label,
+ **kwargs)
diff --git a/projects/medical/2d_image/dermoscopy/isic2016_task1/tools/prepare_dataset.py b/projects/medical/2d_image/dermoscopy/isic2016_task1/tools/prepare_dataset.py
new file mode 100755
index 00000000000..ef4dad54086
--- /dev/null
+++ b/projects/medical/2d_image/dermoscopy/isic2016_task1/tools/prepare_dataset.py
@@ -0,0 +1,120 @@
+import glob
+import os
+import shutil
+
+import numpy as np
+from PIL import Image
+
+
+def check_maskid(train_imgs):
+ for i in train_masks:
+ img = Image.open(i)
+ print(np.unique(np.array(img)))
+
+
+def reformulate_file(image_list, mask_list):
+ file_list = []
+ for idx, (imgp,
+ maskp) in enumerate(zip(sorted(image_list), sorted(mask_list))):
+ item = {'image': imgp, 'label': maskp}
+ file_list.append(item)
+ return file_list
+
+
+def check_file_exist(pair_list):
+ rel_path = os.getcwd()
+ for idx, sample in enumerate(pair_list):
+ image_path = sample['image']
+ assert os.path.exists(os.path.join(rel_path, image_path))
+ if 'label' in sample:
+ mask_path = sample['label']
+ assert os.path.exists(os.path.join(rel_path, mask_path))
+ print('all file path ok!')
+
+
+def convert_maskid(mask):
+ # add mask id conversion
+ arr_mask = np.array(mask).astype(np.uint8)
+ arr_mask[arr_mask == 255] = 1
+ return Image.fromarray(arr_mask)
+
+
+def process_dataset(file_lists, part_dir_dict):
+ for ith, part in enumerate(file_lists):
+ part_dir = part_dir_dict[ith]
+ for sample in part:
+ # read image and mask
+ image_path = sample['image']
+ if 'label' in sample:
+ mask_path = sample['label']
+
+ basename = os.path.basename(image_path)
+ targetname = basename.split('.')[0] # from image name
+
+ # check image file
+ img_save_path = os.path.join(root_path, 'images', part_dir,
+ targetname + save_img_suffix)
+ if not os.path.exists(img_save_path):
+ if not image_path.endswith('.png'):
+ src = Image.open(image_path)
+ src.save(img_save_path)
+ else:
+ shutil.copy(image_path, img_save_path)
+
+ if mask_path is not None:
+ mask_save_path = os.path.join(root_path, 'masks', part_dir,
+ targetname + save_seg_map_suffix)
+ if not os.path.exists(mask_save_path):
+ # check mask file
+ mask = Image.open(mask_path).convert('L')
+ # convert mask id
+ mask = convert_maskid(mask)
+ if not mask_path.endswith('.png'):
+ mask.save(mask_save_path)
+ else:
+ mask.save(mask_save_path)
+
+ # print image num
+ part_dir_folder = os.path.join(root_path, 'images', part_dir)
+ print(
+ f'{part_dir} has {len(os.listdir(part_dir_folder))} images completed!' # noqa
+ )
+
+
+if __name__ == '__main__':
+
+ root_path = 'data/' # original file
+ img_suffix = '.jpg'
+ seg_map_suffix = '.png'
+ save_img_suffix = '.png'
+ save_seg_map_suffix = '.png'
+
+ train_imgs = glob.glob('data/ISBI2016_ISIC_Part1_Training_Data/*' # noqa
+ + img_suffix)
+ train_masks = glob.glob(
+ 'data/ISBI2016_ISIC_Part1_Training_GroundTruth/*' # noqa
+ + seg_map_suffix)
+
+ test_imgs = glob.glob('data/ISBI2016_ISIC_Part1_Test_Data/*' + img_suffix)
+ test_masks = glob.glob(
+ 'data/ISBI2016_ISIC_Part1_Test_GroundTruth/*' # noqa
+ + seg_map_suffix)
+
+ assert len(train_imgs) == len(train_masks)
+ assert len(test_imgs) == len(test_masks)
+
+ print(f'training images: {len(train_imgs)}, test images: {len(test_imgs)}')
+
+ os.system('mkdir -p ' + root_path + 'images/train/')
+ os.system('mkdir -p ' + root_path + 'images/test/')
+ os.system('mkdir -p ' + root_path + 'masks/train/')
+ os.system('mkdir -p ' + root_path + 'masks/test/')
+
+ train_pair_list = reformulate_file(train_imgs, train_masks)
+ test_pair_list = reformulate_file(test_imgs, test_masks)
+
+ check_file_exist(train_pair_list)
+ check_file_exist(test_pair_list)
+
+ part_dir_dict = {0: 'train/', 1: 'test/'}
+ process_dataset([train_pair_list, test_pair_list], part_dir_dict)
diff --git a/projects/medical/2d_image/dermoscopy/isic2017_task1/README.md b/projects/medical/2d_image/dermoscopy/isic2017_task1/README.md
new file mode 100644
index 00000000000..c7cc27096be
--- /dev/null
+++ b/projects/medical/2d_image/dermoscopy/isic2017_task1/README.md
@@ -0,0 +1,158 @@
+# ISIC-2017 Task1
+
+## Description
+
+This project support **`ISIC-2017 Task1 `**, and the dataset used in this project can be downloaded from [here](https://challenge.isic-archive.com/data/#2017).
+
+### Dataset Overview
+
+The goal of the challenge is to help participants develop image analysis tools to enable the automated diagnosis of melanoma from dermoscopic images.
+
+This challenge provides training data (~2000 images) for participants to engage in all 3 components of lesion image analysis. A separate public validation dataset (~150 images) and blind held-out test dataset (~600 images) will be provided for participants to generate and submit automated results.
+
+### Original Statistic Information
+
+| Dataset name | Anatomical region | Task type | Modality | Num. Classes | Train/Val/Test Images | Train/Val/Test Labeled | Release Date | License |
+| ---------------------------------------------------------------- | ----------------- | ------------ | ---------- | ------------ | --------------------- | ---------------------- | ------------ | ---------------------------------------------------------------------- |
+| [ISIC-2017 Task1](https://challenge.isic-archive.com/data/#2017) | full body | segmentation | dermoscopy | 2 | 2000/150/600 | yes/yes/yes | 2017 | [CC-0](https://creativecommons.org/share-your-work/public-domain/cc0/) |
+
+| Class Name | Num. Train | Pct. Train | Num. Val | Pct. Val | Num. Test | Pct. Test |
+| :---------: | :--------: | :--------: | :------: | :------: | :-------: | :-------: |
+| normal | 2000 | 82.86 | 150 | 73.88 | 600 | 70.62 |
+| skin lesion | 2000 | 17.14 | 150 | 26.12 | 600 | 29.38 |
+
+Note:
+
+- `Pct` means percentage of pixels in this category in all pixels.
+
+### Visualization
+
+
+
+### Prerequisites
+
+- Python 3.8
+- PyTorch 1.10.0
+- pillow(PIL) 9.3.0
+- scikit-learn(sklearn) 1.2.0
+- [MIM](https://github.com/open-mmlab/mim) v0.3.4
+- [MMCV](https://github.com/open-mmlab/mmcv) v2.0.0rc4
+- [MMEngine](https://github.com/open-mmlab/mmengine) v0.2.0 or higher
+- [MMSegmentation](https://github.com/open-mmlab/mmsegmentation) v1.0.0rc5
+
+All the commands below rely on the correct configuration of PYTHONPATH, which should point to the project's directory so that Python can locate the module files. In isic2017_task1/ root directory, run the following line to add the current directory to PYTHONPATH:
+
+```shell
+export PYTHONPATH=`pwd`:$PYTHONPATH
+```
+
+### Dataset preparing
+
+- download dataset from [here](https://challenge.isic-archive.com/data/#2017) and decompression data to path 'data/'.
+- run script `"python tools/prepare_dataset.py"` to split dataset and change folder structure as below.
+- run script `"python ../../tools/split_seg_dataset.py"` to split dataset and generate `train.txt` and `test.txt`. If the label of official validation set and test set can't be obtained, we generate `train.txt` and `val.txt` from the training set randomly.
+
+```none
+ mmsegmentation
+ ├── mmseg
+ ├── projects
+ │ ├── medical
+ │ │ ├── 2d_image
+ │ │ │ ├── dermoscopy
+ │ │ │ │ ├── isic2017_task1
+ │ │ │ │ │ ├── configs
+ │ │ │ │ │ ├── datasets
+ │ │ │ │ │ ├── tools
+ │ │ │ │ │ ├── data
+ │ │ │ │ │ │ ├── train.txt
+ │ │ │ │ │ │ ├── val.txt
+ │ │ │ │ │ │ ├── test.txt
+ │ │ │ │ │ │ ├── images
+ │ │ │ │ │ │ │ ├── train
+ │ │ │ │ | │ │ │ ├── xxx.png
+ │ │ │ │ | │ │ │ ├── ...
+ │ │ │ │ | │ │ │ └── xxx.png
+ │ │ │ │ │ │ │ ├── val
+ │ │ │ │ | │ │ │ ├── yyy.png
+ │ │ │ │ | │ │ │ ├── ...
+ │ │ │ │ | │ │ │ └── yyy.png
+ │ │ │ │ │ │ │ ├── test
+ │ │ │ │ | │ │ │ ├── yyy.png
+ │ │ │ │ | │ │ │ ├── ...
+ │ │ │ │ | │ │ │ └── yyy.png
+ │ │ │ │ │ │ ├── masks
+ │ │ │ │ │ │ │ ├── train
+ │ │ │ │ | │ │ │ ├── xxx.png
+ │ │ │ │ | │ │ │ ├── ...
+ │ │ │ │ | │ │ │ └── xxx.png
+ │ │ │ │ │ │ │ ├── val
+ │ │ │ │ | │ │ │ ├── yyy.png
+ │ │ │ │ | │ │ │ ├── ...
+ │ │ │ │ | │ │ │ └── yyy.png
+ │ │ │ │ │ │ │ ├── test
+ │ │ │ │ | │ │ │ ├── yyy.png
+ │ │ │ │ | │ │ │ ├── ...
+ │ │ │ │ | │ │ │ └── yyy.png
+```
+
+### Training commands
+
+```shell
+mim train mmseg ./configs/${CONFIG_PATH}
+```
+
+To train on multiple GPUs, e.g. 8 GPUs, run the following command:
+
+```shell
+mim train mmseg ./configs/${CONFIG_PATH} --launcher pytorch --gpus 8
+```
+
+### Testing commands
+
+```shell
+mim test mmseg ./configs/${CONFIG_PATH} --checkpoint ${CHECKPOINT_PATH}
+```
+
+
+
+## Results
+
+### ISIC-2017 Task1
+
+| Method | Backbone | Crop Size | lr | mIoU | mDice | config |
+| :-------------: | :------: | :-------: | :----: | :--: | :---: | :---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------: |
+| fcn_unet_s5-d16 | unet | 512x512 | 0.01 | - | - | [config](https://github.com/open-mmlab/mmsegmentation/tree/dev-1.x/projects/medical/2d_image/dermoscopy/isic2017_task1/configs/fcn-unet-s5-d16_unet_1xb16-0.01-20k_isic2017-task1-512x512.py) |
+| fcn_unet_s5-d16 | unet | 512x512 | 0.001 | - | - | [config](https://github.com/open-mmlab/mmsegmentation/tree/dev-1.x/projects/medical/2d_image/dermoscopy/isic2017_task1/configs/fcn-unet-s5-d16_unet_1xb16-0.001-20k_isic2017-task1-512x512.py) |
+| fcn_unet_s5-d16 | unet | 512x512 | 0.0001 | - | - | [config](https://github.com/open-mmlab/mmsegmentation/tree/dev-1.x/projects/medical/2d_image/dermoscopy/isic2017_task1/configs/fcn-unet-s5-d16_unet_1xb16-0.0001-20k_isic2017-task1-512x512.py) |
+
+## Checklist
+
+- [x] Milestone 1: PR-ready, and acceptable to be one of the `projects/`.
+
+ - [x] Finish the code
+
+ - [x] Basic docstrings & proper citation
+
+ - [ ] Test-time correctness
+
+ - [x] A full README
+
+- [ ] Milestone 2: Indicates a successful model implementation.
+
+ - [ ] Training-time correctness
+
+- [ ] Milestone 3: Good to be a part of our core package!
+
+ - [ ] Type hints and docstrings
+
+ - [ ] Unit tests
+
+ - [ ] Code polishing
+
+ - [ ] Metafile.yml
+
+- [ ] Move your modules into the core package following the codebase's file hierarchy structure.
+
+- [ ] Refactor your modules into the core package following the codebase's file hierarchy structure.
diff --git a/projects/medical/2d_image/dermoscopy/isic2017_task1/configs/fcn-unet-s5-d16_unet_1xb16-0.0001-20k_isic2017-task1-512x512.py b/projects/medical/2d_image/dermoscopy/isic2017_task1/configs/fcn-unet-s5-d16_unet_1xb16-0.0001-20k_isic2017-task1-512x512.py
new file mode 100644
index 00000000000..58d0a125d33
--- /dev/null
+++ b/projects/medical/2d_image/dermoscopy/isic2017_task1/configs/fcn-unet-s5-d16_unet_1xb16-0.0001-20k_isic2017-task1-512x512.py
@@ -0,0 +1,17 @@
+_base_ = [
+ 'mmseg::_base_/models/fcn_unet_s5-d16.py', './isic2017-task1_512x512.py',
+ 'mmseg::_base_/default_runtime.py',
+ 'mmseg::_base_/schedules/schedule_20k.py'
+]
+custom_imports = dict(imports='datasets.isic2017-task1_dataset')
+img_scale = (512, 512)
+data_preprocessor = dict(size=img_scale)
+optimizer = dict(lr=0.0001)
+optim_wrapper = dict(optimizer=optimizer)
+model = dict(
+ data_preprocessor=data_preprocessor,
+ decode_head=dict(num_classes=2),
+ auxiliary_head=None,
+ test_cfg=dict(mode='whole', _delete_=True))
+vis_backends = None
+visualizer = dict(vis_backends=vis_backends)
diff --git a/projects/medical/2d_image/dermoscopy/isic2017_task1/configs/fcn-unet-s5-d16_unet_1xb16-0.001-20k_isic2017-task1-512x512.py b/projects/medical/2d_image/dermoscopy/isic2017_task1/configs/fcn-unet-s5-d16_unet_1xb16-0.001-20k_isic2017-task1-512x512.py
new file mode 100644
index 00000000000..3becacf64fb
--- /dev/null
+++ b/projects/medical/2d_image/dermoscopy/isic2017_task1/configs/fcn-unet-s5-d16_unet_1xb16-0.001-20k_isic2017-task1-512x512.py
@@ -0,0 +1,17 @@
+_base_ = [
+ 'mmseg::_base_/models/fcn_unet_s5-d16.py', './isic2017-task1_512x512.py',
+ 'mmseg::_base_/default_runtime.py',
+ 'mmseg::_base_/schedules/schedule_20k.py'
+]
+custom_imports = dict(imports='datasets.isic2017-task1_dataset')
+img_scale = (512, 512)
+data_preprocessor = dict(size=img_scale)
+optimizer = dict(lr=0.001)
+optim_wrapper = dict(optimizer=optimizer)
+model = dict(
+ data_preprocessor=data_preprocessor,
+ decode_head=dict(num_classes=2),
+ auxiliary_head=None,
+ test_cfg=dict(mode='whole', _delete_=True))
+vis_backends = None
+visualizer = dict(vis_backends=vis_backends)
diff --git a/projects/medical/2d_image/dermoscopy/isic2017_task1/configs/fcn-unet-s5-d16_unet_1xb16-0.01-20k_isic2017-task1-512x512.py b/projects/medical/2d_image/dermoscopy/isic2017_task1/configs/fcn-unet-s5-d16_unet_1xb16-0.01-20k_isic2017-task1-512x512.py
new file mode 100644
index 00000000000..654ef4dc3d5
--- /dev/null
+++ b/projects/medical/2d_image/dermoscopy/isic2017_task1/configs/fcn-unet-s5-d16_unet_1xb16-0.01-20k_isic2017-task1-512x512.py
@@ -0,0 +1,17 @@
+_base_ = [
+ 'mmseg::_base_/models/fcn_unet_s5-d16.py', './isic2017-task1_512x512.py',
+ 'mmseg::_base_/default_runtime.py',
+ 'mmseg::_base_/schedules/schedule_20k.py'
+]
+custom_imports = dict(imports='datasets.isic2017-task1_dataset')
+img_scale = (512, 512)
+data_preprocessor = dict(size=img_scale)
+optimizer = dict(lr=0.01)
+optim_wrapper = dict(optimizer=optimizer)
+model = dict(
+ data_preprocessor=data_preprocessor,
+ decode_head=dict(num_classes=2),
+ auxiliary_head=None,
+ test_cfg=dict(mode='whole', _delete_=True))
+vis_backends = None
+visualizer = dict(vis_backends=vis_backends)
diff --git a/projects/medical/2d_image/dermoscopy/isic2017_task1/configs/isic2017-task1_512x512.py b/projects/medical/2d_image/dermoscopy/isic2017_task1/configs/isic2017-task1_512x512.py
new file mode 100644
index 00000000000..95997a10997
--- /dev/null
+++ b/projects/medical/2d_image/dermoscopy/isic2017_task1/configs/isic2017-task1_512x512.py
@@ -0,0 +1,41 @@
+dataset_type = 'ISIC2017Task1'
+data_root = 'data/'
+img_scale = (512, 512)
+train_pipeline = [
+ dict(type='LoadImageFromFile'),
+ dict(type='LoadAnnotations'),
+ dict(type='Resize', scale=img_scale, keep_ratio=False),
+ dict(type='RandomFlip', prob=0.5),
+ dict(type='PhotoMetricDistortion'),
+ dict(type='PackSegInputs')
+]
+test_pipeline = [
+ dict(type='LoadImageFromFile'),
+ dict(type='Resize', scale=img_scale, keep_ratio=False),
+ dict(type='LoadAnnotations'),
+ dict(type='PackSegInputs')
+]
+train_dataloader = dict(
+ batch_size=16,
+ num_workers=4,
+ persistent_workers=True,
+ sampler=dict(type='InfiniteSampler', shuffle=True),
+ dataset=dict(
+ type=dataset_type,
+ data_root=data_root,
+ data_prefix=dict(
+ img_path='images/train/', seg_map_path='masks/train/'),
+ pipeline=train_pipeline))
+val_dataloader = dict(
+ batch_size=1,
+ num_workers=4,
+ persistent_workers=True,
+ sampler=dict(type='DefaultSampler', shuffle=False),
+ dataset=dict(
+ type=dataset_type,
+ data_root=data_root,
+ data_prefix=dict(img_path='images/val/', seg_map_path='masks/val/'),
+ pipeline=test_pipeline))
+test_dataloader = val_dataloader
+val_evaluator = dict(type='IoUMetric', iou_metrics=['mIoU', 'mDice'])
+test_evaluator = dict(type='IoUMetric', iou_metrics=['mIoU', 'mDice'])
diff --git a/projects/medical/2d_image/dermoscopy/isic2017_task1/datasets/isic2017-task1_dataset.py b/projects/medical/2d_image/dermoscopy/isic2017_task1/datasets/isic2017-task1_dataset.py
new file mode 100644
index 00000000000..8f11bdd0ba9
--- /dev/null
+++ b/projects/medical/2d_image/dermoscopy/isic2017_task1/datasets/isic2017-task1_dataset.py
@@ -0,0 +1,30 @@
+from mmseg.datasets import BaseSegDataset
+from mmseg.registry import DATASETS
+
+
+@DATASETS.register_module()
+class ISIC2017Task1(BaseSegDataset):
+ """ISIC2017Task1 dataset.
+
+ In segmentation map annotation for ISIC2017Task1,
+ ``reduce_zero_label`` is fixed to False. The ``img_suffix``
+ is fixed to '.png' and ``seg_map_suffix`` is fixed to '.png'.
+
+ Args:
+ img_suffix (str): Suffix of images. Default: '.png'
+ seg_map_suffix (str): Suffix of segmentation maps. Default: '.png'
+ reduce_zero_label (bool): Whether to mark label zero as ignored.
+ Default to False.
+ """
+ METAINFO = dict(classes=('normal', 'skin lesion'))
+
+ def __init__(self,
+ img_suffix='.png',
+ seg_map_suffix='.png',
+ reduce_zero_label=False,
+ **kwargs) -> None:
+ super().__init__(
+ img_suffix=img_suffix,
+ seg_map_suffix=seg_map_suffix,
+ reduce_zero_label=reduce_zero_label,
+ **kwargs)
diff --git a/projects/medical/2d_image/dermoscopy/isic2017_task1/tools/prepare_dataset.py b/projects/medical/2d_image/dermoscopy/isic2017_task1/tools/prepare_dataset.py
new file mode 100755
index 00000000000..b3643c93590
--- /dev/null
+++ b/projects/medical/2d_image/dermoscopy/isic2017_task1/tools/prepare_dataset.py
@@ -0,0 +1,127 @@
+import glob
+import os
+import shutil
+
+import numpy as np
+from PIL import Image
+
+
+def check_maskid(train_imgs):
+ for i in train_masks:
+ img = Image.open(i)
+ print(np.unique(np.array(img)))
+
+
+def reformulate_file(image_list, mask_list):
+ file_list = []
+ for idx, (imgp,
+ maskp) in enumerate(zip(sorted(image_list), sorted(mask_list))):
+ item = {'image': imgp, 'label': maskp}
+ file_list.append(item)
+ return file_list
+
+
+def convert_maskid(mask):
+ # add mask id conversion
+ arr_mask = np.array(mask).astype(np.uint8)
+ arr_mask[arr_mask == 255] = 1
+ return Image.fromarray(arr_mask)
+
+
+def check_file_exist(pair_list):
+ rel_path = os.getcwd()
+ for idx, sample in enumerate(pair_list):
+ image_path = sample['image']
+ assert os.path.exists(os.path.join(rel_path, image_path))
+ if 'label' in sample:
+ mask_path = sample['label']
+ assert os.path.exists(os.path.join(rel_path, mask_path))
+ print('all file path ok!')
+
+
+def process_dataset(file_lists, part_dir_dict):
+ for ith, part in enumerate(file_lists):
+ part_dir = part_dir_dict[ith]
+ for sample in part:
+ # read image and mask
+ image_path = sample['image']
+ if 'label' in sample:
+ mask_path = sample['label']
+
+ basename = os.path.basename(image_path)
+ targetname = basename.split('.')[0] # from image name
+
+ # check image file
+ img_save_path = os.path.join(root_path, 'images', part_dir,
+ targetname + save_img_suffix)
+ if not os.path.exists(img_save_path):
+ if not image_path.endswith('.png'):
+ src = Image.open(image_path)
+ src.save(img_save_path)
+ else:
+ shutil.copy(image_path, img_save_path)
+
+ if mask_path is not None:
+ mask_save_path = os.path.join(root_path, 'masks', part_dir,
+ targetname + save_seg_map_suffix)
+ if not os.path.exists(mask_save_path):
+ # check mask file
+ mask = Image.open(mask_path).convert('L')
+ # convert mask id
+ mask = convert_maskid(mask)
+ if not mask_path.endswith('.png'):
+ mask.save(mask_save_path)
+ else:
+ mask.save(mask_save_path)
+
+ # print image num
+ part_dir_folder = os.path.join(root_path, 'images', part_dir)
+ print(
+ f'{part_dir} has {len(os.listdir(part_dir_folder))} images completed!' # noqa
+ )
+
+
+if __name__ == '__main__':
+
+ root_path = 'data/' # original file
+ img_suffix = '.jpg'
+ seg_map_suffix = '.png'
+ save_img_suffix = '.png'
+ save_seg_map_suffix = '.png'
+
+ train_imgs = glob.glob('data/ISIC-2017_Training_Data/*' + img_suffix)
+ train_masks = glob.glob('data/ISIC-2017_Training_Part1_GroundTruth/*' +
+ seg_map_suffix)
+
+ val_imgs = glob.glob('data/ISIC-2017_Validation_Data/*' + img_suffix)
+ val_masks = glob.glob('data/ISIC-2017_Validation_Part1_GroundTruth/*' +
+ seg_map_suffix)
+
+ test_imgs = glob.glob('data/ISIC-2017_Test_v2_Data/*' + img_suffix)
+ test_masks = glob.glob('data/ISIC-2017_Test_v2_Part1_GroundTruth/*' +
+ seg_map_suffix)
+
+ assert len(train_imgs) == len(train_masks)
+ assert len(val_imgs) == len(val_masks)
+ assert len(test_imgs) == len(test_masks)
+
+ os.system('mkdir -p ' + root_path + 'images/train/')
+ os.system('mkdir -p ' + root_path + 'images/val/')
+ os.system('mkdir -p ' + root_path + 'images/test/')
+ os.system('mkdir -p ' + root_path + 'masks/train/')
+ os.system('mkdir -p ' + root_path + 'masks/val/')
+ os.system('mkdir -p ' + root_path + 'masks/test/')
+
+ part_dir_dict = {0: 'train/', 1: 'val/', 2: 'test/'}
+
+ train_pair_list = reformulate_file(train_imgs, train_masks)
+ val_pair_list = reformulate_file(val_imgs, val_masks)
+ test_pair_list = reformulate_file(test_imgs, test_masks)
+
+ check_file_exist(train_pair_list)
+ check_file_exist(val_pair_list)
+ check_file_exist(test_pair_list)
+
+ part_dir_dict = {0: 'train/', 1: 'val/', 2: 'test/'}
+ process_dataset([train_pair_list, val_pair_list, test_pair_list],
+ part_dir_dict)
diff --git a/projects/medical/2d_image/endoscopy/kvasir_seg/README.md b/projects/medical/2d_image/endoscopy/kvasir_seg/README.md
new file mode 100644
index 00000000000..ea597bc4401
--- /dev/null
+++ b/projects/medical/2d_image/endoscopy/kvasir_seg/README.md
@@ -0,0 +1,145 @@
+# Kvasir-Sessile Dataset (Kvasir SEG)
+
+## Description
+
+This project supports **`Kvasir-Sessile Dataset (Kvasir SEG) `**, which can be downloaded from [here](https://opendatalab.com/Kvasir-Sessile_dataset).
+
+## Dataset Overview
+
+The Kvasir-SEG dataset contains polyp images and their corresponding ground truth from the Kvasir Dataset v2. The resolution of the images contained in Kvasir-SEG varies from 332x487 to 1920x1072 pixels.
+
+
+
+### Information Statistics
+
+| Dataset Name | Anatomical Region | Task Type | Modality | Num. Classes | Train/Val/Test Images | Train/Val/Test Labeled | Release Date | License |
+| ------------------------------------------------------------- | ----------------- | ------------ | --------- | ------------ | --------------------- | ---------------------- | ------------ | --------------------------------------------------------- |
+| [Kvarsir-SEG](https://opendatalab.com/Kvasir-Sessile_dataset) | abdomen | segmentation | endoscopy | 2 | 196/-/- | yes/-/- | 2020 | [CC-BY 4.0](https://creativecommons.org/licenses/by/4.0/) |
+
+| Class Name | Num. Train | Pct. Train | Num. Val | Pct. Val | Num. Test | Pct. Test |
+| :--------: | :--------: | :--------: | :------: | :------: | :-------: | :-------: |
+| background | 196 | 92.31 | - | - | - | - |
+| polyp | 196 | 7.69 | - | - | - | - |
+
+Note:
+
+- `Pct` means percentage of pixels in this category in all pixels.
+
+### Visualization
+
+
+
+### Dataset Citation
+
+```
+@inproceedings{jha2020kvasir,
+ title={Kvasir-seg: A segmented polyp dataset},
+ author={Jha, Debesh and Smedsrud, Pia H and Riegler, Michael A and Halvorsen, P{\aa}l and Lange, Thomas de and Johansen, Dag and Johansen, H{\aa}vard D},
+ booktitle={International Conference on Multimedia Modeling},
+ pages={451--462},
+ year={2020},
+ organization={Springer}
+ }
+```
+
+### Prerequisites
+
+- Python v3.8
+- PyTorch v1.10.0
+- pillow(PIL) v9.3.0
+- scikit-learn(sklearn) v1.2.0
+- [MIM](https://github.com/open-mmlab/mim) v0.3.4
+- [MMCV](https://github.com/open-mmlab/mmcv) v2.0.0rc4
+- [MMEngine](https://github.com/open-mmlab/mmengine) v0.2.0 or higher
+- [MMSegmentation](https://github.com/open-mmlab/mmsegmentation) v1.0.0rc5
+
+All the commands below rely on the correct configuration of `PYTHONPATH`, which should point to the project's directory so that Python can locate the module files. In `kvasir_seg/` root directory, run the following line to add the current directory to `PYTHONPATH`:
+
+```shell
+export PYTHONPATH=`pwd`:$PYTHONPATH
+```
+
+### Dataset Preparing
+
+- download dataset from [here](https://opendatalab.com/Kvasir-Sessile_dataset) and decompress data to path `'data/'`.
+- run script `"python tools/prepare_dataset.py"` to format data and change folder structure as below.
+- run script `"python ../../tools/split_seg_dataset.py"` to split dataset and generate `train.txt`, `val.txt` and `test.txt`. If the label of official validation set and test set cannot be obtained, we generate `train.txt` and `val.txt` from the training set randomly.
+
+```none
+ mmsegmentation
+ ├── mmseg
+ ├── projects
+ │ ├── medical
+ │ │ ├── 2d_image
+ │ │ │ ├── endoscopy
+ │ │ │ │ ├── kvasir_seg
+ │ │ │ │ │ ├── configs
+ │ │ │ │ │ ├── datasets
+ │ │ │ │ │ ├── tools
+ │ │ │ │ │ ├── data
+ │ │ │ │ │ │ ├── train.txt
+ │ │ │ │ │ │ ├── val.txt
+ │ │ │ │ │ │ ├── images
+ │ │ │ │ │ │ │ ├── train
+ │ │ │ │ | │ │ │ ├── xxx.png
+ │ │ │ │ | │ │ │ ├── ...
+ │ │ │ │ | │ │ │ └── xxx.png
+ │ │ │ │ │ │ ├── masks
+ │ │ │ │ │ │ │ ├── train
+ │ │ │ │ | │ │ │ ├── xxx.png
+ │ │ │ │ | │ │ │ ├── ...
+ │ │ │ │ | │ │ │ └── xxx.png
+```
+
+### Divided Dataset Information
+
+***Note: The table information below is divided by ourselves.***
+
+| Class Name | Num. Train | Pct. Train | Num. Val | Pct. Val | Num. Test | Pct. Test |
+| :--------: | :--------: | :--------: | :------: | :------: | :-------: | :-------: |
+| background | 156 | 92.28 | 40 | 92.41 | - | - |
+| polyp | 156 | 7.72 | 40 | 7.59 | - | - |
+
+### Training commands
+
+To train models on a single server with one GPU. (default)
+
+```shell
+mim train mmseg .configs/${CONFIG_FILE}
+```
+
+### Testing commands
+
+To test models on a single server with one GPU. (default)
+
+```shell
+mim test mmseg ./configs/${CONFIG_FILE} --checkpoint ${CHECKPOINT_PATH}
+```
+
+
+
+## Checklist
+
+- [x] Milestone 1: PR-ready, and acceptable to be one of the `projects/`.
+
+ - [x] Finish the code
+ - [x] Basic docstrings & proper citation
+ - [ ] Test-time correctness
+ - [x] A full README
+
+- [x] Milestone 2: Indicates a successful model implementation.
+
+ - [x] Training-time correctness
+
+- [ ] Milestone 3: Good to be a part of our core package!
+
+ - [ ] Type hints and docstrings
+ - [ ] Unit tests
+ - [ ] Code polishing
+ - [ ] Metafile.yml
+
+- [ ] Move your modules into the core package following the codebase's file hierarchy structure.
+
+- [ ] Refactor your modules into the core package following the codebase's file hierarchy structure.
diff --git a/projects/medical/2d_image/endoscopy/kvasir_seg/configs/fcn-unet-s5-d16_unet-{use-sigmoid}_1xb16-0.01-20k_kvasir-seg-512x512.py b/projects/medical/2d_image/endoscopy/kvasir_seg/configs/fcn-unet-s5-d16_unet-{use-sigmoid}_1xb16-0.01-20k_kvasir-seg-512x512.py
new file mode 100644
index 00000000000..145d5a7a172
--- /dev/null
+++ b/projects/medical/2d_image/endoscopy/kvasir_seg/configs/fcn-unet-s5-d16_unet-{use-sigmoid}_1xb16-0.01-20k_kvasir-seg-512x512.py
@@ -0,0 +1,18 @@
+_base_ = [
+ 'mmseg::_base_/models/fcn_unet_s5-d16.py', './kvasir-seg_512x512.py',
+ 'mmseg::_base_/default_runtime.py',
+ 'mmseg::_base_/schedules/schedule_20k.py'
+]
+custom_imports = dict(imports='datasets.kvasir-seg_dataset')
+img_scale = (512, 512)
+data_preprocessor = dict(size=img_scale)
+optimizer = dict(lr=0.01)
+optim_wrapper = dict(optimizer=optimizer)
+model = dict(
+ data_preprocessor=data_preprocessor,
+ decode_head=dict(
+ num_classes=2, loss_decode=dict(use_sigmoid=True), out_channels=1),
+ auxiliary_head=None,
+ test_cfg=dict(mode='whole', _delete_=True))
+vis_backends = None
+visualizer = dict(vis_backends=vis_backends)
diff --git a/projects/medical/2d_image/endoscopy/kvasir_seg/configs/fcn-unet-s5-d16_unet_1xb16-0.0001-20k_kvasir-seg-512x512.py b/projects/medical/2d_image/endoscopy/kvasir_seg/configs/fcn-unet-s5-d16_unet_1xb16-0.0001-20k_kvasir-seg-512x512.py
new file mode 100644
index 00000000000..3ea05c51090
--- /dev/null
+++ b/projects/medical/2d_image/endoscopy/kvasir_seg/configs/fcn-unet-s5-d16_unet_1xb16-0.0001-20k_kvasir-seg-512x512.py
@@ -0,0 +1,17 @@
+_base_ = [
+ 'mmseg::_base_/models/fcn_unet_s5-d16.py', './kvasir-seg_512x512.py',
+ 'mmseg::_base_/default_runtime.py',
+ 'mmseg::_base_/schedules/schedule_20k.py'
+]
+custom_imports = dict(imports='datasets.kvasir-seg_dataset')
+img_scale = (512, 512)
+data_preprocessor = dict(size=img_scale)
+optimizer = dict(lr=0.0001)
+optim_wrapper = dict(optimizer=optimizer)
+model = dict(
+ data_preprocessor=data_preprocessor,
+ decode_head=dict(num_classes=2),
+ auxiliary_head=None,
+ test_cfg=dict(mode='whole', _delete_=True))
+vis_backends = None
+visualizer = dict(vis_backends=vis_backends)
diff --git a/projects/medical/2d_image/endoscopy/kvasir_seg/configs/fcn-unet-s5-d16_unet_1xb16-0.001-20k_kvasir-seg-512x512.py b/projects/medical/2d_image/endoscopy/kvasir_seg/configs/fcn-unet-s5-d16_unet_1xb16-0.001-20k_kvasir-seg-512x512.py
new file mode 100644
index 00000000000..7e064a716aa
--- /dev/null
+++ b/projects/medical/2d_image/endoscopy/kvasir_seg/configs/fcn-unet-s5-d16_unet_1xb16-0.001-20k_kvasir-seg-512x512.py
@@ -0,0 +1,17 @@
+_base_ = [
+ 'mmseg::_base_/models/fcn_unet_s5-d16.py', './kvasir-seg_512x512.py',
+ 'mmseg::_base_/default_runtime.py',
+ 'mmseg::_base_/schedules/schedule_20k.py'
+]
+custom_imports = dict(imports='datasets.kvasir-seg_dataset')
+img_scale = (512, 512)
+data_preprocessor = dict(size=img_scale)
+optimizer = dict(lr=0.001)
+optim_wrapper = dict(optimizer=optimizer)
+model = dict(
+ data_preprocessor=data_preprocessor,
+ decode_head=dict(num_classes=2),
+ auxiliary_head=None,
+ test_cfg=dict(mode='whole', _delete_=True))
+vis_backends = None
+visualizer = dict(vis_backends=vis_backends)
diff --git a/projects/medical/2d_image/endoscopy/kvasir_seg/configs/fcn-unet-s5-d16_unet_1xb16-0.01-20k_kvasir-seg-512x512.py b/projects/medical/2d_image/endoscopy/kvasir_seg/configs/fcn-unet-s5-d16_unet_1xb16-0.01-20k_kvasir-seg-512x512.py
new file mode 100644
index 00000000000..0fc1d6e99d7
--- /dev/null
+++ b/projects/medical/2d_image/endoscopy/kvasir_seg/configs/fcn-unet-s5-d16_unet_1xb16-0.01-20k_kvasir-seg-512x512.py
@@ -0,0 +1,17 @@
+_base_ = [
+ 'mmseg::_base_/models/fcn_unet_s5-d16.py', './kvasir-seg_512x512.py',
+ 'mmseg::_base_/default_runtime.py',
+ 'mmseg::_base_/schedules/schedule_20k.py'
+]
+custom_imports = dict(imports='datasets.kvasir-seg_dataset')
+img_scale = (512, 512)
+data_preprocessor = dict(size=img_scale)
+optimizer = dict(lr=0.01)
+optim_wrapper = dict(optimizer=optimizer)
+model = dict(
+ data_preprocessor=data_preprocessor,
+ decode_head=dict(num_classes=2),
+ auxiliary_head=None,
+ test_cfg=dict(mode='whole', _delete_=True))
+vis_backends = None
+visualizer = dict(vis_backends=vis_backends)
diff --git a/projects/medical/2d_image/endoscopy/kvasir_seg/configs/kvasir-seg_512x512.py b/projects/medical/2d_image/endoscopy/kvasir_seg/configs/kvasir-seg_512x512.py
new file mode 100644
index 00000000000..e8b2467f8cf
--- /dev/null
+++ b/projects/medical/2d_image/endoscopy/kvasir_seg/configs/kvasir-seg_512x512.py
@@ -0,0 +1,42 @@
+dataset_type = 'KvasirSEGDataset'
+data_root = 'data/'
+img_scale = (512, 512)
+train_pipeline = [
+ dict(type='LoadImageFromFile'),
+ dict(type='LoadAnnotations'),
+ dict(type='Resize', scale=img_scale, keep_ratio=False),
+ dict(type='RandomFlip', prob=0.5),
+ dict(type='PhotoMetricDistortion'),
+ dict(type='PackSegInputs')
+]
+test_pipeline = [
+ dict(type='LoadImageFromFile'),
+ dict(type='Resize', scale=img_scale, keep_ratio=False),
+ dict(type='LoadAnnotations'),
+ dict(type='PackSegInputs')
+]
+train_dataloader = dict(
+ batch_size=16,
+ num_workers=4,
+ persistent_workers=True,
+ sampler=dict(type='InfiniteSampler', shuffle=True),
+ dataset=dict(
+ type=dataset_type,
+ data_root=data_root,
+ ann_file='train.txt',
+ data_prefix=dict(img_path='images/', seg_map_path='masks/'),
+ pipeline=train_pipeline))
+val_dataloader = dict(
+ batch_size=1,
+ num_workers=4,
+ persistent_workers=True,
+ sampler=dict(type='DefaultSampler', shuffle=False),
+ dataset=dict(
+ type=dataset_type,
+ data_root=data_root,
+ ann_file='val.txt',
+ data_prefix=dict(img_path='images/', seg_map_path='masks/'),
+ pipeline=test_pipeline))
+test_dataloader = val_dataloader
+val_evaluator = dict(type='IoUMetric', iou_metrics=['mIoU', 'mDice'])
+test_evaluator = dict(type='IoUMetric', iou_metrics=['mIoU', 'mDice'])
diff --git a/projects/medical/2d_image/endoscopy/kvasir_seg/datasets/kvasir-seg_dataset.py b/projects/medical/2d_image/endoscopy/kvasir_seg/datasets/kvasir-seg_dataset.py
new file mode 100644
index 00000000000..9d601328eb6
--- /dev/null
+++ b/projects/medical/2d_image/endoscopy/kvasir_seg/datasets/kvasir-seg_dataset.py
@@ -0,0 +1,30 @@
+from mmseg.datasets import BaseSegDataset
+from mmseg.registry import DATASETS
+
+
+@DATASETS.register_module()
+class KvasirSEGDataset(BaseSegDataset):
+ """KvasirSEGDataset dataset.
+
+ In segmentation map annotation for KvasirSEGDataset, 0 stands for
+ background, which is included in 2 categories.
+ ``reduce_zero_label`` is fixed to False. The ``img_suffix`` is
+ fixed to '.png' and ``seg_map_suffix`` is fixed to '.png'.
+ Args:
+ img_suffix (str): Suffix of images. Default: '.png'
+ seg_map_suffix (str): Suffix of segmentation maps. Default: '.png'
+ reduce_zero_label (bool): Whether to mark label zero as ignored.
+ Default to False..
+ """
+ METAINFO = dict(classes=('background', 'polyp'))
+
+ def __init__(self,
+ img_suffix='.png',
+ seg_map_suffix='.png',
+ reduce_zero_label=False,
+ **kwargs) -> None:
+ super().__init__(
+ img_suffix=img_suffix,
+ seg_map_suffix=seg_map_suffix,
+ reduce_zero_label=reduce_zero_label,
+ **kwargs)
diff --git a/projects/medical/2d_image/endoscopy/kvasir_seg/tools/prepare_dataset.py b/projects/medical/2d_image/endoscopy/kvasir_seg/tools/prepare_dataset.py
new file mode 100644
index 00000000000..74c43e96351
--- /dev/null
+++ b/projects/medical/2d_image/endoscopy/kvasir_seg/tools/prepare_dataset.py
@@ -0,0 +1,87 @@
+import glob
+import os
+
+import numpy as np
+from PIL import Image
+
+root_path = 'data/'
+img_suffix = '.jpg'
+seg_map_suffix = '.jpg'
+save_img_suffix = '.png'
+save_seg_map_suffix = '.png'
+tgt_img_dir = os.path.join(root_path, 'images/train/')
+tgt_mask_dir = os.path.join(root_path, 'masks/train/')
+os.system('mkdir -p ' + tgt_img_dir)
+os.system('mkdir -p ' + tgt_mask_dir)
+
+
+def filter_suffix_recursive(src_dir, suffix):
+ # filter out file names and paths in source directory
+ suffix = '.' + suffix if '.' not in suffix else suffix
+ file_paths = glob.glob(
+ os.path.join(src_dir, '**', '*' + suffix), recursive=True)
+ file_names = [_.split('/')[-1] for _ in file_paths]
+ return sorted(file_paths), sorted(file_names)
+
+
+def convert_label(img, convert_dict):
+ arr = np.zeros_like(img, dtype=np.uint8)
+ for c, i in convert_dict.items():
+ arr[img == c] = i
+ return arr
+
+
+def convert_pics_into_pngs(src_dir, tgt_dir, suffix, convert='RGB'):
+ if not os.path.exists(tgt_dir):
+ os.makedirs(tgt_dir)
+
+ src_paths, src_names = filter_suffix_recursive(src_dir, suffix=suffix)
+ for i, (src_name, src_path) in enumerate(zip(src_names, src_paths)):
+ tgt_name = src_name.replace(suffix, save_img_suffix)
+ tgt_path = os.path.join(tgt_dir, tgt_name)
+ num = len(src_paths)
+ img = np.array(Image.open(src_path))
+ if len(img.shape) == 2:
+ pil = Image.fromarray(img).convert(convert)
+ elif len(img.shape) == 3:
+ pil = Image.fromarray(img)
+ else:
+ raise ValueError('Input image not 2D/3D: ', img.shape)
+
+ pil.save(tgt_path)
+ print(f'processed {i+1}/{num}.')
+
+
+def convert_label_pics_into_pngs(src_dir,
+ tgt_dir,
+ suffix,
+ convert_dict={
+ 0: 0,
+ 255: 1
+ }):
+ if not os.path.exists(tgt_dir):
+ os.makedirs(tgt_dir)
+
+ src_paths, src_names = filter_suffix_recursive(src_dir, suffix=suffix)
+ num = len(src_paths)
+ for i, (src_name, src_path) in enumerate(zip(src_names, src_paths)):
+ tgt_name = src_name.replace(suffix, save_seg_map_suffix)
+ tgt_path = os.path.join(tgt_dir, tgt_name)
+
+ img = np.array(Image.open(src_path))
+ img = convert_label(img, convert_dict)
+ Image.fromarray(img).save(tgt_path)
+ print(f'processed {i+1}/{num}.')
+
+
+if __name__ == '__main__':
+
+ convert_pics_into_pngs(
+ os.path.join(root_path, 'sessile-main-Kvasir-SEG/images'),
+ tgt_img_dir,
+ suffix=img_suffix)
+
+ convert_label_pics_into_pngs(
+ os.path.join(root_path, 'sessile-main-Kvasir-SEG/masks'),
+ tgt_mask_dir,
+ suffix=seg_map_suffix)
diff --git a/projects/medical/2d_image/endoscopy/kvasir_seg_aliyun/README.md b/projects/medical/2d_image/endoscopy/kvasir_seg_aliyun/README.md
new file mode 100644
index 00000000000..80eb00f51bd
--- /dev/null
+++ b/projects/medical/2d_image/endoscopy/kvasir_seg_aliyun/README.md
@@ -0,0 +1,145 @@
+# Kvasir-SEG Segmented Polyp Dataset from Aliyun (Kvasir SEG Aliyun)
+
+## Description
+
+This project supports **`Kvasir-SEG Segmented Polyp Dataset from Aliyun (Kvasir SEG Aliyun) `**, which can be downloaded from [here](https://tianchi.aliyun.com/dataset/84385).
+
+### Dataset Overview
+
+Colorectal cancer is the second most common cancer type among women and third most common among men. Polyps are precursors to colorectal cancer and therefore important to detect and remove at an early stage. Polyps are found in nearly half of the individuals at age 50 that undergo a colonoscopy screening, and their frequency increase with age.Polyps are abnormal tissue growth from the mucous membrane, which is lining the inside of the GI tract, and can sometimes be cancerous. Colonoscopy is the gold standard for detection and assessment of these polyps with subsequent biopsy and removal of the polyps. Early disease detection has a huge impact on survival from colorectal cancer. Increasing the detection of polyps has been shown to decrease risk of colorectal cancer. Thus, automatic detection of more polyps at an early stage can play a crucial role in prevention and survival from colorectal cancer.
+
+The Kvasir-SEG dataset is based on the previous Kvasir dataset, which is the first multi-class dataset for gastrointestinal (GI) tract disease detection and classification. It contains annotated polyp images and their corresponding masks. The pixels depicting polyp tissue, the ROI, are represented by the foreground (white mask), while the background (in black) does not contain positive pixels. These images were collected and verified by experienced gastroenterologists from Vestre Viken Health Trust in Norway. The classes include anatomical landmarks, pathological findings and endoscopic procedures.
+
+### Information Statistics
+
+| Dataset Name | Anatomical Region | Task Type | Modality | Num. Classes | Train/Val/Test Images | Train/Val/Test Labeled | Release Date | License |
+| ------------------------------------------------------ | ----------------- | ------------ | --------- | ------------ | --------------------- | ---------------------- | ------------ | --------------------------------------------------------- |
+| [kvasir-seg](https://tianchi.aliyun.com/dataset/84385) | abdomen | segmentation | endoscopy | 2 | 1000/-/- | yes/-/- | 2020 | [CC-BY 4.0](https://creativecommons.org/licenses/by/4.0/) |
+
+| Class Name | Num. Train | Pct. Train | Num. Val | Pct. Val | Num. Test | Pct. Test |
+| :--------: | :--------: | :--------: | :------: | :------: | :-------: | :-------: |
+| background | 1000 | 84.72 | - | - | - | - |
+| polyp | 1000 | 15.28 | - | - | - | - |
+
+Note:
+
+- `Pct` means percentage of pixels in this category in all pixels.
+
+### Visualization
+
+
+
+### Dataset Citation
+
+```
+@inproceedings{jha2020kvasir,
+ title={Kvasir-seg: A segmented polyp dataset},
+ author={Jha, Debesh and Smedsrud, Pia H and Riegler, Michael A and Halvorsen, P{\aa}l and Lange, Thomas de and Johansen, Dag and Johansen, H{\aa}vard D},
+ booktitle={International Conference on Multimedia Modeling},
+ pages={451--462},
+ year={2020},
+ organization={Springer}
+ }
+```
+
+### Prerequisites
+
+- Python v3.8
+- PyTorch v1.10.0
+- pillow(PIL) v9.3.0
+- scikit-learn(sklearn) v1.2.0
+- [MIM](https://github.com/open-mmlab/mim) v0.3.4
+- [MMCV](https://github.com/open-mmlab/mmcv) v2.0.0rc4
+- [MMEngine](https://github.com/open-mmlab/mmengine) v0.2.0 or higher
+- [MMSegmentation](https://github.com/open-mmlab/mmsegmentation) v1.0.0rc5
+
+All the commands below rely on the correct configuration of `PYTHONPATH`, which should point to the project's directory so that Python can locate the module files. In `kvasir_seg_aliyun/` root directory, run the following line to add the current directory to `PYTHONPATH`:
+
+```shell
+export PYTHONPATH=`pwd`:$PYTHONPATH
+```
+
+### Dataset Preparing
+
+- download dataset from [here](https://tianchi.aliyun.com/dataset/84385) and decompression data to path 'data/.'.
+- run script `"python tools/prepare_dataset.py"` to format data and change folder structure as below.
+- run script `"python ../../tools/split_seg_dataset.py"` to split dataset and generate `train.txt`, `val.txt` and `test.txt`. If the label of official validation set and test set cannot be obtained, we generate `train.txt` and `val.txt` from the training set randomly.
+
+```none
+ mmsegmentation
+ ├── mmseg
+ ├── projects
+ │ ├── medical
+ │ │ ├── 2d_image
+ │ │ │ ├── endoscopy
+ │ │ │ │ ├── kvasir_seg_aliyun
+ │ │ │ │ │ ├── configs
+ │ │ │ │ │ ├── datasets
+ │ │ │ │ │ ├── tools
+ │ │ │ │ │ ├── data
+ │ │ │ │ │ │ ├── train.txt
+ │ │ │ │ │ │ ├── val.txt
+ │ │ │ │ │ │ ├── images
+ │ │ │ │ │ │ │ ├── train
+ │ │ │ │ | │ │ │ ├── xxx.png
+ │ │ │ │ | │ │ │ ├── ...
+ │ │ │ │ | │ │ │ └── xxx.png
+ │ │ │ │ │ │ ├── masks
+ │ │ │ │ │ │ │ ├── train
+ │ │ │ │ | │ │ │ ├── xxx.png
+ │ │ │ │ | │ │ │ ├── ...
+ │ │ │ │ | │ │ │ └── xxx.png
+```
+
+### Divided Dataset Information
+
+***Note: The table information below is divided by ourselves.***
+
+| Class Name | Num. Train | Pct. Train | Num. Val | Pct. Val | Num. Test | Pct. Test |
+| :--------: | :--------: | :--------: | :------: | :------: | :-------: | :-------: |
+| background | 800 | 84.66 | 200 | 84.94 | - | - |
+| polyp | 800 | 15.34 | 200 | 15.06 | - | - |
+
+### Training commands
+
+To train models on a single server with one GPU. (default)
+
+```shell
+mim train mmseg ./configs/${CONFIG_FILE}
+```
+
+### Testing commands
+
+To test models on a single server with one GPU. (default)
+
+```shell
+mim test mmseg ./configs/${CONFIG_FILE} --checkpoint ${CHECKPOINT_PATH}
+```
+
+
+
+## Checklist
+
+- [x] Milestone 1: PR-ready, and acceptable to be one of the `projects/`.
+
+ - [x] Finish the code
+ - [x] Basic docstrings & proper citation
+ - [ ] Test-time correctness
+ - [x] A full README
+
+- [ ] Milestone 2: Indicates a successful model implementation.
+
+ - [ ] Training-time correctness
+
+- [ ] Milestone 3: Good to be a part of our core package!
+
+ - [ ] Type hints and docstrings
+ - [ ] Unit tests
+ - [ ] Code polishing
+ - [ ] Metafile.yml
+
+- [ ] Move your modules into the core package following the codebase's file hierarchy structure.
+
+- [ ] Refactor your modules into the core package following the codebase's file hierarchy structure.
diff --git a/projects/medical/2d_image/endoscopy/kvasir_seg_aliyun/configs/fcn-unet-s5-d16_unet_1xb16-0.0001-20k_kvasir-seg-aliyun-512x512.py b/projects/medical/2d_image/endoscopy/kvasir_seg_aliyun/configs/fcn-unet-s5-d16_unet_1xb16-0.0001-20k_kvasir-seg-aliyun-512x512.py
new file mode 100644
index 00000000000..b59db95232b
--- /dev/null
+++ b/projects/medical/2d_image/endoscopy/kvasir_seg_aliyun/configs/fcn-unet-s5-d16_unet_1xb16-0.0001-20k_kvasir-seg-aliyun-512x512.py
@@ -0,0 +1,17 @@
+_base_ = [
+ 'mmseg::_base_/models/fcn_unet_s5-d16.py',
+ './kvasir-seg-aliyun_512x512.py', 'mmseg::_base_/default_runtime.py',
+ 'mmseg::_base_/schedules/schedule_20k.py'
+]
+custom_imports = dict(imports='datasets.kvasir-seg-aliyun_dataset')
+img_scale = (512, 512)
+data_preprocessor = dict(size=img_scale)
+optimizer = dict(lr=0.0001)
+optim_wrapper = dict(optimizer=optimizer)
+model = dict(
+ data_preprocessor=data_preprocessor,
+ decode_head=dict(num_classes=2),
+ auxiliary_head=None,
+ test_cfg=dict(mode='whole', _delete_=True))
+vis_backends = None
+visualizer = dict(vis_backends=vis_backends)
diff --git a/projects/medical/2d_image/endoscopy/kvasir_seg_aliyun/configs/fcn-unet-s5-d16_unet_1xb16-0.001-20k_kvasir-seg-aliyun-512x512.py b/projects/medical/2d_image/endoscopy/kvasir_seg_aliyun/configs/fcn-unet-s5-d16_unet_1xb16-0.001-20k_kvasir-seg-aliyun-512x512.py
new file mode 100644
index 00000000000..6c526680cd4
--- /dev/null
+++ b/projects/medical/2d_image/endoscopy/kvasir_seg_aliyun/configs/fcn-unet-s5-d16_unet_1xb16-0.001-20k_kvasir-seg-aliyun-512x512.py
@@ -0,0 +1,17 @@
+_base_ = [
+ 'mmseg::_base_/models/fcn_unet_s5-d16.py',
+ './kvasir-seg-aliyun_512x512.py', 'mmseg::_base_/default_runtime.py',
+ 'mmseg::_base_/schedules/schedule_20k.py'
+]
+custom_imports = dict(imports='datasets.kvasir-seg-aliyun_dataset')
+img_scale = (512, 512)
+data_preprocessor = dict(size=img_scale)
+optimizer = dict(lr=0.001)
+optim_wrapper = dict(optimizer=optimizer)
+model = dict(
+ data_preprocessor=data_preprocessor,
+ decode_head=dict(num_classes=2),
+ auxiliary_head=None,
+ test_cfg=dict(mode='whole', _delete_=True))
+vis_backends = None
+visualizer = dict(vis_backends=vis_backends)
diff --git a/projects/medical/2d_image/endoscopy/kvasir_seg_aliyun/configs/fcn-unet-s5-d16_unet_1xb16-0.01-20k_kvasir-seg-aliyun-512x512.py b/projects/medical/2d_image/endoscopy/kvasir_seg_aliyun/configs/fcn-unet-s5-d16_unet_1xb16-0.01-20k_kvasir-seg-aliyun-512x512.py
new file mode 100644
index 00000000000..a192a5bd240
--- /dev/null
+++ b/projects/medical/2d_image/endoscopy/kvasir_seg_aliyun/configs/fcn-unet-s5-d16_unet_1xb16-0.01-20k_kvasir-seg-aliyun-512x512.py
@@ -0,0 +1,17 @@
+_base_ = [
+ 'mmseg::_base_/models/fcn_unet_s5-d16.py',
+ './kvasir-seg-aliyun_512x512.py', 'mmseg::_base_/default_runtime.py',
+ 'mmseg::_base_/schedules/schedule_20k.py'
+]
+custom_imports = dict(imports='datasets.kvasir-seg-aliyun_dataset')
+img_scale = (512, 512)
+data_preprocessor = dict(size=img_scale)
+optimizer = dict(lr=0.01)
+optim_wrapper = dict(optimizer=optimizer)
+model = dict(
+ data_preprocessor=data_preprocessor,
+ decode_head=dict(num_classes=2),
+ auxiliary_head=None,
+ test_cfg=dict(mode='whole', _delete_=True))
+vis_backends = None
+visualizer = dict(vis_backends=vis_backends)
diff --git a/projects/medical/2d_image/endoscopy/kvasir_seg_aliyun/configs/fcn-unet-s5-d16_unet_1xb16-0.01lr-sigmoid-20k_kvasir-seg-aliyun-512x512.py b/projects/medical/2d_image/endoscopy/kvasir_seg_aliyun/configs/fcn-unet-s5-d16_unet_1xb16-0.01lr-sigmoid-20k_kvasir-seg-aliyun-512x512.py
new file mode 100644
index 00000000000..5325e1f0807
--- /dev/null
+++ b/projects/medical/2d_image/endoscopy/kvasir_seg_aliyun/configs/fcn-unet-s5-d16_unet_1xb16-0.01lr-sigmoid-20k_kvasir-seg-aliyun-512x512.py
@@ -0,0 +1,18 @@
+_base_ = [
+ 'mmseg::_base_/models/fcn_unet_s5-d16.py',
+ './kvasir-seg-aliyun_512x512.py', 'mmseg::_base_/default_runtime.py',
+ 'mmseg::_base_/schedules/schedule_20k.py'
+]
+custom_imports = dict(imports='datasets.kvasir-seg-aliyun_dataset')
+img_scale = (512, 512)
+data_preprocessor = dict(size=img_scale)
+optimizer = dict(lr=0.01)
+optim_wrapper = dict(optimizer=optimizer)
+model = dict(
+ data_preprocessor=data_preprocessor,
+ decode_head=dict(
+ num_classes=2, loss_decode=dict(use_sigmoid=True), out_channels=1),
+ auxiliary_head=None,
+ test_cfg=dict(mode='whole', _delete_=True))
+vis_backends = None
+visualizer = dict(vis_backends=vis_backends)
diff --git a/projects/medical/2d_image/endoscopy/kvasir_seg_aliyun/configs/kvasir-seg-aliyun_512x512.py b/projects/medical/2d_image/endoscopy/kvasir_seg_aliyun/configs/kvasir-seg-aliyun_512x512.py
new file mode 100644
index 00000000000..5f868804679
--- /dev/null
+++ b/projects/medical/2d_image/endoscopy/kvasir_seg_aliyun/configs/kvasir-seg-aliyun_512x512.py
@@ -0,0 +1,42 @@
+dataset_type = 'KvasirSEGAliyunDataset'
+data_root = 'data/'
+img_scale = (512, 512)
+train_pipeline = [
+ dict(type='LoadImageFromFile'),
+ dict(type='LoadAnnotations'),
+ dict(type='Resize', scale=img_scale, keep_ratio=False),
+ dict(type='RandomFlip', prob=0.5),
+ dict(type='PhotoMetricDistortion'),
+ dict(type='PackSegInputs')
+]
+test_pipeline = [
+ dict(type='LoadImageFromFile'),
+ dict(type='Resize', scale=img_scale, keep_ratio=False),
+ dict(type='LoadAnnotations'),
+ dict(type='PackSegInputs')
+]
+train_dataloader = dict(
+ batch_size=16,
+ num_workers=4,
+ persistent_workers=True,
+ sampler=dict(type='InfiniteSampler', shuffle=True),
+ dataset=dict(
+ type=dataset_type,
+ data_root=data_root,
+ ann_file='train.txt',
+ data_prefix=dict(img_path='images/', seg_map_path='masks/'),
+ pipeline=train_pipeline))
+val_dataloader = dict(
+ batch_size=1,
+ num_workers=4,
+ persistent_workers=True,
+ sampler=dict(type='DefaultSampler', shuffle=False),
+ dataset=dict(
+ type=dataset_type,
+ data_root=data_root,
+ ann_file='val.txt',
+ data_prefix=dict(img_path='images/', seg_map_path='masks/'),
+ pipeline=test_pipeline))
+test_dataloader = val_dataloader
+val_evaluator = dict(type='IoUMetric', iou_metrics=['mIoU', 'mDice'])
+test_evaluator = dict(type='IoUMetric', iou_metrics=['mIoU', 'mDice'])
diff --git a/projects/medical/2d_image/endoscopy/kvasir_seg_aliyun/datasets/kvasir-seg-aliyun_dataset.py b/projects/medical/2d_image/endoscopy/kvasir_seg_aliyun/datasets/kvasir-seg-aliyun_dataset.py
new file mode 100644
index 00000000000..198caf07bcd
--- /dev/null
+++ b/projects/medical/2d_image/endoscopy/kvasir_seg_aliyun/datasets/kvasir-seg-aliyun_dataset.py
@@ -0,0 +1,30 @@
+from mmseg.datasets import BaseSegDataset
+from mmseg.registry import DATASETS
+
+
+@DATASETS.register_module()
+class KvasirSEGAliyunDataset(BaseSegDataset):
+ """KvasirSEGAliyunDataset dataset.
+
+ In segmentation map annotation for KvasirSEGAliyunDataset,
+ 0 stands for background,which is included in 2 categories.
+ ``reduce_zero_label`` is fixed to False. The ``img_suffix``
+ is fixed to '.png' and ``seg_map_suffix`` is fixed to '.png'.
+ Args:
+ img_suffix (str): Suffix of images. Default: '.png'
+ seg_map_suffix (str): Suffix of segmentation maps. Default: '.png'
+ reduce_zero_label (bool): Whether to mark label zero as ignored.
+ Default to False..
+ """
+ METAINFO = dict(classes=('background', 'polyp'))
+
+ def __init__(self,
+ img_suffix='.png',
+ seg_map_suffix='.png',
+ reduce_zero_label=False,
+ **kwargs) -> None:
+ super().__init__(
+ img_suffix=img_suffix,
+ seg_map_suffix=seg_map_suffix,
+ reduce_zero_label=reduce_zero_label,
+ **kwargs)
diff --git a/projects/medical/2d_image/endoscopy/kvasir_seg_aliyun/tools/prepare_dataset.py b/projects/medical/2d_image/endoscopy/kvasir_seg_aliyun/tools/prepare_dataset.py
new file mode 100644
index 00000000000..b230e7fef58
--- /dev/null
+++ b/projects/medical/2d_image/endoscopy/kvasir_seg_aliyun/tools/prepare_dataset.py
@@ -0,0 +1,86 @@
+import glob
+import os
+
+import numpy as np
+from PIL import Image
+
+root_path = 'data/'
+img_suffix = '.jpg'
+seg_map_suffix = '.jpg'
+save_img_suffix = '.png'
+save_seg_map_suffix = '.png'
+tgt_img_dir = os.path.join(root_path, 'images/train/')
+tgt_mask_dir = os.path.join(root_path, 'masks/train/')
+os.system('mkdir -p ' + tgt_img_dir)
+os.system('mkdir -p ' + tgt_mask_dir)
+
+
+def filter_suffix_recursive(src_dir, suffix):
+ # filter out file names and paths in source directory
+ suffix = '.' + suffix if '.' not in suffix else suffix
+ file_paths = glob.glob(
+ os.path.join(src_dir, '**', '*' + suffix), recursive=True)
+ file_names = [_.split('/')[-1] for _ in file_paths]
+ return sorted(file_paths), sorted(file_names)
+
+
+def convert_label(img, convert_dict):
+ arr = np.zeros_like(img, dtype=np.uint8)
+ for c, i in convert_dict.items():
+ arr[img == c] = i
+ return arr
+
+
+def convert_pics_into_pngs(src_dir, tgt_dir, suffix, convert='RGB'):
+ if not os.path.exists(tgt_dir):
+ os.makedirs(tgt_dir)
+
+ src_paths, src_names = filter_suffix_recursive(src_dir, suffix=suffix)
+ for i, (src_name, src_path) in enumerate(zip(src_names, src_paths)):
+ tgt_name = src_name.replace(suffix, save_img_suffix)
+ tgt_path = os.path.join(tgt_dir, tgt_name)
+ num = len(src_paths)
+ img = np.array(Image.open(src_path))
+ if len(img.shape) == 2:
+ pil = Image.fromarray(img).convert(convert)
+ elif len(img.shape) == 3:
+ pil = Image.fromarray(img)
+ else:
+ raise ValueError('Input image not 2D/3D: ', img.shape)
+
+ pil.save(tgt_path)
+ print(f'processed {i+1}/{num}.')
+
+
+def convert_label_pics_into_pngs(src_dir,
+ tgt_dir,
+ suffix,
+ convert_dict={
+ 0: 0,
+ 255: 1
+ }):
+ if not os.path.exists(tgt_dir):
+ os.makedirs(tgt_dir)
+
+ src_paths, src_names = filter_suffix_recursive(src_dir, suffix=suffix)
+ num = len(src_paths)
+ for i, (src_name, src_path) in enumerate(zip(src_names, src_paths)):
+ tgt_name = src_name.replace(suffix, save_seg_map_suffix)
+ tgt_path = os.path.join(tgt_dir, tgt_name)
+
+ img = np.array(Image.open(src_path).convert('L'))
+ img = convert_label(img, convert_dict)
+ Image.fromarray(img).save(tgt_path)
+ print(f'processed {i+1}/{num}.')
+
+
+if __name__ == '__main__':
+ convert_pics_into_pngs(
+ os.path.join(root_path, 'Kvasir-SEG/images'),
+ tgt_img_dir,
+ suffix=img_suffix)
+
+ convert_label_pics_into_pngs(
+ os.path.join(root_path, 'Kvasir-SEG/masks'),
+ tgt_mask_dir,
+ suffix=seg_map_suffix)
diff --git a/projects/medical/2d_image/fluorescein_angriogram/vampire/README.md b/projects/medical/2d_image/fluorescein_angriogram/vampire/README.md
new file mode 100644
index 00000000000..c2c61c46a0b
--- /dev/null
+++ b/projects/medical/2d_image/fluorescein_angriogram/vampire/README.md
@@ -0,0 +1,158 @@
+# Vessel Assessment and Measurement Platform for Images of the REtina
+
+## Description
+
+This project support **`Vessel Assessment and Measurement Platform for Images of the REtina`**, and the dataset used in this project can be downloaded from [here](https://vampire.computing.dundee.ac.uk/vesselseg.html).
+
+### Dataset Overview
+
+In order to promote evaluation of vessel segmentation on ultra-wide field-of-view (UWFV) fluorescein angriogram (FA) frames, we make public 8 frames from two different sequences, the manually annotated images and the result of our automatic vessel segmentation algorithm.
+
+### Original Statistic Information
+
+| Dataset name | Anatomical region | Task type | Modality | Num. Classes | Train/Val/Test Images | Train/Val/Test Labeled | Release Date | License |
+| ---------------------------------------------------------------- | ----------------- | ------------ | ---------------------- | ------------ | --------------------- | ---------------------- | ------------ | --------------------------------------------------------------- |
+| [Vampire](https://vampire.computing.dundee.ac.uk/vesselseg.html) | vessel | segmentation | fluorescein angriogram | 2 | 8/-/- | yes/-/- | 2017 | [CC-BY-NC 4.0](https://creativecommons.org/licenses/by-sa/4.0/) |
+
+| Class Name | Num. Train | Pct. Train | Num. Val | Pct. Val | Num. Test | Pct. Test |
+| :--------: | :--------: | :--------: | :------: | :------: | :-------: | :-------: |
+| background | 8 | 96.75 | - | - | - | - |
+| vessel | 8 | 3.25 | - | - | - | - |
+
+Note:
+
+- `Pct` means percentage of pixels in this category in all pixels.
+
+### Visualization
+
+
+
+## Dataset Citation
+
+```bibtex
+
+@inproceedings{perez2011improving,
+ title={Improving vessel segmentation in ultra-wide field-of-view retinal fluorescein angiograms},
+ author={Perez-Rovira, Adria and Zutis, K and Hubschman, Jean Pierre and Trucco, Emanuele},
+ booktitle={2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society},
+ pages={2614--2617},
+ year={2011},
+ organization={IEEE}
+}
+
+@article{perez2011rerbee,
+ title={RERBEE: robust efficient registration via bifurcations and elongated elements applied to retinal fluorescein angiogram sequences},
+ author={Perez-Rovira, Adria and Cabido, Raul and Trucco, Emanuele and McKenna, Stephen J and Hubschman, Jean Pierre},
+ journal={IEEE Transactions on Medical Imaging},
+ volume={31},
+ number={1},
+ pages={140--150},
+ year={2011},
+ publisher={IEEE}
+}
+
+```
+
+### Prerequisites
+
+- Python v3.8
+- PyTorch v1.10.0
+- pillow(PIL) v9.3.0
+- scikit-learn(sklearn) v1.2.0
+- [MIM](https://github.com/open-mmlab/mim) v0.3.4
+- [MMCV](https://github.com/open-mmlab/mmcv) v2.0.0rc4
+- [MMEngine](https://github.com/open-mmlab/mmengine) v0.2.0 or higher
+- [MMSegmentation](https://github.com/open-mmlab/mmsegmentation) v1.0.0rc5
+
+All the commands below rely on the correct configuration of `PYTHONPATH`, which should point to the project's directory so that Python can locate the module files. In `vampire/` root directory, run the following line to add the current directory to `PYTHONPATH`:
+
+```shell
+export PYTHONPATH=`pwd`:$PYTHONPATH
+```
+
+### Dataset preparing
+
+- download dataset from [here](https://vampire.computing.dundee.ac.uk/vesselseg.html) and decompression data to path `'data/'`.
+- run script `"python tools/prepare_dataset.py"` to split dataset and change folder structure as below.
+- run script `python ../../tools/split_seg_dataset.py` to split dataset. For the Bacteria_detection dataset, as there is no test or validation dataset, we sample 20% samples from the whole dataset as the validation dataset and 80% samples for training data and make two filename lists `train.txt` and `val.txt`. As we set the random seed as the hard code, we eliminated the randomness, the dataset split actually can be reproducible.
+
+```none
+ mmsegmentation
+ ├── mmseg
+ ├── projects
+ │ ├── medical
+ │ │ ├── 2d_image
+ │ │ │ ├── fluorescein_angriogram
+ │ │ │ │ ├── vampire
+ │ │ │ │ │ ├── configs
+ │ │ │ │ │ ├── datasets
+ │ │ │ │ │ ├── tools
+ │ │ │ │ │ ├── data
+ │ │ │ │ │ │ ├── train.txt
+ │ │ │ │ │ │ ├── val.txt
+ │ │ │ │ │ │ ├── images
+ │ │ │ │ │ │ │ ├── train
+ │ │ │ │ | │ │ │ ├── xxx.png
+ │ │ │ │ | │ │ │ ├── ...
+ │ │ │ │ | │ │ │ └── xxx.png
+ │ │ │ │ │ │ ├── masks
+ │ │ │ │ │ │ │ ├── train
+ │ │ │ │ | │ │ │ ├── xxx.png
+ │ │ │ │ | │ │ │ ├── ...
+ │ │ │ │ | │ │ │ └── xxx.png
+```
+
+### Divided Dataset Information
+
+***Note: The table information below is divided by ourselves.***
+
+| Class Name | Num. Train | Pct. Train | Num. Val | Pct. Val | Num. Test | Pct. Test |
+| :--------: | :--------: | :--------: | :------: | :------: | :-------: | :-------: |
+| background | 6 | 97.48 | 2 | 94.54 | - | - |
+| vessel | 6 | 2.52 | 2 | 5.46 | - | - |
+
+### Training commands
+
+To train models on a single server with one GPU. (default）
+
+```shell
+mim train mmseg ./configs/${CONFIG_PATH}
+```
+
+### Testing commands
+
+To test models on a single server with one GPU. (default)
+
+```shell
+mim test mmseg ./configs/${CONFIG_PATH} --checkpoint ${CHECKPOINT_PATH}
+```
+
+## Checklist
+
+- [x] Milestone 1: PR-ready, and acceptable to be one of the `projects/`.
+
+ - [x] Finish the code
+
+ - [x] Basic docstrings & proper citation
+
+ - [ ] Test-time correctness
+
+ - [x] A full README
+
+- [ ] Milestone 2: Indicates a successful model implementation.
+
+ - [ ] Training-time correctness
+
+- [ ] Milestone 3: Good to be a part of our core package!
+
+ - [ ] Type hints and docstrings
+
+ - [ ] Unit tests
+
+ - [ ] Code polishing
+
+ - [ ] Metafile.yml
+
+- [ ] Move your modules into the core package following the codebase's file hierarchy structure.
+
+- [ ] Refactor your modules into the core package following the codebase's file hierarchy structure.
diff --git a/projects/medical/2d_image/fluorescein_angriogram/vampire/configs/fcn-unet-s5-d16_unet_1xb16-0.0001-20k_vampire-512x512.py b/projects/medical/2d_image/fluorescein_angriogram/vampire/configs/fcn-unet-s5-d16_unet_1xb16-0.0001-20k_vampire-512x512.py
new file mode 100755
index 00000000000..7f5273aaff9
--- /dev/null
+++ b/projects/medical/2d_image/fluorescein_angriogram/vampire/configs/fcn-unet-s5-d16_unet_1xb16-0.0001-20k_vampire-512x512.py
@@ -0,0 +1,19 @@
+_base_ = [
+ 'mmseg::_base_/models/fcn_unet_s5-d16.py', './vampire_512x512.py',
+ 'mmseg::_base_/default_runtime.py',
+ 'mmseg::_base_/schedules/schedule_20k.py'
+]
+custom_imports = dict(imports='datasets.vampire_dataset')
+img_scale = (512, 512)
+data_preprocessor = dict(size=img_scale)
+optimizer = dict(lr=0.0001)
+optim_wrapper = dict(optimizer=optimizer)
+model = dict(
+ type='EncoderDecoder',
+ data_preprocessor=data_preprocessor,
+ pretrained=None,
+ decode_head=dict(num_classes=2),
+ auxiliary_head=None,
+ test_cfg=dict(mode='whole', _delete_=True))
+vis_backends = None
+visualizer = dict(vis_backends=vis_backends)
diff --git a/projects/medical/2d_image/fluorescein_angriogram/vampire/configs/fcn-unet-s5-d16_unet_1xb16-0.001-20k_vampire-512x512.py b/projects/medical/2d_image/fluorescein_angriogram/vampire/configs/fcn-unet-s5-d16_unet_1xb16-0.001-20k_vampire-512x512.py
new file mode 100755
index 00000000000..4382229989b
--- /dev/null
+++ b/projects/medical/2d_image/fluorescein_angriogram/vampire/configs/fcn-unet-s5-d16_unet_1xb16-0.001-20k_vampire-512x512.py
@@ -0,0 +1,19 @@
+_base_ = [
+ 'mmseg::_base_/models/fcn_unet_s5-d16.py', './vampire_512x512.py',
+ 'mmseg::_base_/default_runtime.py',
+ 'mmseg::_base_/schedules/schedule_20k.py'
+]
+custom_imports = dict(imports='datasets.vampire_dataset')
+img_scale = (512, 512)
+data_preprocessor = dict(size=img_scale)
+optimizer = dict(lr=0.001)
+optim_wrapper = dict(optimizer=optimizer)
+model = dict(
+ type='EncoderDecoder',
+ data_preprocessor=dict(size=img_scale),
+ pretrained=None,
+ decode_head=dict(num_classes=2),
+ auxiliary_head=None,
+ test_cfg=dict(mode='whole', _delete_=True))
+vis_backends = None
+visualizer = dict(vis_backends=vis_backends)
diff --git a/projects/medical/2d_image/fluorescein_angriogram/vampire/configs/fcn-unet-s5-d16_unet_1xb16-0.01-20k_vampire-512x512.py b/projects/medical/2d_image/fluorescein_angriogram/vampire/configs/fcn-unet-s5-d16_unet_1xb16-0.01-20k_vampire-512x512.py
new file mode 100755
index 00000000000..8d93e17627f
--- /dev/null
+++ b/projects/medical/2d_image/fluorescein_angriogram/vampire/configs/fcn-unet-s5-d16_unet_1xb16-0.01-20k_vampire-512x512.py
@@ -0,0 +1,22 @@
+_base_ = [
+ 'mmseg::_base_/models/fcn_unet_s5-d16.py', './vampire_512x512.py',
+ 'mmseg::_base_/default_runtime.py',
+ 'mmseg::_base_/schedules/schedule_20k.py'
+]
+custom_imports = dict(imports='datasets.vampire_dataset')
+img_scale = (512, 512)
+data_preprocessor = dict(size=img_scale)
+optimizer = dict(lr=0.01)
+optim_wrapper = dict(optimizer=optimizer)
+model = dict(
+ type='EncoderDecoder',
+ data_preprocessor=data_preprocessor,
+ pretrained=None,
+ decode_head=dict(
+ num_classes=2,
+ loss_decode=dict(type='CrossEntropyLoss', use_sigmoid=True),
+ out_channels=1),
+ auxiliary_head=None,
+ test_cfg=dict(mode='whole', _delete_=True))
+vis_backends = None
+visualizer = dict(vis_backends=vis_backends)
diff --git a/projects/medical/2d_image/fluorescein_angriogram/vampire/configs/vampire_512x512.py b/projects/medical/2d_image/fluorescein_angriogram/vampire/configs/vampire_512x512.py
new file mode 100755
index 00000000000..4eda92f9f22
--- /dev/null
+++ b/projects/medical/2d_image/fluorescein_angriogram/vampire/configs/vampire_512x512.py
@@ -0,0 +1,42 @@
+dataset_type = 'VampireDataset'
+data_root = 'data'
+img_scale = (512, 512)
+train_pipeline = [
+ dict(type='LoadImageFromFile'),
+ dict(type='LoadAnnotations'),
+ dict(type='Resize', scale=img_scale, keep_ratio=False),
+ dict(type='RandomFlip', prob=0.5),
+ dict(type='PhotoMetricDistortion'),
+ dict(type='PackSegInputs')
+]
+test_pipeline = [
+ dict(type='LoadImageFromFile'),
+ dict(type='Resize', scale=img_scale, keep_ratio=False),
+ dict(type='LoadAnnotations'),
+ dict(type='PackSegInputs')
+]
+train_dataloader = dict(
+ batch_size=16,
+ num_workers=4,
+ persistent_workers=True,
+ sampler=dict(type='InfiniteSampler', shuffle=True),
+ dataset=dict(
+ type=dataset_type,
+ data_root=data_root,
+ ann_file='train.txt',
+ data_prefix=dict(img_path='images/', seg_map_path='masks/'),
+ pipeline=train_pipeline))
+val_dataloader = dict(
+ batch_size=1,
+ num_workers=4,
+ persistent_workers=True,
+ sampler=dict(type='DefaultSampler', shuffle=False),
+ dataset=dict(
+ type=dataset_type,
+ data_root=data_root,
+ ann_file='val.txt',
+ data_prefix=dict(img_path='images/', seg_map_path='masks/'),
+ pipeline=test_pipeline))
+test_dataloader = val_dataloader
+val_evaluator = dict(type='IoUMetric', iou_metrics=['mIoU', 'mDice'])
+test_evaluator = dict(type='IoUMetric', iou_metrics=['mIoU', 'mDice'])
diff --git a/projects/medical/2d_image/fluorescein_angriogram/vampire/datasets/__init__.py b/projects/medical/2d_image/fluorescein_angriogram/vampire/datasets/__init__.py
new file mode 100755
index 00000000000..93f9cbf0506
--- /dev/null
+++ b/projects/medical/2d_image/fluorescein_angriogram/vampire/datasets/__init__.py
@@ -0,0 +1,3 @@
+from .vampire_dataset import VampireDataset
+
+__all__ = ['VampireDataset']
diff --git a/projects/medical/2d_image/fluorescein_angriogram/vampire/datasets/vampire_dataset.py b/projects/medical/2d_image/fluorescein_angriogram/vampire/datasets/vampire_dataset.py
new file mode 100755
index 00000000000..4d38040f7f1
--- /dev/null
+++ b/projects/medical/2d_image/fluorescein_angriogram/vampire/datasets/vampire_dataset.py
@@ -0,0 +1,28 @@
+from mmseg.datasets import BaseSegDataset
+from mmseg.registry import DATASETS
+
+
+@DATASETS.register_module()
+class VampireDataset(BaseSegDataset):
+ """VampireDataset dataset.
+
+ In segmentation map annotation for VampireDataset, 0 stands for background,
+ which is included in 2 categories. ``reduce_zero_label`` is fixed to
+ False. The ``img_suffix`` is fixed to '.png' and ``seg_map_suffix`` is
+ fixed to '.png'.
+ Args:
+ img_suffix (str): Suffix of images. Default: '.png'
+ seg_map_suffix (str): Suffix of segmentation maps. Default: '.png'
+ """
+ METAINFO = dict(classes=('background', 'vessel'))
+
+ def __init__(self,
+ img_suffix='.png',
+ seg_map_suffix='.png',
+ reduce_zero_label=False,
+ **kwargs) -> None:
+ super().__init__(
+ img_suffix=img_suffix,
+ seg_map_suffix=seg_map_suffix,
+ reduce_zero_label=reduce_zero_label,
+ **kwargs)
diff --git a/projects/medical/2d_image/fluorescein_angriogram/vampire/tools/prepare_dataset.py b/projects/medical/2d_image/fluorescein_angriogram/vampire/tools/prepare_dataset.py
new file mode 100644
index 00000000000..2755b5d28bd
--- /dev/null
+++ b/projects/medical/2d_image/fluorescein_angriogram/vampire/tools/prepare_dataset.py
@@ -0,0 +1,44 @@
+import os
+import shutil
+
+from PIL import Image
+
+path = 'data'
+
+if not os.path.exists(os.path.join(path, 'images', 'train')):
+ os.system(f'mkdir -p {os.path.join(path, "images", "train")}')
+
+if not os.path.exists(os.path.join(path, 'masks', 'train')):
+ os.system(f'mkdir -p {os.path.join(path, "masks", "train")}')
+
+origin_data_path = os.path.join(path, 'vesselSegmentation')
+
+imgs_amd14 = os.listdir(os.path.join(origin_data_path, 'AMD14'))
+imgs_ger7 = os.listdir(os.path.join(origin_data_path, 'GER7'))
+
+for img in imgs_amd14:
+ shutil.copy(
+ os.path.join(origin_data_path, 'AMD14', img),
+ os.path.join(path, 'images', 'train', img))
+ # copy GT
+ img_gt = img.replace('.png', '-GT.png')
+ shutil.copy(
+ os.path.join(origin_data_path, 'AMD14-GT', f'{img_gt}'),
+ os.path.join(path, 'masks', 'train', img))
+
+for img in imgs_ger7:
+ shutil.copy(
+ os.path.join(origin_data_path, 'GER7', img),
+ os.path.join(path, 'images', 'train', img))
+ # copy GT
+ img_gt = img.replace('.bmp', '-GT.png')
+ img = img.replace('bmp', 'png')
+ shutil.copy(
+ os.path.join(origin_data_path, 'GER7-GT', img_gt),
+ os.path.join(path, 'masks', 'train', img))
+
+imgs = os.listdir(os.path.join(path, 'images', 'train'))
+for img in imgs:
+ if not img.endswith('.png'):
+ im = Image.open(os.path.join(path, 'images', 'train', img))
+ im.save(os.path.join(path, 'images', 'train', img[:-4] + '.png'))
diff --git a/projects/medical/2d_image/fundus_photography/dr_hagis/README.md b/projects/medical/2d_image/fundus_photography/dr_hagis/README.md
new file mode 100644
index 00000000000..85d8a3e271c
--- /dev/null
+++ b/projects/medical/2d_image/fundus_photography/dr_hagis/README.md
@@ -0,0 +1,155 @@
+# DR HAGIS: Diabetic Retinopathy, Hypertension, Age-related macular degeneration and Glacuoma ImageS
+
+## Description
+
+This project supports **`DR HAGIS: Diabetic Retinopathy, Hypertension, Age-related macular degeneration and Glacuoma ImageS`**, which can be downloaded from [here](https://paperswithcode.com/dataset/dr-hagis).
+
+### Dataset Overview
+
+The DR HAGIS database has been created to aid the development of vessel extraction algorithms suitable for retinal screening programmes. Researchers are encouraged to test their segmentation algorithms using this database. All thirty-nine fundus images were obtained from a diabetic retinopathy screening programme in the UK. Hence, all images were taken from diabetic patients.
+
+Besides the fundus images, the manual segmentation of the retinal surface vessels is provided by an expert grader. These manually segmented images can be used as the ground truth to compare and assess the automatic vessel extraction algorithms. Masks of the FOV are provided as well to quantify the accuracy of vessel extraction within the FOV only. The images were acquired in different screening centers, therefore reflecting the range of image resolutions, digital cameras and fundus cameras used in the clinic. The fundus images were captured using a Topcon TRC-NW6s, Topcon TRC-NW8 or a Canon CR DGi fundus camera with a horizontal 45 degree field-of-view (FOV). The images are 4752x3168 pixels, 3456x2304 pixels, 3126x2136 pixels, 2896x1944 pixels or 2816x1880 pixels in size.
+
+### Original Statistic Information
+
+| Dataset name | Anatomical region | Task type | Modality | Num. Classes | Train/Val/Test Images | Train/Val/Test Labeled | Release Date | License |
+| ------------------------------------------------------- | ----------------- | ------------ | ------------------ | ------------ | --------------------- | ---------------------- | ------------ | ------- |
+| [DR HAGIS](https://paperswithcode.com/dataset/dr-hagis) | head and neck | segmentation | fundus photography | 2 | 40/-/- | yes/-/- | 2017 | - |
+
+| Class Name | Num. Train | Pct. Train | Num. Val | Pct. Val | Num. Test | Pct. Test |
+| :--------: | :--------: | :--------: | :------: | :------: | :-------: | :-------: |
+| background | 40 | 96.38 | - | - | - | - |
+| vessel | 40 | 3.62 | - | - | - | - |
+
+Note:
+
+- `Pct` means percentage of pixels in this category in all pixels.
+
+### Visualization
+
+
+
+## Usage
+
+### Prerequisites
+
+- Python v3.8
+- PyTorch v1.10.0
+- [MIM](https://github.com/open-mmlab/mim) v0.3.4
+- [MMCV](https://github.com/open-mmlab/mmcv) v2.0.0rc4
+- [MMEngine](https://github.com/open-mmlab/mmengine) v0.2.0 or higher
+- [MMSegmentation](https://github.com/open-mmlab/mmsegmentation) v1.0.0rc5
+
+All the commands below rely on the correct configuration of `PYTHONPATH`, which should point to the project's directory so that Python can locate the module files. In `dr_hagis/` root directory, run the following line to add the current directory to `PYTHONPATH`:
+
+```shell
+export PYTHONPATH=`pwd`:$PYTHONPATH
+```
+
+### Dataset preparing
+
+- download dataset from [here](https://paperswithcode.com/dataset/dr-hagis) and decompress data to path `'data/'`.
+- run script `"python tools/prepare_dataset.py"` to format data and change folder structure as below.
+- run script `"python ../../tools/split_seg_dataset.py"` to split dataset and generate `train.txt`, `val.txt` and `test.txt`. If the label of official validation set and test set can't be obtained, we generate `train.txt` and `val.txt` from the training set randomly.
+
+```none
+ mmsegmentation
+ ├── mmseg
+ ├── projects
+ │ ├── medical
+ │ │ ├── 2d_image
+ │ │ │ ├── fundus_photography
+ │ │ │ │ ├── dr_hagis
+ │ │ │ │ │ ├── configs
+ │ │ │ │ │ ├── datasets
+ │ │ │ │ │ ├── tools
+ │ │ │ │ │ ├── data
+ │ │ │ │ │ │ ├── train.txt
+ │ │ │ │ │ │ ├── val.txt
+ │ │ │ │ │ │ ├── images
+ │ │ │ │ │ │ │ ├── train
+ │ │ │ │ | │ │ │ ├── xxx.png
+ │ │ │ │ | │ │ │ ├── ...
+ │ │ │ │ | │ │ │ └── xxx.png
+ │ │ │ │ │ │ ├── masks
+ │ │ │ │ │ │ │ ├── train
+ │ │ │ │ | │ │ │ ├── xxx.png
+ │ │ │ │ | │ │ │ ├── ...
+ │ │ │ │ | │ │ │ └── xxx.png
+```
+
+### Divided Dataset Information
+
+***Note: The table information below is divided by ourselves.***
+
+| Class Name | Num. Train | Pct. Train | Num. Val | Pct. Val | Num. Test | Pct. Test |
+| :--------: | :--------: | :--------: | :------: | :------: | :-------: | :-------: |
+| background | 32 | 96.21 | 8 | 97.12 | - | - |
+| vessel | 32 | 3.79 | 8 | 2.88 | - | - |
+
+### Training commands
+
+Train models on a single server with one GPU.
+
+```shell
+mim train mmseg ./configs/${CONFIG_FILE}
+```
+
+### Testing commands
+
+Test models on a single server with one GPU.
+
+```shell
+mim test mmseg ./configs/${CONFIG_FILE} --checkpoint ${CHECKPOINT_PATH}
+```
+
+
+
+## Dataset Citation
+
+If this work is helpful for your research, please consider citing the below paper.
+
+```
+@article{holm2017dr,
+ title={DR HAGIS—a fundus image database for the automatic extraction of retinal surface vessels from diabetic patients},
+ author={Holm, Sven and Russell, Greg and Nourrit, Vincent and McLoughlin, Niall},
+ journal={Journal of Medical Imaging},
+ volume={4},
+ number={1},
+ pages={014503--014503},
+ year={2017},
+ publisher={Society of Photo-Optical Instrumentation Engineers}
+}
+```
+
+## Checklist
+
+- [x] Milestone 1: PR-ready, and acceptable to be one of the `projects/`.
+
+ - [x] Finish the code
+
+ - [x] Basic docstrings & proper citation
+
+ - [ ] Test-time correctness
+
+ - [x] A full README
+
+- [ ] Milestone 2: Indicates a successful model implementation.
+
+ - [ ] Training-time correctness
+
+- [ ] Milestone 3: Good to be a part of our core package!
+
+ - [ ] Type hints and docstrings
+
+ - [ ] Unit tests
+
+ - [ ] Code polishing
+
+ - [ ] Metafile.yml
+
+- [ ] Move your modules into the core package following the codebase's file hierarchy structure.
+
+- [ ] Refactor your modules into the core package following the codebase's file hierarchy structure.
diff --git a/projects/medical/2d_image/fundus_photography/dr_hagis/configs/dr-hagis_512x512.py b/projects/medical/2d_image/fundus_photography/dr_hagis/configs/dr-hagis_512x512.py
new file mode 100644
index 00000000000..93b96384109
--- /dev/null
+++ b/projects/medical/2d_image/fundus_photography/dr_hagis/configs/dr-hagis_512x512.py
@@ -0,0 +1,42 @@
+dataset_type = 'DRHAGISDataset'
+data_root = 'data/'
+img_scale = (512, 512)
+train_pipeline = [
+ dict(type='LoadImageFromFile'),
+ dict(type='LoadAnnotations'),
+ dict(type='Resize', scale=img_scale, keep_ratio=False),
+ dict(type='RandomFlip', prob=0.5),
+ dict(type='PhotoMetricDistortion'),
+ dict(type='PackSegInputs')
+]
+test_pipeline = [
+ dict(type='LoadImageFromFile'),
+ dict(type='Resize', scale=img_scale, keep_ratio=False),
+ dict(type='LoadAnnotations'),
+ dict(type='PackSegInputs')
+]
+train_dataloader = dict(
+ batch_size=16,
+ num_workers=4,
+ persistent_workers=True,
+ sampler=dict(type='InfiniteSampler', shuffle=True),
+ dataset=dict(
+ type=dataset_type,
+ data_root=data_root,
+ ann_file='train.txt',
+ data_prefix=dict(img_path='images/', seg_map_path='masks/'),
+ pipeline=train_pipeline))
+val_dataloader = dict(
+ batch_size=1,
+ num_workers=4,
+ persistent_workers=True,
+ sampler=dict(type='DefaultSampler', shuffle=False),
+ dataset=dict(
+ type=dataset_type,
+ data_root=data_root,
+ ann_file='val.txt',
+ data_prefix=dict(img_path='images/', seg_map_path='masks/'),
+ pipeline=test_pipeline))
+test_dataloader = val_dataloader
+val_evaluator = dict(type='IoUMetric', iou_metrics=['mIoU', 'mDice'])
+test_evaluator = dict(type='IoUMetric', iou_metrics=['mIoU', 'mDice'])
diff --git a/projects/medical/2d_image/fundus_photography/dr_hagis/configs/fcn-unet-s5-d16_unet_1xb16-0.0001-20k_dr-hagis-512x512.py b/projects/medical/2d_image/fundus_photography/dr_hagis/configs/fcn-unet-s5-d16_unet_1xb16-0.0001-20k_dr-hagis-512x512.py
new file mode 100644
index 00000000000..9d14427c45f
--- /dev/null
+++ b/projects/medical/2d_image/fundus_photography/dr_hagis/configs/fcn-unet-s5-d16_unet_1xb16-0.0001-20k_dr-hagis-512x512.py
@@ -0,0 +1,17 @@
+_base_ = [
+ './dr-hagis_512x512.py', 'mmseg::_base_/models/fcn_unet_s5-d16.py',
+ 'mmseg::_base_/default_runtime.py',
+ 'mmseg::_base_/schedules/schedule_20k.py'
+]
+custom_imports = dict(imports='datasets.dr-hagis_dataset')
+img_scale = (512, 512)
+data_preprocessor = dict(size=img_scale)
+optimizer = dict(lr=0.0001)
+optim_wrapper = dict(optimizer=optimizer)
+model = dict(
+ data_preprocessor=data_preprocessor,
+ decode_head=dict(num_classes=2),
+ auxiliary_head=None,
+ test_cfg=dict(mode='whole', _delete_=True))
+vis_backends = None
+visualizer = dict(vis_backends=vis_backends)
diff --git a/projects/medical/2d_image/fundus_photography/dr_hagis/configs/fcn-unet-s5-d16_unet_1xb16-0.001-20k_dr-hagis-512x512.py b/projects/medical/2d_image/fundus_photography/dr_hagis/configs/fcn-unet-s5-d16_unet_1xb16-0.001-20k_dr-hagis-512x512.py
new file mode 100644
index 00000000000..507ec748bf5
--- /dev/null
+++ b/projects/medical/2d_image/fundus_photography/dr_hagis/configs/fcn-unet-s5-d16_unet_1xb16-0.001-20k_dr-hagis-512x512.py
@@ -0,0 +1,17 @@
+_base_ = [
+ './dr-hagis_512x512.py', 'mmseg::_base_/models/fcn_unet_s5-d16.py',
+ 'mmseg::_base_/default_runtime.py',
+ 'mmseg::_base_/schedules/schedule_20k.py'
+]
+custom_imports = dict(imports='datasets.dr-hagis_dataset')
+img_scale = (512, 512)
+data_preprocessor = dict(size=img_scale)
+optimizer = dict(lr=0.001)
+optim_wrapper = dict(optimizer=optimizer)
+model = dict(
+ data_preprocessor=data_preprocessor,
+ decode_head=dict(num_classes=2),
+ auxiliary_head=None,
+ test_cfg=dict(mode='whole', _delete_=True))
+vis_backends = None
+visualizer = dict(vis_backends=vis_backends)
diff --git a/projects/medical/2d_image/fundus_photography/dr_hagis/configs/fcn-unet-s5-d16_unet_1xb16-0.01-20k_dr-hagis-512x512.py b/projects/medical/2d_image/fundus_photography/dr_hagis/configs/fcn-unet-s5-d16_unet_1xb16-0.01-20k_dr-hagis-512x512.py
new file mode 100644
index 00000000000..092ae00a7d3
--- /dev/null
+++ b/projects/medical/2d_image/fundus_photography/dr_hagis/configs/fcn-unet-s5-d16_unet_1xb16-0.01-20k_dr-hagis-512x512.py
@@ -0,0 +1,17 @@
+_base_ = [
+ './dr-hagis_512x512.py', 'mmseg::_base_/models/fcn_unet_s5-d16.py',
+ 'mmseg::_base_/default_runtime.py',
+ 'mmseg::_base_/schedules/schedule_20k.py'
+]
+custom_imports = dict(imports='datasets.dr-hagis_dataset')
+img_scale = (512, 512)
+data_preprocessor = dict(size=img_scale)
+optimizer = dict(lr=0.01)
+optim_wrapper = dict(optimizer=optimizer)
+model = dict(
+ data_preprocessor=data_preprocessor,
+ decode_head=dict(num_classes=2),
+ auxiliary_head=None,
+ test_cfg=dict(mode='whole', _delete_=True))
+vis_backends = None
+visualizer = dict(vis_backends=vis_backends)
diff --git a/projects/medical/2d_image/fundus_photography/dr_hagis/datasets/dr-hagis_dataset.py b/projects/medical/2d_image/fundus_photography/dr_hagis/datasets/dr-hagis_dataset.py
new file mode 100644
index 00000000000..9659f0b8d77
--- /dev/null
+++ b/projects/medical/2d_image/fundus_photography/dr_hagis/datasets/dr-hagis_dataset.py
@@ -0,0 +1,27 @@
+from mmseg.datasets import BaseSegDataset
+from mmseg.registry import DATASETS
+
+
+@DATASETS.register_module()
+class DRHAGISDataset(BaseSegDataset):
+ """DRHAGISDataset dataset.
+
+ In segmentation map annotation for DRHAGISDataset,
+ ``reduce_zero_label`` is fixed to False. The ``img_suffix``
+ is fixed to '.png' and ``seg_map_suffix`` is fixed to '.png'.
+
+ Args:
+ img_suffix (str): Suffix of images. Default: '.png'
+ seg_map_suffix (str): Suffix of segmentation maps. Default: '.png'
+ """
+ METAINFO = dict(classes=('background', 'vessel'))
+
+ def __init__(self,
+ img_suffix='.png',
+ seg_map_suffix='.png',
+ **kwargs) -> None:
+ super().__init__(
+ img_suffix=img_suffix,
+ seg_map_suffix=seg_map_suffix,
+ reduce_zero_label=False,
+ **kwargs)
diff --git a/projects/medical/2d_image/fundus_photography/dr_hagis/tools/prepare_dataset.py b/projects/medical/2d_image/fundus_photography/dr_hagis/tools/prepare_dataset.py
new file mode 100755
index 00000000000..51f4df7dac2
--- /dev/null
+++ b/projects/medical/2d_image/fundus_photography/dr_hagis/tools/prepare_dataset.py
@@ -0,0 +1,41 @@
+import glob
+import os
+import shutil
+
+import mmengine
+import numpy as np
+from PIL import Image
+
+root_path = 'data/'
+img_suffix = '.jpg'
+seg_map_suffix = '_manual_orig.png'
+save_img_suffix = '.png'
+save_seg_map_suffix = '.png'
+
+x_train = glob.glob(os.path.join('data/DRHAGIS/**/*' + img_suffix))
+
+mmengine.mkdir_or_exist(root_path + 'images/train/')
+mmengine.mkdir_or_exist(root_path + 'masks/train/')
+
+D3_palette = {0: (0, 0, 0), 1: (1, 1, 1)}
+D3_invert_palette = {v: k for k, v in D3_palette.items()}
+D2_255_convert_dict = {0: 0, 255: 1}
+
+part_dir_dict = {0: 'train/', 1: 'val/'}
+for ith, part in enumerate([x_train]):
+ part_dir = part_dir_dict[ith]
+ for img in part:
+ basename = os.path.basename(img)
+ shutil.copy(
+ img, root_path + 'images/' + part_dir + basename.split('.')[0] +
+ save_img_suffix)
+ mask_path = root_path + 'DRHAGIS/Manual_Segmentations/' + basename.split( # noqa
+ '.')[0] + seg_map_suffix
+ label = np.array(Image.open(mask_path))
+
+ save_mask_path = root_path + 'masks/' + part_dir + basename.split(
+ '.')[0] + save_seg_map_suffix # noqa
+ mask = np.array(Image.open(mask_path)).astype(np.uint8)
+ mask[mask == 255] = 1
+ mask = Image.fromarray(mask)
+ mask.save(save_mask_path)
diff --git a/projects/medical/2d_image/fundus_photography/gamma3/README.md b/projects/medical/2d_image/fundus_photography/gamma3/README.md
new file mode 100644
index 00000000000..e834508fcb5
--- /dev/null
+++ b/projects/medical/2d_image/fundus_photography/gamma3/README.md
@@ -0,0 +1,167 @@
+# Glaucoma grAding from Multi-Modality imAges Task3
+
+## Description
+
+This project support **`Glaucoma grAding from Multi-Modality imAges Task3`**, and the dataset used in this project can be downloaded from [here](https://aistudio.baidu.com/aistudio/competition/detail/121/0/datasets).
+
+### Dataset Overview
+
+This regular-challenge dataset was provided by Sun Yat-sen Ophthalmic Center, Sun Yat-sen University, Guangzhou, China. The dataset contains 200 fundus color images: 100 pairs in the training set and 100 pairs in the test set.
+
+### Original Statistic Information
+
+| Dataset name | Anatomical region | Task type | Modality | Num. Classes | Train/Val/Test Images | Train/Val/Test Labeled | Release Date | License |
+| ----------------------------------------------------------------------------------- | ----------------- | ------------ | --------------- | ------------ | --------------------- | ---------------------- | ------------ | --------------------------------------------------------------- |
+| [GammaTask3](https://aistudio.baidu.com/aistudio/competition/detail/121/0/datasets) | eye | segmentation | fundus photophy | 3 | 100/-/100 | yes/-/- | 2021 | [CC-BY-NC 4.0](https://creativecommons.org/licenses/by-sa/4.0/) |
+
+| Class Name | Num. Train | Pct. Train | Num. Val | Pct. Val | Num. Test | Pct. Test |
+| :--------: | :--------: | :--------: | :------: | :------: | :-------: | :-------: |
+| background | 100 | 99.02 | - | - | - | - |
+| optic disc | 100 | 0.67 | - | - | - | - |
+| optic cup | 100 | 0.31 | - | - | - | - |
+
+Note:
+
+- `Pct` means percentage of pixels in this category in all pixels.
+
+### Visualization
+
+
+
+## Dataset Citation
+
+```bibtex
+@article{fu2018joint,
+ title={Joint optic disc and cup segmentation based on multi-label deep network and polar transformation},
+ author={Fu, Huazhu and Cheng, Jun and Xu, Yanwu and Wong, Damon Wing Kee and Liu, Jiang and Cao, Xiaochun},
+ journal={IEEE transactions on medical imaging},
+ volume={37},
+ number={7},
+ pages={1597--1605},
+ year={2018},
+ publisher={IEEE}
+}
+
+@article{sevastopolsky2017optic,
+ title={Optic disc and cup segmentation methods for glaucoma detection with modification of U-Net convolutional neural network},
+ author={Sevastopolsky, Artem},
+ journal={Pattern Recognition and Image Analysis},
+ volume={27},
+ pages={618--624},
+ year={2017},
+ publisher={Springer}
+}
+```
+
+### Prerequisites
+
+- Python v3.8
+- PyTorch v1.10.0
+- pillow(PIL) v9.3.0
+- scikit-learn(sklearn) v1.2.0
+- [MIM](https://github.com/open-mmlab/mim) v0.3.4
+- [MMCV](https://github.com/open-mmlab/mmcv) v2.0.0rc4
+- [MMEngine](https://github.com/open-mmlab/mmengine) v0.2.0 or higher
+- [MMSegmentation](https://github.com/open-mmlab/mmsegmentation) v1.0.0rc5
+
+All the commands below rely on the correct configuration of `PYTHONPATH`, which should point to the project's directory so that Python can locate the module files. In `gammm3/` root directory, run the following line to add the current directory to `PYTHONPATH`:
+
+```shell
+export PYTHONPATH=`pwd`:$PYTHONPATH
+```
+
+### Dataset preparing
+
+- download dataset from [here](https://aistudio.baidu.com/aistudio/competition/detail/121/0/datasets) and decompression data to path `'data/'`.
+- run script `"python tools/prepare_dataset.py"` to split dataset and change folder structure as below.
+- run script `"python ../../tools/split_seg_dataset.py"` to split dataset and generate `train.txt`, `val.txt` and `test.txt`. If the label of official validation set and test set can't be obtained, we generate `train.txt` and `val.txt` from the training set randomly.
+
+```none
+ mmsegmentation
+ ├── mmseg
+ ├── projects
+ │ ├── medical
+ │ │ ├── 2d_image
+ │ │ │ ├── fundus_photography
+ │ │ │ │ ├── gamma3
+ │ │ │ │ │ ├── configs
+ │ │ │ │ │ ├── datasets
+ │ │ │ │ │ ├── tools
+ │ │ │ │ │ ├── data
+ │ │ │ │ │ │ ├── train.txt
+ │ │ │ │ │ │ ├── val.txt
+ │ │ │ │ │ │ ├── images
+ │ │ │ │ │ │ │ ├── train
+ │ │ │ │ | │ │ │ ├── xxx.png
+ │ │ │ │ | │ │ │ ├── ...
+ │ │ │ │ | │ │ │ └── xxx.png
+ │ │ │ │ │ │ │ ├── test
+ │ │ │ │ | │ │ │ ├── yyy.png
+ │ │ │ │ | │ │ │ ├── ...
+ │ │ │ │ | │ │ │ └── yyy.png
+ │ │ │ │ │ │ ├── masks
+ │ │ │ │ │ │ │ ├── train
+ │ │ │ │ | │ │ │ ├── xxx.png
+ │ │ │ │ | │ │ │ ├── ...
+ │ │ │ │ | │ │ │ └── xxx.png
+```
+
+### Divided Dataset Information
+
+***Note: The table information below is divided by ourselves.***
+
+| Class Name | Num. Train | Pct. Train | Num. Val | Pct. Val | Num. Test | Pct. Test |
+| :--------: | :--------: | :--------: | :------: | :------: | :-------: | :-------: |
+| background | 80 | 99.01 | 20 | 99.07 | - | - |
+| optic disc | 80 | 0.68 | 20 | 0.63 | - | - |
+| optic cup | 80 | 0.32 | 20 | 0.31 | - | - |
+
+### Training commands
+
+To train models on a single server with one GPU. (default）
+
+```shell
+mim train mmseg ./configs/${CONFIG_PATH}
+```
+
+### Testing commands
+
+To test models on a single server with one GPU. (default）
+
+```shell
+mim test mmseg ./configs/${CONFIG_PATH} --checkpoint ${CHECKPOINT_PATH}
+```
+
+
+
+## Checklist
+
+- [x] Milestone 1: PR-ready, and acceptable to be one of the `projects/`.
+
+ - [x] Finish the code
+
+ - [x] Basic docstrings & proper citation
+
+ - [ ] Test-time correctness
+
+ - [x] A full README
+
+- [ ] Milestone 2: Indicates a successful model implementation.
+
+ - [ ] Training-time correctness
+
+- [ ] Milestone 3: Good to be a part of our core package!
+
+ - [ ] Type hints and docstrings
+
+ - [ ] Unit tests
+
+ - [ ] Code polishing
+
+ - [ ] Metafile.yml
+
+- [ ] Move your modules into the core package following the codebase's file hierarchy structure.
+
+- [ ] Refactor your modules into the core package following the codebase's file hierarchy structure.
diff --git a/projects/medical/2d_image/fundus_photography/gamma3/configs/fcn-unet-s5-d16_unet_1xb16-0.0001-20k_gamma3-512x512.py b/projects/medical/2d_image/fundus_photography/gamma3/configs/fcn-unet-s5-d16_unet_1xb16-0.0001-20k_gamma3-512x512.py
new file mode 100644
index 00000000000..0daac51e10f
--- /dev/null
+++ b/projects/medical/2d_image/fundus_photography/gamma3/configs/fcn-unet-s5-d16_unet_1xb16-0.0001-20k_gamma3-512x512.py
@@ -0,0 +1,17 @@
+_base_ = [
+ 'mmseg::_base_/models/fcn_unet_s5-d16.py', './gamma3_512x512.py',
+ 'mmseg::_base_/default_runtime.py',
+ 'mmseg::_base_/schedules/schedule_20k.py'
+]
+custom_imports = dict(imports='datasets.gamma3_dataset')
+img_scale = (512, 512)
+data_preprocessor = dict(size=img_scale)
+optimizer = dict(lr=0.0001)
+optim_wrapper = dict(optimizer=optimizer)
+model = dict(
+ data_preprocessor=data_preprocessor,
+ decode_head=dict(num_classes=3),
+ auxiliary_head=None,
+ test_cfg=dict(mode='whole', _delete_=True))
+vis_backends = None
+visualizer = dict(vis_backends=vis_backends)
diff --git a/projects/medical/2d_image/fundus_photography/gamma3/configs/fcn-unet-s5-d16_unet_1xb16-0.001-20k_gamma3-512x512.py b/projects/medical/2d_image/fundus_photography/gamma3/configs/fcn-unet-s5-d16_unet_1xb16-0.001-20k_gamma3-512x512.py
new file mode 100644
index 00000000000..8a25cd0d266
--- /dev/null
+++ b/projects/medical/2d_image/fundus_photography/gamma3/configs/fcn-unet-s5-d16_unet_1xb16-0.001-20k_gamma3-512x512.py
@@ -0,0 +1,17 @@
+_base_ = [
+ 'mmseg::_base_/models/fcn_unet_s5-d16.py', './gamma3_512x512.py',
+ 'mmseg::_base_/default_runtime.py',
+ 'mmseg::_base_/schedules/schedule_20k.py'
+]
+custom_imports = dict(imports='datasets.gamma3_dataset')
+img_scale = (512, 512)
+data_preprocessor = dict(size=img_scale)
+optimizer = dict(lr=0.001)
+optim_wrapper = dict(optimizer=optimizer)
+model = dict(
+ data_preprocessor=data_preprocessor,
+ decode_head=dict(num_classes=3),
+ auxiliary_head=None,
+ test_cfg=dict(mode='whole', _delete_=True))
+vis_backends = None
+visualizer = dict(vis_backends=vis_backends)
diff --git a/projects/medical/2d_image/fundus_photography/gamma3/configs/fcn-unet-s5-d16_unet_1xb16-0.01-20k_gamma3-512x512.py b/projects/medical/2d_image/fundus_photography/gamma3/configs/fcn-unet-s5-d16_unet_1xb16-0.01-20k_gamma3-512x512.py
new file mode 100644
index 00000000000..ea648438672
--- /dev/null
+++ b/projects/medical/2d_image/fundus_photography/gamma3/configs/fcn-unet-s5-d16_unet_1xb16-0.01-20k_gamma3-512x512.py
@@ -0,0 +1,17 @@
+_base_ = [
+ 'mmseg::_base_/models/fcn_unet_s5-d16.py', './gamma3_512x512.py',
+ 'mmseg::_base_/default_runtime.py',
+ 'mmseg::_base_/schedules/schedule_20k.py'
+]
+custom_imports = dict(imports='datasets.gamma3_dataset')
+img_scale = (512, 512)
+data_preprocessor = dict(size=img_scale)
+optimizer = dict(lr=0.01)
+optim_wrapper = dict(optimizer=optimizer)
+model = dict(
+ data_preprocessor=data_preprocessor,
+ decode_head=dict(num_classes=3),
+ auxiliary_head=None,
+ test_cfg=dict(mode='whole', _delete_=True))
+vis_backends = None
+visualizer = dict(vis_backends=vis_backends)
diff --git a/projects/medical/2d_image/fundus_photography/gamma3/configs/gamma3_512x512.py b/projects/medical/2d_image/fundus_photography/gamma3/configs/gamma3_512x512.py
new file mode 100644
index 00000000000..d23ab55ca71
--- /dev/null
+++ b/projects/medical/2d_image/fundus_photography/gamma3/configs/gamma3_512x512.py
@@ -0,0 +1,42 @@
+dataset_type = 'Gamma3Dataset'
+data_root = 'data/'
+img_scale = (512, 512)
+train_pipeline = [
+ dict(type='LoadImageFromFile'),
+ dict(type='LoadAnnotations'),
+ dict(type='Resize', scale=img_scale, keep_ratio=False),
+ dict(type='RandomFlip', prob=0.5),
+ dict(type='PhotoMetricDistortion'),
+ dict(type='PackSegInputs')
+]
+test_pipeline = [
+ dict(type='LoadImageFromFile'),
+ dict(type='Resize', scale=img_scale, keep_ratio=False),
+ dict(type='LoadAnnotations'),
+ dict(type='PackSegInputs')
+]
+train_dataloader = dict(
+ batch_size=16,
+ num_workers=4,
+ persistent_workers=True,
+ sampler=dict(type='InfiniteSampler', shuffle=True),
+ dataset=dict(
+ type=dataset_type,
+ data_root=data_root,
+ ann_file='train.txt',
+ data_prefix=dict(img_path='images/', seg_map_path='masks/'),
+ pipeline=train_pipeline))
+val_dataloader = dict(
+ batch_size=1,
+ num_workers=4,
+ persistent_workers=True,
+ sampler=dict(type='DefaultSampler', shuffle=False),
+ dataset=dict(
+ type=dataset_type,
+ data_root=data_root,
+ ann_file='val.txt',
+ data_prefix=dict(img_path='images/', seg_map_path='masks/'),
+ pipeline=test_pipeline))
+test_dataloader = val_dataloader
+val_evaluator = dict(type='IoUMetric', iou_metrics=['mIoU', 'mDice'])
+test_evaluator = dict(type='IoUMetric', iou_metrics=['mIoU', 'mDice'])
diff --git a/projects/medical/2d_image/fundus_photography/gamma3/datasets/gamma3_dataset.py b/projects/medical/2d_image/fundus_photography/gamma3/datasets/gamma3_dataset.py
new file mode 100644
index 00000000000..56cbdd63e61
--- /dev/null
+++ b/projects/medical/2d_image/fundus_photography/gamma3/datasets/gamma3_dataset.py
@@ -0,0 +1,30 @@
+from mmseg.datasets import BaseSegDataset
+from mmseg.registry import DATASETS
+
+
+@DATASETS.register_module()
+class Gamma3Dataset(BaseSegDataset):
+ """Gamma3Dataset dataset.
+
+ In segmentation map annotation for Gamma3Dataset,
+ ``reduce_zero_label`` is fixed to False. The ``img_suffix``
+ is fixed to '.png' and ``seg_map_suffix`` is fixed to '.png'.
+
+ Args:
+ img_suffix (str): Suffix of images. Default: '.png'
+ seg_map_suffix (str): Suffix of segmentation maps. Default: '.png'
+ reduce_zero_label (bool): Whether to mark label zero as ignored.
+ Default to False.
+ """
+ METAINFO = dict(classes=('background', 'disc', 'cup'))
+
+ def __init__(self,
+ img_suffix='.png',
+ seg_map_suffix='.png',
+ reduce_zero_label=False,
+ **kwargs) -> None:
+ super().__init__(
+ img_suffix=img_suffix,
+ seg_map_suffix=seg_map_suffix,
+ reduce_zero_label=reduce_zero_label,
+ **kwargs)
diff --git a/projects/medical/2d_image/fundus_photography/gamma3/tools/prepare_dataset.py b/projects/medical/2d_image/fundus_photography/gamma3/tools/prepare_dataset.py
new file mode 100644
index 00000000000..eb820b6b740
--- /dev/null
+++ b/projects/medical/2d_image/fundus_photography/gamma3/tools/prepare_dataset.py
@@ -0,0 +1,107 @@
+import glob
+import os
+
+import numpy as np
+from PIL import Image
+
+root_path = 'data/'
+img_suffix = '.jpg'
+seg_map_suffix = '.png'
+save_img_suffix = '.png'
+save_seg_map_suffix = '.png'
+tgt_img_train_dir = os.path.join(root_path, 'images/train/')
+tgt_mask_train_dir = os.path.join(root_path, 'masks/train/')
+tgt_img_test_dir = os.path.join(root_path, 'images/test/')
+os.system('mkdir -p ' + tgt_img_train_dir)
+os.system('mkdir -p ' + tgt_mask_train_dir)
+os.system('mkdir -p ' + tgt_img_test_dir)
+
+
+def filter_suffix_recursive(src_dir, suffix):
+ # filter out file names and paths in source directory
+ suffix = '.' + suffix if '.' not in suffix else suffix
+ file_paths = glob.glob(
+ os.path.join(src_dir, '**/*' + suffix), recursive=True)
+ file_names = [_.split('/')[-1] for _ in file_paths]
+ return sorted(file_paths), sorted(file_names)
+
+
+def convert_label(img, convert_dict):
+ arr = np.zeros_like(img, dtype=np.uint8)
+ for c, i in convert_dict.items():
+ arr[img == c] = i
+ return arr
+
+
+def convert_pics_into_pngs(src_dir, tgt_dir, suffix, convert='RGB'):
+ if not os.path.exists(tgt_dir):
+ os.makedirs(tgt_dir)
+ src_paths, src_names = filter_suffix_recursive(src_dir, suffix=suffix)
+
+ for i, (src_name, src_path) in enumerate(zip(src_names, src_paths)):
+ tgt_name = src_name.replace(suffix, save_img_suffix)
+ tgt_path = os.path.join(tgt_dir, tgt_name)
+ num = len(src_paths)
+ img = np.array(Image.open(src_path))
+ if len(img.shape) == 2:
+ pil = Image.fromarray(img).convert(convert)
+ elif len(img.shape) == 3:
+ pil = Image.fromarray(img)
+ else:
+ raise ValueError('Input image not 2D/3D: ', img.shape)
+
+ pil.save(tgt_path)
+ print(f'processed {i+1}/{num}.')
+
+
+def convert_label_pics_into_pngs(src_dir,
+ tgt_dir,
+ suffix,
+ convert_dict={
+ 0: 2,
+ 128: 1,
+ 255: 0
+ }):
+ if not os.path.exists(tgt_dir):
+ os.makedirs(tgt_dir)
+
+ src_paths, src_names = filter_suffix_recursive(src_dir, suffix=suffix)
+ num = len(src_paths)
+ for i, (src_name, src_path) in enumerate(zip(src_names, src_paths)):
+ tgt_name = src_name.replace(suffix, save_seg_map_suffix)
+ tgt_path = os.path.join(tgt_dir, tgt_name)
+
+ img = np.array(Image.open(src_path))
+ img = convert_label(img, convert_dict)
+ Image.fromarray(img).save(tgt_path)
+ print(f'processed {i+1}/{num}.')
+
+
+if __name__ == '__main__':
+
+ convert_pics_into_pngs(
+ os.path.join(
+ root_path,
+ 'task3_disc_cup_segmentation/training/fundus color images/'),
+ tgt_img_train_dir,
+ suffix=img_suffix)
+
+ convert_pics_into_pngs(
+ os.path.join(
+ root_path,
+ 'task3_disc_cup_segmentation/testing/fundus color images/'),
+ tgt_img_test_dir,
+ suffix=img_suffix)
+
+ convert_label_pics_into_pngs(
+ os.path.join(root_path,
+ 'task3_disc_cup_segmentation/training/Disc_Cup_Mask/'),
+ tgt_mask_train_dir,
+ suffix=seg_map_suffix,
+ convert_dict={
+ 0: 2,
+ 128: 1,
+ 255: 0
+ })
+ # original: [0, 128, 255] for ['optic cup', 'optic disc', 'background']
+ # converted: [0, 1, 2] for ['background', 'optic disc', 'optic cup']
diff --git a/projects/medical/2d_image/fundus_photography/orvs/README.md b/projects/medical/2d_image/fundus_photography/orvs/README.md
new file mode 100644
index 00000000000..6f09203ac4d
--- /dev/null
+++ b/projects/medical/2d_image/fundus_photography/orvs/README.md
@@ -0,0 +1,140 @@
+# ORVS (Online Retinal image for Vessel Segmentation (ORVS))
+
+## Description
+
+This project supports **`ORVS (Online Retinal image for Vessel Segmentation (ORVS))`**, which can be downloaded from [here](https://opendatalab.org.cn/ORVS).
+
+### Dataset Overview
+
+The ORVS dataset is a newly established collaboration between the Department of Computer Science and the Department of Vision Science at the University of Calgary. The dataset contains 49 images collected from a clinic in Calgary, Canada, consisting of 42 training images and 7 testing images. All images were obtained using a Zeiss Visucam 200 with a 30-degree field of view (FOV). The image size is 1444×1444 pixels with 24 bits per pixel. The images are stored in JPEG format with low compression, which is common in ophthalmic practice. All images were manually traced by an expert who has been working in the field of retinal image analysis and has been trained to mark all pixels belonging to retinal vessels. The Windows Paint 3D tool was used for manual image annotation.
+
+### Original Statistic Information
+
+| Dataset name | Anatomical region | Task type | Modality | Num. Classes | Train/Val/Test Images | Train/Val/Test Labeled | Release Date | License |
+| ------------------------------------------------------ | ----------------- | ------------ | ------------------ | ------------ | --------------------- | ---------------------- | ------------ | ------- |
+| [Bactteria detection](https://opendatalab.org.cn/ORVS) | bacteria | segmentation | fundus photography | 2 | 130/-/72 | yes/-/yes | 2020 | - |
+
+| Class Name | Num. Train | Pct. Train | Num. Val | Pct. Val | Num. Test | Pct. Test |
+| :--------: | :--------: | :--------: | :------: | :------: | :-------: | :-------: |
+| background | 130 | 94.83 | - | - | 72 | 94.25 |
+| vessel | 130 | 5.17 | - | - | 72 | 5.75 |
+
+Note:
+
+- `Pct` means percentage of pixels in this category in all pixels.
+
+### Visualization
+
+
+
+### Prerequisites
+
+- Python v3.8
+- PyTorch v1.10.0
+- [MIM](https://github.com/open-mmlab/mim) v0.3.4
+- [MMCV](https://github.com/open-mmlab/mmcv) v2.0.0rc4
+- [MMEngine](https://github.com/open-mmlab/mmengine) v0.2.0 or higher
+- [MMSegmentation](https://github.com/open-mmlab/mmsegmentation) v1.0.0rc5
+
+All the commands below rely on the correct configuration of `PYTHONPATH`, which should point to the project's directory so that Python can locate the module files. In `orvs/` root directory, run the following line to add the current directory to `PYTHONPATH`:
+
+```shell
+export PYTHONPATH=`pwd`:$PYTHONPATH
+```
+
+### Dataset preparing
+
+- Clone this [repository](https://github.com/AbdullahSarhan/ICPRVessels), then move `Vessels-Datasets` to `data/`.
+- run script `"python tools/prepare_dataset.py"` to format data and change folder structure as below.
+- run script `"python ../../tools/split_seg_dataset.py"` to split dataset and generate `train.txt`, `val.txt` and `test.txt`. If the label of official validation set and test set can't be obtained, we generate `train.txt` and `val.txt` from the training set randomly.
+
+```none
+ mmsegmentation
+ ├── mmseg
+ ├── projects
+ │ ├── medical
+ │ │ ├── 2d_image
+ │ │ │ ├── fundus_photography
+ │ │ │ │ ├── orvs
+ │ │ │ │ │ ├── configs
+ │ │ │ │ │ ├── datasets
+ │ │ │ │ │ ├── tools
+ │ │ │ │ │ ├── data
+ │ │ │ │ │ │ ├── train.txt
+ │ │ │ │ │ │ ├── test.txt
+ │ │ │ │ │ │ ├── images
+ │ │ │ │ │ │ │ ├── train
+ │ │ │ │ | │ │ │ ├── xxx.png
+ │ │ │ │ | │ │ │ ├── ...
+ │ │ │ │ | │ │ │ └── xxx.png
+ │ │ │ │ │ │ ├── masks
+ │ │ │ │ │ │ │ ├── train
+ │ │ │ │ | │ │ │ ├── xxx.png
+ │ │ │ │ | │ │ │ ├── ...
+ │ │ │ │ | │ │ │ └── xxx.png
+```
+
+### Training commands
+
+Train models on a single server with one GPU.
+
+```shell
+mim train mmseg ./configs/${CONFIG_FILE}
+```
+
+### Testing commands
+
+Test models on a single server with one GPU.
+
+```shell
+mim test mmseg ./configs/${CONFIG_FILE} --checkpoint ${CHECKPOINT_PATH}
+```
+
+
+
+## Dataset Citation
+
+If this work is helpful for your research, please consider citing the below paper.
+
+```
+@inproceedings{sarhan2021transfer,
+ title={Transfer learning through weighted loss function and group normalization for vessel segmentation from retinal images},
+ author={Sarhan, Abdullah and Rokne, Jon and Alhajj, Reda and Crichton, Andrew},
+ booktitle={2020 25th International Conference on Pattern Recognition (ICPR)},
+ pages={9211--9218},
+ year={2021},
+ organization={IEEE}
+}
+```
+
+## Checklist
+
+- [x] Milestone 1: PR-ready, and acceptable to be one of the `projects/`.
+
+ - [x] Finish the code
+
+ - [x] Basic docstrings & proper citation
+
+ - [ ] Test-time correctness
+
+ - [x] A full README
+
+- [ ] Milestone 2: Indicates a successful model implementation.
+
+ - [ ] Training-time correctness
+
+- [ ] Milestone 3: Good to be a part of our core package!
+
+ - [ ] Type hints and docstrings
+
+ - [ ] Unit tests
+
+ - [ ] Code polishing
+
+ - [ ] Metafile.yml
+
+- [ ] Move your modules into the core package following the codebase's file hierarchy structure.
+
+- [ ] Refactor your modules into the core package following the codebase's file hierarchy structure.
diff --git a/projects/medical/2d_image/fundus_photography/orvs/configs/fcn-unet-s5-d16_unet_1xb16-0.0001-20k_orvs-512x512.py b/projects/medical/2d_image/fundus_photography/orvs/configs/fcn-unet-s5-d16_unet_1xb16-0.0001-20k_orvs-512x512.py
new file mode 100644
index 00000000000..662f837158a
--- /dev/null
+++ b/projects/medical/2d_image/fundus_photography/orvs/configs/fcn-unet-s5-d16_unet_1xb16-0.0001-20k_orvs-512x512.py
@@ -0,0 +1,17 @@
+_base_ = [
+ './orvs_512x512.py', 'mmseg::_base_/models/fcn_unet_s5-d16.py',
+ 'mmseg::_base_/default_runtime.py',
+ 'mmseg::_base_/schedules/schedule_20k.py'
+]
+custom_imports = dict(imports='datasets.orvs_dataset')
+img_scale = (512, 512)
+data_preprocessor = dict(size=img_scale)
+optimizer = dict(lr=0.0001)
+optim_wrapper = dict(optimizer=optimizer)
+model = dict(
+ data_preprocessor=data_preprocessor,
+ decode_head=dict(num_classes=2),
+ auxiliary_head=None,
+ test_cfg=dict(mode='whole', _delete_=True))
+vis_backends = None
+visualizer = dict(vis_backends=vis_backends)
diff --git a/projects/medical/2d_image/fundus_photography/orvs/configs/fcn-unet-s5-d16_unet_1xb16-0.001-20k_orvs-512x512.py b/projects/medical/2d_image/fundus_photography/orvs/configs/fcn-unet-s5-d16_unet_1xb16-0.001-20k_orvs-512x512.py
new file mode 100644
index 00000000000..c47cdb6b24d
--- /dev/null
+++ b/projects/medical/2d_image/fundus_photography/orvs/configs/fcn-unet-s5-d16_unet_1xb16-0.001-20k_orvs-512x512.py
@@ -0,0 +1,17 @@
+_base_ = [
+ './orvs_512x512.py', 'mmseg::_base_/models/fcn_unet_s5-d16.py',
+ 'mmseg::_base_/default_runtime.py',
+ 'mmseg::_base_/schedules/schedule_20k.py'
+]
+custom_imports = dict(imports='datasets.orvs_dataset')
+img_scale = (512, 512)
+data_preprocessor = dict(size=img_scale)
+optimizer = dict(lr=0.001)
+optim_wrapper = dict(optimizer=optimizer)
+model = dict(
+ data_preprocessor=data_preprocessor,
+ decode_head=dict(num_classes=2),
+ auxiliary_head=None,
+ test_cfg=dict(mode='whole', _delete_=True))
+vis_backends = None
+visualizer = dict(vis_backends=vis_backends)
diff --git a/projects/medical/2d_image/fundus_photography/orvs/configs/fcn-unet-s5-d16_unet_1xb16-0.01-20k_orvs-512x512.py b/projects/medical/2d_image/fundus_photography/orvs/configs/fcn-unet-s5-d16_unet_1xb16-0.01-20k_orvs-512x512.py
new file mode 100644
index 00000000000..1097aade286
--- /dev/null
+++ b/projects/medical/2d_image/fundus_photography/orvs/configs/fcn-unet-s5-d16_unet_1xb16-0.01-20k_orvs-512x512.py
@@ -0,0 +1,17 @@
+_base_ = [
+ './orvs_512x512.py', 'mmseg::_base_/models/fcn_unet_s5-d16.py',
+ 'mmseg::_base_/default_runtime.py',
+ 'mmseg::_base_/schedules/schedule_20k.py'
+]
+custom_imports = dict(imports='datasets.orvs_dataset')
+img_scale = (512, 512)
+data_preprocessor = dict(size=img_scale)
+optimizer = dict(lr=0.01)
+optim_wrapper = dict(optimizer=optimizer)
+model = dict(
+ data_preprocessor=data_preprocessor,
+ decode_head=dict(num_classes=2),
+ auxiliary_head=None,
+ test_cfg=dict(mode='whole', _delete_=True))
+vis_backends = None
+visualizer = dict(vis_backends=vis_backends)
diff --git a/projects/medical/2d_image/fundus_photography/orvs/configs/orvs_512x512.py b/projects/medical/2d_image/fundus_photography/orvs/configs/orvs_512x512.py
new file mode 100644
index 00000000000..a5594dec388
--- /dev/null
+++ b/projects/medical/2d_image/fundus_photography/orvs/configs/orvs_512x512.py
@@ -0,0 +1,42 @@
+dataset_type = 'ORVSDataset'
+data_root = 'data/'
+img_scale = (512, 512)
+train_pipeline = [
+ dict(type='LoadImageFromFile'),
+ dict(type='LoadAnnotations'),
+ dict(type='Resize', scale=img_scale, keep_ratio=False),
+ dict(type='RandomFlip', prob=0.5),
+ dict(type='PhotoMetricDistortion'),
+ dict(type='PackSegInputs')
+]
+test_pipeline = [
+ dict(type='LoadImageFromFile'),
+ dict(type='Resize', scale=img_scale, keep_ratio=False),
+ dict(type='LoadAnnotations'),
+ dict(type='PackSegInputs')
+]
+train_dataloader = dict(
+ batch_size=16,
+ num_workers=4,
+ persistent_workers=True,
+ sampler=dict(type='InfiniteSampler', shuffle=True),
+ dataset=dict(
+ type=dataset_type,
+ data_root=data_root,
+ ann_file='train.txt',
+ data_prefix=dict(img_path='images/', seg_map_path='masks/'),
+ pipeline=train_pipeline))
+val_dataloader = dict(
+ batch_size=1,
+ num_workers=4,
+ persistent_workers=True,
+ sampler=dict(type='DefaultSampler', shuffle=False),
+ dataset=dict(
+ type=dataset_type,
+ data_root=data_root,
+ ann_file='test.txt',
+ data_prefix=dict(img_path='images/', seg_map_path='masks/'),
+ pipeline=test_pipeline))
+test_dataloader = val_dataloader
+val_evaluator = dict(type='IoUMetric', iou_metrics=['mIoU', 'mDice'])
+test_evaluator = dict(type='IoUMetric', iou_metrics=['mIoU', 'mDice'])
diff --git a/projects/medical/2d_image/fundus_photography/orvs/datasets/orvs_dataset.py b/projects/medical/2d_image/fundus_photography/orvs/datasets/orvs_dataset.py
new file mode 100644
index 00000000000..e915ae4cd2b
--- /dev/null
+++ b/projects/medical/2d_image/fundus_photography/orvs/datasets/orvs_dataset.py
@@ -0,0 +1,27 @@
+from mmseg.datasets import BaseSegDataset
+from mmseg.registry import DATASETS
+
+
+@DATASETS.register_module()
+class ORVSDataset(BaseSegDataset):
+ """ORVSDataset dataset.
+
+ In segmentation map annotation for ORVSDataset,
+ ``reduce_zero_label`` is fixed to False. The ``img_suffix``
+ is fixed to '.png' and ``seg_map_suffix`` is fixed to '.png'.
+
+ Args:
+ img_suffix (str): Suffix of images. Default: '.png'
+ seg_map_suffix (str): Suffix of segmentation maps. Default: '.png'
+ """
+ METAINFO = dict(classes=('background', 'vessel'))
+
+ def __init__(self,
+ img_suffix='.png',
+ seg_map_suffix='.png',
+ **kwargs) -> None:
+ super().__init__(
+ img_suffix=img_suffix,
+ seg_map_suffix=seg_map_suffix,
+ reduce_zero_label=False,
+ **kwargs)
diff --git a/projects/medical/2d_image/fundus_photography/orvs/tools/prepare_dataset.py b/projects/medical/2d_image/fundus_photography/orvs/tools/prepare_dataset.py
new file mode 100755
index 00000000000..f902d871010
--- /dev/null
+++ b/projects/medical/2d_image/fundus_photography/orvs/tools/prepare_dataset.py
@@ -0,0 +1,55 @@
+import glob
+import os
+
+import numpy as np
+from PIL import Image
+
+root_path = 'data/'
+img_suffix = '.jpg'
+seg_map_suffix_list = ['.jpg', '.png', '.tif']
+save_img_suffix = '.png'
+save_seg_map_suffix = '.png'
+
+x_train = glob.glob(
+ os.path.join('data/Vessels-Datasets/*/Train/Original/Images/*' +
+ img_suffix))
+x_test = glob.glob(
+ os.path.join('data/Vessels-Datasets/*/Test/Original/Images/*' +
+ img_suffix))
+
+os.system('mkdir -p ' + root_path + 'images/train/')
+os.system('mkdir -p ' + root_path + 'images/test/')
+os.system('mkdir -p ' + root_path + 'masks/train/')
+os.system('mkdir -p ' + root_path + 'masks/test/')
+
+part_dir_dict = {0: 'train/', 1: 'test/'}
+for ith, part in enumerate([x_train, x_test]):
+ part_dir = part_dir_dict[ith]
+ for img in part:
+ type_name = img.split('/')[-5]
+ basename = type_name + '_' + os.path.basename(img)
+ save_img_path = root_path + 'images/' + part_dir + basename.split(
+ '.')[0] + save_img_suffix
+ Image.open(img).save(save_img_path)
+
+ for seg_map_suffix in seg_map_suffix_list:
+ if os.path.exists('/'.join(img.split('/')[:-1]).replace(
+ 'Images', 'Labels')):
+ mask_path = img.replace('Images', 'Labels').replace(
+ img_suffix, seg_map_suffix)
+ else:
+ mask_path = img.replace('Images', 'labels').replace(
+ img_suffix, seg_map_suffix)
+ if os.path.exists(mask_path):
+ break
+ save_mask_path = root_path + 'masks/' + part_dir + basename.split(
+ '.')[0] + save_seg_map_suffix
+ masks = np.array(Image.open(mask_path).convert('L')).astype(np.uint8)
+ if len(np.unique(masks)) == 2 and 1 in np.unique(masks):
+ print(np.unique(masks))
+ pass
+ else:
+ masks[masks < 128] = 0
+ masks[masks >= 128] = 1
+ masks = Image.fromarray(masks)
+ masks.save(save_mask_path)
diff --git a/projects/medical/2d_image/fundus_photography/rite/README.md b/projects/medical/2d_image/fundus_photography/rite/README.md
new file mode 100644
index 00000000000..0aea9b00d17
--- /dev/null
+++ b/projects/medical/2d_image/fundus_photography/rite/README.md
@@ -0,0 +1,135 @@
+# Retinal Images vessel Tree Extraction (RITE)
+
+## Description
+
+This project supports **`Retinal Images vessel Tree Extraction (RITE) `**, which can be downloaded from [here](https://opendatalab.com/RITE).
+
+### Dataset Overview
+
+The RITE (Retinal Images vessel Tree Extraction) is a database that enables comparative studies on segmentation or classification of arteries and veins on retinal fundus images, which is established based on the public available DRIVE database (Digital Retinal Images for Vessel Extraction). RITE contains 40 sets of images, equally separated into a training subset and a test subset, the same as DRIVE. The two subsets are built from the corresponding two subsets in DRIVE. For each set, there is a fundus photograph, a vessel reference standard. The fundus photograph is inherited from DRIVE. For the training set, the vessel reference standard is a modified version of 1st_manual from DRIVE. For the test set, the vessel reference standard is 2nd_manual from DRIVE.
+
+### Statistic Information
+
+| Dataset Name | Anatomical Region | Task Type | Modality | Num. Classes | Train/Val/Test Images | Train/Val/Test Labeled | Release Date | License |
+| ------------------------------------ | ----------------- | ------------ | ------------------ | ------------ | --------------------- | ---------------------- | ------------ | --------------------------------------------------------------- |
+| [Rite](https://opendatalab.com/RITE) | head_and_neck | segmentation | fundus_photography | 2 | 20/-/20 | yes/-/yes | 2013 | [CC-BY-NC 4.0](https://creativecommons.org/licenses/by-sa/4.0/) |
+
+| Class Name | Num. Train | Pct. Train | Num. Val | Pct. Val | Num. Test | Pct. Test |
+| :--------: | :--------: | :--------: | :------: | :------: | :-------: | :-------: |
+| background | 20 | 91.61 | - | - | 20 | 91.58 |
+| vessel | 20 | 8.39 | - | - | 20 | 8.42 |
+
+Note:
+
+- `Pct` means percentage of pixels in this category in all pixels.
+
+### Visualization
+
+
+
+### Dataset Citation
+
+```
+@InProceedings{10.1007/978-3-642-40763-5_54,
+ author={Hu, Qiao and Abr{\`a}moff, Michael D. and Garvin, Mona K.},
+ title={Automated Separation of Binary Overlapping Trees in Low-Contrast Color Retinal Images},
+ booktitle={Medical Image Computing and Computer-Assisted Intervention -- MICCAI 2013},
+ year={2013},
+ pages={436--443},
+}
+
+
+```
+
+### Prerequisites
+
+- Python v3.8
+- PyTorch v1.10.0
+- pillow(PIL) v9.3.0 9.3.0
+- scikit-learn(sklearn) v1.2.0 1.2.0
+- [MIM](https://github.com/open-mmlab/mim) v0.3.4
+- [MMCV](https://github.com/open-mmlab/mmcv) v2.0.0rc4
+- [MMEngine](https://github.com/open-mmlab/mmengine) v0.2.0 or higher
+- [MMSegmentation](https://github.com/open-mmlab/mmsegmentation) v1.0.0rc5
+
+All the commands below rely on the correct configuration of `PYTHONPATH`, which should point to the project's directory so that Python can locate the module files. In `rite/` root directory, run the following line to add the current directory to `PYTHONPATH`:
+
+```shell
+export PYTHONPATH=`pwd`:$PYTHONPATH
+```
+
+### Dataset Preparing
+
+- download dataset from [here](https://opendatalab.com/RITE) and decompress data to path `'data/'`.
+- run script `"python tools/prepare_dataset.py"` to format data and change folder structure as below.
+- run script `"python ../../tools/split_seg_dataset.py"` to split dataset and generate `train.txt`, `val.txt` and `test.txt`. If the label of official validation set and test set cannot be obtained, we generate `train.txt` and `val.txt` from the training set randomly.
+
+```none
+ mmsegmentation
+ ├── mmseg
+ ├── projects
+ │ ├── medical
+ │ │ ├── 2d_image
+ │ │ │ ├── fundus_photography
+ │ │ │ │ ├── rite
+ │ │ │ │ │ ├── configs
+ │ │ │ │ │ ├── datasets
+ │ │ │ │ │ ├── tools
+ │ │ │ │ │ ├── data
+ │ │ │ │ │ │ ├── train.txt
+ │ │ │ │ │ │ ├── val.txt
+ │ │ │ │ │ │ ├── images
+ │ │ │ │ │ │ │ ├── train
+ │ │ │ │ | │ │ │ ├── xxx.png
+ │ │ │ │ | │ │ │ ├── ...
+ │ │ │ │ | │ │ │ └── xxx.png
+ │ │ │ │ │ │ ├── masks
+ │ │ │ │ │ │ │ ├── train
+ │ │ │ │ | │ │ │ ├── xxx.png
+ │ │ │ │ | │ │ │ ├── ...
+ │ │ │ │ | │ │ │ └── xxx.png
+```
+
+### Training commands
+
+To train models on a single server with one GPU. (default)
+
+```shell
+mim train mmseg ./configs/${CONFIG_FILE}
+```
+
+### Testing commands
+
+To test models on a single server with one GPU. (default)
+
+```shell
+mim test mmseg ./configs/${CONFIG_FILE} --checkpoint ${CHECKPOINT_PATH}
+```
+
+
+
+## Checklist
+
+- [x] Milestone 1: PR-ready, and acceptable to be one of the `projects/`.
+
+ - [x] Finish the code
+ - [x] Basic docstrings & proper citation
+ - [ ] Test-time correctness
+ - [x] A full README
+
+- [ ] Milestone 2: Indicates a successful model implementation.
+
+ - [ ] Training-time correctness
+
+- [ ] Milestone 3: Good to be a part of our core package!
+
+ - [ ] Type hints and docstrings
+ - [ ] Unit tests
+ - [ ] Code polishing
+ - [ ] Metafile.yml
+
+- [ ] Move your modules into the core package following the codebase's file hierarchy structure.
+
+- [ ] Refactor your modules into the core package following the codebase's file hierarchy structure.
diff --git a/projects/medical/2d_image/fundus_photography/rite/configs/fcn-unet-s5-d16_unet_1xb16-0.0001-20k_rite-512x512.py b/projects/medical/2d_image/fundus_photography/rite/configs/fcn-unet-s5-d16_unet_1xb16-0.0001-20k_rite-512x512.py
new file mode 100644
index 00000000000..27dd4363b16
--- /dev/null
+++ b/projects/medical/2d_image/fundus_photography/rite/configs/fcn-unet-s5-d16_unet_1xb16-0.0001-20k_rite-512x512.py
@@ -0,0 +1,17 @@
+_base_ = [
+ 'mmseg::_base_/models/fcn_unet_s5-d16.py', './rite_512x512.py',
+ 'mmseg::_base_/default_runtime.py',
+ 'mmseg::_base_/schedules/schedule_20k.py'
+]
+custom_imports = dict(imports='datasets.rite_dataset')
+img_scale = (512, 512)
+data_preprocessor = dict(size=img_scale)
+optimizer = dict(lr=0.0001)
+optim_wrapper = dict(optimizer=optimizer)
+model = dict(
+ data_preprocessor=data_preprocessor,
+ decode_head=dict(num_classes=2),
+ auxiliary_head=None,
+ test_cfg=dict(mode='whole', _delete_=True))
+vis_backends = None
+visualizer = dict(vis_backends=vis_backends)
diff --git a/projects/medical/2d_image/fundus_photography/rite/configs/fcn-unet-s5-d16_unet_1xb16-0.001-20k_rite-512x512.py b/projects/medical/2d_image/fundus_photography/rite/configs/fcn-unet-s5-d16_unet_1xb16-0.001-20k_rite-512x512.py
new file mode 100644
index 00000000000..48f6f973a1a
--- /dev/null
+++ b/projects/medical/2d_image/fundus_photography/rite/configs/fcn-unet-s5-d16_unet_1xb16-0.001-20k_rite-512x512.py
@@ -0,0 +1,17 @@
+_base_ = [
+ 'mmseg::_base_/models/fcn_unet_s5-d16.py', './rite_512x512.py',
+ 'mmseg::_base_/default_runtime.py',
+ 'mmseg::_base_/schedules/schedule_20k.py'
+]
+custom_imports = dict(imports='datasets.rite_dataset')
+img_scale = (512, 512)
+data_preprocessor = dict(size=img_scale)
+optimizer = dict(lr=0.001)
+optim_wrapper = dict(optimizer=optimizer)
+model = dict(
+ data_preprocessor=data_preprocessor,
+ decode_head=dict(num_classes=2),
+ auxiliary_head=None,
+ test_cfg=dict(mode='whole', _delete_=True))
+vis_backends = None
+visualizer = dict(vis_backends=vis_backends)
diff --git a/projects/medical/2d_image/fundus_photography/rite/configs/fcn-unet-s5-d16_unet_1xb16-0.01-20k_rite-512x512.py b/projects/medical/2d_image/fundus_photography/rite/configs/fcn-unet-s5-d16_unet_1xb16-0.01-20k_rite-512x512.py
new file mode 100644
index 00000000000..5f5b24ba6a4
--- /dev/null
+++ b/projects/medical/2d_image/fundus_photography/rite/configs/fcn-unet-s5-d16_unet_1xb16-0.01-20k_rite-512x512.py
@@ -0,0 +1,17 @@
+_base_ = [
+ 'mmseg::_base_/models/fcn_unet_s5-d16.py', './rite_512x512.py',
+ 'mmseg::_base_/default_runtime.py',
+ 'mmseg::_base_/schedules/schedule_20k.py'
+]
+custom_imports = dict(imports='datasets.rite_dataset')
+img_scale = (512, 512)
+data_preprocessor = dict(size=img_scale)
+optimizer = dict(lr=0.01)
+optim_wrapper = dict(optimizer=optimizer)
+model = dict(
+ data_preprocessor=data_preprocessor,
+ decode_head=dict(num_classes=2),
+ auxiliary_head=None,
+ test_cfg=dict(mode='whole', _delete_=True))
+vis_backends = None
+visualizer = dict(vis_backends=vis_backends)
diff --git a/projects/medical/2d_image/fundus_photography/rite/configs/fcn-unet-s5-d16_unet_1xb16-0.01lr-sigmoid-20k_rite-512x512.py b/projects/medical/2d_image/fundus_photography/rite/configs/fcn-unet-s5-d16_unet_1xb16-0.01lr-sigmoid-20k_rite-512x512.py
new file mode 100644
index 00000000000..bf66b6f320c
--- /dev/null
+++ b/projects/medical/2d_image/fundus_photography/rite/configs/fcn-unet-s5-d16_unet_1xb16-0.01lr-sigmoid-20k_rite-512x512.py
@@ -0,0 +1,18 @@
+_base_ = [
+ 'mmseg::_base_/models/fcn_unet_s5-d16.py', './rite_512x512.py',
+ 'mmseg::_base_/default_runtime.py',
+ 'mmseg::_base_/schedules/schedule_20k.py'
+]
+custom_imports = dict(imports='datasets.rite_dataset')
+img_scale = (512, 512)
+data_preprocessor = dict(size=img_scale)
+optimizer = dict(lr=0.01)
+optim_wrapper = dict(optimizer=optimizer)
+model = dict(
+ data_preprocessor=data_preprocessor,
+ decode_head=dict(
+ num_classes=2, loss_decode=dict(use_sigmoid=True), out_channels=1),
+ auxiliary_head=None,
+ test_cfg=dict(mode='whole', _delete_=True))
+vis_backends = None
+visualizer = dict(vis_backends=vis_backends)
diff --git a/projects/medical/2d_image/fundus_photography/rite/configs/rite_512x512.py b/projects/medical/2d_image/fundus_photography/rite/configs/rite_512x512.py
new file mode 100644
index 00000000000..02f620c665f
--- /dev/null
+++ b/projects/medical/2d_image/fundus_photography/rite/configs/rite_512x512.py
@@ -0,0 +1,42 @@
+dataset_type = 'RITEDataset'
+data_root = 'data/'
+img_scale = (512, 512)
+train_pipeline = [
+ dict(type='LoadImageFromFile'),
+ dict(type='LoadAnnotations'),
+ dict(type='Resize', scale=img_scale, keep_ratio=False),
+ dict(type='RandomFlip', prob=0.5),
+ dict(type='PhotoMetricDistortion'),
+ dict(type='PackSegInputs')
+]
+test_pipeline = [
+ dict(type='LoadImageFromFile'),
+ dict(type='Resize', scale=img_scale, keep_ratio=False),
+ dict(type='LoadAnnotations'),
+ dict(type='PackSegInputs')
+]
+train_dataloader = dict(
+ batch_size=16,
+ num_workers=4,
+ persistent_workers=True,
+ sampler=dict(type='InfiniteSampler', shuffle=True),
+ dataset=dict(
+ type=dataset_type,
+ data_root=data_root,
+ ann_file='train.txt',
+ data_prefix=dict(img_path='images/', seg_map_path='masks/'),
+ pipeline=train_pipeline))
+val_dataloader = dict(
+ batch_size=1,
+ num_workers=4,
+ persistent_workers=True,
+ sampler=dict(type='DefaultSampler', shuffle=False),
+ dataset=dict(
+ type=dataset_type,
+ data_root=data_root,
+ ann_file='test.txt',
+ data_prefix=dict(img_path='images/', seg_map_path='masks/'),
+ pipeline=test_pipeline))
+test_dataloader = val_dataloader
+val_evaluator = dict(type='IoUMetric', iou_metrics=['mIoU', 'mDice'])
+test_evaluator = dict(type='IoUMetric', iou_metrics=['mIoU', 'mDice'])
diff --git a/projects/medical/2d_image/fundus_photography/rite/datasets/rite_dataset.py b/projects/medical/2d_image/fundus_photography/rite/datasets/rite_dataset.py
new file mode 100644
index 00000000000..99f688de949
--- /dev/null
+++ b/projects/medical/2d_image/fundus_photography/rite/datasets/rite_dataset.py
@@ -0,0 +1,31 @@
+from mmseg.datasets import BaseSegDataset
+from mmseg.registry import DATASETS
+
+
+@DATASETS.register_module()
+class RITEDataset(BaseSegDataset):
+ """RITEDataset dataset.
+
+ In segmentation map annotation for RITEDataset,
+ 0 stands for background, which is included in 2 categories.
+ ``reduce_zero_label`` is fixed to False. The ``img_suffix``
+ is fixed to '.png' and ``seg_map_suffix`` is fixed to '.png'.
+
+ Args:
+ img_suffix (str): Suffix of images. Default: '.png'
+ seg_map_suffix (str): Suffix of segmentation maps. Default: '.png'
+ reduce_zero_label (bool): Whether to mark label zero as ignored.
+ Default to False.
+ """
+ METAINFO = dict(classes=('background', 'vessel'))
+
+ def __init__(self,
+ img_suffix='.png',
+ seg_map_suffix='.png',
+ reduce_zero_label=False,
+ **kwargs) -> None:
+ super().__init__(
+ img_suffix=img_suffix,
+ seg_map_suffix=seg_map_suffix,
+ reduce_zero_label=reduce_zero_label,
+ **kwargs)
diff --git a/projects/medical/2d_image/fundus_photography/rite/tools/prepare_dataset.py b/projects/medical/2d_image/fundus_photography/rite/tools/prepare_dataset.py
new file mode 100644
index 00000000000..ca7e996961f
--- /dev/null
+++ b/projects/medical/2d_image/fundus_photography/rite/tools/prepare_dataset.py
@@ -0,0 +1,98 @@
+import glob
+import os
+
+import numpy as np
+from PIL import Image
+
+root_path = 'data/'
+img_suffix = '.tif'
+seg_map_suffix = '.png'
+save_img_suffix = '.png'
+save_seg_map_suffix = '.png'
+src_img_train_dir = os.path.join(root_path, 'AV_groundTruth/training/images/')
+src_img_test_dir = os.path.join(root_path, 'AV_groundTruth/test/images/')
+src_mask_train_dir = os.path.join(root_path, 'AV_groundTruth/training/vessel/')
+src_mask_test_dir = os.path.join(root_path, 'AV_groundTruth/test/vessel/')
+
+tgt_img_train_dir = os.path.join(root_path, 'images/train/')
+tgt_mask_train_dir = os.path.join(root_path, 'masks/train/')
+tgt_img_test_dir = os.path.join(root_path, 'images/test/')
+tgt_mask_test_dir = os.path.join(root_path, 'masks/test/')
+os.system('mkdir -p ' + tgt_img_train_dir)
+os.system('mkdir -p ' + tgt_mask_train_dir)
+os.system('mkdir -p ' + tgt_img_test_dir)
+os.system('mkdir -p ' + tgt_mask_test_dir)
+
+
+def filter_suffix_recursive(src_dir, suffix):
+ # filter out file names and paths in source directory
+ suffix = '.' + suffix if '.' not in suffix else suffix
+ file_paths = glob.glob(
+ os.path.join(src_dir, '**', '*' + suffix), recursive=True)
+ file_names = [_.split('/')[-1] for _ in file_paths]
+ return sorted(file_paths), sorted(file_names)
+
+
+def convert_label(img, convert_dict):
+ arr = np.zeros_like(img, dtype=np.uint8)
+ for c, i in convert_dict.items():
+ arr[img == c] = i
+ return arr
+
+
+def convert_pics_into_pngs(src_dir, tgt_dir, suffix, convert='RGB'):
+ if not os.path.exists(tgt_dir):
+ os.makedirs(tgt_dir)
+
+ src_paths, src_names = filter_suffix_recursive(src_dir, suffix=suffix)
+ for i, (src_name, src_path) in enumerate(zip(src_names, src_paths)):
+ tgt_name = src_name.replace(suffix, save_img_suffix)
+ tgt_path = os.path.join(tgt_dir, tgt_name)
+ num = len(src_paths)
+ img = np.array(Image.open(src_path))
+ if len(img.shape) == 2:
+ pil = Image.fromarray(img).convert(convert)
+ elif len(img.shape) == 3:
+ pil = Image.fromarray(img)
+ else:
+ raise ValueError('Input image not 2D/3D: ', img.shape)
+
+ pil.save(tgt_path)
+ print(f'processed {i+1}/{num}.')
+
+
+def convert_label_pics_into_pngs(src_dir,
+ tgt_dir,
+ suffix,
+ convert_dict={
+ 0: 0,
+ 255: 1
+ }):
+ if not os.path.exists(tgt_dir):
+ os.makedirs(tgt_dir)
+
+ src_paths, src_names = filter_suffix_recursive(src_dir, suffix=suffix)
+ num = len(src_paths)
+ for i, (src_name, src_path) in enumerate(zip(src_names, src_paths)):
+ tgt_name = src_name.replace(suffix, save_seg_map_suffix)
+ tgt_path = os.path.join(tgt_dir, tgt_name)
+
+ img = np.array(Image.open(src_path))
+ img = convert_label(img, convert_dict)
+ Image.fromarray(img).save(tgt_path)
+ print(f'processed {i+1}/{num}.')
+
+
+if __name__ == '__main__':
+
+ convert_pics_into_pngs(
+ src_img_train_dir, tgt_img_train_dir, suffix=img_suffix)
+
+ convert_pics_into_pngs(
+ src_img_test_dir, tgt_img_test_dir, suffix=img_suffix)
+
+ convert_label_pics_into_pngs(
+ src_mask_train_dir, tgt_mask_train_dir, suffix=seg_map_suffix)
+
+ convert_label_pics_into_pngs(
+ src_mask_test_dir, tgt_mask_test_dir, suffix=seg_map_suffix)
diff --git a/projects/medical/2d_image/histopathology/breastCancerCellSegmentation/README.md b/projects/medical/2d_image/histopathology/breastCancerCellSegmentation/README.md
new file mode 100644
index 00000000000..97c4a0f0e55
--- /dev/null
+++ b/projects/medical/2d_image/histopathology/breastCancerCellSegmentation/README.md
@@ -0,0 +1,123 @@
+# breastCancerCellSegmentation
+
+## Description
+
+This project supports **`breastCancerCellSegmentation`**, which can be downloaded from [here](https://www.heywhale.com/mw/dataset/5e9e9b35ebb37f002c625423).
+
+### Dataset Overview
+
+This dataset, with 58 H&E-stained histopathology images was used for breast cancer cell detection and associated real-world data.
+Conventional histology uses a combination of hematoxylin and eosin stains, commonly referred to as H&E. These images are stained because most cells are inherently transparent with little or no intrinsic pigment.
+Certain special stains selectively bind to specific components and can be used to identify biological structures such as cells.
+
+### Original Statistic Information
+
+| Dataset name | Anatomical region | Task type | Modality | Num. Classes | Train/Val/Test Images | Train/Val/Test Labeled | Release Date | License |
+| -------------------------------------------------------------------------------------------- | ----------------- | ------------ | -------------- | ------------ | --------------------- | ---------------------- | ------------ | --------------------------------------------------------------- |
+| [breastCancerCellSegmentation](https://www.heywhale.com/mw/dataset/5e9e9b35ebb37f002c625423) | cell | segmentation | histopathology | 2 | 58/-/- | yes/-/- | 2020 | [CC-BY-NC 4.0](https://creativecommons.org/licenses/by-sa/4.0/) |
+
+| Class Name | Num. Train | Pct. Train | Num. Val | Pct. Val | Num. Test | Pct. Test |
+| :--------------: | :--------: | :--------: | :------: | :------: | :-------: | :-------: |
+| background | 58 | 98.37 | - | - | - | - |
+| breastCancerCell | 58 | 1.63 | - | - | - | - |
+
+Note:
+
+- `Pct` means percentage of pixels in this category in all pixels.
+
+### Visualization
+
+
+
+## Usage
+
+### Prerequisites
+
+- Python v3.8
+- PyTorch v1.10.0
+- pillow (PIL) v9.3.0
+- scikit-learn (sklearn) v1.2.0
+- [MIM](https://github.com/open-mmlab/mim) v0.3.4
+- [MMCV](https://github.com/open-mmlab/mmcv) v2.0.0rc4
+- [MMEngine](https://github.com/open-mmlab/mmengine) v0.2.0 or higher
+- [MMSegmentation](https://github.com/open-mmlab/mmsegmentation) v1.0.0
+
+All the commands below rely on the correct configuration of `PYTHONPATH`, which should point to the project's directory so that Python can locate the module files. In `breastCancerCellSegmentation/` root directory, run the following line to add the current directory to `PYTHONPATH`:
+
+```shell
+export PYTHONPATH=`pwd`:$PYTHONPATH
+```
+
+### Dataset Preparing
+
+- Download dataset from [here](https://www.heywhale.com/mw/dataset/5e9e9b35ebb37f002c625423) and save it to the `data/` directory .
+- Decompress data to path `data/`. This will create a new folder named `data/breastCancerCellSegmentation/`, which contains the original image data.
+- run script `python tools/prepare_dataset.py` to format data and change folder structure as below.
+
+```none
+ mmsegmentation
+ ├── mmseg
+ ├── projects
+ │ ├── medical
+ │ │ ├── 2d_image
+ │ │ │ ├── histopathology
+ │ │ │ │ ├── breastCancerCellSegmentation
+ │ │ │ │ │ ├── configs
+ │ │ │ │ │ ├── datasets
+ │ │ │ │ │ ├── tools
+ │ │ │ │ │ ├── data
+ │ │ │ │ │ │ ├── breastCancerCellSegmentation
+ | │ │ │ │ │ │ ├── train.txt
+ | │ │ │ │ │ │ ├── val.txt
+ | │ │ │ │ │ │ ├── images
+ | │ │ │ │ │ │ | ├── xxx.tif
+ | │ │ │ │ │ │ ├── masks
+ | │ │ │ │ │ │ | ├── xxx.TIF
+
+```
+
+### Training commands
+
+Train models on a single server with one GPU.
+
+```shell
+mim train mmseg ./configs/${CONFIG_FILE}
+```
+
+### Testing commands
+
+Test models on a single server with one GPU.
+
+```shell
+mim test mmseg ./configs/${CONFIG_FILE} --checkpoint ${CHECKPOINT_PATH}
+```
+
+## Checklist
+
+- [x] Milestone 1: PR-ready, and acceptable to be one of the `projects/`.
+
+ - [x] Finish the code
+
+ - [x] Basic docstrings & proper citation
+
+ - [x] Test-time correctness
+
+ - [x] A full README
+
+- [ ] Milestone 2: Indicates a successful model implementation.
+
+ - [ ] Training-time correctness
+
+- [ ] Milestone 3: Good to be a part of our core package!
+
+ - [ ] Type hints and docstrings
+
+ - [ ] Unit tests
+
+ - [ ] Code polishing
+
+ - [ ] Metafile.yml
+
+- [ ] Move your modules into the core package following the codebase's file hierarchy structure.
+
+- [ ] Refactor your modules into the core package following the codebase's file hierarchy structure.
diff --git a/projects/medical/2d_image/histopathology/breastCancerCellSegmentation/configs/breastCancerCellSegmentation_512x512.py b/projects/medical/2d_image/histopathology/breastCancerCellSegmentation/configs/breastCancerCellSegmentation_512x512.py
new file mode 100644
index 00000000000..1cf0fccf5be
--- /dev/null
+++ b/projects/medical/2d_image/histopathology/breastCancerCellSegmentation/configs/breastCancerCellSegmentation_512x512.py
@@ -0,0 +1,42 @@
+dataset_type = 'breastCancerCellSegmentationDataset'
+data_root = 'data/breastCancerCellSegmentation'
+img_scale = (512, 512)
+train_pipeline = [
+ dict(type='LoadImageFromFile', imdecode_backend='tifffile'),
+ dict(type='LoadAnnotations', imdecode_backend='tifffile'),
+ dict(type='Resize', scale=img_scale, keep_ratio=False),
+ dict(type='RandomFlip', prob=0.5),
+ dict(type='PhotoMetricDistortion'),
+ dict(type='PackSegInputs')
+]
+test_pipeline = [
+ dict(type='LoadImageFromFile', imdecode_backend='tifffile'),
+ dict(type='Resize', scale=img_scale, keep_ratio=False),
+ dict(type='LoadAnnotations', imdecode_backend='tifffile'),
+ dict(type='PackSegInputs')
+]
+train_dataloader = dict(
+ batch_size=16,
+ num_workers=4,
+ persistent_workers=True,
+ sampler=dict(type='InfiniteSampler', shuffle=True),
+ dataset=dict(
+ type=dataset_type,
+ data_root=data_root,
+ ann_file='train.txt',
+ data_prefix=dict(img_path='images', seg_map_path='masks'),
+ pipeline=train_pipeline))
+val_dataloader = dict(
+ batch_size=1,
+ num_workers=4,
+ persistent_workers=True,
+ sampler=dict(type='DefaultSampler', shuffle=False),
+ dataset=dict(
+ type=dataset_type,
+ data_root=data_root,
+ ann_file='val.txt',
+ data_prefix=dict(img_path='images', seg_map_path='masks'),
+ pipeline=test_pipeline))
+test_dataloader = val_dataloader
+val_evaluator = dict(type='IoUMetric', iou_metrics=['mIoU', 'mDice'])
+test_evaluator = dict(type='IoUMetric', iou_metrics=['mIoU', 'mDice'])
diff --git a/projects/medical/2d_image/histopathology/breastCancerCellSegmentation/configs/fcn-unet-s5-d16_unet_1xb16-0.0001-20k_breastCancerCellSegmentation-512x512.py b/projects/medical/2d_image/histopathology/breastCancerCellSegmentation/configs/fcn-unet-s5-d16_unet_1xb16-0.0001-20k_breastCancerCellSegmentation-512x512.py
new file mode 100644
index 00000000000..55d17089686
--- /dev/null
+++ b/projects/medical/2d_image/histopathology/breastCancerCellSegmentation/configs/fcn-unet-s5-d16_unet_1xb16-0.0001-20k_breastCancerCellSegmentation-512x512.py
@@ -0,0 +1,18 @@
+_base_ = [
+ 'mmseg::_base_/models/fcn_unet_s5-d16.py',
+ './breastCancerCellSegmentation_512x512.py',
+ 'mmseg::_base_/default_runtime.py',
+ 'mmseg::_base_/schedules/schedule_20k.py'
+]
+custom_imports = dict(imports='datasets.breastCancerCellSegmentation_dataset')
+img_scale = (512, 512)
+data_preprocessor = dict(size=img_scale)
+optimizer = dict(lr=0.0001)
+optim_wrapper = dict(optimizer=optimizer)
+model = dict(
+ data_preprocessor=data_preprocessor,
+ decode_head=dict(num_classes=2),
+ auxiliary_head=None,
+ test_cfg=dict(mode='whole', _delete_=True))
+vis_backends = None
+visualizer = dict(vis_backends=vis_backends)
diff --git a/projects/medical/2d_image/histopathology/breastCancerCellSegmentation/configs/fcn-unet-s5-d16_unet_1xb16-0.001-20k_breastCancerCellSegmentation-512x512.py b/projects/medical/2d_image/histopathology/breastCancerCellSegmentation/configs/fcn-unet-s5-d16_unet_1xb16-0.001-20k_breastCancerCellSegmentation-512x512.py
new file mode 100644
index 00000000000..cf28aad739f
--- /dev/null
+++ b/projects/medical/2d_image/histopathology/breastCancerCellSegmentation/configs/fcn-unet-s5-d16_unet_1xb16-0.001-20k_breastCancerCellSegmentation-512x512.py
@@ -0,0 +1,18 @@
+_base_ = [
+ 'mmseg::_base_/models/fcn_unet_s5-d16.py',
+ './breastCancerCellSegmentation_512x512.py',
+ 'mmseg::_base_/default_runtime.py',
+ 'mmseg::_base_/schedules/schedule_20k.py'
+]
+custom_imports = dict(imports='datasets.breastCancerCellSegmentation_dataset')
+img_scale = (512, 512)
+data_preprocessor = dict(size=img_scale)
+optimizer = dict(lr=0.001)
+optim_wrapper = dict(optimizer=optimizer)
+model = dict(
+ data_preprocessor=data_preprocessor,
+ decode_head=dict(num_classes=2),
+ auxiliary_head=None,
+ test_cfg=dict(mode='whole', _delete_=True))
+vis_backends = None
+visualizer = dict(vis_backends=vis_backends)
diff --git a/projects/medical/2d_image/histopathology/breastCancerCellSegmentation/configs/fcn-unet-s5-d16_unet_1xb16-0.01-20k_breastCancerCellSegmentation-512x512.py b/projects/medical/2d_image/histopathology/breastCancerCellSegmentation/configs/fcn-unet-s5-d16_unet_1xb16-0.01-20k_breastCancerCellSegmentation-512x512.py
new file mode 100644
index 00000000000..29aaff38941
--- /dev/null
+++ b/projects/medical/2d_image/histopathology/breastCancerCellSegmentation/configs/fcn-unet-s5-d16_unet_1xb16-0.01-20k_breastCancerCellSegmentation-512x512.py
@@ -0,0 +1,18 @@
+_base_ = [
+ 'mmseg::_base_/models/fcn_unet_s5-d16.py',
+ './breastCancerCellSegmentation_512x512.py',
+ 'mmseg::_base_/default_runtime.py',
+ 'mmseg::_base_/schedules/schedule_20k.py'
+]
+custom_imports = dict(imports='datasets.breastCancerCellSegmentation_dataset')
+img_scale = (512, 512)
+data_preprocessor = dict(size=img_scale)
+optimizer = dict(lr=0.01)
+optim_wrapper = dict(optimizer=optimizer)
+model = dict(
+ data_preprocessor=data_preprocessor,
+ decode_head=dict(num_classes=2),
+ auxiliary_head=None,
+ test_cfg=dict(mode='whole', _delete_=True))
+vis_backends = None
+visualizer = dict(vis_backends=vis_backends)
diff --git a/projects/medical/2d_image/histopathology/breastCancerCellSegmentation/datasets/breastCancerCellSegmentation_dataset.py b/projects/medical/2d_image/histopathology/breastCancerCellSegmentation/datasets/breastCancerCellSegmentation_dataset.py
new file mode 100644
index 00000000000..eeceb6318c0
--- /dev/null
+++ b/projects/medical/2d_image/histopathology/breastCancerCellSegmentation/datasets/breastCancerCellSegmentation_dataset.py
@@ -0,0 +1,30 @@
+from mmseg.datasets import BaseSegDataset
+from mmseg.registry import DATASETS
+
+
+@DATASETS.register_module()
+class breastCancerCellSegmentationDataset(BaseSegDataset):
+ """breastCancerCellSegmentationDataset dataset.
+
+ In segmentation map annotation for breastCancerCellSegmentationDataset,
+ ``reduce_zero_label`` is fixed to False. The ``img_suffix``
+ is fixed to '.png' and ``seg_map_suffix`` is fixed to '.png'.
+
+ Args:
+ img_suffix (str): Suffix of images. Default: '.png'
+ seg_map_suffix (str): Suffix of segmentation maps. Default: '.png'
+ reduce_zero_label (bool): Whether to mark label zero as ignored.
+ Default to False.
+ """
+ METAINFO = dict(classes=('background', 'breastCancerCell'))
+
+ def __init__(self,
+ img_suffix='_ccd.tif',
+ seg_map_suffix='.TIF',
+ reduce_zero_label=False,
+ **kwargs) -> None:
+ super().__init__(
+ img_suffix=img_suffix,
+ seg_map_suffix=seg_map_suffix,
+ reduce_zero_label=reduce_zero_label,
+ **kwargs)
diff --git a/projects/medical/2d_image/histopathology/breastCancerCellSegmentation/tools/prepare_dataset.py b/projects/medical/2d_image/histopathology/breastCancerCellSegmentation/tools/prepare_dataset.py
new file mode 100644
index 00000000000..09cc689c862
--- /dev/null
+++ b/projects/medical/2d_image/histopathology/breastCancerCellSegmentation/tools/prepare_dataset.py
@@ -0,0 +1,36 @@
+import argparse
+import glob
+import os
+
+from sklearn.model_selection import train_test_split
+
+
+def save_anno(img_list, file_path, suffix):
+ # 只保留文件名，不保留后缀
+ img_list = [x.split('/')[-1][:-len(suffix)] for x in img_list]
+
+ with open(file_path, 'w') as file_:
+ for x in list(img_list):
+ file_.write(x + '\n')
+
+
+if __name__ == '__main__':
+ parser = argparse.ArgumentParser()
+ parser.add_argument(
+ '--data_root', default='data/breastCancerCellSegmentation/')
+ args = parser.parse_args()
+ data_root = args.data_root
+
+ # 1. 划分训练集、验证集
+ # 1.1 获取所有图片路径
+ img_list = glob.glob(os.path.join(data_root, 'images', '*.tif'))
+ img_list.sort()
+ mask_list = glob.glob(os.path.join(data_root, 'masks', '*.TIF'))
+ mask_list.sort()
+ assert len(img_list) == len(mask_list)
+ # 1.2 划分训练集、验证集、测试集
+ train_img_list, val_img_list, train_mask_list, val_mask_list = train_test_split( # noqa
+ img_list, mask_list, test_size=0.2, random_state=42)
+ # 1.3 保存划分结果
+ save_anno(train_img_list, os.path.join(data_root, 'train.txt'), '_ccd.tif')
+ save_anno(val_img_list, os.path.join(data_root, 'val.txt'), '_ccd.tif')
diff --git a/projects/medical/2d_image/histopathology/breast_cancer_cell_seg/README.md b/projects/medical/2d_image/histopathology/breast_cancer_cell_seg/README.md
new file mode 100644
index 00000000000..b6f1ca63419
--- /dev/null
+++ b/projects/medical/2d_image/histopathology/breast_cancer_cell_seg/README.md
@@ -0,0 +1,147 @@
+# Breast Cancer Cell Segmentation
+
+## Description
+
+This project support **`Breast Cancer Cell Segmentation`**, and the dataset used in this project can be downloaded from [here](https://tianchi.aliyun.com/dataset/dataDetail?dataId=90152).
+
+### Dataset Overview
+
+In this dataset, there are 58 H&E stained histopathology images used in breast cancer cell detection with associated ground truth data available. Routine histology uses the stain combination of hematoxylin and eosin, commonly referred to as H&E. These images are stained since most cells are essentially transparent, with little or no intrinsic pigment. Certain special stains, which bind selectively to particular components, are be used to identify biological structures such as cells. In those images, the challenging problem is cell segmentation for subsequent classification in benign and malignant cells.
+
+### Original Statistic Information
+
+| Dataset name | Anatomical region | Task type | Modality | Num. Classes | Train/Val/Test Images | Train/Val/Test Labeled | Release Date | License |
+| --------------------------------------------------------------------------------------------- | ----------------- | ------------ | -------------- | ------------ | --------------------- | ---------------------- | ------------ | ------------------------------------------------------------------------------------------------------ |
+| [Breast Cancer Cell Segmentation](https://tianchi.aliyun.com/dataset/dataDetail?dataId=90152) | thorax | segmentation | histopathology | 2 | 58/-/- | yes/-/- | 2021 | [CC-BY-SA-NC 4.0](http://creativecommons.org/licenses/by-sa/4.0/?spm=5176.12282016.0.0.3f5b5291ypBxb2) |
+
+| Class Name | Num. Train | Pct. Train | Num. Val | Pct. Val | Num. Test | Pct. Test |
+| :----------------: | :--------: | :--------: | :------: | :------: | :-------: | :-------: |
+| normal | 58 | 98.37 | - | - | - | - |
+| breast cancer cell | 58 | 1.63 | - | - | - | - |
+
+Note:
+
+- `Pct` means percentage of pixels in this category in all pixels.
+
+### Visualization
+
+
+
+## Dataset Citation
+
+```
+@inproceedings{gelasca2008evaluation,
+ title={Evaluation and benchmark for biological image segmentation},
+ author={Gelasca, Elisa Drelie and Byun, Jiyun and Obara, Boguslaw and Manjunath, BS},
+ booktitle={2008 15th IEEE international conference on image processing},
+ pages={1816--1819},
+ year={2008},
+ organization={IEEE}
+}
+```
+
+### Prerequisites
+
+- Python v3.8
+- PyTorch v1.10.0
+- [MIM](https://github.com/open-mmlab/mim) v0.3.4
+- [MMCV](https://github.com/open-mmlab/mmcv) v2.0.0rc4
+- [MMEngine](https://github.com/open-mmlab/mmengine) v0.2.0 or higher
+- [MMSegmentation](https://github.com/open-mmlab/mmsegmentation) v1.0.0rc5
+
+All the commands below rely on the correct configuration of `PYTHONPATH`, which should point to the project's directory so that Python can locate the module files. In `breast_cancer_cell_seg/` root directory, run the following line to add the current directory to `PYTHONPATH`:
+
+```shell
+export PYTHONPATH=`pwd`:$PYTHONPATH
+```
+
+### Dataset preparing
+
+- download dataset from [here](https://tianchi.aliyun.com/dataset/dataDetail?dataId=90152) and decompression data to path `'data/'`.
+- run script `"python tools/prepare_dataset.py"` to format data and change folder structure as below.
+- run script `"python ../../tools/split_seg_dataset.py"` to split dataset and generate `train.txt`, `val.txt` and `test.txt`. If the label of official validation set and test set can't be obtained, we generate `train.txt` and `val.txt` from the training set randomly.
+
+```none
+ mmsegmentation
+ ├── mmseg
+ ├── projects
+ │ ├── medical
+ │ │ ├── 2d_image
+ │ │ │ ├── histopathology
+ │ │ │ │ ├── breast_cancer_cell_seg
+ │ │ │ │ │ ├── configs
+ │ │ │ │ │ ├── datasets
+ │ │ │ │ │ ├── tools
+ │ │ │ │ │ ├── data
+ │ │ │ │ │ │ ├── train.txt
+ │ │ │ │ │ │ ├── val.txt
+ │ │ │ │ │ │ ├── images
+ │ │ │ │ │ │ │ ├── train
+ │ │ │ │ | │ │ │ ├── xxx.png
+ │ │ │ │ | │ │ │ ├── ...
+ │ │ │ │ | │ │ │ └── xxx.png
+ │ │ │ │ │ │ ├── masks
+ │ │ │ │ │ │ │ ├── train
+ │ │ │ │ | │ │ │ ├── xxx.png
+ │ │ │ │ | │ │ │ ├── ...
+ │ │ │ │ | │ │ │ └── xxx.png
+```
+
+### Divided Dataset Information
+
+***Note: The table information below is divided by ourselves.***
+
+| Class Name | Num. Train | Pct. Train | Num. Val | Pct. Val | Num. Test | Pct. Test |
+| :----------: | :--------: | :--------: | :------: | :------: | :-------: | :-------: |
+| background | 46 | 98.36 | 12 | 98.41 | - | - |
+| erythrocytes | 46 | 1.64 | 12 | 1.59 | - | - |
+
+### Training commands
+
+Train models on a single server with one GPU.
+
+```shell
+mim train mmseg ./configs/${CONFIG_FILE}
+```
+
+### Testing commands
+
+Test models on a single server with one GPU.
+
+```shell
+mim test mmseg ./configs/${CONFIG_FILE} --checkpoint ${CHECKPOINT_PATH}
+```
+
+
+
+## Checklist
+
+- [x] Milestone 1: PR-ready, and acceptable to be one of the `projects/`.
+
+ - [x] Finish the code
+
+ - [x] Basic docstrings & proper citation
+
+ - [x] Test-time correctness
+
+ - [x] A full README
+
+- [ ] Milestone 2: Indicates a successful model implementation.
+
+ - [ ] Training-time correctness
+
+- [ ] Milestone 3: Good to be a part of our core package!
+
+ - [ ] Type hints and docstrings
+
+ - [ ] Unit tests
+
+ - [ ] Code polishing
+
+ - [ ] Metafile.yml
+
+- [ ] Move your modules into the core package following the codebase's file hierarchy structure.
+
+- [ ] Refactor your modules into the core package following the codebase's file hierarchy structure.
diff --git a/projects/medical/2d_image/histopathology/breast_cancer_cell_seg/configs/breast-cancer-cell-seg_512x512.py b/projects/medical/2d_image/histopathology/breast_cancer_cell_seg/configs/breast-cancer-cell-seg_512x512.py
new file mode 100644
index 00000000000..ead40e4345c
--- /dev/null
+++ b/projects/medical/2d_image/histopathology/breast_cancer_cell_seg/configs/breast-cancer-cell-seg_512x512.py
@@ -0,0 +1,42 @@
+dataset_type = 'BreastCancerCellSegDataset'
+data_root = 'data/'
+img_scale = (512, 512)
+train_pipeline = [
+ dict(type='LoadImageFromFile'),
+ dict(type='LoadAnnotations'),
+ dict(type='Resize', scale=img_scale, keep_ratio=False),
+ dict(type='RandomFlip', prob=0.5),
+ dict(type='PhotoMetricDistortion'),
+ dict(type='PackSegInputs')
+]
+test_pipeline = [
+ dict(type='LoadImageFromFile'),
+ dict(type='Resize', scale=img_scale, keep_ratio=False),
+ dict(type='LoadAnnotations'),
+ dict(type='PackSegInputs')
+]
+train_dataloader = dict(
+ batch_size=16,
+ num_workers=4,
+ persistent_workers=True,
+ sampler=dict(type='InfiniteSampler', shuffle=True),
+ dataset=dict(
+ type=dataset_type,
+ data_root=data_root,
+ ann_file='train.txt',
+ data_prefix=dict(img_path='images/', seg_map_path='masks/'),
+ pipeline=train_pipeline))
+val_dataloader = dict(
+ batch_size=1,
+ num_workers=4,
+ persistent_workers=True,
+ sampler=dict(type='DefaultSampler', shuffle=False),
+ dataset=dict(
+ type=dataset_type,
+ data_root=data_root,
+ ann_file='val.txt',
+ data_prefix=dict(img_path='images/', seg_map_path='masks/'),
+ pipeline=test_pipeline))
+test_dataloader = val_dataloader
+val_evaluator = dict(type='IoUMetric', iou_metrics=['mIoU', 'mDice'])
+test_evaluator = dict(type='IoUMetric', iou_metrics=['mIoU', 'mDice'])
diff --git a/projects/medical/2d_image/histopathology/breast_cancer_cell_seg/configs/fcn-unet-s5-d16_unet_1xb16-0.0001-20k_breast-cancer-cell-seg-512x512.py b/projects/medical/2d_image/histopathology/breast_cancer_cell_seg/configs/fcn-unet-s5-d16_unet_1xb16-0.0001-20k_breast-cancer-cell-seg-512x512.py
new file mode 100644
index 00000000000..691a0ff613d
--- /dev/null
+++ b/projects/medical/2d_image/histopathology/breast_cancer_cell_seg/configs/fcn-unet-s5-d16_unet_1xb16-0.0001-20k_breast-cancer-cell-seg-512x512.py
@@ -0,0 +1,18 @@
+_base_ = [
+ './breast-cancer-cell-seg_512x512.py',
+ 'mmseg::_base_/models/fcn_unet_s5-d16.py',
+ 'mmseg::_base_/default_runtime.py',
+ 'mmseg::_base_/schedules/schedule_20k.py'
+]
+custom_imports = dict(imports='datasets.breast-cancer-cell-seg_dataset')
+img_scale = (512, 512)
+data_preprocessor = dict(size=img_scale)
+optimizer = dict(lr=0.0001)
+optim_wrapper = dict(optimizer=optimizer)
+model = dict(
+ data_preprocessor=data_preprocessor,
+ decode_head=dict(num_classes=2),
+ auxiliary_head=None,
+ test_cfg=dict(mode='whole', _delete_=True))
+vis_backends = None
+visualizer = dict(vis_backends=vis_backends)
diff --git a/projects/medical/2d_image/histopathology/breast_cancer_cell_seg/configs/fcn-unet-s5-d16_unet_1xb16-0.001-20k_breast-cancer-cell-seg-512x512.py b/projects/medical/2d_image/histopathology/breast_cancer_cell_seg/configs/fcn-unet-s5-d16_unet_1xb16-0.001-20k_breast-cancer-cell-seg-512x512.py
new file mode 100644
index 00000000000..719b767ab1f
--- /dev/null
+++ b/projects/medical/2d_image/histopathology/breast_cancer_cell_seg/configs/fcn-unet-s5-d16_unet_1xb16-0.001-20k_breast-cancer-cell-seg-512x512.py
@@ -0,0 +1,18 @@
+_base_ = [
+ './breast-cancer-cell-seg_512x512.py',
+ 'mmseg::_base_/models/fcn_unet_s5-d16.py',
+ 'mmseg::_base_/default_runtime.py',
+ 'mmseg::_base_/schedules/schedule_20k.py'
+]
+custom_imports = dict(imports='datasets.breast-cancer-cell-seg_dataset')
+img_scale = (512, 512)
+data_preprocessor = dict(size=img_scale)
+optimizer = dict(lr=0.001)
+optim_wrapper = dict(optimizer=optimizer)
+model = dict(
+ data_preprocessor=data_preprocessor,
+ decode_head=dict(num_classes=2),
+ auxiliary_head=None,
+ test_cfg=dict(mode='whole', _delete_=True))
+vis_backends = None
+visualizer = dict(vis_backends=vis_backends)
diff --git a/projects/medical/2d_image/histopathology/breast_cancer_cell_seg/configs/fcn-unet-s5-d16_unet_1xb16-0.01-20k_breast-cancer-cell-seg-512x512.py b/projects/medical/2d_image/histopathology/breast_cancer_cell_seg/configs/fcn-unet-s5-d16_unet_1xb16-0.01-20k_breast-cancer-cell-seg-512x512.py
new file mode 100644
index 00000000000..9dfe70f761f
--- /dev/null
+++ b/projects/medical/2d_image/histopathology/breast_cancer_cell_seg/configs/fcn-unet-s5-d16_unet_1xb16-0.01-20k_breast-cancer-cell-seg-512x512.py
@@ -0,0 +1,18 @@
+_base_ = [
+ './breast-cancer-cell-seg_512x512.py',
+ 'mmseg::_base_/models/fcn_unet_s5-d16.py',
+ 'mmseg::_base_/default_runtime.py',
+ 'mmseg::_base_/schedules/schedule_20k.py'
+]
+custom_imports = dict(imports='datasets.breast-cancer-cell-seg_dataset')
+img_scale = (512, 512)
+data_preprocessor = dict(size=img_scale)
+optimizer = dict(lr=0.01)
+optim_wrapper = dict(optimizer=optimizer)
+model = dict(
+ data_preprocessor=data_preprocessor,
+ decode_head=dict(num_classes=2),
+ auxiliary_head=None,
+ test_cfg=dict(mode='whole', _delete_=True))
+vis_backends = None
+visualizer = dict(vis_backends=vis_backends)
diff --git a/projects/medical/2d_image/histopathology/breast_cancer_cell_seg/datasets/breast-cancer-cell-seg_dataset.py b/projects/medical/2d_image/histopathology/breast_cancer_cell_seg/datasets/breast-cancer-cell-seg_dataset.py
new file mode 100644
index 00000000000..6f27029d396
--- /dev/null
+++ b/projects/medical/2d_image/histopathology/breast_cancer_cell_seg/datasets/breast-cancer-cell-seg_dataset.py
@@ -0,0 +1,29 @@
+from mmseg.datasets import BaseSegDataset
+from mmseg.registry import DATASETS
+
+
+@DATASETS.register_module()
+class BreastCancerCellSegDataset(BaseSegDataset):
+ """BreastCancerCellSegDataset dataset.
+
+ In segmentation map annotation for BreastCancerCellSegDataset,
+ ``reduce_zero_label`` is fixed to False. The ``img_suffix``
+ is fixed to '.png' and ``seg_map_suffix`` is fixed to '.png'.
+
+ Args:
+ img_suffix (str): Suffix of images. Default: '.png'
+ seg_map_suffix (str): Suffix of segmentation maps. Default: '.png'
+ reduce_zero_label (bool): Whether to mark label zero as ignored.
+ Default to False.
+ """
+ METAINFO = dict(classes=('normal', 'breast cancer cell'))
+
+ def __init__(self,
+ img_suffix='.png',
+ seg_map_suffix='.png',
+ **kwargs) -> None:
+ super().__init__(
+ img_suffix=img_suffix,
+ seg_map_suffix=seg_map_suffix,
+ reduce_zero_label=False,
+ **kwargs)
diff --git a/projects/medical/2d_image/histopathology/breast_cancer_cell_seg/tools/prepare_dataset.py b/projects/medical/2d_image/histopathology/breast_cancer_cell_seg/tools/prepare_dataset.py
new file mode 100755
index 00000000000..775f2eed18a
--- /dev/null
+++ b/projects/medical/2d_image/histopathology/breast_cancer_cell_seg/tools/prepare_dataset.py
@@ -0,0 +1,47 @@
+import glob
+import os
+
+import numpy as np
+from PIL import Image
+
+root_path = 'data/'
+img_suffix = '.tif'
+seg_map_suffix = '.TIF'
+save_img_suffix = '.png'
+save_seg_map_suffix = '.png'
+
+x_train = glob.glob(
+ os.path.join('data/Breast Cancer Cell Segmentation_datasets/Images/*' +
+ img_suffix))
+
+os.system('mkdir -p ' + root_path + 'images/train/')
+os.system('mkdir -p ' + root_path + 'masks/train/')
+
+D2_255_convert_dict = {0: 0, 255: 1}
+
+
+def convert_2d(img, convert_dict=D2_255_convert_dict):
+ arr_2d = np.zeros((img.shape[0], img.shape[1]), dtype=np.uint8)
+ for c, i in convert_dict.items():
+ arr_2d[img == c] = i
+ return arr_2d
+
+
+part_dir_dict = {0: 'train/'}
+for ith, part in enumerate([x_train]):
+ part_dir = part_dir_dict[ith]
+ for img in part:
+ basename = os.path.basename(img)
+ img_save_path = root_path + 'images/' + part_dir + basename.split(
+ '.')[0] + save_img_suffix
+ Image.open(img).save(img_save_path)
+ mask_path = root_path + 'Breast Cancer Cell Segmentation_datasets/Masks/' + '_'.join( # noqa
+ basename.split('_')[:-1]) + seg_map_suffix
+ label = np.array(Image.open(mask_path))
+
+ save_mask_path = root_path + 'masks/' + part_dir + basename.split(
+ '.')[0] + save_seg_map_suffix
+ assert len(label.shape) == 2 and 255 in label and 1 not in label
+ mask = convert_2d(label)
+ mask = Image.fromarray(mask.astype(np.uint8))
+ mask.save(save_mask_path)
diff --git a/projects/medical/2d_image/histopathology/conic2022_seg/README.md b/projects/medical/2d_image/histopathology/conic2022_seg/README.md
new file mode 100644
index 00000000000..1f55b44ed6d
--- /dev/null
+++ b/projects/medical/2d_image/histopathology/conic2022_seg/README.md
@@ -0,0 +1,207 @@
+# CoNIC: Colon Nuclei Identification and Counting Challenge
+
+## Description
+
+This project supports **`CoNIC: Colon Nuclei Identification and Counting Challenge`**, which can be downloaded from [here](https://drive.google.com/drive/folders/1il9jG7uA4-ebQ_lNmXbbF2eOK9uNwheb).
+
+### Dataset Overview
+
+Nuclear segmentation, classification and quantification within Haematoxylin & Eosin stained histology images enables the extraction of interpretable cell-based features that can be used in downstream explainable models in computational pathology (CPath). To help drive forward research and innovation for automatic nuclei recognition in CPath, we organise the Colon Nuclei Identification and Counting (CoNIC) Challenge. The challenge requires researchers to develop algorithms that perform segmentation, classification and counting of 6 different types of nuclei within the current largest known publicly available nuclei-level dataset in CPath, containing around half a million labelled nuclei.
+
+### Task Information
+
+The CONIC challenge has 2 tasks:
+
+- Task 1: Nuclear segmentation and classification.
+
+The first task requires participants to segment nuclei within the tissue, while also classifying each nucleus into one of the following categories: epithelial, lymphocyte, plasma, eosinophil, neutrophil or connective tissue.
+
+- Task 2: Prediction of cellular composition.
+
+For the second task, we ask participants to predict how many nuclei of each class are present in each input image.
+
+The output of Task 1 can be directly used to perform Task 2, but these can be treated as independent tasks. Therefore, if it is preferred, prediction of cellular composition can be treated as a stand alone regression task.
+
+***NOTE：We only consider `Task 1` in the following sections.***
+
+### Original Statistic Information
+
+| Dataset name | Anatomical region | Task type | Modality | Num. Classes | Train/Val/Test Images | Train/Val/Test Labeled | Release Date | License |
+| -------------------------------------------------------- | ----------------- | ------------ | -------------- | ------------ | --------------------- | ---------------------- | ------------ | ------------------------------------------------------------------------------------------------------------ |
+| [CoNIC202](https://conic-challenge.grand-challenge.org/) | abdomen | segmentation | histopathology | 7 | 4981/-/- | yes/-/- | 2022 | [Attribution-NonCommercial-ShareAlike 4.0 International](https://creativecommons.org/licenses/by-nc-sa/4.0/) |
+
+| Class Name | Num. Train | Pct. Train | Num. Val | Pct. Val | Num. Test | Pct. Test |
+| :--------: | :--------: | :--------: | :------: | :------: | :-------: | :-------: |
+| background | 4981 | 83.97 | - | - | - | - |
+| neutrophil | 1218 | 0.13 | - | - | - | - |
+| epithelial | 4256 | 10.31 | - | - | - | - |
+| lymphocyte | 4473 | 1.85 | - | - | - | - |
+| plasma | 3316 | 0.55 | - | - | - | - |
+| eosinophil | 1456 | 0.1 | - | - | - | - |
+| connective | 4613 | 3.08 | - | - | - | - |
+
+Note:
+
+- `Pct` means percentage of pixels in this category in all pixels.
+
+### Visualization
+
+
+
+### Prerequisites
+
+- Python v3.8
+- PyTorch v1.10.0
+- pillow(PIL) v9.3.0
+- scikit-learn(sklearn) v1.2.0
+- [MIM](https://github.com/open-mmlab/mim) v0.3.4
+- [MMCV](https://github.com/open-mmlab/mmcv) v2.0.0rc4
+- [MMEngine](https://github.com/open-mmlab/mmengine) v0.2.0 or higher
+- [MMSegmentation](https://github.com/open-mmlab/mmsegmentation) v1.0.0rc5
+
+All the commands below rely on the correct configuration of `PYTHONPATH`, which should point to the project's directory so that Python can locate the module files. In `conic2022_seg/` root directory, run the following line to add the current directory to `PYTHONPATH`:
+
+```shell
+export PYTHONPATH=`pwd`:$PYTHONPATH
+```
+
+### Dataset preparing
+
+- download dataset from [here](https://drive.google.com/drive/folders/1il9jG7uA4-ebQ_lNmXbbF2eOK9uNwheb/) and move data to path `'data/CoNIC_Challenge'`. The directory should be like:
+ ```shell
+ data/CoNIC_Challenge
+ ├── README.txt
+ ├── by-nc-sa.md
+ ├── counts.csv
+ ├── images.npy
+ ├── labels.npy
+ └── patch_info.csv
+ ```
+- run script `"python tools/prepare_dataset.py"` to format data and change folder structure as below.
+- run script `"python ../../tools/split_seg_dataset.py"` to split dataset and generate `train.txt`, `val.txt` and `test.txt`. If the label of official validation set and test set can't be obtained, we generate `train.txt` and `val.txt` from the training set randomly.
+
+```none
+ mmsegmentation
+ ├── mmseg
+ ├── projects
+ │ ├── medical
+ │ │ ├── 2d_image
+ │ │ │ ├── histopathology
+ │ │ │ │ ├── conic2022_seg
+ │ │ │ │ │ ├── configs
+ │ │ │ │ │ ├── datasets
+ │ │ │ │ │ ├── tools
+ │ │ │ │ │ ├── data
+ │ │ │ │ │ │ ├── train.txt
+ │ │ │ │ │ │ ├── val.txt
+ │ │ │ │ │ │ ├── images
+ │ │ │ │ │ │ │ ├── train
+ │ │ │ │ | │ │ │ ├── xxx.png
+ │ │ │ │ | │ │ │ ├── ...
+ │ │ │ │ | │ │ │ └── xxx.png
+ │ │ │ │ │ │ ├── masks
+ │ │ │ │ │ │ │ ├── train
+ │ │ │ │ | │ │ │ ├── xxx.png
+ │ │ │ │ | │ │ │ ├── ...
+ │ │ │ │ | │ │ │ └── xxx.png
+```
+
+### Divided Dataset Information
+
+***Note: The table information below is divided by ourselves.***
+
+| Class Name | Num. Train | Pct. Train | Num. Val | Pct. Val | Num. Test | Pct. Test |
+| :--------: | :--------: | :--------: | :------: | :------: | :-------: | :-------: |
+| background | 3984 | 84.06 | 997 | 83.65 | - | - |
+| neutrophil | 956 | 0.12 | 262 | 0.13 | - | - |
+| epithelial | 3400 | 10.26 | 856 | 10.52 | - | - |
+| lymphocyte | 3567 | 1.83 | 906 | 1.96 | - | - |
+| plasma | 2645 | 0.55 | 671 | 0.56 | - | - |
+| eosinophil | 1154 | 0.1 | 302 | 0.1 | - | - |
+| connective | 3680 | 3.08 | 933 | 3.08 | - | - |
+
+### Training commands
+
+Train models on a single server with one GPU.
+
+```shell
+mim train mmseg ./configs/${CONFIG_FILE}
+```
+
+### Testing commands
+
+Test models on a single server with one GPU.
+
+```shell
+mim test mmseg ./configs/${CONFIG_FILE} --checkpoint ${CHECKPOINT_PATH}
+```
+
+
+
+## Organizers
+
+- Simon Graham (TIA, PathLAKE)
+- Mostafa Jahanifar (TIA, PathLAKE)
+- Dang Vu (TIA)
+- Giorgos Hadjigeorghiou (TIA, PathLAKE)
+- Thomas Leech (TIA, PathLAKE)
+- David Snead (UHCW, PathLAKE)
+- Shan Raza (TIA, PathLAKE)
+- Fayyaz Minhas (TIA, PathLAKE)
+- Nasir Rajpoot (TIA, PathLAKE)
+
+TIA: Tissue Image Analytics Centre, Department of Computer Science, University of Warwick, United Kingdom
+
+UHCW: Department of Pathology, University Hospitals Coventry and Warwickshire, United Kingdom
+
+PathLAKE: Pathology Image Data Lake for Analytics Knowledge & Education, , University Hospitals Coventry and Warwickshire, United Kingdom
+
+## Dataset Citation
+
+If this work is helpful for your research, please consider citing the below paper.
+
+```
+@inproceedings{graham2021lizard,
+ title={Lizard: A large-scale dataset for colonic nuclear instance segmentation and classification},
+ author={Graham, Simon and Jahanifar, Mostafa and Azam, Ayesha and Nimir, Mohammed and Tsang, Yee-Wah and Dodd, Katherine and Hero, Emily and Sahota, Harvir and Tank, Atisha and Benes, Ksenija and others},
+ booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision},
+ pages={684--693},
+ year={2021}
+}
+@article{graham2021conic,
+ title={Conic: Colon nuclei identification and counting challenge 2022},
+ author={Graham, Simon and Jahanifar, Mostafa and Vu, Quoc Dang and Hadjigeorghiou, Giorgos and Leech, Thomas and Snead, David and Raza, Shan E Ahmed and Minhas, Fayyaz and Rajpoot, Nasir},
+ journal={arXiv preprint arXiv:2111.14485},
+ year={2021}
+}
+```
+
+## Checklist
+
+- [x] Milestone 1: PR-ready, and acceptable to be one of the `projects/`.
+
+ - [x] Finish the code
+
+ - [x] Basic docstrings & proper citation
+
+ - [x] A full README
+
+- [ ] Milestone 2: Indicates a successful model implementation.
+
+ - [ ] Training-time correctness
+
+- [ ] Milestone 3: Good to be a part of our core package!
+
+ - [ ] Type hints and docstrings
+
+ - [ ] Unit tests
+
+ - [ ] Code polishing
+
+ - [ ] Metafile.yml
+
+- [ ] Move your modules into the core package following the codebase's file hierarchy structure.
+
+- [ ] Refactor your modules into the core package following the codebase's file hierarchy structure.
diff --git a/projects/medical/2d_image/histopathology/conic2022_seg/configs/conic2022-seg_512x512.py b/projects/medical/2d_image/histopathology/conic2022_seg/configs/conic2022-seg_512x512.py
new file mode 100644
index 00000000000..51b4e5782ab
--- /dev/null
+++ b/projects/medical/2d_image/histopathology/conic2022_seg/configs/conic2022-seg_512x512.py
@@ -0,0 +1,42 @@
+dataset_type = 'Conic2022SegDataset'
+data_root = 'data/'
+img_scale = (512, 512)
+train_pipeline = [
+ dict(type='LoadImageFromFile'),
+ dict(type='LoadAnnotations'),
+ dict(type='Resize', scale=img_scale, keep_ratio=False),
+ dict(type='RandomFlip', prob=0.5),
+ dict(type='PhotoMetricDistortion'),
+ dict(type='PackSegInputs')
+]
+test_pipeline = [
+ dict(type='LoadImageFromFile'),
+ dict(type='Resize', scale=img_scale, keep_ratio=False),
+ dict(type='LoadAnnotations'),
+ dict(type='PackSegInputs')
+]
+train_dataloader = dict(
+ batch_size=16,
+ num_workers=4,
+ persistent_workers=True,
+ sampler=dict(type='InfiniteSampler', shuffle=True),
+ dataset=dict(
+ type=dataset_type,
+ data_root=data_root,
+ ann_file='train.txt',
+ data_prefix=dict(img_path='images/', seg_map_path='masks/'),
+ pipeline=train_pipeline))
+val_dataloader = dict(
+ batch_size=1,
+ num_workers=4,
+ persistent_workers=True,
+ sampler=dict(type='DefaultSampler', shuffle=False),
+ dataset=dict(
+ type=dataset_type,
+ data_root=data_root,
+ ann_file='val.txt',
+ data_prefix=dict(img_path='images/', seg_map_path='masks/'),
+ pipeline=test_pipeline))
+test_dataloader = val_dataloader
+val_evaluator = dict(type='IoUMetric', iou_metrics=['mIoU', 'mDice'])
+test_evaluator = dict(type='IoUMetric', iou_metrics=['mIoU', 'mDice'])
diff --git a/projects/medical/2d_image/histopathology/conic2022_seg/configs/fcn-unet-s5-d16_unet_1xb16-0.0001-20k_conic2022-512x512.py b/projects/medical/2d_image/histopathology/conic2022_seg/configs/fcn-unet-s5-d16_unet_1xb16-0.0001-20k_conic2022-512x512.py
new file mode 100644
index 00000000000..3e0248c78ce
--- /dev/null
+++ b/projects/medical/2d_image/histopathology/conic2022_seg/configs/fcn-unet-s5-d16_unet_1xb16-0.0001-20k_conic2022-512x512.py
@@ -0,0 +1,17 @@
+_base_ = [
+ './conic2022-seg_512x512.py', 'mmseg::_base_/models/fcn_unet_s5-d16.py',
+ 'mmseg::_base_/default_runtime.py',
+ 'mmseg::_base_/schedules/schedule_20k.py'
+]
+custom_imports = dict(imports='datasets.conic2022-seg_dataset')
+img_scale = (512, 512)
+data_preprocessor = dict(size=img_scale)
+optimizer = dict(lr=0.0001)
+optim_wrapper = dict(optimizer=optimizer)
+model = dict(
+ data_preprocessor=data_preprocessor,
+ decode_head=dict(num_classes=7),
+ auxiliary_head=None,
+ test_cfg=dict(mode='whole', _delete_=True))
+vis_backends = None
+visualizer = dict(vis_backends=vis_backends)
diff --git a/projects/medical/2d_image/histopathology/conic2022_seg/configs/fcn-unet-s5-d16_unet_1xb16-0.001-20k_conic2022-512x512.py b/projects/medical/2d_image/histopathology/conic2022_seg/configs/fcn-unet-s5-d16_unet_1xb16-0.001-20k_conic2022-512x512.py
new file mode 100644
index 00000000000..fd0e9d8d28b
--- /dev/null
+++ b/projects/medical/2d_image/histopathology/conic2022_seg/configs/fcn-unet-s5-d16_unet_1xb16-0.001-20k_conic2022-512x512.py
@@ -0,0 +1,17 @@
+_base_ = [
+ './conic2022-seg_512x512.py', 'mmseg::_base_/models/fcn_unet_s5-d16.py',
+ 'mmseg::_base_/default_runtime.py',
+ 'mmseg::_base_/schedules/schedule_20k.py'
+]
+custom_imports = dict(imports='datasets.conic2022-seg_dataset')
+img_scale = (512, 512)
+data_preprocessor = dict(size=img_scale)
+optimizer = dict(lr=0.001)
+optim_wrapper = dict(optimizer=optimizer)
+model = dict(
+ data_preprocessor=data_preprocessor,
+ decode_head=dict(num_classes=7),
+ auxiliary_head=None,
+ test_cfg=dict(mode='whole', _delete_=True))
+vis_backends = None
+visualizer = dict(vis_backends=vis_backends)
diff --git a/projects/medical/2d_image/histopathology/conic2022_seg/configs/fcn-unet-s5-d16_unet_1xb16-0.01-20k_conic2022-512x512.py b/projects/medical/2d_image/histopathology/conic2022_seg/configs/fcn-unet-s5-d16_unet_1xb16-0.01-20k_conic2022-512x512.py
new file mode 100644
index 00000000000..bb667f14fd4
--- /dev/null
+++ b/projects/medical/2d_image/histopathology/conic2022_seg/configs/fcn-unet-s5-d16_unet_1xb16-0.01-20k_conic2022-512x512.py
@@ -0,0 +1,17 @@
+_base_ = [
+ './conic2022-seg_512x512.py', 'mmseg::_base_/models/fcn_unet_s5-d16.py',
+ 'mmseg::_base_/default_runtime.py',
+ 'mmseg::_base_/schedules/schedule_20k.py'
+]
+custom_imports = dict(imports='datasets.conic2022-seg_dataset')
+img_scale = (512, 512)
+data_preprocessor = dict(size=img_scale)
+optimizer = dict(lr=0.01)
+optim_wrapper = dict(optimizer=optimizer)
+model = dict(
+ data_preprocessor=data_preprocessor,
+ decode_head=dict(num_classes=7),
+ auxiliary_head=None,
+ test_cfg=dict(mode='whole', _delete_=True))
+vis_backends = None
+visualizer = dict(vis_backends=vis_backends)
diff --git a/projects/medical/2d_image/histopathology/conic2022_seg/conic2022_seg_dataset.png b/projects/medical/2d_image/histopathology/conic2022_seg/conic2022_seg_dataset.png
new file mode 100644
index 00000000000..65bb0bbe0a5
Binary files /dev/null and b/projects/medical/2d_image/histopathology/conic2022_seg/conic2022_seg_dataset.png differ
diff --git a/projects/medical/2d_image/histopathology/conic2022_seg/datasets/conic2022-seg_dataset.py b/projects/medical/2d_image/histopathology/conic2022_seg/datasets/conic2022-seg_dataset.py
new file mode 100644
index 00000000000..9af0958ab34
--- /dev/null
+++ b/projects/medical/2d_image/histopathology/conic2022_seg/datasets/conic2022-seg_dataset.py
@@ -0,0 +1,29 @@
+from mmseg.datasets import BaseSegDataset
+from mmseg.registry import DATASETS
+
+
+@DATASETS.register_module()
+class Conic2022SegDataset(BaseSegDataset):
+ """Conic2022SegDataset dataset.
+
+ In segmentation map annotation for Conic2022SegDataset,
+ ``reduce_zero_label`` is fixed to False. The ``img_suffix``
+ is fixed to '.png' and ``seg_map_suffix`` is fixed to '.png'.
+
+ Args:
+ img_suffix (str): Suffix of images. Default: '.png'
+ seg_map_suffix (str): Suffix of segmentation maps. Default: '.png'
+ """
+ METAINFO = dict(
+ classes=('background', 'neutrophil', 'epithelial', 'lymphocyte',
+ 'plasma', 'eosinophil', 'connective'))
+
+ def __init__(self,
+ img_suffix='.png',
+ seg_map_suffix='.png',
+ **kwargs) -> None:
+ super().__init__(
+ img_suffix=img_suffix,
+ seg_map_suffix=seg_map_suffix,
+ reduce_zero_label=False,
+ **kwargs)
diff --git a/projects/medical/2d_image/histopathology/conic2022_seg/tools/prepare_dataset.py b/projects/medical/2d_image/histopathology/conic2022_seg/tools/prepare_dataset.py
new file mode 100755
index 00000000000..89cfb4aae24
--- /dev/null
+++ b/projects/medical/2d_image/histopathology/conic2022_seg/tools/prepare_dataset.py
@@ -0,0 +1,65 @@
+import glob
+import os
+import shutil
+
+import numpy as np
+from PIL import Image
+
+img_save_root = 'data/'
+root_path = 'data/'
+img_suffix = '.png'
+seg_map_suffix = '.png'
+save_img_suffix = '.png'
+save_seg_map_suffix = '.png'
+
+label_set = set()
+
+
+def save_masks_from_npz(data, save_root, part='masks/'):
+ global label_set
+ num = data.shape[0]
+ for i in range(num):
+ # np_img = data[i, :, :, :]
+ np_mask = data[i, :, :, 1]
+ label_set = set.union(label_set, set(np.unique(np_mask)))
+ img = Image.fromarray(np_mask)
+ save_path = os.path.join(save_root, part, str(i) + save_seg_map_suffix)
+ img.save(save_path)
+
+
+def save_images_from_npz(data, save_root, part='images/'):
+ num = data.shape[0]
+ for i in range(num):
+ np_img = data[i, :, :, :]
+ img = Image.fromarray(np_img)
+ save_path = os.path.join(save_root, part, str(i) + save_img_suffix)
+ img.save(save_path)
+
+
+images_npy = np.load('data/CoNIC_Challenge/images.npy')
+labels_npy = np.load('data/CoNIC_Challenge/labels.npy')
+
+os.system('mkdir -p ' + img_save_root + 'images_ori')
+os.system('mkdir -p ' + img_save_root + 'labels')
+save_images_from_npz(images_npy, img_save_root, 'images_ori')
+save_masks_from_npz(labels_npy, img_save_root, 'labels')
+print(label_set)
+
+x_train = glob.glob(os.path.join('data/images_ori/*' + img_suffix))
+
+os.system('mkdir -p ' + root_path + 'images/train/')
+os.system('mkdir -p ' + root_path + 'masks/train/')
+
+part_dir_dict = {0: 'train/', 1: 'val/'}
+for ith, part in enumerate([x_train]):
+ part_dir = part_dir_dict[ith]
+ for img in part:
+ basename = os.path.basename(img)
+ shutil.copy(
+ img, root_path + 'images/' + part_dir + basename.split('.')[0] +
+ save_img_suffix)
+ mask_path = root_path + 'labels/' + basename.split(
+ '.')[0] + seg_map_suffix
+ save_mask_path = root_path + 'masks/' + part_dir + basename.split(
+ '.')[0] + save_seg_map_suffix
+ shutil.copy(mask_path, save_mask_path)
diff --git a/projects/medical/2d_image/histopathology/consep/README.md b/projects/medical/2d_image/histopathology/consep/README.md
new file mode 100644
index 00000000000..ca3d7aa1089
--- /dev/null
+++ b/projects/medical/2d_image/histopathology/consep/README.md
@@ -0,0 +1,147 @@
+# Colorectal Nuclear Segmentation and Phenotypes (CoNSeP) Dataset
+
+## Description
+
+This project supports **`Colorectal Nuclear Segmentation and Phenotypes (CoNSeP) Dataset`**, which can be downloaded from [here](https://warwick.ac.uk/fac/cross_fac/tia/data/hovernet/).
+
+### Dataset Overview
+
+The CoNSeP (Colon Segmentation and Phenotyping) dataset consists of 41 H&E stained image tiles, each with a size of 1,000×1,000 pixels and a magnification of 40x. These images were extracted from 16 colorectal adenocarcinoma (CRA) whole slide images (WSI), each of which belonged to a separate patient and was scanned using an Omnyx VL120 scanner at the Pathology Department of the University Hospitals Coventry and Warwickshire NHS Trust, UK. This dataset was first used in paper named, "HoVer-Net: Simultaneous Segmentation and Classification of Nuclei in Multi-Tissue Histology Images".
+
+### Original Statistic Information
+
+| Dataset name | Anatomical region | Task type | Modality | Num. Classes | Train/Val/Test Images | Train/Val/Test Labeled | Release Date | License |
+| -------------------------------------------------------- | ----------------- | ------------ | -------------- | ------------ | --------------------- | ---------------------- | ------------ | ------- |
+| [CoNIC202](https://conic-challenge.grand-challenge.org/) | abdomen | segmentation | histopathology | 7 | 4981/-/- | yes/-/- | 2022 | - |
+
+| Class Name | Num. Train | Pct. Train | Num. Val | Pct. Val | Num. Test | Pct. Test |
+| :-----------------------------: | :--------: | :--------: | :------: | :------: | :-------: | :-------: |
+| background | 27 | 83.61 | 14 | 80.4 | - | - |
+| other | 17 | 0.17 | 9 | 0.52 | - | - |
+| inflammatory | 25 | 2.66 | 14 | 2.14 | - | - |
+| healthy epithelial | 3 | 1.47 | 2 | 1.58 | - | - |
+| dysplastic/malignant epithelial | 10 | 7.17 | 8 | 9.16 | - | - |
+| fibroblast | 23 | 3.84 | 14 | 4.63 | - | - |
+| muscle | 8 | 1.05 | 3 | 1.42 | - | - |
+| endothelial | 7 | 0.02 | 4 | 0.15 | - | - |
+
+Note:
+
+- `Pct` means percentage of pixels in this category in all pixels.
+
+### Visualization
+
+
+
+### Prerequisites
+
+- Python v3.8
+- PyTorch v1.10.0
+- [MIM](https://github.com/open-mmlab/mim) v0.3.4
+- [MMCV](https://github.com/open-mmlab/mmcv) v2.0.0rc4
+- [MMEngine](https://github.com/open-mmlab/mmengine) v0.2.0 or higher
+- [MMSegmentation](https://github.com/open-mmlab/mmsegmentation) v1.0.0rc5
+
+All the commands below rely on the correct configuration of `PYTHONPATH`, which should point to the project's directory so that Python can locate the module files. In `conic2022_seg/` root directory, run the following line to add the current directory to `PYTHONPATH`:
+
+```shell
+export PYTHONPATH=`pwd`:$PYTHONPATH
+```
+
+### Dataset preparing
+
+- download dataset from [here](https://opendatalab.com/CoNSeP) and decompress data to path `'data/'`.
+- run script `"python tools/prepare_dataset.py"` to format data and change folder structure as below.
+- run script `"python ../../tools/split_seg_dataset.py"` to split dataset and generate `train.txt`, `val.txt` and `test.txt`. If the label of official validation set and test set can't be obtained, we generate `train.txt` and `val.txt` from the training set randomly.
+
+```none
+ mmsegmentation
+ ├── mmseg
+ ├── projects
+ │ ├── medical
+ │ │ ├── 2d_image
+ │ │ │ ├── histopathology
+ │ │ │ │ ├── consep
+ │ │ │ │ │ ├── configs
+ │ │ │ │ │ ├── datasets
+ │ │ │ │ │ ├── tools
+ │ │ │ │ │ ├── data
+ │ │ │ │ │ │ ├── train.txt
+ │ │ │ │ │ │ ├── val.txt
+ │ │ │ │ │ │ ├── images
+ │ │ │ │ │ │ │ ├── train
+ │ │ │ │ | │ │ │ ├── xxx.png
+ │ │ │ │ | │ │ │ ├── ...
+ │ │ │ │ | │ │ │ └── xxx.png
+ │ │ │ │ │ │ ├── masks
+ │ │ │ │ │ │ │ ├── train
+ │ │ │ │ | │ │ │ ├── xxx.png
+ │ │ │ │ | │ │ │ ├── ...
+ │ │ │ │ | │ │ │ └── xxx.png
+```
+
+### Training commands
+
+Train models on a single server with one GPU.
+
+```shell
+mim train mmseg ./configs/${CONFIG_FILE}
+```
+
+### Testing commands
+
+Test models on a single server with one GPU.
+
+```shell
+mim test mmseg ./configs/${CONFIG_FILE} --checkpoint ${CHECKPOINT_PATH}
+```
+
+
+
+## Dataset Citation
+
+If this work is helpful for your research, please consider citing the below paper.
+
+```
+@article{graham2019hover,
+ title={Hover-net: Simultaneous segmentation and classification of nuclei in multi-tissue histology images},
+ author={Graham, Simon and Vu, Quoc Dang and Raza, Shan E Ahmed and Azam, Ayesha and Tsang, Yee Wah and Kwak, Jin Tae and Rajpoot, Nasir},
+ journal={Medical Image Analysis},
+ volume={58},
+ pages={101563},
+ year={2019},
+ publisher={Elsevier}
+}
+```
+
+## Checklist
+
+- [x] Milestone 1: PR-ready, and acceptable to be one of the `projects/`.
+
+ - [x] Finish the code
+
+ - [x] Basic docstrings & proper citation
+
+ - [x] Test-time correctness
+
+ - [x] A full README
+
+- [ ] Milestone 2: Indicates a successful model implementation.
+
+ - [ ] Training-time correctness
+
+- [ ] Milestone 3: Good to be a part of our core package!
+
+ - [ ] Type hints and docstrings
+
+ - [ ] Unit tests
+
+ - [ ] Code polishing
+
+ - [ ] Metafile.yml
+
+- [ ] Move your modules into the core package following the codebase's file hierarchy structure.
+
+- [ ] Refactor your modules into the core package following the codebase's file hierarchy structure.
diff --git a/projects/medical/2d_image/histopathology/consep/configs/consep_512x512.py b/projects/medical/2d_image/histopathology/consep/configs/consep_512x512.py
new file mode 100644
index 00000000000..0d9b8948b01
--- /dev/null
+++ b/projects/medical/2d_image/histopathology/consep/configs/consep_512x512.py
@@ -0,0 +1,42 @@
+dataset_type = 'ConsepDataset'
+data_root = 'data/'
+img_scale = (512, 512)
+train_pipeline = [
+ dict(type='LoadImageFromFile'),
+ dict(type='LoadAnnotations'),
+ dict(type='Resize', scale=img_scale, keep_ratio=False),
+ dict(type='RandomFlip', prob=0.5),
+ dict(type='PhotoMetricDistortion'),
+ dict(type='PackSegInputs')
+]
+test_pipeline = [
+ dict(type='LoadImageFromFile'),
+ dict(type='Resize', scale=img_scale, keep_ratio=False),
+ dict(type='LoadAnnotations'),
+ dict(type='PackSegInputs')
+]
+train_dataloader = dict(
+ batch_size=16,
+ num_workers=4,
+ persistent_workers=True,
+ sampler=dict(type='InfiniteSampler', shuffle=True),
+ dataset=dict(
+ type=dataset_type,
+ data_root=data_root,
+ ann_file='train.txt',
+ data_prefix=dict(img_path='images/', seg_map_path='masks/'),
+ pipeline=train_pipeline))
+val_dataloader = dict(
+ batch_size=1,
+ num_workers=4,
+ persistent_workers=True,
+ sampler=dict(type='DefaultSampler', shuffle=False),
+ dataset=dict(
+ type=dataset_type,
+ data_root=data_root,
+ ann_file='val.txt',
+ data_prefix=dict(img_path='images/', seg_map_path='masks/'),
+ pipeline=test_pipeline))
+test_dataloader = val_dataloader
+val_evaluator = dict(type='IoUMetric', iou_metrics=['mIoU', 'mDice'])
+test_evaluator = dict(type='IoUMetric', iou_metrics=['mIoU', 'mDice'])
diff --git a/projects/medical/2d_image/histopathology/consep/configs/fcn-unet-s5-d16_unet_1xb16-0.0001-20k_consep-512x512.py b/projects/medical/2d_image/histopathology/consep/configs/fcn-unet-s5-d16_unet_1xb16-0.0001-20k_consep-512x512.py
new file mode 100644
index 00000000000..cbcf5db775b
--- /dev/null
+++ b/projects/medical/2d_image/histopathology/consep/configs/fcn-unet-s5-d16_unet_1xb16-0.0001-20k_consep-512x512.py
@@ -0,0 +1,17 @@
+_base_ = [
+ './consep_512x512.py', 'mmseg::_base_/models/fcn_unet_s5-d16.py',
+ 'mmseg::_base_/default_runtime.py',
+ 'mmseg::_base_/schedules/schedule_20k.py'
+]
+custom_imports = dict(imports='datasets.consep_dataset')
+img_scale = (512, 512)
+data_preprocessor = dict(size=img_scale)
+optimizer = dict(lr=0.0001)
+optim_wrapper = dict(optimizer=optimizer)
+model = dict(
+ data_preprocessor=data_preprocessor,
+ decode_head=dict(num_classes=8),
+ auxiliary_head=None,
+ test_cfg=dict(mode='whole', _delete_=True))
+vis_backends = None
+visualizer = dict(vis_backends=vis_backends)
diff --git a/projects/medical/2d_image/histopathology/consep/configs/fcn-unet-s5-d16_unet_1xb16-0.001-20k_consep-512x512.py b/projects/medical/2d_image/histopathology/consep/configs/fcn-unet-s5-d16_unet_1xb16-0.001-20k_consep-512x512.py
new file mode 100644
index 00000000000..b374566e6ef
--- /dev/null
+++ b/projects/medical/2d_image/histopathology/consep/configs/fcn-unet-s5-d16_unet_1xb16-0.001-20k_consep-512x512.py
@@ -0,0 +1,17 @@
+_base_ = [
+ './consep_512x512.py', 'mmseg::_base_/models/fcn_unet_s5-d16.py',
+ 'mmseg::_base_/default_runtime.py',
+ 'mmseg::_base_/schedules/schedule_20k.py'
+]
+custom_imports = dict(imports='datasets.consep_dataset')
+img_scale = (512, 512)
+data_preprocessor = dict(size=img_scale)
+optimizer = dict(lr=0.001)
+optim_wrapper = dict(optimizer=optimizer)
+model = dict(
+ data_preprocessor=data_preprocessor,
+ decode_head=dict(num_classes=8),
+ auxiliary_head=None,
+ test_cfg=dict(mode='whole', _delete_=True))
+vis_backends = None
+visualizer = dict(vis_backends=vis_backends)
diff --git a/projects/medical/2d_image/histopathology/consep/configs/fcn-unet-s5-d16_unet_1xb16-0.01-20k_consep-512x512.py b/projects/medical/2d_image/histopathology/consep/configs/fcn-unet-s5-d16_unet_1xb16-0.01-20k_consep-512x512.py
new file mode 100644
index 00000000000..35bdaa34c84
--- /dev/null
+++ b/projects/medical/2d_image/histopathology/consep/configs/fcn-unet-s5-d16_unet_1xb16-0.01-20k_consep-512x512.py
@@ -0,0 +1,17 @@
+_base_ = [
+ './consep_512x512.py', 'mmseg::_base_/models/fcn_unet_s5-d16.py',
+ 'mmseg::_base_/default_runtime.py',
+ 'mmseg::_base_/schedules/schedule_20k.py'
+]
+custom_imports = dict(imports='datasets.consep_dataset')
+img_scale = (512, 512)
+data_preprocessor = dict(size=img_scale)
+optimizer = dict(lr=0.01)
+optim_wrapper = dict(optimizer=optimizer)
+model = dict(
+ data_preprocessor=data_preprocessor,
+ decode_head=dict(num_classes=8),
+ auxiliary_head=None,
+ test_cfg=dict(mode='whole', _delete_=True))
+vis_backends = None
+visualizer = dict(vis_backends=vis_backends)
diff --git a/projects/medical/2d_image/histopathology/consep/datasets/consep_dataset.py b/projects/medical/2d_image/histopathology/consep/datasets/consep_dataset.py
new file mode 100644
index 00000000000..ceb2b3ab25b
--- /dev/null
+++ b/projects/medical/2d_image/histopathology/consep/datasets/consep_dataset.py
@@ -0,0 +1,30 @@
+from mmseg.datasets import BaseSegDataset
+from mmseg.registry import DATASETS
+
+
+@DATASETS.register_module()
+class ConsepDataset(BaseSegDataset):
+ """ConsepDataset dataset.
+
+ In segmentation map annotation for ConsepDataset,
+ ``reduce_zero_label`` is fixed to False. The ``img_suffix``
+ is fixed to '.png' and ``seg_map_suffix`` is fixed to '.png'.
+
+ Args:
+ img_suffix (str): Suffix of images. Default: '.png'
+ seg_map_suffix (str): Suffix of segmentation maps. Default: '.png'
+ """
+ METAINFO = dict(
+ classes=('background', 'other', 'inflammatory', 'healthy epithelial',
+ 'dysplastic/malignant epithelial', 'fibroblast', 'muscle',
+ 'endothelial'))
+
+ def __init__(self,
+ img_suffix='.png',
+ seg_map_suffix='.png',
+ **kwargs) -> None:
+ super().__init__(
+ img_suffix=img_suffix,
+ seg_map_suffix=seg_map_suffix,
+ reduce_zero_label=False,
+ **kwargs)
diff --git a/projects/medical/2d_image/histopathology/consep/tools/prepare_dataset.py b/projects/medical/2d_image/histopathology/consep/tools/prepare_dataset.py
new file mode 100755
index 00000000000..83a2e18ce10
--- /dev/null
+++ b/projects/medical/2d_image/histopathology/consep/tools/prepare_dataset.py
@@ -0,0 +1,54 @@
+import glob
+import os
+import shutil
+
+import numpy as np
+from PIL import Image
+from scipy.io import loadmat
+
+root_path = 'data/'
+img_suffix = '.png'
+seg_map_suffix = '.mat'
+save_img_suffix = '.png'
+save_seg_map_suffix = '.png'
+
+x_train = glob.glob(os.path.join('data/CoNSeP/Train/Images/*' + img_suffix))
+x_test = glob.glob(os.path.join('data/CoNSeP/Test/Images/*' + img_suffix))
+
+os.system('mkdir -p ' + root_path + 'images/train/')
+os.system('mkdir -p ' + root_path + 'images/val/')
+os.system('mkdir -p ' + root_path + 'masks/train/')
+os.system('mkdir -p ' + root_path + 'masks/val/')
+D2_255_convert_dict = {0: 0, 255: 1}
+
+
+def convert_2d(img, convert_dict=D2_255_convert_dict):
+ arr_2d = np.zeros((img.shape[0], img.shape[1]), dtype=np.uint8)
+ for c, i in convert_dict.items():
+ arr_2d[img == c] = i
+ return arr_2d
+
+
+part_dir_dict = {0: 'CoNSeP/Train/', 1: 'CoNSeP/Test/'}
+save_dir_dict = {0: 'train/', 1: 'val/'}
+for ith, part in enumerate([x_train, x_test]):
+ part_dir = part_dir_dict[ith]
+ for img in part:
+ basename = os.path.basename(img)
+ shutil.copy(
+ img, root_path + 'images/' + save_dir_dict[ith] +
+ basename.split('.')[0] + save_img_suffix)
+
+ mask_path = root_path + part_dir + 'Labels/' + basename.split(
+ '.')[0] + seg_map_suffix
+ label_ = loadmat(mask_path)
+ label = label_['inst_map']
+ label_type = label_['inst_type']
+ label_dict = {i + 1: int(val) for i, val in enumerate(label_type)}
+
+ save_mask_path = root_path + 'masks/' + save_dir_dict[
+ ith] + basename.split('.')[0] + save_seg_map_suffix
+
+ res = convert_2d(label, convert_dict=label_dict)
+ res = Image.fromarray(res.astype(np.uint8))
+ res.save(save_mask_path)
diff --git a/projects/medical/2d_image/histopathology/fusc2021/README.md b/projects/medical/2d_image/histopathology/fusc2021/README.md
new file mode 100644
index 00000000000..8130d593503
--- /dev/null
+++ b/projects/medical/2d_image/histopathology/fusc2021/README.md
@@ -0,0 +1,136 @@
+# Foot Ulcer Segmentation Challenge 2021 (FUSC 2021)
+
+## Description
+
+This project supports **`Foot Ulcer Segmentation Challenge 2021 (FUSC 2021) `**, which can be downloaded from [here](https://fusc.grand-challenge.org/).
+
+### Dataset Overview
+
+This chronic wound dataset was collected over 2 years from October 2019 to April 2021 at the center and contains 1,210 foot ulcer images taken from 889 patients during multiple clinical visits. The raw images were taken by Canon SX 620 HS digital camera and iPad Pro under uncontrolled illumination conditions,
+with various backgrounds. The images (shown in Figure 1) are randomly split into 3 subsets: a training set with 810 images, a validation set with 200 images, and a testing set with 200 images. Of course, the annotations of the testing set are kept private. The data collected were de-identified and in accordance with relevant guidelines and regulations and the patient’s informed consent is waived by the institutional review board of the University of Wisconsin-Milwaukee.
+
+### Information Statistics
+
+| Dataset Name | Anatomical Region | Task Type | Modality | Num. Classes | Train/Val/Test Images | Train/Val/Test Labeled | Release Date | License |
+| --------------------------------------------- | ----------------- | ------------ | -------------- | ------------ | --------------------- | ---------------------- | ------------ | ------------------------------------------------------------- |
+| [fusc2021](https://fusc.grand-challenge.org/) | lower limb | segmentation | histopathology | 2 | 810/200/200 | yes/yes/no | 2021 | [CC0 1.0](https://creativecommons.org/publicdomain/zero/1.0/) |
+
+| Class Name | Num. Train | Pct. Train | Num. Val | Pct. Val | Num. Test | Pct. Test |
+| :--------: | :--------: | :--------: | :------: | :------: | :-------: | :-------: |
+| background | 810 | 98.71 | 200 | 98.78 | - | - |
+| wound | 791 | 1.29 | 195 | 1.22 | - | - |
+
+Note:
+
+- `Pct` means percentage of pixels in this category in all pixels.
+
+### Visualization
+
+
+
+### Dataset Citation
+
+```
+@article{s41598-020-78799-w,
+ title={Fully automatic wound segmentation with deep convolutional neural networks},
+ author={Chuanbo Wang and D. M. Anisuzzaman and Victor Williamson and Mrinal Kanti Dhar and Behrouz Rostami and Jeffrey Niezgoda and Sandeep Gopalakrishnan and Zeyun Yu},
+ journal={Scientific Reports},
+ volume={10},
+ number={1},
+ pages={21897},
+ year={2020}
+}
+```
+
+### Prerequisites
+
+- Python v3.8
+- PyTorch v1.10.0
+- pillow(PIL) v9.3.0
+- scikit-learn(sklearn) v1.2.0
+- [MIM](https://github.com/open-mmlab/mim) v0.3.4
+- [MMCV](https://github.com/open-mmlab/mmcv) v2.0.0rc4
+- [MMEngine](https://github.com/open-mmlab/mmengine) v0.2.0 or higher
+- [MMSegmentation](https://github.com/open-mmlab/mmsegmentation) v1.0.0rc5
+
+All the commands below rely on the correct configuration of `PYTHONPATH`, which should point to the project's directory so that Python can locate the module files. In `fusc2021/` root directory, run the following line to add the current directory to `PYTHONPATH`:
+
+```shell
+export PYTHONPATH=`pwd`:$PYTHONPATH
+```
+
+### Dataset Preparing
+
+- download dataset from [here](https://fusc.grand-challenge.org/) and decompress data to path `'data/'`.
+- run script `"python tools/prepare_dataset.py"` to format data and change folder structure as below.
+- run script `"python ../../tools/split_seg_dataset.py"` to split dataset and generate `train.txt`, `val.txt` and `test.txt`. If the label of official validation set and test set cannot be obtained, we generate `train.txt` and `val.txt` from the training set randomly.
+
+```none
+ mmsegmentation
+ ├── mmseg
+ ├── projects
+ │ ├── medical
+ │ │ ├── 2d_image
+ │ │ │ ├── histopathology
+ │ │ │ │ ├── fusc2021
+ │ │ │ │ │ ├── configs
+ │ │ │ │ │ ├── datasets
+ │ │ │ │ │ ├── tools
+ │ │ │ │ │ ├── data
+ │ │ │ │ │ │ ├── train.txt
+ │ │ │ │ │ │ ├── val.txt
+ │ │ │ │ │ │ ├── images
+ │ │ │ │ │ │ │ ├── train
+ │ │ │ │ | │ │ │ ├── xxx.png
+ │ │ │ │ | │ │ │ ├── ...
+ │ │ │ │ | │ │ │ └── xxx.png
+ │ │ │ │ │ │ ├── masks
+ │ │ │ │ │ │ │ ├── train
+ │ │ │ │ | │ │ │ ├── xxx.png
+ │ │ │ │ | │ │ │ ├── ...
+ │ │ │ │ | │ │ │ └── xxx.png
+```
+
+### Training commands
+
+To train models on a single server with one GPU. (default)
+
+```shell
+mim train mmseg ./configs/${CONFIG_FILE}
+```
+
+### Testing commands
+
+To test models on a single server with one GPU. (default)
+
+```shell
+mim test mmseg ./configs/${CONFIG_FILE} --checkpoint ${CHECKPOINT_PATH}
+```
+
+
+
+## Checklist
+
+- [x] Milestone 1: PR-ready, and acceptable to be one of the `projects/`.
+
+ - [x] Finish the code
+ - [x] Basic docstrings & proper citation
+ - [ ] Test-time correctness
+ - [x] A full README
+
+- [ ] Milestone 2: Indicates a successful model implementation.
+
+ - [ ] Training-time correctness
+
+- [ ] Milestone 3: Good to be a part of our core package!
+
+ - [ ] Type hints and docstrings
+ - [ ] Unit tests
+ - [ ] Code polishing
+ - [ ] Metafile.yml
+
+- [ ] Move your modules into the core package following the codebase's file hierarchy structure.
+
+- [ ] Refactor your modules into the core package following the codebase's file hierarchy structure.
diff --git a/projects/medical/2d_image/histopathology/fusc2021/configs/fcn-unet-s5-d16_unet_1xb16-0.0001-20k_fusc2021-512x512.py b/projects/medical/2d_image/histopathology/fusc2021/configs/fcn-unet-s5-d16_unet_1xb16-0.0001-20k_fusc2021-512x512.py
new file mode 100644
index 00000000000..c3f42751124
--- /dev/null
+++ b/projects/medical/2d_image/histopathology/fusc2021/configs/fcn-unet-s5-d16_unet_1xb16-0.0001-20k_fusc2021-512x512.py
@@ -0,0 +1,17 @@
+_base_ = [
+ 'mmseg::_base_/models/fcn_unet_s5-d16.py', './fusc2021_512x512.py',
+ 'mmseg::_base_/default_runtime.py',
+ 'mmseg::_base_/schedules/schedule_20k.py'
+]
+custom_imports = dict(imports='datasets.fusc2021_dataset')
+img_scale = (512, 512)
+data_preprocessor = dict(size=img_scale)
+optimizer = dict(lr=0.0001)
+optim_wrapper = dict(optimizer=optimizer)
+model = dict(
+ data_preprocessor=data_preprocessor,
+ decode_head=dict(num_classes=2),
+ auxiliary_head=None,
+ test_cfg=dict(mode='whole', _delete_=True))
+vis_backends = None
+visualizer = dict(vis_backends=vis_backends)
diff --git a/projects/medical/2d_image/histopathology/fusc2021/configs/fcn-unet-s5-d16_unet_1xb16-0.001-20k_fusc2021-512x512.py b/projects/medical/2d_image/histopathology/fusc2021/configs/fcn-unet-s5-d16_unet_1xb16-0.001-20k_fusc2021-512x512.py
new file mode 100644
index 00000000000..ed870303fff
--- /dev/null
+++ b/projects/medical/2d_image/histopathology/fusc2021/configs/fcn-unet-s5-d16_unet_1xb16-0.001-20k_fusc2021-512x512.py
@@ -0,0 +1,17 @@
+_base_ = [
+ 'mmseg::_base_/models/fcn_unet_s5-d16.py', './fusc2021_512x512.py',
+ 'mmseg::_base_/default_runtime.py',
+ 'mmseg::_base_/schedules/schedule_20k.py'
+]
+custom_imports = dict(imports='datasets.fusc2021_dataset')
+img_scale = (512, 512)
+data_preprocessor = dict(size=img_scale)
+optimizer = dict(lr=0.001)
+optim_wrapper = dict(optimizer=optimizer)
+model = dict(
+ data_preprocessor=data_preprocessor,
+ decode_head=dict(num_classes=2),
+ auxiliary_head=None,
+ test_cfg=dict(mode='whole', _delete_=True))
+vis_backends = None
+visualizer = dict(vis_backends=vis_backends)
diff --git a/projects/medical/2d_image/histopathology/fusc2021/configs/fcn-unet-s5-d16_unet_1xb16-0.01-20k_fusc2021-512x512.py b/projects/medical/2d_image/histopathology/fusc2021/configs/fcn-unet-s5-d16_unet_1xb16-0.01-20k_fusc2021-512x512.py
new file mode 100644
index 00000000000..cbc09ae6cdd
--- /dev/null
+++ b/projects/medical/2d_image/histopathology/fusc2021/configs/fcn-unet-s5-d16_unet_1xb16-0.01-20k_fusc2021-512x512.py
@@ -0,0 +1,17 @@
+_base_ = [
+ 'mmseg::_base_/models/fcn_unet_s5-d16.py', './fusc2021_512x512.py',
+ 'mmseg::_base_/default_runtime.py',
+ 'mmseg::_base_/schedules/schedule_20k.py'
+]
+custom_imports = dict(imports='datasets.fusc2021_dataset')
+img_scale = (512, 512)
+data_preprocessor = dict(size=img_scale)
+optimizer = dict(lr=0.01)
+optim_wrapper = dict(optimizer=optimizer)
+model = dict(
+ data_preprocessor=data_preprocessor,
+ decode_head=dict(num_classes=2),
+ auxiliary_head=None,
+ test_cfg=dict(mode='whole', _delete_=True))
+vis_backends = None
+visualizer = dict(vis_backends=vis_backends)
diff --git a/projects/medical/2d_image/histopathology/fusc2021/configs/fcn-unet-s5-d16_unet_1xb16-0.01lr-sigmoid-20k_fusc2021-512x512.py b/projects/medical/2d_image/histopathology/fusc2021/configs/fcn-unet-s5-d16_unet_1xb16-0.01lr-sigmoid-20k_fusc2021-512x512.py
new file mode 100644
index 00000000000..f1477ee725e
--- /dev/null
+++ b/projects/medical/2d_image/histopathology/fusc2021/configs/fcn-unet-s5-d16_unet_1xb16-0.01lr-sigmoid-20k_fusc2021-512x512.py
@@ -0,0 +1,18 @@
+_base_ = [
+ 'mmseg::_base_/models/fcn_unet_s5-d16.py', './fusc2021_512x512.py',
+ 'mmseg::_base_/default_runtime.py',
+ 'mmseg::_base_/schedules/schedule_20k.py'
+]
+custom_imports = dict(imports='datasets.fusc2021_dataset')
+img_scale = (512, 512)
+data_preprocessor = dict(size=img_scale)
+optimizer = dict(lr=0.01)
+optim_wrapper = dict(optimizer=optimizer)
+model = dict(
+ data_preprocessor=data_preprocessor,
+ decode_head=dict(
+ num_classes=2, loss_decode=dict(use_sigmoid=True), out_channels=1),
+ auxiliary_head=None,
+ test_cfg=dict(mode='whole', _delete_=True))
+vis_backends = None
+visualizer = dict(vis_backends=vis_backends)
diff --git a/projects/medical/2d_image/histopathology/fusc2021/configs/fusc2021_512x512.py b/projects/medical/2d_image/histopathology/fusc2021/configs/fusc2021_512x512.py
new file mode 100644
index 00000000000..e650474cea8
--- /dev/null
+++ b/projects/medical/2d_image/histopathology/fusc2021/configs/fusc2021_512x512.py
@@ -0,0 +1,42 @@
+dataset_type = 'FUSC2021Dataset'
+data_root = 'data/'
+img_scale = (512, 512)
+train_pipeline = [
+ dict(type='LoadImageFromFile'),
+ dict(type='LoadAnnotations'),
+ dict(type='Resize', scale=img_scale, keep_ratio=False),
+ dict(type='RandomFlip', prob=0.5),
+ dict(type='PhotoMetricDistortion'),
+ dict(type='PackSegInputs')
+]
+test_pipeline = [
+ dict(type='LoadImageFromFile'),
+ dict(type='Resize', scale=img_scale, keep_ratio=False),
+ dict(type='LoadAnnotations'),
+ dict(type='PackSegInputs')
+]
+train_dataloader = dict(
+ batch_size=16,
+ num_workers=4,
+ persistent_workers=True,
+ sampler=dict(type='InfiniteSampler', shuffle=True),
+ dataset=dict(
+ type=dataset_type,
+ data_root=data_root,
+ ann_file='train.txt',
+ data_prefix=dict(img_path='images/', seg_map_path='masks/'),
+ pipeline=train_pipeline))
+val_dataloader = dict(
+ batch_size=1,
+ num_workers=4,
+ persistent_workers=True,
+ sampler=dict(type='DefaultSampler', shuffle=False),
+ dataset=dict(
+ type=dataset_type,
+ data_root=data_root,
+ ann_file='val.txt',
+ data_prefix=dict(img_path='images/', seg_map_path='masks/'),
+ pipeline=test_pipeline))
+test_dataloader = val_dataloader
+val_evaluator = dict(type='IoUMetric', iou_metrics=['mIoU', 'mDice'])
+test_evaluator = dict(type='IoUMetric', iou_metrics=['mIoU', 'mDice'])
diff --git a/projects/medical/2d_image/histopathology/fusc2021/datasets/fusc2021_dataset.py b/projects/medical/2d_image/histopathology/fusc2021/datasets/fusc2021_dataset.py
new file mode 100644
index 00000000000..d331ac8c3a2
--- /dev/null
+++ b/projects/medical/2d_image/histopathology/fusc2021/datasets/fusc2021_dataset.py
@@ -0,0 +1,30 @@
+from mmseg.datasets import BaseSegDataset
+from mmseg.registry import DATASETS
+
+
+@DATASETS.register_module()
+class FUSC2021Dataset(BaseSegDataset):
+ """FUSC2021Dataset dataset.
+
+ In segmentation map annotation for FUSC2021Dataset, 0 stands for background
+ , which is included in 2 categories. ``reduce_zero_label``
+ is fixed to False. The ``img_suffix`` is fixed to '.png' and
+ ``seg_map_suffix`` is fixed to '.png'.
+ Args:
+ img_suffix (str): Suffix of images. Default: '.png'
+ seg_map_suffix (str): Suffix of segmentation maps. Default: '.png'
+ reduce_zero_label (bool): Whether to mark label zero as ignored.
+ Default to False..
+ """
+ METAINFO = dict(classes=('background', 'wound'))
+
+ def __init__(self,
+ img_suffix='.png',
+ seg_map_suffix='.png',
+ reduce_zero_label=False,
+ **kwargs) -> None:
+ super().__init__(
+ img_suffix=img_suffix,
+ seg_map_suffix=seg_map_suffix,
+ reduce_zero_label=reduce_zero_label,
+ **kwargs)
diff --git a/projects/medical/2d_image/histopathology/fusc2021/tools/prepare_dataset.py b/projects/medical/2d_image/histopathology/fusc2021/tools/prepare_dataset.py
new file mode 100644
index 00000000000..8f2de3daa95
--- /dev/null
+++ b/projects/medical/2d_image/histopathology/fusc2021/tools/prepare_dataset.py
@@ -0,0 +1,114 @@
+import glob
+import os
+
+import numpy as np
+from PIL import Image
+
+root_path = 'data/'
+img_suffix = '.png'
+seg_map_suffix = '.png'
+save_img_suffix = '.png'
+save_seg_map_suffix = '.png'
+src_img_train_dir = os.path.join(
+ root_path, 'wound-segmentation/data/' +
+ 'Foot Ulcer Segmentation Challenge/train/images')
+src_img_val_dir = os.path.join(
+ root_path, 'wound-segmentation/data/' +
+ 'Foot Ulcer Segmentation Challenge/validation/images')
+src_img_test_dir = os.path.join(
+ root_path, 'wound-segmentation/data/' +
+ 'Foot Ulcer Segmentation Challenge/test/images')
+src_mask_train_dir = os.path.join(
+ root_path, 'wound-segmentation/data/' +
+ 'Foot Ulcer Segmentation Challenge/train/labels')
+src_mask_val_dir = os.path.join(
+ root_path, 'wound-segmentation/data/' +
+ 'Foot Ulcer Segmentation Challenge/validation/labels')
+
+tgt_img_train_dir = os.path.join(root_path, 'images/train/')
+tgt_mask_train_dir = os.path.join(root_path, 'masks/train/')
+tgt_img_val_dir = os.path.join(root_path, 'images/val/')
+tgt_mask_val_dir = os.path.join(root_path, 'masks/val/')
+tgt_img_test_dir = os.path.join(root_path, 'images/test/')
+os.system('mkdir -p ' + tgt_img_train_dir)
+os.system('mkdir -p ' + tgt_img_val_dir)
+os.system('mkdir -p ' + tgt_img_test_dir)
+os.system('mkdir -p ' + tgt_mask_train_dir)
+os.system('mkdir -p ' + tgt_mask_val_dir)
+
+
+def filter_suffix_recursive(src_dir, suffix):
+ # filter out file names and paths in source directory
+ suffix = '.' + suffix if '.' not in suffix else suffix
+ file_paths = glob.glob(
+ os.path.join(src_dir, '**', '*' + suffix), recursive=True)
+ file_names = [_.split('/')[-1] for _ in file_paths]
+ return sorted(file_paths), sorted(file_names)
+
+
+def convert_label(img, convert_dict):
+ arr = np.zeros_like(img, dtype=np.uint8)
+ for c, i in convert_dict.items():
+ arr[img == c] = i
+ return arr
+
+
+def convert_pics_into_pngs(src_dir, tgt_dir, suffix, convert='RGB'):
+ if not os.path.exists(tgt_dir):
+ os.makedirs(tgt_dir)
+
+ src_paths, src_names = filter_suffix_recursive(src_dir, suffix=suffix)
+
+ for i, (src_name, src_path) in enumerate(zip(src_names, src_paths)):
+ tgt_name = src_name.replace(suffix, save_img_suffix)
+ tgt_path = os.path.join(tgt_dir, tgt_name)
+ num = len(src_paths)
+ img = np.array(Image.open(src_path))
+ if len(img.shape) == 2:
+ pil = Image.fromarray(img).convert(convert)
+ elif len(img.shape) == 3:
+ pil = Image.fromarray(img)
+ else:
+ raise ValueError('Input image not 2D/3D: ', img.shape)
+
+ pil.save(tgt_path)
+ print(f'processed {i+1}/{num}.')
+
+
+def convert_label_pics_into_pngs(src_dir,
+ tgt_dir,
+ suffix,
+ convert_dict={
+ 0: 0,
+ 255: 1
+ }):
+ if not os.path.exists(tgt_dir):
+ os.makedirs(tgt_dir)
+
+ src_paths, src_names = filter_suffix_recursive(src_dir, suffix=suffix)
+ num = len(src_paths)
+ for i, (src_name, src_path) in enumerate(zip(src_names, src_paths)):
+ tgt_name = src_name.replace(suffix, save_seg_map_suffix)
+ tgt_path = os.path.join(tgt_dir, tgt_name)
+
+ img = np.array(Image.open(src_path).convert('L'))
+ img = convert_label(img, convert_dict)
+ Image.fromarray(img).save(tgt_path)
+ print(f'processed {i+1}/{num}.')
+
+
+if __name__ == '__main__':
+
+ convert_pics_into_pngs(
+ src_img_train_dir, tgt_img_train_dir, suffix=img_suffix)
+
+ convert_pics_into_pngs(src_img_val_dir, tgt_img_val_dir, suffix=img_suffix)
+
+ convert_pics_into_pngs(
+ src_img_test_dir, tgt_img_test_dir, suffix=img_suffix)
+
+ convert_label_pics_into_pngs(
+ src_mask_train_dir, tgt_mask_train_dir, suffix=seg_map_suffix)
+
+ convert_label_pics_into_pngs(
+ src_mask_val_dir, tgt_mask_val_dir, suffix=seg_map_suffix)
diff --git a/projects/medical/2d_image/histopathology/pannuke/README.md b/projects/medical/2d_image/histopathology/pannuke/README.md
new file mode 100644
index 00000000000..e0cade7536d
--- /dev/null
+++ b/projects/medical/2d_image/histopathology/pannuke/README.md
@@ -0,0 +1,146 @@
+# Pan-Cancer Histology Dataset for Nuclei Instance Segmentation and Classification (PanNuke)
+
+## Description
+
+This project supports **`Pan-Cancer Histology Dataset for Nuclei Instance Segmentation and Classification (PanNuke)`**, which can be downloaded from [here](https://academictorrents.com/details/99f2c7b57b95500711e33f2ee4d14c9fd7c7366c).
+
+### Dataset Overview
+
+Semi automatically generated nuclei instance segmentation and classification dataset with exhaustive nuclei labels across 19 different tissue types. The dataset consists of 481 visual fields, of which 312 are randomly sampled from more than 20K whole slide images at different magnifications, from multiple data sources. In total the dataset contains 205,343 labeled nuclei, each with an instance segmentation mask. Models trained on pannuke can aid in whole slide image tissue type segmentation, and generalise to new tissues. PanNuke demonstrates one of the first successfully semi-automatically generated datasets.
+
+### Statistic Information
+
+| Dataset Name | Anatomical Region | Task Type | Modality | Num. Classes | Train/Val/Test Images | Train/Val/Test Labeled | Release Date | License |
+| ---------------------------------------------------------------------------------------- | ----------------- | ------------ | -------------- | ------------ | --------------------- | ---------------------- | ------------ | --------------------------------------------------------------- |
+| [Pannuke](https://academictorrents.com/details/99f2c7b57b95500711e33f2ee4d14c9fd7c7366c) | full_body | segmentation | histopathology | 6 | 7901/-/- | yes/-/- | 2019 | [CC-BY-NC 4.0](https://creativecommons.org/licenses/by-sa/4.0/) |
+
+| Class Name | Num. Train | Pct. Train | Num. Val | Pct. Val | Num. Test | Pct. Test |
+| :-----------------------: | :--------: | :--------: | :------: | :------: | :-------: | :-------: |
+| background | 7901 | 83.32 | - | - | - | - |
+| neoplastic | 4190 | 8.64 | - | - | - | - |
+| non-neoplastic epithelial | 4126 | 1.77 | - | - | - | - |
+| inflammatory | 6137 | 3.73 | - | - | - | - |
+| connective | 232 | 0.07 | - | - | - | - |
+| dead | 1528 | 2.47 | - | - | - | - |
+
+Note:
+
+- `Pct` means percentage of pixels in this category in all pixels.
+
+### Visualization
+
+
+
+### Dataset Citation
+
+```
+@inproceedings{gamper2019pannuke,
+ title={PanNuke: an open pan-cancer histology dataset for nuclei instance segmentation and classification},
+ author={Gamper, Jevgenij and Koohbanani, Navid Alemi and Benet, Ksenija and Khuram, Ali and Rajpoot, Nasir},
+ booktitle={European Congress on Digital Pathology},
+ pages={11--19},
+ year={2019},
+}
+```
+
+### Prerequisites
+
+- Python v3.8
+- PyTorch v1.10.0
+- pillow(PIL) v9.3.0 9.3.0
+- scikit-learn(sklearn) v1.2.0 1.2.0
+- [MIM](https://github.com/open-mmlab/mim) v0.3.4
+- [MMCV](https://github.com/open-mmlab/mmcv) v2.0.0rc4
+- [MMEngine](https://github.com/open-mmlab/mmengine) v0.2.0 or higher
+- [MMSegmentation](https://github.com/open-mmlab/mmsegmentation) v1.0.0rc5
+
+All the commands below rely on the correct configuration of `PYTHONPATH`, which should point to the project's directory so that Python can locate the module files. In `pannuke/` root directory, run the following line to add the current directory to `PYTHONPATH`:
+
+```shell
+export PYTHONPATH=`pwd`:$PYTHONPATH
+```
+
+### Dataset Preparing
+
+- download dataset from [here](https://academictorrents.com/details/99f2c7b57b95500711e33f2ee4d14c9fd7c7366c) and decompress data to path `'data/'`.
+- run script `"python tools/prepare_dataset.py"` to format data and change folder structure as below.
+- run script `"python ../../tools/split_seg_dataset.py"` to split dataset and generate `train.txt`, `val.txt` and `test.txt`. If the label of official validation set and test set cannot be obtained, we generate `train.txt` and `val.txt` from the training set randomly.
+
+```none
+ mmsegmentation
+ ├── mmseg
+ ├── projects
+ │ ├── medical
+ │ │ ├── 2d_image
+ │ │ │ ├── histopathology
+ │ │ │ │ ├── pannuke
+ │ │ │ │ │ ├── configs
+ │ │ │ │ │ ├── datasets
+ │ │ │ │ │ ├── tools
+ │ │ │ │ │ ├── data
+ │ │ │ │ │ │ ├── train.txt
+ │ │ │ │ │ │ ├── val.txt
+ │ │ │ │ │ │ ├── images
+ │ │ │ │ │ │ │ ├── train
+ │ │ │ │ | │ │ │ ├── xxx.png
+ │ │ │ │ | │ │ │ ├── ...
+ │ │ │ │ | │ │ │ └── xxx.png
+ │ │ │ │ │ │ ├── masks
+ │ │ │ │ │ │ │ ├── train
+ │ │ │ │ | │ │ │ ├── xxx.png
+ │ │ │ │ | │ │ │ ├── ...
+ │ │ │ │ | │ │ │ └── xxx.png
+```
+
+### Divided Dataset Information
+
+***Note: The table information below is divided by ourselves.***
+
+| Class Name | Num. Train | Pct. Train | Num. Val | Pct. Val | Num. Test | Pct. Test |
+| :-----------------------: | :--------: | :--------: | :------: | :------: | :-------: | :-------: |
+| background | 6320 | 83.38 | 1581 | 83.1 | - | - |
+| neoplastic | 3339 | 8.55 | 851 | 9.0 | - | - |
+| non-neoplastic epithelial | 3293 | 1.77 | 833 | 1.76 | - | - |
+| inflammatory | 4914 | 3.72 | 1223 | 3.76 | - | - |
+| connective | 170 | 0.06 | 62 | 0.09 | - | - |
+| dead | 1235 | 2.51 | 293 | 2.29 | - | - |
+
+### Training commands
+
+To train models on a single server with one GPU. (default)
+
+```shell
+mim train mmseg ./configs/${CONFIG_FILE}
+```
+
+### Testing commands
+
+To test models on a single server with one GPU. (default)
+
+```shell
+mim test mmseg ./configs/${CONFIG_FILE} --checkpoint ${CHECKPOINT_PATH}
+```
+
+## Checklist
+
+- [x] Milestone 1: PR-ready, and acceptable to be one of the `projects/`.
+
+ - [x] Finish the code
+ - [x] Basic docstrings & proper citation
+ - [ ] Test-time correctness
+ - [x] A full README
+
+- [ ] Milestone 2: Indicates a successful model implementation.
+
+ - [ ] Training-time correctness
+
+- [ ] Milestone 3: Good to be a part of our core package!
+
+ - [ ] Type hints and docstrings
+ - [ ] Unit tests
+ - [ ] Code polishing
+ - [ ] Metafile.yml
+
+- [ ] Move your modules into the core package following the codebase's file hierarchy structure.
+
+- [ ] Refactor your modules into the core package following the codebase's file hierarchy structure.
diff --git a/projects/medical/2d_image/histopathology/pannuke/configs/fcn-unet-s5-d16_unet-{use-sigmoid}_1xb16-0.01-20k_bactteria-detection-512x512.py b/projects/medical/2d_image/histopathology/pannuke/configs/fcn-unet-s5-d16_unet-{use-sigmoid}_1xb16-0.01-20k_bactteria-detection-512x512.py
new file mode 100644
index 00000000000..92584e9a684
--- /dev/null
+++ b/projects/medical/2d_image/histopathology/pannuke/configs/fcn-unet-s5-d16_unet-{use-sigmoid}_1xb16-0.01-20k_bactteria-detection-512x512.py
@@ -0,0 +1,18 @@
+_base_ = [
+ 'mmseg::_base_/models/fcn_unet_s5-d16.py',
+ './bactteria-detection_512x512.py', 'mmseg::_base_/default_runtime.py',
+ 'mmseg::_base_/schedules/schedule_20k.py'
+]
+custom_imports = dict(imports='datasets.bactteria-detection_dataset')
+img_scale = (512, 512)
+data_preprocessor = dict(size=img_scale)
+optimizer = dict(lr=0.01)
+optim_wrapper = dict(optimizer=optimizer)
+model = dict(
+ data_preprocessor=data_preprocessor,
+ decode_head=dict(
+ num_classes=2, loss_decode=dict(use_sigmoid=True), out_channels=1),
+ auxiliary_head=None,
+ test_cfg=dict(mode='whole', _delete_=True))
+vis_backends = None
+visualizer = dict(vis_backends=vis_backends)
diff --git a/projects/medical/2d_image/histopathology/pannuke/configs/fcn-unet-s5-d16_unet_1xb16-0.0001-20k_pannuke-512x512.py b/projects/medical/2d_image/histopathology/pannuke/configs/fcn-unet-s5-d16_unet_1xb16-0.0001-20k_pannuke-512x512.py
new file mode 100644
index 00000000000..042a08ce008
--- /dev/null
+++ b/projects/medical/2d_image/histopathology/pannuke/configs/fcn-unet-s5-d16_unet_1xb16-0.0001-20k_pannuke-512x512.py
@@ -0,0 +1,17 @@
+_base_ = [
+ 'mmseg::_base_/models/fcn_unet_s5-d16.py', './pannuke_512x512.py',
+ 'mmseg::_base_/default_runtime.py',
+ 'mmseg::_base_/schedules/schedule_20k.py'
+]
+custom_imports = dict(imports='datasets.pannuke_dataset')
+img_scale = (512, 512)
+data_preprocessor = dict(size=img_scale)
+optimizer = dict(lr=0.0001)
+optim_wrapper = dict(optimizer=optimizer)
+model = dict(
+ data_preprocessor=data_preprocessor,
+ decode_head=dict(num_classes=6),
+ auxiliary_head=None,
+ test_cfg=dict(mode='whole', _delete_=True))
+vis_backends = None
+visualizer = dict(vis_backends=vis_backends)
diff --git a/projects/medical/2d_image/histopathology/pannuke/configs/fcn-unet-s5-d16_unet_1xb16-0.001-20k_pannuke-512x512.py b/projects/medical/2d_image/histopathology/pannuke/configs/fcn-unet-s5-d16_unet_1xb16-0.001-20k_pannuke-512x512.py
new file mode 100644
index 00000000000..e92514c9132
--- /dev/null
+++ b/projects/medical/2d_image/histopathology/pannuke/configs/fcn-unet-s5-d16_unet_1xb16-0.001-20k_pannuke-512x512.py
@@ -0,0 +1,17 @@
+_base_ = [
+ 'mmseg::_base_/models/fcn_unet_s5-d16.py', './pannuke_512x512.py',
+ 'mmseg::_base_/default_runtime.py',
+ 'mmseg::_base_/schedules/schedule_20k.py'
+]
+custom_imports = dict(imports='datasets.pannuke_dataset')
+img_scale = (512, 512)
+data_preprocessor = dict(size=img_scale)
+optimizer = dict(lr=0.001)
+optim_wrapper = dict(optimizer=optimizer)
+model = dict(
+ data_preprocessor=data_preprocessor,
+ decode_head=dict(num_classes=6),
+ auxiliary_head=None,
+ test_cfg=dict(mode='whole', _delete_=True))
+vis_backends = None
+visualizer = dict(vis_backends=vis_backends)
diff --git a/projects/medical/2d_image/histopathology/pannuke/configs/fcn-unet-s5-d16_unet_1xb16-0.01-20k_pannuke-512x512.py b/projects/medical/2d_image/histopathology/pannuke/configs/fcn-unet-s5-d16_unet_1xb16-0.01-20k_pannuke-512x512.py
new file mode 100644
index 00000000000..a9403c849fa
--- /dev/null
+++ b/projects/medical/2d_image/histopathology/pannuke/configs/fcn-unet-s5-d16_unet_1xb16-0.01-20k_pannuke-512x512.py
@@ -0,0 +1,17 @@
+_base_ = [
+ 'mmseg::_base_/models/fcn_unet_s5-d16.py', './pannuke_512x512.py',
+ 'mmseg::_base_/default_runtime.py',
+ 'mmseg::_base_/schedules/schedule_20k.py'
+]
+custom_imports = dict(imports='datasets.pannuke_dataset')
+img_scale = (512, 512)
+data_preprocessor = dict(size=img_scale)
+optimizer = dict(lr=0.01)
+optim_wrapper = dict(optimizer=optimizer)
+model = dict(
+ data_preprocessor=data_preprocessor,
+ decode_head=dict(num_classes=6),
+ auxiliary_head=None,
+ test_cfg=dict(mode='whole', _delete_=True))
+vis_backends = None
+visualizer = dict(vis_backends=vis_backends)
diff --git a/projects/medical/2d_image/histopathology/pannuke/configs/pannuke_512x512.py b/projects/medical/2d_image/histopathology/pannuke/configs/pannuke_512x512.py
new file mode 100644
index 00000000000..316ac1ac443
--- /dev/null
+++ b/projects/medical/2d_image/histopathology/pannuke/configs/pannuke_512x512.py
@@ -0,0 +1,42 @@
+dataset_type = 'PanNukeDataset'
+data_root = 'data/'
+img_scale = (512, 512)
+train_pipeline = [
+ dict(type='LoadImageFromFile'),
+ dict(type='LoadAnnotations'),
+ dict(type='Resize', scale=img_scale, keep_ratio=False),
+ dict(type='RandomFlip', prob=0.5),
+ dict(type='PhotoMetricDistortion'),
+ dict(type='PackSegInputs')
+]
+test_pipeline = [
+ dict(type='LoadImageFromFile'),
+ dict(type='Resize', scale=img_scale, keep_ratio=False),
+ dict(type='LoadAnnotations'),
+ dict(type='PackSegInputs')
+]
+train_dataloader = dict(
+ batch_size=16,
+ num_workers=4,
+ persistent_workers=True,
+ sampler=dict(type='InfiniteSampler', shuffle=True),
+ dataset=dict(
+ type=dataset_type,
+ data_root=data_root,
+ ann_file='train.txt',
+ data_prefix=dict(img_path='images/', seg_map_path='masks/'),
+ pipeline=train_pipeline))
+val_dataloader = dict(
+ batch_size=1,
+ num_workers=4,
+ persistent_workers=True,
+ sampler=dict(type='DefaultSampler', shuffle=False),
+ dataset=dict(
+ type=dataset_type,
+ data_root=data_root,
+ ann_file='val.txt',
+ data_prefix=dict(img_path='images/', seg_map_path='masks/'),
+ pipeline=test_pipeline))
+test_dataloader = val_dataloader
+val_evaluator = dict(type='IoUMetric', iou_metrics=['mIoU', 'mDice'])
+test_evaluator = dict(type='IoUMetric', iou_metrics=['mIoU', 'mDice'])
diff --git a/projects/medical/2d_image/histopathology/pannuke/datasets/pannuke_dataset.py b/projects/medical/2d_image/histopathology/pannuke/datasets/pannuke_dataset.py
new file mode 100644
index 00000000000..4d3c687ff31
--- /dev/null
+++ b/projects/medical/2d_image/histopathology/pannuke/datasets/pannuke_dataset.py
@@ -0,0 +1,33 @@
+from mmseg.datasets import BaseSegDataset
+from mmseg.registry import DATASETS
+
+
+@DATASETS.register_module()
+class PanNukeDataset(BaseSegDataset):
+ """PanNukeDataset dataset.
+
+ In segmentation map annotation for PanNukeDataset,
+ 0 stands for background, which is included in 6 categories.
+ ``reduce_zero_label`` is fixed to False. The ``img_suffix``
+ is fixed to '.png' and ``seg_map_suffix`` is fixed to '.png'.
+
+ Args:
+ img_suffix (str): Suffix of images. Default: '.png'
+ seg_map_suffix (str): Suffix of segmentation maps. Default: '.png'
+ reduce_zero_label (bool): Whether to mark label zero as ignored.
+ Default to False.
+ """
+ METAINFO = dict(
+ classes=('background', 'neoplastic', 'non-neoplastic epithelial',
+ 'inflammatory', 'connective', 'dead'))
+
+ def __init__(self,
+ img_suffix='.png',
+ seg_map_suffix='.png',
+ reduce_zero_label=False,
+ **kwargs) -> None:
+ super().__init__(
+ img_suffix=img_suffix,
+ seg_map_suffix=seg_map_suffix,
+ reduce_zero_label=reduce_zero_label,
+ **kwargs)
diff --git a/projects/medical/2d_image/histopathology/pannuke/tools/prepare_dataset.py b/projects/medical/2d_image/histopathology/pannuke/tools/prepare_dataset.py
new file mode 100644
index 00000000000..7213b181f40
--- /dev/null
+++ b/projects/medical/2d_image/histopathology/pannuke/tools/prepare_dataset.py
@@ -0,0 +1,49 @@
+import os
+
+import numpy as np
+from PIL import Image
+
+root_path = 'data/'
+
+tgt_img_dir = os.path.join(root_path, 'images/train')
+tgt_mask_dir = os.path.join(root_path, 'masks/train')
+os.system('mkdir -p ' + tgt_img_dir)
+os.system('mkdir -p ' + tgt_mask_dir)
+
+fold_img_paths = sorted([
+ os.path.join(root_path, 'pannuke/Fold 1/images/fold1/images.npy'),
+ os.path.join(root_path, 'pannuke/Fold 2/images/fold2/images.npy'),
+ os.path.join(root_path, 'pannuke/Fold 3/images/fold3/images.npy')
+])
+
+fold_mask_paths = sorted([
+ os.path.join(root_path, 'pannuke/Fold 1/masks/fold1/masks.npy'),
+ os.path.join(root_path, 'pannuke/Fold 2/masks/fold2/masks.npy'),
+ os.path.join(root_path, 'pannuke/Fold 3/masks/fold3/masks.npy')
+])
+
+for n, (img_path,
+ mask_path) in enumerate(zip(fold_img_paths, fold_mask_paths)):
+ fold_name = str(n + 1)
+ imgs = np.load(img_path)
+ masks = np.load(mask_path)
+
+ for i in range(imgs.shape[0]):
+ img = np.uint8(imgs[i])
+ mask_multichannel = np.minimum(np.uint8(masks[i]), 1)
+ mask = np.zeros((img.shape[0], img.shape[1]), dtype=np.uint8)
+ for j in range(mask_multichannel.shape[-1]):
+ factor = (j + 1) % mask_multichannel.shape[-1]
+ # convert [0,1,2,3,4,5] to [1,2,3,4,5,0],
+ # with the last label being background
+ mask[mask_multichannel[..., j] == 1] = factor
+
+ file_name = 'fold' + fold_name + '_' + str(i).rjust(4, '0') + '.png'
+ print('Processing: ', file_name)
+ tgt_img_path = os.path.join(tgt_img_dir, file_name)
+ tgt_mask_path = os.path.join(tgt_mask_dir, file_name)
+ Image.fromarray(img).save(tgt_img_path)
+ Image.fromarray(mask).save(tgt_mask_path)
+
+ del imgs
+ del masks
diff --git a/projects/medical/2d_image/histopathology/pcam/README.md b/projects/medical/2d_image/histopathology/pcam/README.md
new file mode 100644
index 00000000000..5a8094950c5
--- /dev/null
+++ b/projects/medical/2d_image/histopathology/pcam/README.md
@@ -0,0 +1,153 @@
+# PCam (PatchCamelyon)
+
+## Description
+
+This project supports **`Patch Camelyon (PCam) `**, which can be downloaded from [here](https://opendatalab.com/PCam).
+
+### Dataset Overview
+
+PatchCamelyon is an image classification dataset. It consists of 327680 color images (96 x 96px) extracted from histopathologic scans of lymph node sections. Each image is annotated with a binary label indicating presence of metastatic tissue. PCam provides a new benchmark for machine learning models: bigger than CIFAR10, smaller than ImageNet, trainable on a single GPU.
+
+### Statistic Information
+
+| Dataset Name | Anatomical Region | Task Type | Modality | Num. Classes | Train/Val/Test images | Train/Val/Test Labeled | Release Date | License |
+| ------------------------------------ | ----------------- | ------------ | -------------- | ------------ | --------------------- | ---------------------- | ------------ | ------------------------------------------------------------- |
+| [Pcam](https://opendatalab.com/PCam) | throax | segmentation | histopathology | 2 | 327680/-/- | yes/-/- | 2018 | [CC0 1.0](https://creativecommons.org/publicdomain/zero/1.0/) |
+
+| Class Name | Num. Train | Pct. Train | Num. Val | Pct. Val | Num. Test | Pct. Test |
+| :---------------: | :--------: | :--------: | :------: | :------: | :-------: | :-------: |
+| background | 214849 | 63.77 | - | - | - | - |
+| metastatic tissue | 131832 | 36.22 | - | - | - | - |
+
+Note:
+
+- `Pct` means percentage of pixels in this category in all pixels.
+
+### Visualization
+
+
+
+### Dataset Citation
+
+```
+@inproceedings{veeling2018rotation,
+ title={Rotation equivariant CNNs for digital pathology},
+ author={Veeling, Bastiaan S and Linmans, Jasper and Winkens, Jim and Cohen, Taco and Welling, Max},
+ booktitle={International Conference on Medical image computing and computer-assisted intervention},
+ pages={210--218},
+ year={2018},
+}
+```
+
+### Prerequisites
+
+- Python v3.8
+- PyTorch v1.10.0
+- pillow(PIL) v9.3.0 9.3.0
+- scikit-learn(sklearn) v1.2.0 1.2.0
+- [MIM](https://github.com/open-mmlab/mim) v0.3.4
+- [MMCV](https://github.com/open-mmlab/mmcv) v2.0.0rc4
+- [MMEngine](https://github.com/open-mmlab/mmengine) v0.2.0 or higher
+- [MMSegmentation](https://github.com/open-mmlab/mmsegmentation) v1.0.0rc5
+
+All the commands below rely on the correct configuration of `PYTHONPATH`, which should point to the project's directory so that Python can locate the module files. In `pcam/` root directory, run the following line to add the current directory to `PYTHONPATH`:
+
+```shell
+export PYTHONPATH=`pwd`:$PYTHONPATH
+```
+
+### Dataset Preparing
+
+- download dataset from [here](https://opendatalab.com/PCam) and decompress data to path `'data/'`.
+- run script `"python tools/prepare_dataset.py"` to format data and change folder structure as below.
+- run script `"python ../../tools/split_seg_dataset.py"` to split dataset and generate `train.txt`, `val.txt` and `test.txt`. If the label of official validation set and test set cannot be obtained, we generate `train.txt` and `val.txt` from the training set randomly.
+
+```shell
+mkdir data & cd data
+pip install opendatalab
+odl get PCam
+mv ./PCam/raw/pcamv1 ./
+rm -rf PCam
+cd ..
+python tools/prepare_dataset.py
+python ../../tools/split_seg_dataset.py
+```
+
+```none
+ mmsegmentation
+ ├── mmseg
+ ├── projects
+ │ ├── medical
+ │ │ ├── 2d_image
+ │ │ │ ├── histopathology
+ │ │ │ │ ├── pcam
+ │ │ │ │ │ ├── configs
+ │ │ │ │ │ ├── datasets
+ │ │ │ │ │ ├── tools
+ │ │ │ │ │ ├── data
+ │ │ │ │ │ │ ├── train.txt
+ │ │ │ │ │ │ ├── val.txt
+ │ │ │ │ │ │ ├── images
+ │ │ │ │ │ │ │ ├── train
+ │ │ │ │ | │ │ │ ├── xxx.png
+ │ │ │ │ | │ │ │ ├── ...
+ │ │ │ │ | │ │ │ └── xxx.png
+ │ │ │ │ │ │ ├── masks
+ │ │ │ │ │ │ │ ├── train
+ │ │ │ │ | │ │ │ ├── xxx.png
+ │ │ │ │ | │ │ │ ├── ...
+ │ │ │ │ | │ │ │ └── xxx.png
+```
+
+### Divided Dataset Information
+
+***Note: The table information below is divided by ourselves.***
+
+| Class Name | Num. Train | Pct. Train | Num. Val | Pct. Val | Num. Test | Pct. Test |
+| :---------------: | :--------: | :--------: | :------: | :------: | :-------: | :-------: |
+| background | 171948 | 63.82 | 42901 | 63.6 | - | - |
+| metastatic tissue | 105371 | 36.18 | 26461 | 36.4 | - | - |
+
+### Training commands
+
+To train models on a single server with one GPU. (default)
+
+```shell
+mim train mmseg ./configs/${CONFIG_FILE}
+```
+
+### Testing commands
+
+To test models on a single server with one GPU. (default)
+
+```shell
+mim test mmseg ./configs/${CONFIG_FILE} --checkpoint ${CHECKPOINT_PATH}
+```
+
+
+
+## Checklist
+
+- [x] Milestone 1: PR-ready, and acceptable to be one of the `projects/`.
+
+ - [x] Finish the code
+ - [x] Basic docstrings & proper citation
+ - [ ] Test-time correctness
+ - [x] A full README
+
+- [ ] Milestone 2: Indicates a successful model implementation.
+
+ - [ ] Training-time correctness
+
+- [ ] Milestone 3: Good to be a part of our core package!
+
+ - [ ] Type hints and docstrings
+ - [ ] Unit tests
+ - [ ] Code polishing
+ - [ ] Metafile.yml
+
+- [ ] Move your modules into the core package following the codebase's file hierarchy structure.
+
+- [ ] Refactor your modules into the core package following the codebase's file hierarchy structure.
diff --git a/projects/medical/2d_image/histopathology/pcam/configs/fcn-unet-s5-d16_unet_1xb16-0.0001-20k_pcam-512x512.py b/projects/medical/2d_image/histopathology/pcam/configs/fcn-unet-s5-d16_unet_1xb16-0.0001-20k_pcam-512x512.py
new file mode 100644
index 00000000000..20601f1ea56
--- /dev/null
+++ b/projects/medical/2d_image/histopathology/pcam/configs/fcn-unet-s5-d16_unet_1xb16-0.0001-20k_pcam-512x512.py
@@ -0,0 +1,17 @@
+_base_ = [
+ 'mmseg::_base_/models/fcn_unet_s5-d16.py', './pcam_512x512.py',
+ 'mmseg::_base_/default_runtime.py',
+ 'mmseg::_base_/schedules/schedule_20k.py'
+]
+custom_imports = dict(imports='datasets.pcam_dataset')
+img_scale = (512, 512)
+data_preprocessor = dict(size=img_scale)
+optimizer = dict(lr=0.0001)
+optim_wrapper = dict(optimizer=optimizer)
+model = dict(
+ data_preprocessor=data_preprocessor,
+ decode_head=dict(num_classes=2),
+ auxiliary_head=None,
+ test_cfg=dict(mode='whole', _delete_=True))
+vis_backends = None
+visualizer = dict(vis_backends=vis_backends)
diff --git a/projects/medical/2d_image/histopathology/pcam/configs/fcn-unet-s5-d16_unet_1xb16-0.001-20k_pcam-512x512.py b/projects/medical/2d_image/histopathology/pcam/configs/fcn-unet-s5-d16_unet_1xb16-0.001-20k_pcam-512x512.py
new file mode 100644
index 00000000000..c057535409f
--- /dev/null
+++ b/projects/medical/2d_image/histopathology/pcam/configs/fcn-unet-s5-d16_unet_1xb16-0.001-20k_pcam-512x512.py
@@ -0,0 +1,17 @@
+_base_ = [
+ 'mmseg::_base_/models/fcn_unet_s5-d16.py', './pcam_512x512.py',
+ 'mmseg::_base_/default_runtime.py',
+ 'mmseg::_base_/schedules/schedule_20k.py'
+]
+custom_imports = dict(imports='datasets.pcam_dataset')
+img_scale = (512, 512)
+data_preprocessor = dict(size=img_scale)
+optimizer = dict(lr=0.001)
+optim_wrapper = dict(optimizer=optimizer)
+model = dict(
+ data_preprocessor=data_preprocessor,
+ decode_head=dict(num_classes=2),
+ auxiliary_head=None,
+ test_cfg=dict(mode='whole', _delete_=True))
+vis_backends = None
+visualizer = dict(vis_backends=vis_backends)
diff --git a/projects/medical/2d_image/histopathology/pcam/configs/fcn-unet-s5-d16_unet_1xb16-0.01-20k_pcam-512x512.py b/projects/medical/2d_image/histopathology/pcam/configs/fcn-unet-s5-d16_unet_1xb16-0.01-20k_pcam-512x512.py
new file mode 100644
index 00000000000..4c1d5fe421c
--- /dev/null
+++ b/projects/medical/2d_image/histopathology/pcam/configs/fcn-unet-s5-d16_unet_1xb16-0.01-20k_pcam-512x512.py
@@ -0,0 +1,17 @@
+_base_ = [
+ 'mmseg::_base_/models/fcn_unet_s5-d16.py', './pcam_512x512.py',
+ 'mmseg::_base_/default_runtime.py',
+ 'mmseg::_base_/schedules/schedule_20k.py'
+]
+custom_imports = dict(imports='datasets.pcam_dataset')
+img_scale = (512, 512)
+data_preprocessor = dict(size=img_scale)
+optimizer = dict(lr=0.01)
+optim_wrapper = dict(optimizer=optimizer)
+model = dict(
+ data_preprocessor=data_preprocessor,
+ decode_head=dict(num_classes=2),
+ auxiliary_head=None,
+ test_cfg=dict(mode='whole', _delete_=True))
+vis_backends = None
+visualizer = dict(vis_backends=vis_backends)
diff --git a/projects/medical/2d_image/histopathology/pcam/configs/fcn-unet-s5-d16_unet_1xb16-0.01lr-sigmoid-20k_pcam-512x512.py b/projects/medical/2d_image/histopathology/pcam/configs/fcn-unet-s5-d16_unet_1xb16-0.01lr-sigmoid-20k_pcam-512x512.py
new file mode 100644
index 00000000000..25e3734795c
--- /dev/null
+++ b/projects/medical/2d_image/histopathology/pcam/configs/fcn-unet-s5-d16_unet_1xb16-0.01lr-sigmoid-20k_pcam-512x512.py
@@ -0,0 +1,18 @@
+_base_ = [
+ 'mmseg::_base_/models/fcn_unet_s5-d16.py', './pcam_512x512.py',
+ 'mmseg::_base_/default_runtime.py',
+ 'mmseg::_base_/schedules/schedule_20k.py'
+]
+custom_imports = dict(imports='datasets.pcam_dataset')
+img_scale = (512, 512)
+data_preprocessor = dict(size=img_scale)
+optimizer = dict(lr=0.01)
+optim_wrapper = dict(optimizer=optimizer)
+model = dict(
+ data_preprocessor=data_preprocessor,
+ decode_head=dict(
+ num_classes=2, loss_decode=dict(use_sigmoid=True), out_channels=1),
+ auxiliary_head=None,
+ test_cfg=dict(mode='whole', _delete_=True))
+vis_backends = None
+visualizer = dict(vis_backends=vis_backends)
diff --git a/projects/medical/2d_image/histopathology/pcam/configs/pcam_512x512.py b/projects/medical/2d_image/histopathology/pcam/configs/pcam_512x512.py
new file mode 100644
index 00000000000..04efc23eb51
--- /dev/null
+++ b/projects/medical/2d_image/histopathology/pcam/configs/pcam_512x512.py
@@ -0,0 +1,42 @@
+dataset_type = 'PCamDataset'
+data_root = 'data/'
+img_scale = (512, 512)
+train_pipeline = [
+ dict(type='LoadImageFromFile'),
+ dict(type='LoadAnnotations'),
+ dict(type='Resize', scale=img_scale, keep_ratio=False),
+ dict(type='RandomFlip', prob=0.5),
+ dict(type='PhotoMetricDistortion'),
+ dict(type='PackSegInputs')
+]
+test_pipeline = [
+ dict(type='LoadImageFromFile'),
+ dict(type='Resize', scale=img_scale, keep_ratio=False),
+ dict(type='LoadAnnotations'),
+ dict(type='PackSegInputs')
+]
+train_dataloader = dict(
+ batch_size=16,
+ num_workers=4,
+ persistent_workers=True,
+ sampler=dict(type='InfiniteSampler', shuffle=True),
+ dataset=dict(
+ type=dataset_type,
+ data_root=data_root,
+ ann_file='train.txt',
+ data_prefix=dict(img_path='images/', seg_map_path='masks/'),
+ pipeline=train_pipeline))
+val_dataloader = dict(
+ batch_size=1,
+ num_workers=4,
+ persistent_workers=True,
+ sampler=dict(type='DefaultSampler', shuffle=False),
+ dataset=dict(
+ type=dataset_type,
+ data_root=data_root,
+ ann_file='val.txt',
+ data_prefix=dict(img_path='images/', seg_map_path='masks/'),
+ pipeline=test_pipeline))
+test_dataloader = val_dataloader
+val_evaluator = dict(type='IoUMetric', iou_metrics=['mIoU', 'mDice'])
+test_evaluator = dict(type='IoUMetric', iou_metrics=['mIoU', 'mDice'])
diff --git a/projects/medical/2d_image/histopathology/pcam/datasets/pcam_dataset.py b/projects/medical/2d_image/histopathology/pcam/datasets/pcam_dataset.py
new file mode 100644
index 00000000000..1c27de543ab
--- /dev/null
+++ b/projects/medical/2d_image/histopathology/pcam/datasets/pcam_dataset.py
@@ -0,0 +1,31 @@
+from mmseg.datasets import BaseSegDataset
+from mmseg.registry import DATASETS
+
+
+@DATASETS.register_module()
+class PCamDataset(BaseSegDataset):
+ """PCamDataset dataset.
+
+ In segmentation map annotation for PCamDataset,
+ 0 stands for background, which is included in 2 categories.
+ ``reduce_zero_label`` is fixed to False. The ``img_suffix``
+ is fixed to '.png' and ``seg_map_suffix`` is fixed to '.png'.
+
+ Args:
+ img_suffix (str): Suffix of images. Default: '.png'
+ seg_map_suffix (str): Suffix of segmentation maps. Default: '.png'
+ reduce_zero_label (bool): Whether to mark label zero as ignored.
+ Default to False.
+ """
+ METAINFO = dict(classes=('background', 'metastatic tissue'))
+
+ def __init__(self,
+ img_suffix='.png',
+ seg_map_suffix='.png',
+ reduce_zero_label=False,
+ **kwargs) -> None:
+ super().__init__(
+ img_suffix=img_suffix,
+ seg_map_suffix=seg_map_suffix,
+ reduce_zero_label=reduce_zero_label,
+ **kwargs)
diff --git a/projects/medical/2d_image/histopathology/pcam/tools/prepare_dataset.py b/projects/medical/2d_image/histopathology/pcam/tools/prepare_dataset.py
new file mode 100644
index 00000000000..75038e6fb49
--- /dev/null
+++ b/projects/medical/2d_image/histopathology/pcam/tools/prepare_dataset.py
@@ -0,0 +1,49 @@
+import os
+
+import h5py
+import numpy as np
+from PIL import Image
+
+root_path = 'data/'
+
+tgt_img_train_dir = os.path.join(root_path, 'images/train/')
+tgt_mask_train_dir = os.path.join(root_path, 'masks/train/')
+tgt_img_val_dir = os.path.join(root_path, 'images/val/')
+tgt_img_test_dir = os.path.join(root_path, 'images/test/')
+
+os.system('mkdir -p ' + tgt_img_train_dir)
+os.system('mkdir -p ' + tgt_mask_train_dir)
+os.system('mkdir -p ' + tgt_img_val_dir)
+os.system('mkdir -p ' + tgt_img_test_dir)
+
+
+def extract_pics_from_h5(h5_path, h5_key, save_dir):
+ f = h5py.File(h5_path, 'r')
+ for i, img in enumerate(f[h5_key]):
+ img = img.astype(np.uint8).squeeze()
+ img = Image.fromarray(img)
+ save_image_path = os.path.join(save_dir, str(i).zfill(8) + '.png')
+ img.save(save_image_path)
+
+
+if __name__ == '__main__':
+
+ extract_pics_from_h5(
+ 'data/pcamv1/camelyonpatch_level_2_split_train_x.h5',
+ h5_key='x',
+ save_dir=tgt_img_train_dir)
+
+ extract_pics_from_h5(
+ 'data/pcamv1/camelyonpatch_level_2_split_valid_x.h5',
+ h5_key='x',
+ save_dir=tgt_img_val_dir)
+
+ extract_pics_from_h5(
+ 'data/pcamv1/camelyonpatch_level_2_split_test_x.h5',
+ h5_key='x',
+ save_dir=tgt_img_test_dir)
+
+ extract_pics_from_h5(
+ 'data/pcamv1/camelyonpatch_level_2_split_train_mask.h5',
+ h5_key='mask',
+ save_dir=tgt_mask_train_dir)
diff --git a/projects/medical/2d_image/infrared_reflectance_imaging/ravir/README.md b/projects/medical/2d_image/infrared_reflectance_imaging/ravir/README.md
new file mode 100644
index 00000000000..ca95921ba36
--- /dev/null
+++ b/projects/medical/2d_image/infrared_reflectance_imaging/ravir/README.md
@@ -0,0 +1,167 @@
+# RAVIR: A Dataset and Methodology for the Semantic Segmentation and Quantitative Analysis of Retinal Arteries and Veins in Infrared Reflectance Imaging
+
+## Description
+
+This project support **`RAVIR: A Dataset and Methodology for the Semantic Segmentation and Quantitative Analysis of Retinal Arteries and Veins in Infrared Reflectance Imaging`**, and the dataset used in this project can be downloaded from [here](https://ravir.grand-challenge.org/).
+
+### Dataset Overview
+
+The retinal vasculature provides important clues in the diagnosis and monitoring of systemic diseases including hypertension and diabetes. The microvascular system is of primary involvement in such conditions, and the retina is the only anatomical site where the microvasculature can be directly observed. The objective assessment of retinal vessels has long been considered a surrogate biomarker for systemic vascular diseases, and with recent advancements in retinal imaging and computer vision technologies, this topic has become the subject of renewed attention. In this paper, we present a novel dataset, dubbed RAVIR, for the semantic segmentation of Retinal Arteries and Veins in Infrared Reflectance (IR) imaging. It enables the creation of deep learning-based models that distinguish extracted vessel type without extensive post-processing.
+
+### Original Statistic Information
+
+| Dataset name | Anatomical region | Task type | Modality | Num. Classes | Train/Val/Test Images | Train/Val/Test Labeled | Release Date | License |
+| ------------------------------------------- | ----------------- | ------------ | ---------------------------- | ------------ | --------------------- | ---------------------- | ------------ | --------------------------------------------------------------- |
+| [Ravir](https://ravir.grand-challenge.org/) | eye | segmentation | infrared reflectance imaging | 3 | 23/-/19 | yes/-/- | 2022 | [CC-BY-NC 4.0](https://creativecommons.org/licenses/by-sa/4.0/) |
+
+| Class Name | Num. Train | Pct. Train | Num. Val | Pct. Val | Num. Test | Pct. Test |
+| :--------: | :--------: | :--------: | :------: | :------: | :-------: | :-------: |
+| background | 23 | 87.22 | - | - | - | - |
+| artery | 23 | 5.45 | - | - | - | - |
+| vein | 23 | 7.33 | - | - | - | - |
+
+Note:
+
+- `Pct` means percentage of pixels in this category in all pixels.
+
+### Visualization
+
+
+
+## Dataset Citation
+
+```bibtex
+@article{hatamizadeh2022ravir,
+ title={RAVIR: A dataset and methodology for the semantic segmentation and quantitative analysis of retinal arteries and veins in infrared reflectance imaging},
+ author={Hatamizadeh, Ali and Hosseini, Hamid and Patel, Niraj and Choi, Jinseo and Pole, Cameron C and Hoeferlin, Cory M and Schwartz, Steven D and Terzopoulos, Demetri},
+ journal={IEEE Journal of Biomedical and Health Informatics},
+ volume={26},
+ number={7},
+ pages={3272--3283},
+ year={2022},
+ publisher={IEEE}
+}
+```
+
+### Prerequisites
+
+- Python v3.8
+- PyTorch v1.10.0
+- pillow(PIL) v9.3.0
+- scikit-learn(sklearn) v1.2.0
+- [MIM](https://github.com/open-mmlab/mim) v0.3.4
+- [MMCV](https://github.com/open-mmlab/mmcv) v2.0.0rc4
+- [MMEngine](https://github.com/open-mmlab/mmengine) v0.2.0 or higher
+- [MMSegmentation](https://github.com/open-mmlab/mmsegmentation) v1.0.0rc5
+
+All the commands below rely on the correct configuration of `PYTHONPATH`, which should point to the project's directory so that Python can locate the module files. In `ravir/` root directory, run the following line to add the current directory to `PYTHONPATH`:
+
+```shell
+export PYTHONPATH=`pwd`:$PYTHONPATH
+```
+
+### Dataset preparing
+
+- download dataset from [here](https://ravir.grand-challenge.org/) and decompression data to path `'data/ravir/'`.
+- run script `"python tools/prepare_dataset.py"` to split dataset and change folder structure as below.
+- run script `"python ../../tools/split_seg_dataset.py --data_root data/ravir"` to split dataset and generate `train.txt`, `val.txt` and `test.txt`. If the label of official validation set and test set can't be obtained, we generate `train.txt` and `val.txt` from the training set randomly.
+
+```none
+ mmsegmentation
+ ├── mmseg
+ ├── projects
+ │ ├── medical
+ │ │ ├── 2d_image
+ │ │ │ ├── infrared_reflectance_imaging
+ │ │ │ │ ├── ravir
+ │ │ │ │ │ ├── configs
+ │ │ │ │ │ ├── datasets
+ │ │ │ │ │ ├── tools
+ │ │ │ │ │ ├── data
+ │ │ │ │ │ │ ├── train.txt
+ │ │ │ │ │ │ ├── val.txt
+ │ │ │ │ │ │ ├── images
+ │ │ │ │ │ │ │ ├── train
+ │ │ │ │ | │ │ │ ├── xxx.png
+ │ │ │ │ | │ │ │ ├── ...
+ │ │ │ │ | │ │ │ └── xxx.png
+ │ │ │ │ │ │ │ ├── test
+ │ │ │ │ | │ │ │ ├── yyy.png
+ │ │ │ │ | │ │ │ ├── ...
+ │ │ │ │ | │ │ │ └── yyy.png
+ │ │ │ │ │ │ ├── masks
+ │ │ │ │ │ │ │ ├── train
+ │ │ │ │ | │ │ │ ├── xxx.png
+ │ │ │ │ | │ │ │ ├── ...
+ │ │ │ │ | │ │ │ └── xxx.png
+```
+
+### Divided Dataset Information
+
+***Note: The table information below is divided by ourselves.***
+
+| Class Name | Num. Train | Pct. Train | Num. Val | Pct. Val | Num. Test | Pct. Test |
+| :--------: | :--------: | :--------: | :------: | :------: | :-------: | :-------: |
+| background | 18 | 87.41 | 5 | 86.53 | - | - |
+| artery | 18 | 5.44 | 5 | 5.50 | - | - |
+| vein | 18 | 7.15 | 5 | 7.97 | - | - |
+
+### Training commands
+
+To train models on a single server with one GPU. (default）
+
+```shell
+mim train mmseg ./configs/${CONFIG_PATH}
+```
+
+### Testing commands
+
+To train models on a single server with one GPU. (default）
+
+```shell
+mim test mmseg ./configs/${CONFIG_PATH} --checkpoint ${CHECKPOINT_PATH}
+```
+
+
+
+## Results
+
+### Ravir
+
+| Method | Backbone | Crop Size | lr | config |
+| :-------------: | :------: | :-------: | :----: | :------------------------------------------------------------------------: |
+| fcn_unet_s5-d16 | unet | 512x512 | 0.01 | [config](./configs/fcn-unet-s5-d16_unet_1xb16-0.01-20k_ravir-512x512.py) |
+| fcn_unet_s5-d16 | unet | 512x512 | 0.001 | [config](./configs/fcn-unet-s5-d16_unet_1xb16-0.001-20k_ravir-512x512.py) |
+| fcn_unet_s5-d16 | unet | 512x512 | 0.0001 | [config](./configs/fcn-unet-s5-d16_unet_1xb16-0.0001-20k_ravir-512x512.py) |
+
+## Checklist
+
+- [x] Milestone 1: PR-ready, and acceptable to be one of the `projects/`.
+
+ - [x] Finish the code
+
+ - [x] Basic docstrings & proper citation
+
+ - [x] Test-time correctness
+
+ - [x] A full README
+
+- [x] Milestone 2: Indicates a successful model implementation.
+
+ - [x] Training-time correctness
+
+- [ ] Milestone 3: Good to be a part of our core package!
+
+ - [ ] Type hints and docstrings
+
+ - [ ] Unit tests
+
+ - [ ] Code polishing
+
+ - [ ] Metafile.yml
+
+- [ ] Move your modules into the core package following the codebase's file hierarchy structure.
+
+- [ ] Refactor your modules into the core package following the codebase's file hierarchy structure.
diff --git a/projects/medical/2d_image/infrared_reflectance_imaging/ravir/configs/fcn-unet-s5-d16_unet_1xb16-0.0001-20k_ravir-512x512.py b/projects/medical/2d_image/infrared_reflectance_imaging/ravir/configs/fcn-unet-s5-d16_unet_1xb16-0.0001-20k_ravir-512x512.py
new file mode 100755
index 00000000000..375ad5abf27
--- /dev/null
+++ b/projects/medical/2d_image/infrared_reflectance_imaging/ravir/configs/fcn-unet-s5-d16_unet_1xb16-0.0001-20k_ravir-512x512.py
@@ -0,0 +1,19 @@
+_base_ = [
+ 'mmseg::_base_/models/fcn_unet_s5-d16.py', './ravir_512x512.py',
+ 'mmseg::_base_/default_runtime.py',
+ 'mmseg::_base_/schedules/schedule_20k.py'
+]
+custom_imports = dict(imports='datasets.ravir_dataset')
+img_scale = (512, 512)
+data_preprocessor = dict(size=img_scale)
+optimizer = dict(lr=0.0001)
+optim_wrapper = dict(optimizer=optimizer)
+model = dict(
+ type='EncoderDecoder',
+ data_preprocessor=data_preprocessor,
+ pretrained=None,
+ decode_head=dict(num_classes=3),
+ auxiliary_head=None,
+ test_cfg=dict(mode='whole', _delete_=True))
+vis_backends = None
+visualizer = dict(vis_backends=vis_backends)
diff --git a/projects/medical/2d_image/infrared_reflectance_imaging/ravir/configs/fcn-unet-s5-d16_unet_1xb16-0.001-20k_ravir-512x512.py b/projects/medical/2d_image/infrared_reflectance_imaging/ravir/configs/fcn-unet-s5-d16_unet_1xb16-0.001-20k_ravir-512x512.py
new file mode 100755
index 00000000000..a7ecf6dd453
--- /dev/null
+++ b/projects/medical/2d_image/infrared_reflectance_imaging/ravir/configs/fcn-unet-s5-d16_unet_1xb16-0.001-20k_ravir-512x512.py
@@ -0,0 +1,19 @@
+_base_ = [
+ 'mmseg::_base_/models/fcn_unet_s5-d16.py', './ravir_512x512.py',
+ 'mmseg::_base_/default_runtime.py',
+ 'mmseg::_base_/schedules/schedule_20k.py'
+]
+custom_imports = dict(imports='datasets.ravir_dataset')
+img_scale = (512, 512)
+data_preprocessor = dict(size=img_scale)
+optimizer = dict(lr=0.001)
+optim_wrapper = dict(optimizer=optimizer)
+model = dict(
+ type='EncoderDecoder',
+ data_preprocessor=dict(size=img_scale),
+ pretrained=None,
+ decode_head=dict(num_classes=3),
+ auxiliary_head=None,
+ test_cfg=dict(mode='whole', _delete_=True))
+vis_backends = None
+visualizer = dict(vis_backends=vis_backends)
diff --git a/projects/medical/2d_image/infrared_reflectance_imaging/ravir/configs/fcn-unet-s5-d16_unet_1xb16-0.01-20k_ravir-512x512.py b/projects/medical/2d_image/infrared_reflectance_imaging/ravir/configs/fcn-unet-s5-d16_unet_1xb16-0.01-20k_ravir-512x512.py
new file mode 100755
index 00000000000..28556df53d1
--- /dev/null
+++ b/projects/medical/2d_image/infrared_reflectance_imaging/ravir/configs/fcn-unet-s5-d16_unet_1xb16-0.01-20k_ravir-512x512.py
@@ -0,0 +1,19 @@
+_base_ = [
+ 'mmseg::_base_/models/fcn_unet_s5-d16.py', './ravir_512x512.py',
+ 'mmseg::_base_/default_runtime.py',
+ 'mmseg::_base_/schedules/schedule_20k.py'
+]
+custom_imports = dict(imports='datasets.ravir_dataset')
+img_scale = (512, 512)
+data_preprocessor = dict(size=img_scale)
+optimizer = dict(lr=0.01)
+optim_wrapper = dict(optimizer=optimizer)
+model = dict(
+ type='EncoderDecoder',
+ data_preprocessor=data_preprocessor,
+ pretrained=None,
+ decode_head=dict(num_classes=3),
+ auxiliary_head=None,
+ test_cfg=dict(mode='whole', _delete_=True))
+vis_backends = None
+visualizer = dict(vis_backends=vis_backends)
diff --git a/projects/medical/2d_image/infrared_reflectance_imaging/ravir/configs/ravir_512x512.py b/projects/medical/2d_image/infrared_reflectance_imaging/ravir/configs/ravir_512x512.py
new file mode 100755
index 00000000000..cb4c292d1f7
--- /dev/null
+++ b/projects/medical/2d_image/infrared_reflectance_imaging/ravir/configs/ravir_512x512.py
@@ -0,0 +1,42 @@
+dataset_type = 'RAVIRDataset'
+data_root = 'data/ravir'
+img_scale = (512, 512)
+train_pipeline = [
+ dict(type='LoadImageFromFile'),
+ dict(type='LoadAnnotations'),
+ dict(type='Resize', scale=img_scale, keep_ratio=False),
+ dict(type='RandomFlip', prob=0.5),
+ dict(type='PhotoMetricDistortion'),
+ dict(type='PackSegInputs')
+]
+test_pipeline = [
+ dict(type='LoadImageFromFile'),
+ dict(type='Resize', scale=img_scale, keep_ratio=False),
+ dict(type='LoadAnnotations'),
+ dict(type='PackSegInputs')
+]
+train_dataloader = dict(
+ batch_size=16,
+ num_workers=4,
+ persistent_workers=True,
+ sampler=dict(type='InfiniteSampler', shuffle=True),
+ dataset=dict(
+ type=dataset_type,
+ data_root=data_root,
+ ann_file='train.txt',
+ data_prefix=dict(img_path='images/', seg_map_path='masks/'),
+ pipeline=train_pipeline))
+val_dataloader = dict(
+ batch_size=1,
+ num_workers=4,
+ persistent_workers=True,
+ sampler=dict(type='DefaultSampler', shuffle=False),
+ dataset=dict(
+ type=dataset_type,
+ data_root=data_root,
+ ann_file='val.txt',
+ data_prefix=dict(img_path='images/', seg_map_path='masks/'),
+ pipeline=test_pipeline))
+test_dataloader = val_dataloader
+val_evaluator = dict(type='IoUMetric', iou_metrics=['mIoU', 'mDice'])
+test_evaluator = dict(type='IoUMetric', iou_metrics=['mIoU', 'mDice'])
diff --git a/projects/medical/2d_image/infrared_reflectance_imaging/ravir/datasets/__init__.py b/projects/medical/2d_image/infrared_reflectance_imaging/ravir/datasets/__init__.py
new file mode 100755
index 00000000000..6f1d051bcff
--- /dev/null
+++ b/projects/medical/2d_image/infrared_reflectance_imaging/ravir/datasets/__init__.py
@@ -0,0 +1,3 @@
+from .ravir_dataset import RAVIRDataset
+
+__all__ = ['RAVIRDataset']
diff --git a/projects/medical/2d_image/infrared_reflectance_imaging/ravir/datasets/ravir_dataset.py b/projects/medical/2d_image/infrared_reflectance_imaging/ravir/datasets/ravir_dataset.py
new file mode 100755
index 00000000000..c9e0a8ed21e
--- /dev/null
+++ b/projects/medical/2d_image/infrared_reflectance_imaging/ravir/datasets/ravir_dataset.py
@@ -0,0 +1,28 @@
+from mmseg.datasets import BaseSegDataset
+from mmseg.registry import DATASETS
+
+
+@DATASETS.register_module()
+class RAVIRDataset(BaseSegDataset):
+ """RAVIRDataset dataset.
+
+ In segmentation map annotation for RAVIRDataset, 0 stands for background,
+ which is included in 3 categories. ``reduce_zero_label`` is fixed to
+ False. The ``img_suffix`` is fixed to '.png' and ``seg_map_suffix`` is
+ fixed to '.png'.
+ Args:
+ img_suffix (str): Suffix of images. Default: '.png'
+ seg_map_suffix (str): Suffix of segmentation maps. Default: '.png'
+ """
+ METAINFO = dict(classes=('background', 'artery', 'vein'))
+
+ def __init__(self,
+ img_suffix='.png',
+ seg_map_suffix='.png',
+ reduce_zero_label=False,
+ **kwargs) -> None:
+ super().__init__(
+ img_suffix=img_suffix,
+ seg_map_suffix=seg_map_suffix,
+ reduce_zero_label=reduce_zero_label,
+ **kwargs)
diff --git a/projects/medical/2d_image/infrared_reflectance_imaging/ravir/tools/prepare_dataset.py b/projects/medical/2d_image/infrared_reflectance_imaging/ravir/tools/prepare_dataset.py
new file mode 100644
index 00000000000..068dcad8145
--- /dev/null
+++ b/projects/medical/2d_image/infrared_reflectance_imaging/ravir/tools/prepare_dataset.py
@@ -0,0 +1,33 @@
+import glob
+import os
+
+import numpy as np
+from PIL import Image
+from tqdm import tqdm
+
+# map = {255:2, 128:1, 0:0}
+
+os.makedirs('data/ravir/images/train', exist_ok=True)
+os.makedirs('data/ravir/images/test', exist_ok=True)
+os.makedirs('data/ravir/masks/train', exist_ok=True)
+
+os.system(
+ r'cp data/ravir/RAVIR\ Dataset/train/training_images/* data/ravir/images/train' # noqa
+)
+os.system(
+ r'cp data/ravir/RAVIR\ Dataset/train/training_masks/* data/ravir/masks/train' # noqa
+)
+os.system(r'cp data/ravir/RAVIR\ Dataset/test/* data/ravir/images/test')
+
+os.system(r'rm -rf data/ravir/RAVIR\ Dataset')
+
+imgs = glob.glob(os.path.join('data/ravir/masks/train', '*.png'))
+
+for im_path in tqdm(imgs):
+ im = Image.open(im_path)
+ imn = np.array(im)
+ imn[imn == 255] = 2
+ imn[imn == 128] = 1
+ imn[imn == 0] = 0
+ new_im = Image.fromarray(imn)
+ new_im.save(im_path)
diff --git a/projects/medical/2d_image/microscopy_images/2pm_vessel/README.md b/projects/medical/2d_image/microscopy_images/2pm_vessel/README.md
new file mode 100644
index 00000000000..1feb433a319
--- /dev/null
+++ b/projects/medical/2d_image/microscopy_images/2pm_vessel/README.md
@@ -0,0 +1,153 @@
+# 2-PM Vessel Dataset
+
+## Description
+
+This project supports **`2-PM Vessel Dataset`**, which can be downloaded from [here](https://opendatalab.org.cn/2-PM_Vessel_Dataset).
+
+### Dataset Overview
+
+An open-source volumetric brain vasculature dataset obtained with two-photon microscopy at Focused Ultrasound Lab, at Sunnybrook Research Institute (affiliated with University of Toronto by Dr. Alison Burgess, Charissa Poon and Marc Santos).
+
+The dataset contains a total of 12 volumetric stacks consisting images of mouse brain vasculature and tumor vasculature.
+
+### Information Statistics
+
+| Dataset Name | Anatomical Region | Task Type | Modality | Num. Classes | Train/Val/Test Images | Train/Val/Test Labeled | Release Date | License |
+| ------------------------------------------------------------ | ----------------- | ------------ | ----------------- | ------------ | --------------------- | ---------------------- | ------------ | ------------------------------------------------------------- |
+| [2pm_vessel](https://opendatalab.org.cn/2-PM_Vessel_Dataset) | vessel | segmentation | microscopy_images | 2 | 216/-/- | yes/-/- | 2021 | [CC0 1.0](https://creativecommons.org/publicdomain/zero/1.0/) |
+
+| Class Name | Num. Train | Pct. Train | Num. Val | Pct. Val | Num. Test | Pct. Test |
+| :--------: | :--------: | :--------: | :------: | :------: | :-------: | :-------: |
+| background | 216 | 85.78 | - | - | - | - |
+| vessel | 180 | 14.22 | - | - | - | - |
+
+Note:
+
+- `Pct` means percentage of pixels in this category in all pixels.
+
+### Visualization
+
+
+
+### Dataset Citation
+
+```
+@article{teikari2016deep,
+ title={Deep learning convolutional networks for multiphoton microscopy vasculature segmentation},
+ author={Teikari, Petteri and Santos, Marc and Poon, Charissa and Hynynen, Kullervo},
+ journal={arXiv preprint arXiv:1606.02382},
+ year={2016}
+}
+```
+
+### Prerequisites
+
+- Python v3.8
+- PyTorch v1.10.0
+- pillow(PIL) v9.3.0
+- scikit-learn(sklearn) v1.2.0
+- [MIM](https://github.com/open-mmlab/mim) v0.3.4
+- [MMCV](https://github.com/open-mmlab/mmcv) v2.0.0rc4
+- [MMEngine](https://github.com/open-mmlab/mmengine) v0.2.0 or higher
+- [MMSegmentation](https://github.com/open-mmlab/mmsegmentation) v1.0.0rc5
+
+All the commands below rely on the correct configuration of `PYTHONPATH`, which should point to the project's directory so that Python can locate the module files. In `2pm_vessel/` root directory, run the following line to add the current directory to `PYTHONPATH`:
+
+```shell
+export PYTHONPATH=`pwd`:$PYTHONPATH
+```
+
+### Dataset Preparing
+
+- download dataset from [here](https://opendatalab.org.cn/2-PM_Vessel_Dataset) and decompress data to path `'data/'`.
+- run script `"python tools/prepare_dataset.py"` to format data and change folder structure as below.
+- run script `"python ../../tools/split_seg_dataset.py"` to split dataset and generate `train.txt`, `val.txt` and `test.txt`. If the label of official validation set and test set can't be obtained, we generate `train.txt` and `val.txt` from the training set randomly.
+
+```shell
+mkdir data & cd data
+pip install opendatalab
+odl get 2-PM_Vessel_Dataset
+cd ..
+python tools/prepare_dataset.py
+python ../../tools/split_seg_dataset.py
+```
+
+```none
+ mmsegmentation
+ ├── mmseg
+ ├── projects
+ │ ├── medical
+ │ │ ├── 2d_image
+ │ │ │ ├── microscopy_images
+ │ │ │ │ ├── 2pm_vessel
+ │ │ │ │ │ ├── configs
+ │ │ │ │ │ ├── datasets
+ │ │ │ │ │ ├── tools
+ │ │ │ │ │ ├── data
+ │ │ │ │ │ │ ├── train.txt
+ │ │ │ │ │ │ ├── val.txt
+ │ │ │ │ │ │ ├── images
+ │ │ │ │ │ │ │ ├── train
+ │ │ │ │ | │ │ │ ├── xxx.png
+ │ │ │ │ | │ │ │ ├── ...
+ │ │ │ │ | │ │ │ └── xxx.png
+ │ │ │ │ │ │ ├── masks
+ │ │ │ │ │ │ │ ├── train
+ │ │ │ │ | │ │ │ ├── xxx.png
+ │ │ │ │ | │ │ │ ├── ...
+ │ │ │ │ | │ │ │ └── xxx.png
+
+```
+
+### Divided Dataset Information
+
+***Note: The table information below is divided by ourselves.***
+
+| Class Name | Num. Train | Pct. Train | Num. Val | Pct. Val | Num. Test | Pct. Test |
+| :--------: | :--------: | :--------: | :------: | :------: | :-------: | :-------: |
+| background | 172 | 85.88 | 44 | 85.4 | - | - |
+| vessel | 142 | 14.12 | 38 | 14.6 | - | - |
+
+### Training commands
+
+To train models on a single server with one GPU. (default)
+
+```shell
+mim train mmseg ./configs/${CONFIG_FILE}
+```
+
+### Testing commands
+
+To test models on a single server with one GPU. (default)
+
+```shell
+mim test mmseg ./configs/${CONFIG_FILE} --checkpoint ${CHECKPOINT_PATH}
+```
+
+
+
+## Checklist
+
+- [x] Milestone 1: PR-ready, and acceptable to be one of the `projects/`.
+
+ - [x] Finish the code
+ - [x] Basic docstrings & proper citation
+ - [ ] Test-time correctness
+ - [x] A full README
+
+- [ ] Milestone 2: Indicates a successful model implementation.
+
+ - [ ] Training-time correctness
+
+- [ ] Milestone 3: Good to be a part of our core package!
+
+ - [ ] Type hints and docstrings
+ - [ ] Unit tests
+ - [ ] Code polishing
+ - [ ] Metafile.yml
+
+- [ ] Move your modules into the core package following the codebase's file hierarchy structure.
+
+- [ ] Refactor your modules into the core package following the codebase's file hierarchy structure.
diff --git a/projects/medical/2d_image/microscopy_images/2pm_vessel/configs/2pm-vessel_512x512.py b/projects/medical/2d_image/microscopy_images/2pm_vessel/configs/2pm-vessel_512x512.py
new file mode 100644
index 00000000000..124403fa973
--- /dev/null
+++ b/projects/medical/2d_image/microscopy_images/2pm_vessel/configs/2pm-vessel_512x512.py
@@ -0,0 +1,42 @@
+dataset_type = 'TwoPMVesselDataset'
+data_root = 'data/'
+img_scale = (512, 512)
+train_pipeline = [
+ dict(type='LoadImageFromFile'),
+ dict(type='LoadAnnotations'),
+ dict(type='Resize', scale=img_scale, keep_ratio=False),
+ dict(type='RandomFlip', prob=0.5),
+ dict(type='PhotoMetricDistortion'),
+ dict(type='PackSegInputs')
+]
+test_pipeline = [
+ dict(type='LoadImageFromFile'),
+ dict(type='Resize', scale=img_scale, keep_ratio=False),
+ dict(type='LoadAnnotations'),
+ dict(type='PackSegInputs')
+]
+train_dataloader = dict(
+ batch_size=16,
+ num_workers=4,
+ persistent_workers=True,
+ sampler=dict(type='InfiniteSampler', shuffle=True),
+ dataset=dict(
+ type=dataset_type,
+ data_root=data_root,
+ ann_file='train.txt',
+ data_prefix=dict(img_path='images/', seg_map_path='masks/'),
+ pipeline=train_pipeline))
+val_dataloader = dict(
+ batch_size=1,
+ num_workers=4,
+ persistent_workers=True,
+ sampler=dict(type='DefaultSampler', shuffle=False),
+ dataset=dict(
+ type=dataset_type,
+ data_root=data_root,
+ ann_file='val.txt',
+ data_prefix=dict(img_path='images/', seg_map_path='masks/'),
+ pipeline=test_pipeline))
+test_dataloader = val_dataloader
+val_evaluator = dict(type='IoUMetric', iou_metrics=['mIoU', 'mDice'])
+test_evaluator = dict(type='IoUMetric', iou_metrics=['mIoU', 'mDice'])
diff --git a/projects/medical/2d_image/microscopy_images/2pm_vessel/configs/fcn-unet-s5-d16_unet_1xb16-0.0001-20k_2pm-vessel-512x512.py b/projects/medical/2d_image/microscopy_images/2pm_vessel/configs/fcn-unet-s5-d16_unet_1xb16-0.0001-20k_2pm-vessel-512x512.py
new file mode 100644
index 00000000000..2a429e9068e
--- /dev/null
+++ b/projects/medical/2d_image/microscopy_images/2pm_vessel/configs/fcn-unet-s5-d16_unet_1xb16-0.0001-20k_2pm-vessel-512x512.py
@@ -0,0 +1,17 @@
+_base_ = [
+ 'mmseg::_base_/models/fcn_unet_s5-d16.py', './2pm-vessel_512x512.py',
+ 'mmseg::_base_/default_runtime.py',
+ 'mmseg::_base_/schedules/schedule_20k.py'
+]
+custom_imports = dict(imports='datasets.2pm-vessel_dataset')
+img_scale = (512, 512)
+data_preprocessor = dict(size=img_scale)
+optimizer = dict(lr=0.0001)
+optim_wrapper = dict(optimizer=optimizer)
+model = dict(
+ data_preprocessor=data_preprocessor,
+ decode_head=dict(num_classes=2),
+ auxiliary_head=None,
+ test_cfg=dict(mode='whole', _delete_=True))
+vis_backends = None
+visualizer = dict(vis_backends=vis_backends)
diff --git a/projects/medical/2d_image/microscopy_images/2pm_vessel/configs/fcn-unet-s5-d16_unet_1xb16-0.001-20k_2pm-vessel-512x512.py b/projects/medical/2d_image/microscopy_images/2pm_vessel/configs/fcn-unet-s5-d16_unet_1xb16-0.001-20k_2pm-vessel-512x512.py
new file mode 100644
index 00000000000..10d9bb82f25
--- /dev/null
+++ b/projects/medical/2d_image/microscopy_images/2pm_vessel/configs/fcn-unet-s5-d16_unet_1xb16-0.001-20k_2pm-vessel-512x512.py
@@ -0,0 +1,17 @@
+_base_ = [
+ 'mmseg::_base_/models/fcn_unet_s5-d16.py', './2pm-vessel_512x512.py',
+ 'mmseg::_base_/default_runtime.py',
+ 'mmseg::_base_/schedules/schedule_20k.py'
+]
+custom_imports = dict(imports='datasets.2pm-vessel_dataset')
+img_scale = (512, 512)
+data_preprocessor = dict(size=img_scale)
+optimizer = dict(lr=0.001)
+optim_wrapper = dict(optimizer=optimizer)
+model = dict(
+ data_preprocessor=data_preprocessor,
+ decode_head=dict(num_classes=2),
+ auxiliary_head=None,
+ test_cfg=dict(mode='whole', _delete_=True))
+vis_backends = None
+visualizer = dict(vis_backends=vis_backends)
diff --git a/projects/medical/2d_image/microscopy_images/2pm_vessel/configs/fcn-unet-s5-d16_unet_1xb16-0.01-20k_2pm-vessel-512x512.py b/projects/medical/2d_image/microscopy_images/2pm_vessel/configs/fcn-unet-s5-d16_unet_1xb16-0.01-20k_2pm-vessel-512x512.py
new file mode 100644
index 00000000000..65c1579ec71
--- /dev/null
+++ b/projects/medical/2d_image/microscopy_images/2pm_vessel/configs/fcn-unet-s5-d16_unet_1xb16-0.01-20k_2pm-vessel-512x512.py
@@ -0,0 +1,17 @@
+_base_ = [
+ 'mmseg::_base_/models/fcn_unet_s5-d16.py', './2pm-vessel_512x512.py',
+ 'mmseg::_base_/default_runtime.py',
+ 'mmseg::_base_/schedules/schedule_20k.py'
+]
+custom_imports = dict(imports='datasets.2pm-vessel_dataset')
+img_scale = (512, 512)
+data_preprocessor = dict(size=img_scale)
+optimizer = dict(lr=0.01)
+optim_wrapper = dict(optimizer=optimizer)
+model = dict(
+ data_preprocessor=data_preprocessor,
+ decode_head=dict(num_classes=2),
+ auxiliary_head=None,
+ test_cfg=dict(mode='whole', _delete_=True))
+vis_backends = None
+visualizer = dict(vis_backends=vis_backends)
diff --git a/projects/medical/2d_image/microscopy_images/2pm_vessel/configs/fcn-unet-s5-d16_unet_1xb16-0.01lr-sigmoid-20k_bactteria-detection-512x512.py b/projects/medical/2d_image/microscopy_images/2pm_vessel/configs/fcn-unet-s5-d16_unet_1xb16-0.01lr-sigmoid-20k_bactteria-detection-512x512.py
new file mode 100644
index 00000000000..91ed6ada3f2
--- /dev/null
+++ b/projects/medical/2d_image/microscopy_images/2pm_vessel/configs/fcn-unet-s5-d16_unet_1xb16-0.01lr-sigmoid-20k_bactteria-detection-512x512.py
@@ -0,0 +1,18 @@
+_base_ = [
+ 'mmseg::_base_/models/fcn_unet_s5-d16.py', './2pm-vessel_512x512.py',
+ 'mmseg::_base_/default_runtime.py',
+ 'mmseg::_base_/schedules/schedule_20k.py'
+]
+custom_imports = dict(imports='datasets.2pm-vessel_dataset')
+img_scale = (512, 512)
+data_preprocessor = dict(size=img_scale)
+optimizer = dict(lr=0.01)
+optim_wrapper = dict(optimizer=optimizer)
+model = dict(
+ data_preprocessor=data_preprocessor,
+ decode_head=dict(
+ num_classes=2, loss_decode=dict(use_sigmoid=True), out_channels=1),
+ auxiliary_head=None,
+ test_cfg=dict(mode='whole', _delete_=True))
+vis_backends = None
+visualizer = dict(vis_backends=vis_backends)
diff --git a/projects/medical/2d_image/microscopy_images/2pm_vessel/datasets/2pm-vessel_dataset.py b/projects/medical/2d_image/microscopy_images/2pm_vessel/datasets/2pm-vessel_dataset.py
new file mode 100644
index 00000000000..984b5a1361d
--- /dev/null
+++ b/projects/medical/2d_image/microscopy_images/2pm_vessel/datasets/2pm-vessel_dataset.py
@@ -0,0 +1,31 @@
+from mmseg.datasets import BaseSegDataset
+from mmseg.registry import DATASETS
+
+
+@DATASETS.register_module()
+class TwoPMVesselDataset(BaseSegDataset):
+ """TwoPMVesselDataset dataset.
+
+ In segmentation map annotation for TwoPMVesselDataset,
+ 0 stands for background, which is included in 2 categories.
+ ``reduce_zero_label`` is fixed to False. The ``img_suffix``
+ is fixed to '.png' and ``seg_map_suffix`` is fixed to '.png'.
+
+ Args:
+ img_suffix (str): Suffix of images. Default: '.png'
+ seg_map_suffix (str): Suffix of segmentation maps. Default: '.png'
+ reduce_zero_label (bool): Whether to mark label zero as ignored.
+ Default to False.
+ """
+ METAINFO = dict(classes=('background', 'vessel'))
+
+ def __init__(self,
+ img_suffix='.png',
+ seg_map_suffix='.png',
+ reduce_zero_label=False,
+ **kwargs) -> None:
+ super().__init__(
+ img_suffix=img_suffix,
+ seg_map_suffix=seg_map_suffix,
+ reduce_zero_label=reduce_zero_label,
+ **kwargs)
diff --git a/projects/medical/2d_image/microscopy_images/2pm_vessel/tools/prepare_dataset.py b/projects/medical/2d_image/microscopy_images/2pm_vessel/tools/prepare_dataset.py
new file mode 100644
index 00000000000..1b46af2cad0
--- /dev/null
+++ b/projects/medical/2d_image/microscopy_images/2pm_vessel/tools/prepare_dataset.py
@@ -0,0 +1,46 @@
+import os
+
+import tifffile as tiff
+from PIL import Image
+
+root_path = 'data/'
+
+image_dir = os.path.join(root_path,
+ '2-PM_Vessel_Dataset/raw/vesselNN_dataset/denoised')
+label_dir = os.path.join(root_path,
+ '2-PM_Vessel_Dataset/raw/vesselNN_dataset/labels')
+tgt_img_train_dir = os.path.join(root_path, 'images/train/')
+tgt_mask_train_dir = os.path.join(root_path, 'masks/train/')
+os.system('mkdir -p ' + tgt_img_train_dir)
+os.system('mkdir -p ' + tgt_mask_train_dir)
+
+
+def filter_suffix(src_dir, suffix):
+ suffix = '.' + suffix if '.' not in suffix else suffix
+ file_names = [_ for _ in os.listdir(src_dir) if _.endswith(suffix)]
+ file_paths = [os.path.join(src_dir, _) for _ in file_names]
+ return sorted(file_paths), sorted(file_names)
+
+
+if __name__ == '__main__':
+
+ image_path_list, _ = filter_suffix(image_dir, suffix='tif')
+ label_path_list, _ = filter_suffix(label_dir, suffix='.tif')
+
+ for img_path, label_path in zip(image_path_list, label_path_list):
+ labels = tiff.imread(label_path)
+ images = tiff.imread(img_path)
+ assert labels.ndim == 3
+ assert images.shape == labels.shape
+ name = img_path.split('/')[-1].replace('.tif', '')
+ # a single .tif file contains multiple slices
+ # as long as it is read by tifffile package.
+ for i in range(labels.shape[0]):
+ slice_name = name + '_' + str(i).rjust(3, '0') + '.png'
+ image = images[i]
+ label = labels[i] // 255
+
+ save_path_label = os.path.join(tgt_mask_train_dir, slice_name)
+ Image.fromarray(label).save(save_path_label)
+ save_path_image = os.path.join(tgt_img_train_dir, slice_name)
+ Image.fromarray(image).convert('RGB').save(save_path_image)
diff --git a/projects/medical/2d_image/microscopy_images/bactteria_detection/README.md b/projects/medical/2d_image/microscopy_images/bactteria_detection/README.md
new file mode 100644
index 00000000000..1cedda715a7
--- /dev/null
+++ b/projects/medical/2d_image/microscopy_images/bactteria_detection/README.md
@@ -0,0 +1,160 @@
+# Bactteria detection with darkfield microscopy
+
+## Description
+
+This project supports **`Bactteria detection with darkfield microscopy`**, which can be downloaded from [here](https://tianchi.aliyun.com/dataset/94411).
+
+### Dataset Overview
+
+Spirochaeta is a genus of bacteria classified within the phylum Spirochaetes. Included in this dataset are 366 darkfield microscopy images and manually annotated masks which can be used for classification and segmentation purposes. Detecting bacteria in blood could have a huge significance for research in both the medical and computer science field.
+
+It was gathered and annotated by students (hand-on experience)
+It has more than just one targeted class (blood cell and bacteria were annotated)
+It is highly imbalanced, so naive loss functions would work less properly
+
+### Original Statistic Information
+
+| Dataset name | Anatomical region | Task type | Modality | Num. Classes | Train/Val/Test Images | Train/Val/Test Labeled | Release Date | License |
+| --------------------------------------------------------------- | ----------------- | ------------ | ---------- | ------------ | --------------------- | ---------------------- | ------------ | --------------------------------------------------------------- |
+| [Bactteria detection](https://tianchi.aliyun.com/dataset/94411) | bacteria | segmentation | microscopy | 3 | 366/-/- | yes/-/- | 2017 | [CC-BY-NC 4.0](https://creativecommons.org/licenses/by-sa/4.0/) |
+
+| Class Name | Num. Train | Pct. Train | Num. Val | Pct. Val | Num. Test | Pct. Test |
+| :----------: | :--------: | :--------: | :------: | :------: | :-------: | :-------: |
+| background | 366 | 85.9 | - | - | - | - |
+| erythrocytes | 345 | 13.03 | - | - | - | - |
+| spirochaete | 288 | 1.07 | - | - | - | - |
+
+Note:
+
+- `Pct` means percentage of pixels in this category in all pixels.
+
+### Visualization
+
+
+
+## Usage
+
+### Prerequisites
+
+- Python v3.8
+- PyTorch v1.10.0
+- pillow (PIL) v9.3.0
+- scikit-learn (sklearn) v1.2.0
+- [MIM](https://github.com/open-mmlab/mim) v0.3.4
+- [MMCV](https://github.com/open-mmlab/mmcv) v2.0.0rc4
+- [MMEngine](https://github.com/open-mmlab/mmengine) v0.2.0 or higher
+- [MMSegmentation](https://github.com/open-mmlab/mmsegmentation) v1.0.0rc5
+
+All the commands below rely on the correct configuration of `PYTHONPATH`, which should point to the project's directory so that Python can locate the module files. In `bactteria_detection/` root directory, run the following line to add the current directory to `PYTHONPATH`:
+
+```shell
+export PYTHONPATH=`pwd`:$PYTHONPATH
+```
+
+### Dataset Preparing
+
+- Download dataset from [here](https://tianchi.aliyun.com/dataset/94411) and save it to the `data/` directory .
+- Decompress data to path `data/`. This will create a new folder named `data/Bacteria_detection_with_darkfield_microscopy_datasets/`, which contains the original image data.
+- run script `python tools/prepare_dataset.py` to format data and change folder structure as below.
+- run script `python ../../tools/split_seg_dataset.py` to split dataset. For the Bacteria_detection dataset, as there is no test or validation dataset, we sample 20% samples from the whole dataset as the validation dataset and 80% samples for training data and make two filename lists `train.txt` and `val.txt`. As we set the random seed as the hard code, we eliminated the randomness, the dataset split actually can be reproducible.
+
+```none
+ mmsegmentation
+ ├── mmseg
+ ├── projects
+ │ ├── medical
+ │ │ ├── 2d_image
+ │ │ │ ├── microscopy_images
+ │ │ │ │ ├── bactteria_detection
+ │ │ │ │ │ ├── configs
+ │ │ │ │ │ ├── datasets
+ │ │ │ │ │ ├── tools
+ │ │ │ │ │ ├── data
+ │ │ │ │ │ │ ├── train.txt
+ │ │ │ │ │ │ ├── val.txt
+ │ │ │ │ │ │ ├── Bacteria_detection_with_darkfield_microscopy_datasets
+ │ │ │ │ │ │ ├── images
+ │ │ │ │ │ │ │ ├── train
+ │ │ │ │ | │ │ │ ├── xxx.png
+ │ │ │ │ | │ │ │ ├── ...
+ │ │ │ │ | │ │ │ └── xxx.png
+ │ │ │ │ │ │ ├── masks
+ │ │ │ │ │ │ │ ├── train
+ │ │ │ │ | │ │ │ ├── xxx.png
+ │ │ │ │ | │ │ │ ├── ...
+ │ │ │ │ | │ │ │ └── xxx.png
+```
+
+### Divided Dataset Information
+
+***Note: The table information below is divided by ourselves.***
+
+| Class Name | Num. Train | Pct. Train | Num. Val | Pct. Val | Num. Test | Pct. Test |
+| :----------: | :--------: | :--------: | :------: | :------: | :-------: | :-------: |
+| background | 292 | 85.66 | 74 | 86.7 | - | - |
+| erythrocytes | 274 | 13.25 | 71 | 12.29 | - | - |
+| spirochaete | 231 | 1.09 | 57 | 1.01 | - | - |
+
+### Training commands
+
+Train models on a single server with one GPU.
+
+```shell
+export PYTHONPATH=`pwd`:$PYTHONPATH
+mim train mmseg ./configs/${CONFIG_FILE}
+```
+
+### Testing commands
+
+Test models on a single server with one GPU.
+
+```shell
+export PYTHONPATH=`pwd`:$PYTHONPATH
+mim test mmseg ./configs/${CONFIG_FILE} --checkpoint ${CHECKPOINT_PATH}
+```
+
+
+
+## Results
+
+### Bactteria detection with darkfield microscopy
+
+***Note: The following experimental results are based on the data randomly partitioned according to the above method described in the dataset preparing section.***
+
+| Method | Backbone | Crop Size | lr | mIoU | mDice | config | download |
+| :-------------: | :------: | :-------: | :----: | :---: | :---: | :--------------------------------------------------------------------------------------: | :----------------------: |
+| fcn_unet_s5-d16 | unet | 512x512 | 0.01 | 76.48 | 84.68 | [config](./configs/fcn-unet-s5-d16_unet_1xb16-0.01-20k_bactteria-detection-512x512.py) | [model](<>) \| [log](<>) |
+| fcn_unet_s5-d16 | unet | 512x512 | 0.001 | 61.06 | 63.69 | [config](./configs/fcn-unet-s5-d16_unet_1xb16-0.001-20k_bactteria-detection-512x512.py) | [model](<>) \| [log](<>) |
+| fcn_unet_s5-d16 | unet | 512x512 | 0.0001 | 58.87 | 62.42 | [config](./configs/fcn-unet-s5-d16_unet_1xb16-0.0001-20k_bactteria-detection-512x512.py) | [model](<>) \| [log](<>) |
+
+## Checklist
+
+- [x] Milestone 1: PR-ready, and acceptable to be one of the `projects/`.
+
+ - [x] Finish the code
+
+ - [x] Basic docstrings & proper citation
+
+ - [x] Test-time correctness
+
+ - [x] A full README
+
+- [x] Milestone 2: Indicates a successful model implementation.
+
+ - [x] Training-time correctness
+
+- [ ] Milestone 3: Good to be a part of our core package!
+
+ - [ ] Type hints and docstrings
+
+ - [ ] Unit tests
+
+ - [ ] Code polishing
+
+ - [ ] Metafile.yml
+
+- [ ] Move your modules into the core package following the codebase's file hierarchy structure.
+
+- [ ] Refactor your modules into the core package following the codebase's file hierarchy structure.
diff --git a/projects/medical/2d_image/microscopy_images/bactteria_detection/configs/bactteria-detection_512x512.py b/projects/medical/2d_image/microscopy_images/bactteria_detection/configs/bactteria-detection_512x512.py
new file mode 100644
index 00000000000..e3eab4e3868
--- /dev/null
+++ b/projects/medical/2d_image/microscopy_images/bactteria_detection/configs/bactteria-detection_512x512.py
@@ -0,0 +1,42 @@
+dataset_type = 'BactteriaDetectionDataset'
+data_root = 'data/'
+img_scale = (512, 512)
+train_pipeline = [
+ dict(type='LoadImageFromFile'),
+ dict(type='LoadAnnotations'),
+ dict(type='Resize', scale=img_scale, keep_ratio=False),
+ dict(type='RandomFlip', prob=0.5),
+ dict(type='PhotoMetricDistortion'),
+ dict(type='PackSegInputs')
+]
+test_pipeline = [
+ dict(type='LoadImageFromFile'),
+ dict(type='Resize', scale=img_scale, keep_ratio=False),
+ dict(type='LoadAnnotations'),
+ dict(type='PackSegInputs')
+]
+train_dataloader = dict(
+ batch_size=16,
+ num_workers=4,
+ persistent_workers=True,
+ sampler=dict(type='InfiniteSampler', shuffle=True),
+ dataset=dict(
+ type=dataset_type,
+ data_root=data_root,
+ ann_file='train.txt',
+ data_prefix=dict(img_path='images/', seg_map_path='masks/'),
+ pipeline=train_pipeline))
+val_dataloader = dict(
+ batch_size=1,
+ num_workers=4,
+ persistent_workers=True,
+ sampler=dict(type='DefaultSampler', shuffle=False),
+ dataset=dict(
+ type=dataset_type,
+ data_root=data_root,
+ ann_file='val.txt',
+ data_prefix=dict(img_path='images/', seg_map_path='masks/'),
+ pipeline=test_pipeline))
+test_dataloader = val_dataloader
+val_evaluator = dict(type='IoUMetric', iou_metrics=['mIoU', 'mDice'])
+test_evaluator = dict(type='IoUMetric', iou_metrics=['mIoU', 'mDice'])
diff --git a/projects/medical/2d_image/microscopy_images/bactteria_detection/configs/fcn-unet-s5-d16_unet_1xb16-0.0001-20k_bactteria-detection-512x512.py b/projects/medical/2d_image/microscopy_images/bactteria_detection/configs/fcn-unet-s5-d16_unet_1xb16-0.0001-20k_bactteria-detection-512x512.py
new file mode 100644
index 00000000000..ede58d785c1
--- /dev/null
+++ b/projects/medical/2d_image/microscopy_images/bactteria_detection/configs/fcn-unet-s5-d16_unet_1xb16-0.0001-20k_bactteria-detection-512x512.py
@@ -0,0 +1,18 @@
+_base_ = [
+ './bactteria-detection_512x512.py',
+ 'mmseg::_base_/models/fcn_unet_s5-d16.py',
+ 'mmseg::_base_/default_runtime.py',
+ 'mmseg::_base_/schedules/schedule_20k.py'
+]
+custom_imports = dict(imports='datasets.bactteria-detection_dataset')
+img_scale = (512, 512)
+data_preprocessor = dict(size=img_scale)
+optimizer = dict(lr=0.0001)
+optim_wrapper = dict(optimizer=optimizer)
+model = dict(
+ data_preprocessor=data_preprocessor,
+ decode_head=dict(num_classes=3),
+ auxiliary_head=None,
+ test_cfg=dict(mode='whole', _delete_=True))
+vis_backends = None
+visualizer = dict(vis_backends=vis_backends)
diff --git a/projects/medical/2d_image/microscopy_images/bactteria_detection/configs/fcn-unet-s5-d16_unet_1xb16-0.001-20k_bactteria-detection-512x512.py b/projects/medical/2d_image/microscopy_images/bactteria_detection/configs/fcn-unet-s5-d16_unet_1xb16-0.001-20k_bactteria-detection-512x512.py
new file mode 100644
index 00000000000..bde3fa14acd
--- /dev/null
+++ b/projects/medical/2d_image/microscopy_images/bactteria_detection/configs/fcn-unet-s5-d16_unet_1xb16-0.001-20k_bactteria-detection-512x512.py
@@ -0,0 +1,18 @@
+_base_ = [
+ './bactteria-detection_512x512.py',
+ 'mmseg::_base_/models/fcn_unet_s5-d16.py',
+ 'mmseg::_base_/default_runtime.py',
+ 'mmseg::_base_/schedules/schedule_20k.py'
+]
+custom_imports = dict(imports='datasets.bactteria-detection_dataset')
+img_scale = (512, 512)
+data_preprocessor = dict(size=img_scale)
+optimizer = dict(lr=0.001)
+optim_wrapper = dict(optimizer=optimizer)
+model = dict(
+ data_preprocessor=data_preprocessor,
+ decode_head=dict(num_classes=3),
+ auxiliary_head=None,
+ test_cfg=dict(mode='whole', _delete_=True))
+vis_backends = None
+visualizer = dict(vis_backends=vis_backends)
diff --git a/projects/medical/2d_image/microscopy_images/bactteria_detection/configs/fcn-unet-s5-d16_unet_1xb16-0.01-20k_bactteria-detection-512x512.py b/projects/medical/2d_image/microscopy_images/bactteria_detection/configs/fcn-unet-s5-d16_unet_1xb16-0.01-20k_bactteria-detection-512x512.py
new file mode 100644
index 00000000000..08e204f3809
--- /dev/null
+++ b/projects/medical/2d_image/microscopy_images/bactteria_detection/configs/fcn-unet-s5-d16_unet_1xb16-0.01-20k_bactteria-detection-512x512.py
@@ -0,0 +1,18 @@
+_base_ = [
+ './bactteria-detection_512x512.py',
+ 'mmseg::_base_/models/fcn_unet_s5-d16.py',
+ 'mmseg::_base_/default_runtime.py',
+ 'mmseg::_base_/schedules/schedule_20k.py'
+]
+custom_imports = dict(imports='datasets.bactteria-detection_dataset')
+img_scale = (512, 512)
+data_preprocessor = dict(size=img_scale)
+optimizer = dict(lr=0.01)
+optim_wrapper = dict(optimizer=optimizer)
+model = dict(
+ data_preprocessor=data_preprocessor,
+ decode_head=dict(num_classes=3),
+ auxiliary_head=None,
+ test_cfg=dict(mode='whole', _delete_=True))
+vis_backends = None
+visualizer = dict(vis_backends=vis_backends)
diff --git a/projects/medical/2d_image/microscopy_images/bactteria_detection/datasets/bactteria-detection_dataset.py b/projects/medical/2d_image/microscopy_images/bactteria_detection/datasets/bactteria-detection_dataset.py
new file mode 100644
index 00000000000..c95097b1acd
--- /dev/null
+++ b/projects/medical/2d_image/microscopy_images/bactteria_detection/datasets/bactteria-detection_dataset.py
@@ -0,0 +1,27 @@
+from mmseg.datasets import BaseSegDataset
+from mmseg.registry import DATASETS
+
+
+@DATASETS.register_module()
+class BactteriaDetectionDataset(BaseSegDataset):
+ """BactteriaDetectionDataset dataset.
+
+ In segmentation map annotation for BactteriaDetectionDataset,
+ ``reduce_zero_label`` is fixed to False. The ``img_suffix``
+ is fixed to '.png' and ``seg_map_suffix`` is fixed to '.png'.
+
+ Args:
+ img_suffix (str): Suffix of images. Default: '.png'
+ seg_map_suffix (str): Suffix of segmentation maps. Default: '.png'
+ """
+ METAINFO = dict(classes=('background', 'erythrocytes', 'spirochaete'))
+
+ def __init__(self,
+ img_suffix='.png',
+ seg_map_suffix='.png',
+ **kwargs) -> None:
+ super().__init__(
+ img_suffix=img_suffix,
+ seg_map_suffix=seg_map_suffix,
+ reduce_zero_label=False,
+ **kwargs)
diff --git a/projects/medical/2d_image/microscopy_images/bactteria_detection/tools/prepare_dataset.py b/projects/medical/2d_image/microscopy_images/bactteria_detection/tools/prepare_dataset.py
new file mode 100755
index 00000000000..8dcc719e262
--- /dev/null
+++ b/projects/medical/2d_image/microscopy_images/bactteria_detection/tools/prepare_dataset.py
@@ -0,0 +1,33 @@
+import glob
+import os
+import shutil
+
+from PIL import Image
+
+root_path = 'data/'
+img_suffix = '.png'
+seg_map_suffix = '.png'
+save_img_suffix = '.png'
+save_seg_map_suffix = '.png'
+
+x_train = glob.glob(
+ 'data/Bacteria_detection_with_darkfield_microscopy_datasets/images/*' +
+ img_suffix) # noqa
+
+os.system('mkdir -p ' + root_path + 'images/train/')
+os.system('mkdir -p ' + root_path + 'masks/train/')
+
+part_dir_dict = {0: 'train/'}
+for ith, part in enumerate([x_train]):
+ part_dir = part_dir_dict[ith]
+ for img in part:
+ basename = os.path.basename(img)
+ img_save_path = os.path.join(root_path, 'images', part_dir,
+ basename.split('.')[0] + save_img_suffix)
+ shutil.copy(img, img_save_path)
+ mask_path = 'data/Bacteria_detection_with_darkfield_microscopy_datasets/masks/' + basename # noqa
+ mask = Image.open(mask_path).convert('L')
+ mask_save_path = os.path.join(
+ root_path, 'masks', part_dir,
+ basename.split('.')[0] + save_seg_map_suffix)
+ mask.save(mask_save_path)
diff --git a/projects/medical/2d_image/tools/split_seg_dataset.py b/projects/medical/2d_image/tools/split_seg_dataset.py
new file mode 100644
index 00000000000..9ab2e9282fa
--- /dev/null
+++ b/projects/medical/2d_image/tools/split_seg_dataset.py
@@ -0,0 +1,42 @@
+import argparse
+import glob
+import os
+
+from sklearn.model_selection import train_test_split
+
+
+def save_anno(img_list, file_path, remove_suffix=True):
+ if remove_suffix:
+ img_list = [
+ '/'.join(img_path.split('/')[-2:]) for img_path in img_list
+ ]
+ img_list = [
+ '.'.join(img_path.split('.')[:-1]) for img_path in img_list
+ ]
+ with open(file_path, 'w') as file_:
+ for x in list(img_list):
+ file_.write(x + '\n')
+
+
+if __name__ == '__main__':
+ parser = argparse.ArgumentParser()
+ parser.add_argument('--data_root', default='data/')
+ args = parser.parse_args()
+ data_root = args.data_root
+ if os.path.exists(os.path.join(data_root, 'masks/val')):
+ x_val = sorted(glob.glob(data_root + '/images/val/*.png'))
+ save_anno(x_val, data_root + '/val.txt')
+ if os.path.exists(os.path.join(data_root, 'masks/test')):
+ x_test = sorted(glob.glob(data_root + '/images/test/*.png'))
+ save_anno(x_test, data_root + '/test.txt')
+ if not os.path.exists(os.path.join(
+ data_root, 'masks/val')) and not os.path.exists(
+ os.path.join(data_root, 'masks/test')):
+ all_imgs = sorted(glob.glob(data_root + '/images/train/*.png'))
+ x_train, x_val = train_test_split(
+ all_imgs, test_size=0.2, random_state=0)
+ save_anno(x_train, data_root + '/train.txt')
+ save_anno(x_val, data_root + '/val.txt')
+ else:
+ x_train = sorted(glob.glob(data_root + '/images/train/*.png'))
+ save_anno(x_train, data_root + '/train.txt')
diff --git a/projects/medical/2d_image/x_ray/chest_image_pneum/README.md b/projects/medical/2d_image/x_ray/chest_image_pneum/README.md
new file mode 100644
index 00000000000..a1cd27ba455
--- /dev/null
+++ b/projects/medical/2d_image/x_ray/chest_image_pneum/README.md
@@ -0,0 +1,147 @@
+# Chest Image Dataset for Pneumothorax Segmentation
+
+## Description
+
+This project supports **`Chest Image Dataset for Pneumothorax Segmentation`**, which can be downloaded from [here](https://tianchi.aliyun.com/dataset/83075).
+
+### Dataset Overview
+
+Pneumothorax can be caused by a blunt chest injury, damage from underlying lung disease, or most horrifying—it may occur for no obvious reason at all. On some occasions, a collapsed lung can be a life-threatening event.
+Pneumothorax is usually diagnosed by a radiologist on a chest x-ray, and can sometimes be very difficult to confirm. An accurate AI algorithm to detect pneumothorax would be useful in a lot of clinical scenarios. AI could be used to triage chest radiographs for priority interpretation, or to provide a more confident diagnosis for non-radiologists.
+
+The dataset is provided by the Society for Imaging Informatics in Medicine(SIIM), American College of Radiology (ACR),Society of Thoracic Radiology (STR) and MD.ai. You can develop a model to classify (and if present, segment) pneumothorax from a set of chest radiographic images. If successful, you could aid in the early recognition of pneumothoraces and save lives.
+
+### Original Statistic Information
+
+| Dataset name | Anatomical region | Task type | Modality | Num. Classes | Train/Val/Test Images | Train/Val/Test Labeled | Release Date | License |
+| --------------------------------------------------------------------- | ----------------- | ------------ | -------- | ------------ | --------------------- | ---------------------- | ------------ | ------------------------------------------------------------------ |
+| [pneumothorax segmentation](https://tianchi.aliyun.com/dataset/83075) | thorax | segmentation | x_ray | 2 | 12089/-/3205 | yes/-/no | - | [CC-BY-SA-NC 4.0](https://creativecommons.org/licenses/by-sa/4.0/) |
+
+| Class Name | Num. Train | Pct. Train | Num. Val | Pct. Val | Num. Test | Pct. Test |
+| :---------------: | :--------: | :--------: | :------: | :------: | :-------: | :-------: |
+| normal | 12089 | 99.75 | - | - | - | - |
+| pneumothorax area | 2669 | 0.25 | - | - | - | - |
+
+Note:
+
+- `Pct` means percentage of pixels in this category in all pixels.
+
+### Visualization
+
+
+
+### Prerequisites
+
+- Python v3.8
+- PyTorch v1.10.0
+- [MIM](https://github.com/open-mmlab/mim) v0.3.4
+- [MMCV](https://github.com/open-mmlab/mmcv) v2.0.0rc4
+- [MMEngine](https://github.com/open-mmlab/mmengine) v0.2.0 or higher
+- [MMSegmentation](https://github.com/open-mmlab/mmsegmentation) v1.0.0rc5
+
+All the commands below rely on the correct configuration of `PYTHONPATH`, which should point to the project's directory so that Python can locate the module files. In `chest_image_pneum/` root directory, run the following line to add the current directory to `PYTHONPATH`:
+
+```shell
+export PYTHONPATH=`pwd`:$PYTHONPATH
+```
+
+### Dataset preparing
+
+- download dataset from [here](https://tianchi.aliyun.com/dataset/83075) and decompress data to path `'data/'`.
+- run script `"python tools/prepare_dataset.py"` to format data and change folder structure as below.
+- run script `"python ../../tools/split_seg_dataset.py"` to split dataset and generate `train.txt`, `val.txt` and `test.txt`. If the label of official validation set and test set can't be obtained, we generate `train.txt` and `val.txt` from the training set randomly.
+
+```none
+ mmsegmentation
+ ├── mmseg
+ ├── projects
+ │ ├── medical
+ │ │ ├── 2d_image
+ │ │ │ ├── x_ray
+ │ │ │ │ ├── chest_image_pneum
+ │ │ │ │ │ ├── configs
+ │ │ │ │ │ ├── datasets
+ │ │ │ │ │ ├── tools
+ │ │ │ │ │ ├── data
+ │ │ │ │ │ │ ├── train.txt
+ │ │ │ │ │ │ ├── test.txt
+ │ │ │ │ │ │ ├── images
+ │ │ │ │ │ │ │ ├── train
+ │ │ │ │ | │ │ │ ├── xxx.png
+ │ │ │ │ | │ │ │ ├── ...
+ │ │ │ │ | │ │ │ └── xxx.png
+ │ │ │ │ │ │ ├── masks
+ │ │ │ │ │ │ │ ├── train
+ │ │ │ │ | │ │ │ ├── xxx.png
+ │ │ │ │ | │ │ │ ├── ...
+ │ │ │ │ | │ │ │ └── xxx.png
+```
+
+### Divided Dataset Information
+
+***Note: The table information below is divided by ourselves.***
+
+| Class Name | Num. Train | Pct. Train | Num. Val | Pct. Val | Num. Test | Pct. Test |
+| :---------------: | :--------: | :--------: | :------: | :------: | :-------: | :-------: |
+| normal | 9637 | 99.75 | 2410 | 99.74 | - | - |
+| pneumothorax area | 2137 | 0.25 | 532 | 0.26 | - | - |
+
+### Training commands
+
+Train models on a single server with one GPU.
+
+```shell
+mim train mmseg ./configs/${CONFIG_FILE}
+```
+
+### Testing commands
+
+Test models on a single server with one GPU.
+
+```shell
+mim test mmseg ./configs/${CONFIG_FILE} --checkpoint ${CHECKPOINT_PATH}
+```
+
+
+
+## Results
+
+### Bactteria detection with darkfield microscopy
+
+| Method | Backbone | Crop Size | lr | mIoU | mDice | config | download |
+| :-------------: | :------: | :-------: | :----: | :--: | :---: | :------------------------------------------------------------------------------------: | :----------------------: |
+| fcn_unet_s5-d16 | unet | 512x512 | 0.01 | - | - | [config](./configs/fcn-unet-s5-d16_unet_1xb16-0.01-20k_chest-image-pneum-512x512.py) | [model](<>) \| [log](<>) |
+| fcn_unet_s5-d16 | unet | 512x512 | 0.001 | - | - | [config](./configs/fcn-unet-s5-d16_unet_1xb16-0.001-20k_chest-image-pneum-512x512.py) | [model](<>) \| [log](<>) |
+| fcn_unet_s5-d16 | unet | 512x512 | 0.0001 | - | - | [config](./configs/fcn-unet-s5-d16_unet_1xb16-0.0001-20k_chest-image-pneum-512x512.py) | [model](<>) \| [log](<>) |
+
+## Checklist
+
+- [x] Milestone 1: PR-ready, and acceptable to be one of the `projects/`.
+
+ - [x] Finish the code
+
+ - [x] Basic docstrings & proper citation
+
+ - [x] Test-time correctness
+
+ - [x] A full README
+
+- [x] Milestone 2: Indicates a successful model implementation.
+
+ - [x] Training-time correctness
+
+- [ ] Milestone 3: Good to be a part of our core package!
+
+ - [ ] Type hints and docstrings
+
+ - [ ] Unit tests
+
+ - [ ] Code polishing
+
+ - [ ] Metafile.yml
+
+- [ ] Move your modules into the core package following the codebase's file hierarchy structure.
+
+- [ ] Refactor your modules into the core package following the codebase's file hierarchy structure.
diff --git a/projects/medical/2d_image/x_ray/chest_image_pneum/configs/chest-image-pneum_512x512.py b/projects/medical/2d_image/x_ray/chest_image_pneum/configs/chest-image-pneum_512x512.py
new file mode 100644
index 00000000000..411229bd413
--- /dev/null
+++ b/projects/medical/2d_image/x_ray/chest_image_pneum/configs/chest-image-pneum_512x512.py
@@ -0,0 +1,42 @@
+dataset_type = 'ChestImagePneumDataset'
+data_root = 'data/'
+img_scale = (512, 512)
+train_pipeline = [
+ dict(type='LoadImageFromFile'),
+ dict(type='LoadAnnotations'),
+ dict(type='Resize', scale=img_scale, keep_ratio=False),
+ dict(type='RandomFlip', prob=0.5),
+ dict(type='PhotoMetricDistortion'),
+ dict(type='PackSegInputs')
+]
+test_pipeline = [
+ dict(type='LoadImageFromFile'),
+ dict(type='Resize', scale=img_scale, keep_ratio=False),
+ dict(type='LoadAnnotations'),
+ dict(type='PackSegInputs')
+]
+train_dataloader = dict(
+ batch_size=16,
+ num_workers=4,
+ persistent_workers=True,
+ sampler=dict(type='InfiniteSampler', shuffle=True),
+ dataset=dict(
+ type=dataset_type,
+ data_root=data_root,
+ ann_file='train.txt',
+ data_prefix=dict(img_path='images/', seg_map_path='masks/'),
+ pipeline=train_pipeline))
+val_dataloader = dict(
+ batch_size=1,
+ num_workers=4,
+ persistent_workers=True,
+ sampler=dict(type='DefaultSampler', shuffle=False),
+ dataset=dict(
+ type=dataset_type,
+ data_root=data_root,
+ ann_file='val.txt',
+ data_prefix=dict(img_path='images/', seg_map_path='masks/'),
+ pipeline=test_pipeline))
+test_dataloader = val_dataloader
+val_evaluator = dict(type='IoUMetric', iou_metrics=['mIoU', 'mDice'])
+test_evaluator = dict(type='IoUMetric', iou_metrics=['mIoU', 'mDice'])
diff --git a/projects/medical/2d_image/x_ray/chest_image_pneum/configs/fcn-unet-s5-d16_unet_1xb16-0.0001-20k_chest-image-pneum-512x512.py b/projects/medical/2d_image/x_ray/chest_image_pneum/configs/fcn-unet-s5-d16_unet_1xb16-0.0001-20k_chest-image-pneum-512x512.py
new file mode 100644
index 00000000000..0f26459467c
--- /dev/null
+++ b/projects/medical/2d_image/x_ray/chest_image_pneum/configs/fcn-unet-s5-d16_unet_1xb16-0.0001-20k_chest-image-pneum-512x512.py
@@ -0,0 +1,18 @@
+_base_ = [
+ './chest-image-pneum_512x512.py',
+ 'mmseg::_base_/models/fcn_unet_s5-d16.py',
+ 'mmseg::_base_/default_runtime.py',
+ 'mmseg::_base_/schedules/schedule_20k.py'
+]
+custom_imports = dict(imports='datasets.chest-image-pneum_dataset')
+img_scale = (512, 512)
+data_preprocessor = dict(size=img_scale)
+optimizer = dict(lr=0.0001)
+optim_wrapper = dict(optimizer=optimizer)
+model = dict(
+ data_preprocessor=data_preprocessor,
+ decode_head=dict(num_classes=2),
+ auxiliary_head=None,
+ test_cfg=dict(mode='whole', _delete_=True))
+vis_backends = None
+visualizer = dict(vis_backends=vis_backends)
diff --git a/projects/medical/2d_image/x_ray/chest_image_pneum/configs/fcn-unet-s5-d16_unet_1xb16-0.001-20k_chest-image-pneum-512x512.py b/projects/medical/2d_image/x_ray/chest_image_pneum/configs/fcn-unet-s5-d16_unet_1xb16-0.001-20k_chest-image-pneum-512x512.py
new file mode 100644
index 00000000000..37b91889d8a
--- /dev/null
+++ b/projects/medical/2d_image/x_ray/chest_image_pneum/configs/fcn-unet-s5-d16_unet_1xb16-0.001-20k_chest-image-pneum-512x512.py
@@ -0,0 +1,18 @@
+_base_ = [
+ './chest-image-pneum_512x512.py',
+ 'mmseg::_base_/models/fcn_unet_s5-d16.py',
+ 'mmseg::_base_/default_runtime.py',
+ 'mmseg::_base_/schedules/schedule_20k.py'
+]
+custom_imports = dict(imports='datasets.chest-image-pneum_dataset')
+img_scale = (512, 512)
+data_preprocessor = dict(size=img_scale)
+optimizer = dict(lr=0.001)
+optim_wrapper = dict(optimizer=optimizer)
+model = dict(
+ data_preprocessor=data_preprocessor,
+ decode_head=dict(num_classes=2),
+ auxiliary_head=None,
+ test_cfg=dict(mode='whole', _delete_=True))
+vis_backends = None
+visualizer = dict(vis_backends=vis_backends)
diff --git a/projects/medical/2d_image/x_ray/chest_image_pneum/configs/fcn-unet-s5-d16_unet_1xb16-0.01-20k_chest-image-pneum-512x512.py b/projects/medical/2d_image/x_ray/chest_image_pneum/configs/fcn-unet-s5-d16_unet_1xb16-0.01-20k_chest-image-pneum-512x512.py
new file mode 100644
index 00000000000..379e8181f3d
--- /dev/null
+++ b/projects/medical/2d_image/x_ray/chest_image_pneum/configs/fcn-unet-s5-d16_unet_1xb16-0.01-20k_chest-image-pneum-512x512.py
@@ -0,0 +1,18 @@
+_base_ = [
+ './chest-image-pneum_512x512.py',
+ 'mmseg::_base_/models/fcn_unet_s5-d16.py',
+ 'mmseg::_base_/default_runtime.py',
+ 'mmseg::_base_/schedules/schedule_20k.py'
+]
+custom_imports = dict(imports='datasets.chest-image-pneum_dataset')
+img_scale = (512, 512)
+data_preprocessor = dict(size=img_scale)
+optimizer = dict(lr=0.01)
+optim_wrapper = dict(optimizer=optimizer)
+model = dict(
+ data_preprocessor=data_preprocessor,
+ decode_head=dict(num_classes=2),
+ auxiliary_head=None,
+ test_cfg=dict(mode='whole', _delete_=True))
+vis_backends = None
+visualizer = dict(vis_backends=vis_backends)
diff --git a/projects/medical/2d_image/x_ray/chest_image_pneum/datasets/chest-image-pneum_dataset.py b/projects/medical/2d_image/x_ray/chest_image_pneum/datasets/chest-image-pneum_dataset.py
new file mode 100644
index 00000000000..aeee60ae92e
--- /dev/null
+++ b/projects/medical/2d_image/x_ray/chest_image_pneum/datasets/chest-image-pneum_dataset.py
@@ -0,0 +1,27 @@
+from mmseg.datasets import BaseSegDataset
+from mmseg.registry import DATASETS
+
+
+@DATASETS.register_module()
+class ChestImagePneumDataset(BaseSegDataset):
+ """ChestImagePneumDataset dataset.
+
+ In segmentation map annotation for ChestImagePneumDataset,
+ ``reduce_zero_label`` is fixed to False. The ``img_suffix``
+ is fixed to '.png' and ``seg_map_suffix`` is fixed to '.png'.
+
+ Args:
+ img_suffix (str): Suffix of images. Default: '.png'
+ seg_map_suffix (str): Suffix of segmentation maps. Default: '.png'
+ """
+ METAINFO = dict(classes=('normal', 'pneumothorax area'))
+
+ def __init__(self,
+ img_suffix='.png',
+ seg_map_suffix='.png',
+ **kwargs) -> None:
+ super().__init__(
+ img_suffix=img_suffix,
+ seg_map_suffix=seg_map_suffix,
+ reduce_zero_label=False,
+ **kwargs)
diff --git a/projects/medical/2d_image/x_ray/chest_image_pneum/tools/prepare_dataset.py b/projects/medical/2d_image/x_ray/chest_image_pneum/tools/prepare_dataset.py
new file mode 100755
index 00000000000..47eddc96dc5
--- /dev/null
+++ b/projects/medical/2d_image/x_ray/chest_image_pneum/tools/prepare_dataset.py
@@ -0,0 +1,73 @@
+import os
+
+import numpy as np
+import pandas as pd
+import pydicom
+from PIL import Image
+
+root_path = 'data/'
+img_suffix = '.dcm'
+seg_map_suffix = '.png'
+save_img_suffix = '.png'
+save_seg_map_suffix = '.png'
+
+x_train = []
+for fpath, dirname, fnames in os.walk('data/chestimage_train_datasets'):
+ for fname in fnames:
+ if fname.endswith('.dcm'):
+ x_train.append(os.path.join(fpath, fname))
+x_test = []
+for fpath, dirname, fnames in os.walk('data/chestimage_test_datasets/'):
+ for fname in fnames:
+ if fname.endswith('.dcm'):
+ x_test.append(os.path.join(fpath, fname))
+
+os.system('mkdir -p ' + root_path + 'images/train/')
+os.system('mkdir -p ' + root_path + 'images/test/')
+os.system('mkdir -p ' + root_path + 'masks/train/')
+
+
+def rle_decode(rle, width, height):
+ mask = np.zeros(width * height, dtype=np.uint8)
+ array = np.asarray([int(x) for x in rle.split()])
+ starts = array[0::2]
+ lengths = array[1::2]
+
+ current_position = 0
+ for index, start in enumerate(starts):
+ current_position += start
+ mask[current_position:current_position + lengths[index]] = 1
+ current_position += lengths[index]
+
+ return mask.reshape(width, height, order='F')
+
+
+part_dir_dict = {0: 'train/', 1: 'test/'}
+dict_from_csv = pd.read_csv(
+ root_path + 'chestimage_train-rle_datasets.csv', sep=',',
+ index_col=0).to_dict()[' EncodedPixels']
+
+for ith, part in enumerate([x_train, x_test]):
+ part_dir = part_dir_dict[ith]
+ for img in part:
+ basename = os.path.basename(img)
+ img_id = '.'.join(basename.split('.')[:-1])
+ if ith == 0 and (img_id not in dict_from_csv.keys()):
+ continue
+ image = pydicom.read_file(img).pixel_array
+ save_img_path = root_path + 'images/' + part_dir + '.'.join(
+ basename.split('.')[:-1]) + save_img_suffix
+ print(save_img_path)
+ img_h, img_w = image.shape[:2]
+ image = Image.fromarray(image)
+ image.save(save_img_path)
+ if ith == 1:
+ continue
+ if dict_from_csv[img_id] == '-1':
+ mask = np.zeros((img_h, img_w), dtype=np.uint8)
+ else:
+ mask = rle_decode(dict_from_csv[img_id], img_h, img_w)
+ save_mask_path = root_path + 'masks/' + part_dir + '.'.join(
+ basename.split('.')[:-1]) + save_seg_map_suffix
+ mask = Image.fromarray(mask)
+ mask.save(save_mask_path)
diff --git a/projects/medical/2d_image/x_ray/chest_x_ray_images_with_pneumothorax_masks/README.md b/projects/medical/2d_image/x_ray/chest_x_ray_images_with_pneumothorax_masks/README.md
new file mode 100644
index 00000000000..7cb099c8a43
--- /dev/null
+++ b/projects/medical/2d_image/x_ray/chest_x_ray_images_with_pneumothorax_masks/README.md
@@ -0,0 +1,119 @@
+# Chest X-ray Images with Pneumothorax Masks
+
+## Description
+
+This project support **`Chest X-ray Images with Pneumothorax Masks `**, and the dataset used in this project can be downloaded from [here](https://www.kaggle.com/datasets/vbookshelf/pneumothorax-chest-xray-images-and-masks).
+
+### Dataset Overview
+
+A pneumothorax (noo-moe-THOR-aks) is a collapsed lung. A pneumothorax occurs when air leaks into the space between your lung and chest wall. This air pushes on the outside of your lung and makes it collapse. Pneumothorax can be a complete lung collapse or a collapse of only a portion of the lung.
+
+A pneumothorax can be caused by a blunt or penetrating chest injury, certain medical procedures, or damage from underlying lung disease. Or it may occur for no obvious reason. Symptoms usually include sudden chest pain and shortness of breath. On some occasions, a collapsed lung can be a life-threatening event.
+
+Treatment for a pneumothorax usually involves inserting a needle or chest tube between the ribs to remove the excess air. However, a small pneumothorax may heal on its own.
+
+### Statistic Information
+
+| Dataset Name | Anatomical Region | Task type | Modality | Num. Classes | Train/Val/Test Images | Train/Val/Test Labeled | Release date | License |
+| --------------------------------------------------------------------------------------------------------------------------------- | ----------------- | ------------ | -------- | ------------ | --------------------- | ---------------------- | ------------ | --------------------------------------------------------------- |
+| [Chest-x-ray-images-with-pneumothorax-masks](https://www.kaggle.com/datasets/vbookshelf/pneumothorax-chest-xray-images-and-masks) | throax | segmentation | x_ray | 2 | 10675/-/1372 | yes/-/yes | 2020 | [CC-BY-NC 4.0](https://creativecommons.org/licenses/by-sa/4.0/) |
+
+| Class Name | Num. Train | Pct. Train | Num. Val | Pct. Val | Num. Test | Pct. Test |
+| :----------: | :--------: | :--------: | :------: | :------: | :-------: | :-------: |
+| background | 10675 | 99.7 | - | - | 1372 | 99.71 |
+| pneumothroax | 2379 | 0.3 | - | - | 290 | 0.29 |
+
+### Visualization
+
+
+
+### Prerequisites
+
+- Python 3.8
+- PyTorch 1.10.0
+- pillow(PIL) 9.3.0
+- scikit-learn(sklearn) 1.2.0
+- [MIM](https://github.com/open-mmlab/mim) v0.3.4
+- [MMCV](https://github.com/open-mmlab/mmcv) v2.0.0rc4
+- [MMEngine](https://github.com/open-mmlab/mmengine) v0.2.0 or higher
+- [MMSegmentation](https://github.com/open-mmlab/mmsegmentation) v1.0.0rc5
+
+All the commands below rely on the correct configuration of PYTHONPATH, which should point to the project's directory so that Python can locate the module files. In chest_x_ray_images_with_pneumothorax_masks/ root directory, run the following line to add the current directory to PYTHONPATH:
+
+```shell
+export PYTHONPATH=`pwd`:$PYTHONPATH
+```
+
+### Dataset preparing
+
+- download dataset from [here](https://www.kaggle.com/datasets/vbookshelf/pneumothorax-chest-xray-images-and-masks) and decompression data to path 'data/'.
+- run script `"python tools/prepare_dataset.py"` to format data and change folder structure as below.
+- run script `"python ../../tools/split_seg_dataset.py"` to split dataset and generate `train.txt`, `val.txt` and `test.txt`. If the label of official validation set and test set cannot be obtained, we generate `train.txt` and `val.txt` from the training set randomly.
+
+```none
+ mmsegmentation
+ ├── mmseg
+ ├── projects
+ │ ├── medical
+ │ │ ├── 2d_image
+ │ │ │ ├── x_ray
+ │ │ │ │ ├── chest_x_ray_images_with_pneumothorax_masks
+ │ │ │ │ │ ├── configs
+ │ │ │ │ │ ├── datasets
+ │ │ │ │ │ ├── tools
+ │ │ │ │ │ ├── data
+ │ │ │ │ │ │ ├── train.txt
+ │ │ │ │ │ │ ├── val.txt
+ │ │ │ │ │ │ ├── images
+ │ │ │ │ │ │ │ ├── train
+ │ │ │ │ | │ │ │ ├── xxx.png
+ │ │ │ │ | │ │ │ ├── ...
+ │ │ │ │ | │ │ │ └── xxx.png
+ │ │ │ │ │ │ ├── masks
+ │ │ │ │ │ │ │ ├── train
+ │ │ │ │ | │ │ │ ├── xxx.png
+ │ │ │ │ | │ │ │ ├── ...
+ │ │ │ │ | │ │ │ └── xxx.png
+```
+
+### Training commands
+
+```shell
+mim train mmseg ./configs/${CONFIG_PATH}
+```
+
+To train on multiple GPUs, e.g. 8 GPUs, run the following command:
+
+```shell
+mim train mmseg ./configs/${CONFIG_PATH} --launcher pytorch --gpus 8
+```
+
+### Testing commands
+
+```shell
+mim test mmseg ./configs/${CONFIG_PATH} --checkpoint ${CHECKPOINT_PATH}
+```
+
+## Checklist
+
+- [x] Milestone 1: PR-ready, and acceptable to be one of the `projects/`.
+
+ - [x] Finish the code
+ - [x] Basic docstrings & proper citation
+ - [x] Test-time correctness
+ - [x] A full README
+
+- [x] Milestone 2: Indicates a successful model implementation.
+
+ - [x] Training-time correctness
+
+- [ ] Milestone 3: Good to be a part of our core package!
+
+ - [ ] Type hints and docstrings
+ - [ ] Unit tests
+ - [ ] Code polishing
+ - [ ] Metafile.yml
+
+- [ ] Move your modules into the core package following the codebase's file hierarchy structure.
+
+- [ ] Refactor your modules into the core package following the codebase's file hierarchy structure.
diff --git a/projects/medical/2d_image/x_ray/chest_x_ray_images_with_pneumothorax_masks/configs/chest-x-ray-images-with-pneumothorax-masks_512x512.py b/projects/medical/2d_image/x_ray/chest_x_ray_images_with_pneumothorax_masks/configs/chest-x-ray-images-with-pneumothorax-masks_512x512.py
new file mode 100644
index 00000000000..96676de8616
--- /dev/null
+++ b/projects/medical/2d_image/x_ray/chest_x_ray_images_with_pneumothorax_masks/configs/chest-x-ray-images-with-pneumothorax-masks_512x512.py
@@ -0,0 +1,42 @@
+dataset_type = 'ChestPenumoMaskDataset'
+data_root = 'data/'
+img_scale = (512, 512)
+train_pipeline = [
+ dict(type='LoadImageFromFile'),
+ dict(type='LoadAnnotations'),
+ dict(type='Resize', scale=img_scale, keep_ratio=False),
+ dict(type='RandomFlip', prob=0.5),
+ dict(type='PhotoMetricDistortion'),
+ dict(type='PackSegInputs')
+]
+test_pipeline = [
+ dict(type='LoadImageFromFile'),
+ dict(type='Resize', scale=img_scale, keep_ratio=False),
+ dict(type='LoadAnnotations'),
+ dict(type='PackSegInputs')
+]
+train_dataloader = dict(
+ batch_size=16,
+ num_workers=4,
+ persistent_workers=True,
+ sampler=dict(type='InfiniteSampler', shuffle=True),
+ dataset=dict(
+ type=dataset_type,
+ data_root=data_root,
+ ann_file='train.txt',
+ data_prefix=dict(img_path='images/', seg_map_path='masks/'),
+ pipeline=train_pipeline))
+val_dataloader = dict(
+ batch_size=1,
+ num_workers=4,
+ persistent_workers=True,
+ sampler=dict(type='DefaultSampler', shuffle=False),
+ dataset=dict(
+ type=dataset_type,
+ data_root=data_root,
+ ann_file='val.txt',
+ data_prefix=dict(img_path='images/', seg_map_path='masks/'),
+ pipeline=test_pipeline))
+test_dataloader = val_dataloader
+val_evaluator = dict(type='IoUMetric', iou_metrics=['mIoU', 'mDice'])
+test_evaluator = dict(type='IoUMetric', iou_metrics=['mIoU', 'mDice'])
diff --git a/projects/medical/2d_image/x_ray/chest_x_ray_images_with_pneumothorax_masks/configs/fcn-unet-s5-d16_unet-{use-sigmoid}_1xb16-0.01-20k_chest-x-ray-images-with-pneumothorax-masks-512x512.py b/projects/medical/2d_image/x_ray/chest_x_ray_images_with_pneumothorax_masks/configs/fcn-unet-s5-d16_unet-{use-sigmoid}_1xb16-0.01-20k_chest-x-ray-images-with-pneumothorax-masks-512x512.py
new file mode 100644
index 00000000000..76c214d04cb
--- /dev/null
+++ b/projects/medical/2d_image/x_ray/chest_x_ray_images_with_pneumothorax_masks/configs/fcn-unet-s5-d16_unet-{use-sigmoid}_1xb16-0.01-20k_chest-x-ray-images-with-pneumothorax-masks-512x512.py
@@ -0,0 +1,20 @@
+_base_ = [
+ 'mmseg::_base_/models/fcn_unet_s5-d16.py',
+ './chest-x-ray-images-with-pneumothorax-masks_512x512.py',
+ 'mmseg::_base_/default_runtime.py',
+ 'mmseg::_base_/schedules/schedule_20k.py'
+]
+custom_imports = dict(
+ imports='datasets.chest-x-ray-images-with-pneumothorax-masks_dataset')
+img_scale = (512, 512)
+data_preprocessor = dict(size=img_scale)
+optimizer = dict(lr=0.01)
+optim_wrapper = dict(optimizer=optimizer)
+model = dict(
+ data_preprocessor=data_preprocessor,
+ decode_head=dict(
+ num_classes=2, loss_decode=dict(use_sigmoid=True), out_channels=1),
+ auxiliary_head=None,
+ test_cfg=dict(mode='whole', _delete_=True))
+vis_backends = None
+visualizer = dict(vis_backends=vis_backends)
diff --git a/projects/medical/2d_image/x_ray/chest_x_ray_images_with_pneumothorax_masks/configs/fcn-unet-s5-d16_unet_1xb16-0.0001-20k_chest-x-ray-images-with-pneumothorax-masks-512x512.py b/projects/medical/2d_image/x_ray/chest_x_ray_images_with_pneumothorax_masks/configs/fcn-unet-s5-d16_unet_1xb16-0.0001-20k_chest-x-ray-images-with-pneumothorax-masks-512x512.py
new file mode 100644
index 00000000000..066996dda94
--- /dev/null
+++ b/projects/medical/2d_image/x_ray/chest_x_ray_images_with_pneumothorax_masks/configs/fcn-unet-s5-d16_unet_1xb16-0.0001-20k_chest-x-ray-images-with-pneumothorax-masks-512x512.py
@@ -0,0 +1,19 @@
+_base_ = [
+ 'mmseg::_base_/models/fcn_unet_s5-d16.py',
+ './chest-x-ray-images-with-pneumothorax-masks_512x512.py',
+ 'mmseg::_base_/default_runtime.py',
+ 'mmseg::_base_/schedules/schedule_20k.py'
+]
+custom_imports = dict(
+ imports='datasets.chest-x-ray-images-with-pneumothorax-masks_dataset')
+img_scale = (512, 512)
+data_preprocessor = dict(size=img_scale)
+optimizer = dict(lr=0.0001)
+optim_wrapper = dict(optimizer=optimizer)
+model = dict(
+ data_preprocessor=data_preprocessor,
+ decode_head=dict(num_classes=2),
+ auxiliary_head=None,
+ test_cfg=dict(mode='whole', _delete_=True))
+vis_backends = None
+visualizer = dict(vis_backends=vis_backends)
diff --git a/projects/medical/2d_image/x_ray/chest_x_ray_images_with_pneumothorax_masks/configs/fcn-unet-s5-d16_unet_1xb16-0.001-20k_chest-x-ray-images-with-pneumothorax-masks-512x512.py b/projects/medical/2d_image/x_ray/chest_x_ray_images_with_pneumothorax_masks/configs/fcn-unet-s5-d16_unet_1xb16-0.001-20k_chest-x-ray-images-with-pneumothorax-masks-512x512.py
new file mode 100644
index 00000000000..a7065b82315
--- /dev/null
+++ b/projects/medical/2d_image/x_ray/chest_x_ray_images_with_pneumothorax_masks/configs/fcn-unet-s5-d16_unet_1xb16-0.001-20k_chest-x-ray-images-with-pneumothorax-masks-512x512.py
@@ -0,0 +1,19 @@
+_base_ = [
+ 'mmseg::_base_/models/fcn_unet_s5-d16.py',
+ './chest-x-ray-images-with-pneumothorax-masks_512x512.py',
+ 'mmseg::_base_/default_runtime.py',
+ 'mmseg::_base_/schedules/schedule_20k.py'
+]
+custom_imports = dict(
+ imports='datasets.chest-x-ray-images-with-pneumothorax-masks_dataset')
+img_scale = (512, 512)
+data_preprocessor = dict(size=img_scale)
+optimizer = dict(lr=0.001)
+optim_wrapper = dict(optimizer=optimizer)
+model = dict(
+ data_preprocessor=data_preprocessor,
+ decode_head=dict(num_classes=2),
+ auxiliary_head=None,
+ test_cfg=dict(mode='whole', _delete_=True))
+vis_backends = None
+visualizer = dict(vis_backends=vis_backends)
diff --git a/projects/medical/2d_image/x_ray/chest_x_ray_images_with_pneumothorax_masks/configs/fcn-unet-s5-d16_unet_1xb16-0.01-20k_chest-x-ray-images-with-pneumothorax-masks-512x512.py b/projects/medical/2d_image/x_ray/chest_x_ray_images_with_pneumothorax_masks/configs/fcn-unet-s5-d16_unet_1xb16-0.01-20k_chest-x-ray-images-with-pneumothorax-masks-512x512.py
new file mode 100644
index 00000000000..e5682ee76b7
--- /dev/null
+++ b/projects/medical/2d_image/x_ray/chest_x_ray_images_with_pneumothorax_masks/configs/fcn-unet-s5-d16_unet_1xb16-0.01-20k_chest-x-ray-images-with-pneumothorax-masks-512x512.py
@@ -0,0 +1,19 @@
+_base_ = [
+ 'mmseg::_base_/models/fcn_unet_s5-d16.py',
+ './chest-x-ray-images-with-pneumothorax-masks_512x512.py',
+ 'mmseg::_base_/default_runtime.py',
+ 'mmseg::_base_/schedules/schedule_20k.py'
+]
+custom_imports = dict(
+ imports='datasets.chest-x-ray-images-with-pneumothorax-masks_dataset')
+img_scale = (512, 512)
+data_preprocessor = dict(size=img_scale)
+optimizer = dict(lr=0.01)
+optim_wrapper = dict(optimizer=optimizer)
+model = dict(
+ data_preprocessor=data_preprocessor,
+ decode_head=dict(num_classes=2),
+ auxiliary_head=None,
+ test_cfg=dict(mode='whole', _delete_=True))
+vis_backends = None
+visualizer = dict(vis_backends=vis_backends)
diff --git a/projects/medical/2d_image/x_ray/chest_x_ray_images_with_pneumothorax_masks/datasets/chest-x-ray-images-with-pneumothorax-masks_dataset.py b/projects/medical/2d_image/x_ray/chest_x_ray_images_with_pneumothorax_masks/datasets/chest-x-ray-images-with-pneumothorax-masks_dataset.py
new file mode 100644
index 00000000000..d32f597a5a0
--- /dev/null
+++ b/projects/medical/2d_image/x_ray/chest_x_ray_images_with_pneumothorax_masks/datasets/chest-x-ray-images-with-pneumothorax-masks_dataset.py
@@ -0,0 +1,31 @@
+from mmseg.datasets import BaseSegDataset
+from mmseg.registry import DATASETS
+
+
+@DATASETS.register_module()
+class ChestPenumoMaskDataset(BaseSegDataset):
+ """ChestPenumoMaskDataset dataset.
+
+ In segmentation map annotation for ChestPenumoMaskDataset,
+ 0 stands for background, which is included in 2 categories.
+ ``reduce_zero_label`` is fixed to False. The ``img_suffix``
+ is fixed to '.png' and ``seg_map_suffix`` is fixed to '.png'.
+
+ Args:
+ img_suffix (str): Suffix of images. Default: '.png'
+ seg_map_suffix (str): Suffix of segmentation maps. Default: '.png'
+ reduce_zero_label (bool): Whether to mark label zero as ignored.
+ Default to False.
+ """
+ METAINFO = dict(classes=('background', 'penumothroax'))
+
+ def __init__(self,
+ img_suffix='.png',
+ seg_map_suffix='.png',
+ reduce_zero_label=False,
+ **kwargs) -> None:
+ super().__init__(
+ img_suffix=img_suffix,
+ seg_map_suffix=seg_map_suffix,
+ reduce_zero_label=reduce_zero_label,
+ **kwargs)
diff --git a/projects/medical/2d_image/x_ray/chest_x_ray_images_with_pneumothorax_masks/tools/prepare_dataset.py b/projects/medical/2d_image/x_ray/chest_x_ray_images_with_pneumothorax_masks/tools/prepare_dataset.py
new file mode 100644
index 00000000000..c7de1f19042
--- /dev/null
+++ b/projects/medical/2d_image/x_ray/chest_x_ray_images_with_pneumothorax_masks/tools/prepare_dataset.py
@@ -0,0 +1,36 @@
+import glob
+import os
+import shutil
+
+from PIL import Image
+from sklearn.model_selection import train_test_split
+
+root_path = 'data/'
+img_suffix = '.png'
+seg_map_suffix = '.png'
+save_img_suffix = '.png'
+save_seg_map_suffix = '.png'
+
+all_imgs = glob.glob('data/siim-acr-pneumothorax/png_images/*' + img_suffix)
+x_train, x_test = train_test_split(all_imgs, test_size=0.2, random_state=0)
+
+print(len(x_train), len(x_test))
+os.system('mkdir -p ' + root_path + 'images/train/')
+os.system('mkdir -p ' + root_path + 'images/val/')
+os.system('mkdir -p ' + root_path + 'masks/train/')
+os.system('mkdir -p ' + root_path + 'masks/val/')
+
+part_dir_dict = {0: 'train/', 1: 'val/'}
+for ith, part in enumerate([x_train, x_test]):
+ part_dir = part_dir_dict[ith]
+ for img in part:
+ basename = os.path.basename(img)
+ img_save_path = os.path.join(root_path, 'images', part_dir,
+ basename.split('.')[0] + save_img_suffix)
+ shutil.copy(img, img_save_path)
+ mask_path = 'data/siim-acr-pneumothorax/png_masks/' + basename
+ mask = Image.open(mask_path).convert('L')
+ mask_save_path = os.path.join(
+ root_path, 'masks', part_dir,
+ basename.split('.')[0] + save_seg_map_suffix)
+ mask.save(mask_save_path)
diff --git a/projects/medical/2d_image/x_ray/covid_19_ct_cxr/README.md b/projects/medical/2d_image/x_ray/covid_19_ct_cxr/README.md
new file mode 100644
index 00000000000..8469219effa
--- /dev/null
+++ b/projects/medical/2d_image/x_ray/covid_19_ct_cxr/README.md
@@ -0,0 +1,158 @@
+# Covid-19 CT Chest X-ray Dataset
+
+## Description
+
+This project supports **`Covid-19 CT Chest X-ray Dataset`**, which can be downloaded from [here](https://github.com/ieee8023/covid-chestxray-dataset).
+
+### Dataset Overview
+
+In the context of a COVID-19 pandemic, we want to improve prognostic predictions to triage and manage patient care. Data is the first step to developing any diagnostic/prognostic tool. While there exist large public datasets of more typical chest X-rays from the NIH \[Wang 2017\], Spain \[Bustos 2019\], Stanford \[Irvin 2019\], MIT \[Johnson 2019\] and Indiana University \[Demner-Fushman 2016\], there is no collection of COVID-19 chest X-rays or CT scans designed to be used for computational analysis.
+
+The 2019 novel coronavirus (COVID-19) presents several unique features [Fang, 2020](https://pubs.rsna.org/doi/10.1148/radiol.2020200432) and [Ai 2020](https://pubs.rsna.org/doi/10.1148/radiol.2020200642). While the diagnosis is confirmed using polymerase chain reaction (PCR), infected patients with pneumonia may present on chest X-ray and computed tomography (CT) images with a pattern that is only moderately characteristic for the human eye [Ng, 2020](https://pubs.rsna.org/doi/10.1148/ryct.2020200034). In late January, a Chinese team published a paper detailing the clinical and paraclinical features of COVID-19. They reported that patients present abnormalities in chest CT images with most having bilateral involvement [Huang 2020](). Bilateral multiple lobular and subsegmental areas of consolidation constitute the typical findings in chest CT images of intensive care unit (ICU) patients on admission [Huang 2020](). In comparison, non-ICU patients show bilateral ground-glass opacity and subsegmental areas of consolidation in their chest CT images [Huang 2020](). In these patients, later chest CT images display bilateral ground-glass opacity with resolved consolidation [Huang 2020]().
+
+### Statistic Information
+
+| Dataset Name | Anatomical Region | Task Type | Modality | Nnum. Classes | Train/Val/Test Images | Train/Val/Test Labeled | Release date | License |
+| ---------------------------------------------------------------------- | ----------------- | ------------ | -------- | ------------- | --------------------- | ---------------------- | ------------ | --------------------------------------------------------------------- |
+| [Covid-19-ct-cxr](https://github.com/ieee8023/covid-chestxray-dataset) | thorax | segmentation | x_ray | 2 | 205/-/714 | yes/-/no | 2021 | [CC-BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/) |
+
+| Class Name | Num. Train | Pct. Train | Num. Val | Pct. Val | Num. Test | Pct. Test |
+| :--------: | :--------: | :--------: | :------: | :------: | :-------: | :-------: |
+| background | 205 | 72.84 | - | - | - | - |
+| lung | 205 | 27.16 | - | - | - | - |
+
+Note:
+
+- `Pct` means percentage of pixels in this category in all pixels.
+
+### Visualization
+
+
+
+### Dataset Citation
+
+```
+@article{cohen2020covidProspective,
+ title={{COVID-19} Image Data Collection: Prospective Predictions Are the Future},
+ author={Joseph Paul Cohen and Paul Morrison and Lan Dao and Karsten Roth and Tim Q Duong and Marzyeh Ghassemi},
+ journal={arXiv 2006.11988},
+ year={2020}
+}
+
+@article{cohen2020covid,
+ title={COVID-19 image data collection},
+ author={Joseph Paul Cohen and Paul Morrison and Lan Dao},
+ journal={arXiv 2003.11597},
+ year={2020}
+}
+```
+
+### Prerequisites
+
+- Python v3.8
+- PyTorch v1.10.0
+- pillow(PIL) v9.3.0 9.3.0
+- scikit-learn(sklearn) v1.2.0 1.2.0
+- [MIM](https://github.com/open-mmlab/mim) v0.3.4
+- [MMCV](https://github.com/open-mmlab/mmcv) v2.0.0rc4
+- [MMEngine](https://github.com/open-mmlab/mmengine) v0.2.0 or higher
+- [MMSegmentation](https://github.com/open-mmlab/mmsegmentation) v1.0.0rc5
+
+All the commands below rely on the correct configuration of `PYTHONPATH`, which should point to the project's directory so that Python can locate the module files. In `covid_19_ct_cxr/` root directory, run the following line to add the current directory to `PYTHONPATH`:
+
+```shell
+export PYTHONPATH=`pwd`:$PYTHONPATH
+```
+
+### Dataset Preparing
+
+- download dataset from [here](https://github.com/ieee8023/covid-chestxray-dataset) and decompress data to path `'data/'`.
+- run script `"python tools/prepare_dataset.py"` to format data and change folder structure as below.
+- run script `"python ../../tools/split_seg_dataset.py"` to split dataset and generate `train.txt`, `val.txt` and `test.txt`. If the label of official validation set and test set cannot be obtained, we generate `train.txt` and `val.txt` from the training set randomly.
+
+```shell
+mkdir data && cd data
+git clone git@github.com:ieee8023/covid-chestxray-dataset.git
+cd ..
+python tools/prepare_dataset.py
+python ../../tools/split_seg_dataset.py
+```
+
+```none
+ mmsegmentation
+ ├── mmseg
+ ├── projects
+ │ ├── medical
+ │ │ ├── 2d_image
+ │ │ │ ├── x_ray
+ │ │ │ │ ├── covid_19_ct_cxr
+ │ │ │ │ │ ├── configs
+ │ │ │ │ │ ├── datasets
+ │ │ │ │ │ ├── tools
+ │ │ │ │ │ ├── data
+ │ │ │ │ │ │ ├── train.txt
+ │ │ │ │ │ │ ├── val.txt
+ │ │ │ │ │ │ ├── images
+ │ │ │ │ │ │ │ ├── train
+ │ │ │ │ | │ │ │ ├── xxx.png
+ │ │ │ │ | │ │ │ ├── ...
+ │ │ │ │ | │ │ │ └── xxx.png
+ │ │ │ │ │ │ ├── masks
+ │ │ │ │ │ │ │ ├── train
+ │ │ │ │ | │ │ │ ├── xxx.png
+ │ │ │ │ | │ │ │ ├── ...
+ │ │ │ │ | │ │ │ └── xxx.png
+```
+
+### Divided Dataset Information
+
+***Note: The table information below is divided by ourselves.***
+
+| Class Name | Num. Train | Pct. Train | Num. Val | Pct. Val | Num. Test | Pct. Test |
+| :--------: | :--------: | :--------: | :------: | :------: | :-------: | :-------: |
+| background | 164 | 72.88 | 41 | 72.69 | - | - |
+| lung | 164 | 27.12 | 41 | 27.31 | - | - |
+
+### Training commands
+
+To train models on a single server with one GPU. (default)
+
+```shell
+mim train mmseg ./configs/${CONFIG_FILE}
+```
+
+### Testing commands
+
+To test models on a single server with one GPU. (default)
+
+```shell
+mim test mmseg ./configs/${CONFIG_FILE} --checkpoint ${CHECKPOINT_PATH}
+```
+
+
+
+## Checklist
+
+- [x] Milestone 1: PR-ready, and acceptable to be one of the `projects/`.
+
+ - [x] Finish the code
+ - [x] Basic docstrings & proper citation
+ - [x] Test-time correctness
+ - [x] A full README
+
+- [x] Milestone 2: Indicates a successful model implementation.
+
+ - [x] Training-time correctness
+
+- [ ] Milestone 3: Good to be a part of our core package!
+
+ - [ ] Type hints and docstrings
+ - [ ] Unit tests
+ - [ ] Code polishing
+ - [ ] Metafile.yml
+
+- [ ] Move your modules into the core package following the codebase's file hierarchy structure.
+
+- [ ] Refactor your modules into the core package following the codebase's file hierarchy structure.
diff --git a/projects/medical/2d_image/x_ray/covid_19_ct_cxr/configs/covid-19-ct-cxr_512x512.py b/projects/medical/2d_image/x_ray/covid_19_ct_cxr/configs/covid-19-ct-cxr_512x512.py
new file mode 100644
index 00000000000..5242d06c374
--- /dev/null
+++ b/projects/medical/2d_image/x_ray/covid_19_ct_cxr/configs/covid-19-ct-cxr_512x512.py
@@ -0,0 +1,42 @@
+dataset_type = 'Covid19CXRDataset'
+data_root = 'data/'
+img_scale = (512, 512)
+train_pipeline = [
+ dict(type='LoadImageFromFile'),
+ dict(type='LoadAnnotations'),
+ dict(type='Resize', scale=img_scale, keep_ratio=False),
+ dict(type='RandomFlip', prob=0.5),
+ dict(type='PhotoMetricDistortion'),
+ dict(type='PackSegInputs')
+]
+test_pipeline = [
+ dict(type='LoadImageFromFile'),
+ dict(type='Resize', scale=img_scale, keep_ratio=False),
+ dict(type='LoadAnnotations'),
+ dict(type='PackSegInputs')
+]
+train_dataloader = dict(
+ batch_size=16,
+ num_workers=4,
+ persistent_workers=True,
+ sampler=dict(type='InfiniteSampler', shuffle=True),
+ dataset=dict(
+ type=dataset_type,
+ data_root=data_root,
+ ann_file='train.txt',
+ data_prefix=dict(img_path='images/', seg_map_path='masks/'),
+ pipeline=train_pipeline))
+val_dataloader = dict(
+ batch_size=1,
+ num_workers=4,
+ persistent_workers=True,
+ sampler=dict(type='DefaultSampler', shuffle=False),
+ dataset=dict(
+ type=dataset_type,
+ data_root=data_root,
+ ann_file='val.txt',
+ data_prefix=dict(img_path='images/', seg_map_path='masks/'),
+ pipeline=test_pipeline))
+test_dataloader = val_dataloader
+val_evaluator = dict(type='IoUMetric', iou_metrics=['mIoU', 'mDice'])
+test_evaluator = dict(type='IoUMetric', iou_metrics=['mIoU', 'mDice'])
diff --git a/projects/medical/2d_image/x_ray/covid_19_ct_cxr/configs/fcn-unet-s5-d16_unet-{use-sigmoid}_1xb16-0.01-20k_covid-19-ct-cxr-512x512.py b/projects/medical/2d_image/x_ray/covid_19_ct_cxr/configs/fcn-unet-s5-d16_unet-{use-sigmoid}_1xb16-0.01-20k_covid-19-ct-cxr-512x512.py
new file mode 100644
index 00000000000..59a7bedaa06
--- /dev/null
+++ b/projects/medical/2d_image/x_ray/covid_19_ct_cxr/configs/fcn-unet-s5-d16_unet-{use-sigmoid}_1xb16-0.01-20k_covid-19-ct-cxr-512x512.py
@@ -0,0 +1,18 @@
+_base_ = [
+ 'mmseg::_base_/models/fcn_unet_s5-d16.py', './covid-19-ct-cxr_512x512.py',
+ 'mmseg::_base_/default_runtime.py',
+ 'mmseg::_base_/schedules/schedule_20k.py'
+]
+custom_imports = dict(imports='datasets.covid-19-ct-cxr_dataset')
+img_scale = (512, 512)
+data_preprocessor = dict(size=img_scale)
+optimizer = dict(lr=0.01)
+optim_wrapper = dict(optimizer=optimizer)
+model = dict(
+ data_preprocessor=data_preprocessor,
+ decode_head=dict(
+ num_classes=2, loss_decode=dict(use_sigmoid=True), out_channels=1),
+ auxiliary_head=None,
+ test_cfg=dict(mode='whole', _delete_=True))
+vis_backends = None
+visualizer = dict(vis_backends=vis_backends)
diff --git a/projects/medical/2d_image/x_ray/covid_19_ct_cxr/configs/fcn-unet-s5-d16_unet_1xb16-0.0001-20k_covid-19-ct-cxr-512x512.py b/projects/medical/2d_image/x_ray/covid_19_ct_cxr/configs/fcn-unet-s5-d16_unet_1xb16-0.0001-20k_covid-19-ct-cxr-512x512.py
new file mode 100644
index 00000000000..83b8527d46d
--- /dev/null
+++ b/projects/medical/2d_image/x_ray/covid_19_ct_cxr/configs/fcn-unet-s5-d16_unet_1xb16-0.0001-20k_covid-19-ct-cxr-512x512.py
@@ -0,0 +1,17 @@
+_base_ = [
+ 'mmseg::_base_/models/fcn_unet_s5-d16.py', './covid-19-ct-cxr_512x512.py',
+ 'mmseg::_base_/default_runtime.py',
+ 'mmseg::_base_/schedules/schedule_20k.py'
+]
+custom_imports = dict(imports='datasets.covid-19-ct-cxr_dataset')
+img_scale = (512, 512)
+data_preprocessor = dict(size=img_scale)
+optimizer = dict(lr=0.0001)
+optim_wrapper = dict(optimizer=optimizer)
+model = dict(
+ data_preprocessor=data_preprocessor,
+ decode_head=dict(num_classes=2),
+ auxiliary_head=None,
+ test_cfg=dict(mode='whole', _delete_=True))
+vis_backends = None
+visualizer = dict(vis_backends=vis_backends)
diff --git a/projects/medical/2d_image/x_ray/covid_19_ct_cxr/configs/fcn-unet-s5-d16_unet_1xb16-0.001-20k_covid-19-ct-cxr-512x512.py b/projects/medical/2d_image/x_ray/covid_19_ct_cxr/configs/fcn-unet-s5-d16_unet_1xb16-0.001-20k_covid-19-ct-cxr-512x512.py
new file mode 100644
index 00000000000..10cfcbda6ec
--- /dev/null
+++ b/projects/medical/2d_image/x_ray/covid_19_ct_cxr/configs/fcn-unet-s5-d16_unet_1xb16-0.001-20k_covid-19-ct-cxr-512x512.py
@@ -0,0 +1,17 @@
+_base_ = [
+ 'mmseg::_base_/models/fcn_unet_s5-d16.py', './covid-19-ct-cxr_512x512.py',
+ 'mmseg::_base_/default_runtime.py',
+ 'mmseg::_base_/schedules/schedule_20k.py'
+]
+custom_imports = dict(imports='datasets.covid-19-ct-cxr_dataset')
+img_scale = (512, 512)
+data_preprocessor = dict(size=img_scale)
+optimizer = dict(lr=0.001)
+optim_wrapper = dict(optimizer=optimizer)
+model = dict(
+ data_preprocessor=data_preprocessor,
+ decode_head=dict(num_classes=2),
+ auxiliary_head=None,
+ test_cfg=dict(mode='whole', _delete_=True))
+vis_backends = None
+visualizer = dict(vis_backends=vis_backends)
diff --git a/projects/medical/2d_image/x_ray/covid_19_ct_cxr/configs/fcn-unet-s5-d16_unet_1xb16-0.01-20k_covid-19-ct-cxr-512x512.py b/projects/medical/2d_image/x_ray/covid_19_ct_cxr/configs/fcn-unet-s5-d16_unet_1xb16-0.01-20k_covid-19-ct-cxr-512x512.py
new file mode 100644
index 00000000000..aaccc8fd8dd
--- /dev/null
+++ b/projects/medical/2d_image/x_ray/covid_19_ct_cxr/configs/fcn-unet-s5-d16_unet_1xb16-0.01-20k_covid-19-ct-cxr-512x512.py
@@ -0,0 +1,17 @@
+_base_ = [
+ 'mmseg::_base_/models/fcn_unet_s5-d16.py', './covid-19-ct-cxr_512x512.py',
+ 'mmseg::_base_/default_runtime.py',
+ 'mmseg::_base_/schedules/schedule_20k.py'
+]
+custom_imports = dict(imports='datasets.covid-19-ct-cxr_dataset')
+img_scale = (512, 512)
+data_preprocessor = dict(size=img_scale)
+optimizer = dict(lr=0.01)
+optim_wrapper = dict(optimizer=optimizer)
+model = dict(
+ data_preprocessor=data_preprocessor,
+ decode_head=dict(num_classes=2),
+ auxiliary_head=None,
+ test_cfg=dict(mode='whole', _delete_=True))
+vis_backends = None
+visualizer = dict(vis_backends=vis_backends)
diff --git a/projects/medical/2d_image/x_ray/covid_19_ct_cxr/datasets/covid-19-ct-cxr_dataset.py b/projects/medical/2d_image/x_ray/covid_19_ct_cxr/datasets/covid-19-ct-cxr_dataset.py
new file mode 100644
index 00000000000..68a1bb331f7
--- /dev/null
+++ b/projects/medical/2d_image/x_ray/covid_19_ct_cxr/datasets/covid-19-ct-cxr_dataset.py
@@ -0,0 +1,31 @@
+from mmseg.datasets import BaseSegDataset
+from mmseg.registry import DATASETS
+
+
+@DATASETS.register_module()
+class Covid19CXRDataset(BaseSegDataset):
+ """Covid19CXRDataset dataset.
+
+ In segmentation map annotation for Covid19CXRDataset,
+ 0 stands for background, which is included in 2 categories.
+ ``reduce_zero_label`` is fixed to False. The ``img_suffix``
+ is fixed to '.png' and ``seg_map_suffix`` is fixed to '.png'.
+
+ Args:
+ img_suffix (str): Suffix of images. Default: '.png'
+ seg_map_suffix (str): Suffix of segmentation maps. Default: '.png'
+ reduce_zero_label (bool): Whether to mark label zero as ignored.
+ Default to False.
+ """
+ METAINFO = dict(classes=('background', 'lung'))
+
+ def __init__(self,
+ img_suffix='.png',
+ seg_map_suffix='.png',
+ reduce_zero_label=False,
+ **kwargs) -> None:
+ super().__init__(
+ img_suffix=img_suffix,
+ seg_map_suffix=seg_map_suffix,
+ reduce_zero_label=reduce_zero_label,
+ **kwargs)
diff --git a/projects/medical/2d_image/x_ray/covid_19_ct_cxr/tools/prepare_dataset.py b/projects/medical/2d_image/x_ray/covid_19_ct_cxr/tools/prepare_dataset.py
new file mode 100644
index 00000000000..72f6435389d
--- /dev/null
+++ b/projects/medical/2d_image/x_ray/covid_19_ct_cxr/tools/prepare_dataset.py
@@ -0,0 +1,52 @@
+import os
+
+import numpy as np
+from PIL import Image
+
+root_path = 'data/'
+src_img_dir = os.path.join(root_path, 'covid-chestxray-dataset', 'images')
+src_mask_dir = os.path.join(root_path, 'covid-chestxray-dataset',
+ 'annotations/lungVAE-masks')
+tgt_img_train_dir = os.path.join(root_path, 'images/train/')
+tgt_mask_train_dir = os.path.join(root_path, 'masks/train/')
+tgt_img_test_dir = os.path.join(root_path, 'images/test/')
+os.system('mkdir -p ' + tgt_img_train_dir)
+os.system('mkdir -p ' + tgt_mask_train_dir)
+os.system('mkdir -p ' + tgt_img_test_dir)
+
+
+def convert_label(img, convert_dict):
+ arr = np.zeros_like(img, dtype=np.uint8)
+ for c, i in convert_dict.items():
+ arr[img == c] = i
+ return arr
+
+
+if __name__ == '__main__':
+
+ all_img_names = os.listdir(src_img_dir)
+ all_mask_names = os.listdir(src_mask_dir)
+
+ for img_name in all_img_names:
+ base_name = img_name.replace('.png', '')
+ base_name = base_name.replace('.jpg', '')
+ base_name = base_name.replace('.jpeg', '')
+ mask_name_orig = base_name + '_mask.png'
+ if mask_name_orig in all_mask_names:
+ mask_name = base_name + '.png'
+ src_img_path = os.path.join(src_img_dir, img_name)
+ src_mask_path = os.path.join(src_mask_dir, mask_name_orig)
+ tgt_img_path = os.path.join(tgt_img_train_dir, img_name)
+ tgt_mask_path = os.path.join(tgt_mask_train_dir, mask_name)
+
+ img = Image.open(src_img_path).convert('RGB')
+ img.save(tgt_img_path)
+ mask = np.array(Image.open(src_mask_path))
+ mask = convert_label(mask, {0: 0, 255: 1})
+ mask = Image.fromarray(mask)
+ mask.save(tgt_mask_path)
+ else:
+ src_img_path = os.path.join(src_img_dir, img_name)
+ tgt_img_path = os.path.join(tgt_img_test_dir, img_name)
+ img = Image.open(src_img_path).convert('RGB')
+ img.save(tgt_img_path)
diff --git a/projects/medical/2d_image/x_ray/crass/README.md b/projects/medical/2d_image/x_ray/crass/README.md
new file mode 100644
index 00000000000..0621205be81
--- /dev/null
+++ b/projects/medical/2d_image/x_ray/crass/README.md
@@ -0,0 +1,144 @@
+# Chest Radiograph Anatomical Structure Segmentation (CRASS)
+
+## Description
+
+This project supports **`Chest Radiograph Anatomical Structure Segmentation (CRASS) `**, which can be downloaded from [here](https://crass.grand-challenge.org/).
+
+### Dataset Overview
+
+A set of consecutively obtained posterior-anterior chest radiograph were selected from a database containing images acquired at two sites in sub Saharan Africa with a high tuberculosis incidence. All subjects were 15 years or older. Images from digital chest radiography units were used (Delft Imaging Systems, The Netherlands) of varying resolutions, with a typical resolution of 1800--2000 pixels, the pixel size was 250 lm isotropic. From the total set of images, 225 were considered to be normal by an expert radiologist, while 333 of the images contained abnormalities. Of the abnormal images, 220 contained abnormalities in the upper area of the lung where the clavicle is located. The data was divided into a training and a test set. The training set consisted of 299 images, the test set of 249 images.
+The current data is still incomplete and to be added later.
+
+### Information Statistics
+
+| Dataset Name | Anatomical Region | Task Type | Modality | Num. Classes | Train/Val/Test Images | Train/Val/Test Labeled | Release Date | License |
+| ------------------------------------------- | ----------------- | ------------ | -------- | ------------ | --------------------- | ---------------------- | ------------ | ------------------------------------------------------------- |
+| [crass](https://crass.grand-challenge.org/) | pulmonary | segmentation | x_ray | 2 | 299/-/234 | yes/-/no | 2021 | [CC0 1.0](https://creativecommons.org/publicdomain/zero/1.0/) |
+
+| Class Name | Num. Train | Pct. Train | Num. Val | Pct. Val | Num. Test | Pct. Test |
+| :--------: | :--------: | :--------: | :------: | :------: | :-------: | :-------: |
+| background | 299 | 98.38 | - | - | - | - |
+| clavicles | 299 | 1.62 | - | - | - | - |
+
+Note:
+
+- `Pct` means percentage of pixels in this category in all pixels.
+
+### Visualization
+
+
+
+### Dataset Citation
+
+```
+@article{HOGEWEG20121490,
+ title={Clavicle segmentation in chest radiographs},
+ journal={Medical Image Analysis},
+ volume={16},
+ number={8},
+ pages={1490-1502},
+ year={2012}
+}
+```
+
+### Prerequisites
+
+- Python v3.8
+- PyTorch v1.10.0
+- pillow(PIL) v9.3.0
+- scikit-learn(sklearn) v1.2.0
+- [MIM](https://github.com/open-mmlab/mim) v0.3.4
+- [MMCV](https://github.com/open-mmlab/mmcv) v2.0.0rc4
+- [MMEngine](https://github.com/open-mmlab/mmengine) v0.2.0 or higher
+- [MMSegmentation](https://github.com/open-mmlab/mmsegmentation) v1.0.0rc5
+
+All the commands below rely on the correct configuration of `PYTHONPATH`, which should point to the project's directory so that Python can locate the module files. In `crass/` root directory, run the following line to add the current directory to `PYTHONPATH`:
+
+```shell
+export PYTHONPATH=`pwd`:$PYTHONPATH
+```
+
+### Dataset Preparing
+
+- download dataset from [here](https://crass.grand-challenge.org/) and decompress data to path `'data/'`.
+- run script `"python tools/prepare_dataset.py"` to format data and change folder structure as below.
+- run script `"python ../../tools/split_seg_dataset.py"` to split dataset and generate `train.txt`, `val.txt` and `test.txt`. If the label of official validation set and test set cannot be obtained, we generate `train.txt` and `val.txt` from the training set randomly.
+
+```none
+ mmsegmentation
+ ├── mmseg
+ ├── projects
+ │ ├── medical
+ │ │ ├── 2d_image
+ │ │ │ ├── x_ray
+ │ │ │ │ ├── crass
+ │ │ │ │ │ ├── configs
+ │ │ │ │ │ ├── datasets
+ │ │ │ │ │ ├── tools
+ │ │ │ │ │ ├── data
+ │ │ │ │ │ │ ├── train.txt
+ │ │ │ │ │ │ ├── val.txt
+ │ │ │ │ │ │ ├── images
+ │ │ │ │ │ │ │ ├── train
+ │ │ │ │ | │ │ │ ├── xxx.png
+ │ │ │ │ | │ │ │ ├── ...
+ │ │ │ │ | │ │ │ └── xxx.png
+ │ │ │ │ │ │ ├── masks
+ │ │ │ │ │ │ │ ├── train
+ │ │ │ │ | │ │ │ ├── xxx.png
+ │ │ │ │ | │ │ │ ├── ...
+ │ │ │ │ | │ │ │ └── xxx.png
+```
+
+### Divided Dataset Information
+
+***Note: The table information below is divided by ourselves.***
+
+| Class Name | Num. Train | Pct. Train | Num. Val | Pct. Val | Num. Test | Pct. Test |
+| :--------: | :--------: | :--------: | :------: | :------: | :-------: | :-------: |
+| background | 227 | 98.38 | 57 | 98.39 | - | - |
+| clavicles | 227 | 1.62 | 57 | 1.61 | - | - |
+
+### Training commands
+
+To train models on a single server with one GPU. (default)
+
+```shell
+mim train mmseg ./configs/${CONFIG_FILE}
+```
+
+### Testing commands
+
+To test models on a single server with one GPU. (default)
+
+```shell
+mim test mmseg ./configs/${CONFIG_FILE} --checkpoint ${CHECKPOINT_PATH}
+```
+
+
+
+## Checklist
+
+- [x] Milestone 1: PR-ready, and acceptable to be one of the `projects/`.
+
+ - [x] Finish the code
+ - [x] Basic docstrings & proper citation
+ - [ ] Test-time correctness
+ - [x] A full README
+
+- [x] Milestone 2: Indicates a successful model implementation.
+
+ - [x] Training-time correctness
+
+- [ ] Milestone 3: Good to be a part of our core package!
+
+ - [ ] Type hints and docstrings
+ - [ ] Unit tests
+ - [ ] Code polishing
+ - [ ] Metafile.yml
+
+- [ ] Move your modules into the core package following the codebase's file hierarchy structure.
+
+- [ ] Refactor your modules into the core package following the codebase's file hierarchy structure.
diff --git a/projects/medical/2d_image/x_ray/crass/configs/crass_512x512.py b/projects/medical/2d_image/x_ray/crass/configs/crass_512x512.py
new file mode 100644
index 00000000000..1425f50cc46
--- /dev/null
+++ b/projects/medical/2d_image/x_ray/crass/configs/crass_512x512.py
@@ -0,0 +1,42 @@
+dataset_type = 'CRASSDataset'
+data_root = 'data/'
+img_scale = (512, 512)
+train_pipeline = [
+ dict(type='LoadImageFromFile'),
+ dict(type='LoadAnnotations'),
+ dict(type='Resize', scale=img_scale, keep_ratio=False),
+ dict(type='RandomFlip', prob=0.5),
+ dict(type='PhotoMetricDistortion'),
+ dict(type='PackSegInputs')
+]
+test_pipeline = [
+ dict(type='LoadImageFromFile'),
+ dict(type='Resize', scale=img_scale, keep_ratio=False),
+ dict(type='LoadAnnotations'),
+ dict(type='PackSegInputs')
+]
+train_dataloader = dict(
+ batch_size=16,
+ num_workers=4,
+ persistent_workers=True,
+ sampler=dict(type='InfiniteSampler', shuffle=True),
+ dataset=dict(
+ type=dataset_type,
+ data_root=data_root,
+ ann_file='train.txt',
+ data_prefix=dict(img_path='images/', seg_map_path='masks/'),
+ pipeline=train_pipeline))
+val_dataloader = dict(
+ batch_size=1,
+ num_workers=4,
+ persistent_workers=True,
+ sampler=dict(type='DefaultSampler', shuffle=False),
+ dataset=dict(
+ type=dataset_type,
+ data_root=data_root,
+ ann_file='tval.txt',
+ data_prefix=dict(img_path='images/', seg_map_path='masks/'),
+ pipeline=test_pipeline))
+test_dataloader = val_dataloader
+val_evaluator = dict(type='IoUMetric', iou_metrics=['mIoU', 'mDice'])
+test_evaluator = dict(type='IoUMetric', iou_metrics=['mIoU', 'mDice'])
diff --git a/projects/medical/2d_image/x_ray/crass/configs/fcn-unet-s5-d16_unet_1xb16-0.0001-20k_crass-512x512.py b/projects/medical/2d_image/x_ray/crass/configs/fcn-unet-s5-d16_unet_1xb16-0.0001-20k_crass-512x512.py
new file mode 100644
index 00000000000..b52bc78f790
--- /dev/null
+++ b/projects/medical/2d_image/x_ray/crass/configs/fcn-unet-s5-d16_unet_1xb16-0.0001-20k_crass-512x512.py
@@ -0,0 +1,17 @@
+_base_ = [
+ 'mmseg::_base_/models/fcn_unet_s5-d16.py', './crass_512x512.py',
+ 'mmseg::_base_/default_runtime.py',
+ 'mmseg::_base_/schedules/schedule_20k.py'
+]
+custom_imports = dict(imports='datasets.crass_dataset')
+img_scale = (512, 512)
+data_preprocessor = dict(size=img_scale)
+optimizer = dict(lr=0.0001)
+optim_wrapper = dict(optimizer=optimizer)
+model = dict(
+ data_preprocessor=data_preprocessor,
+ decode_head=dict(num_classes=2),
+ auxiliary_head=None,
+ test_cfg=dict(mode='whole', _delete_=True))
+vis_backends = None
+visualizer = dict(vis_backends=vis_backends)
diff --git a/projects/medical/2d_image/x_ray/crass/configs/fcn-unet-s5-d16_unet_1xb16-0.001-20k_crass-512x512.py b/projects/medical/2d_image/x_ray/crass/configs/fcn-unet-s5-d16_unet_1xb16-0.001-20k_crass-512x512.py
new file mode 100644
index 00000000000..45242c65b48
--- /dev/null
+++ b/projects/medical/2d_image/x_ray/crass/configs/fcn-unet-s5-d16_unet_1xb16-0.001-20k_crass-512x512.py
@@ -0,0 +1,17 @@
+_base_ = [
+ 'mmseg::_base_/models/fcn_unet_s5-d16.py', './crass_512x512.py',
+ 'mmseg::_base_/default_runtime.py',
+ 'mmseg::_base_/schedules/schedule_20k.py'
+]
+custom_imports = dict(imports='datasets.crass_dataset')
+img_scale = (512, 512)
+data_preprocessor = dict(size=img_scale)
+optimizer = dict(lr=0.001)
+optim_wrapper = dict(optimizer=optimizer)
+model = dict(
+ data_preprocessor=data_preprocessor,
+ decode_head=dict(num_classes=2),
+ auxiliary_head=None,
+ test_cfg=dict(mode='whole', _delete_=True))
+vis_backends = None
+visualizer = dict(vis_backends=vis_backends)
diff --git a/projects/medical/2d_image/x_ray/crass/configs/fcn-unet-s5-d16_unet_1xb16-0.01-20k_crass-512x512.py b/projects/medical/2d_image/x_ray/crass/configs/fcn-unet-s5-d16_unet_1xb16-0.01-20k_crass-512x512.py
new file mode 100644
index 00000000000..bcf9d0a5ca5
--- /dev/null
+++ b/projects/medical/2d_image/x_ray/crass/configs/fcn-unet-s5-d16_unet_1xb16-0.01-20k_crass-512x512.py
@@ -0,0 +1,17 @@
+_base_ = [
+ 'mmseg::_base_/models/fcn_unet_s5-d16.py', './crass_512x512.py',
+ 'mmseg::_base_/default_runtime.py',
+ 'mmseg::_base_/schedules/schedule_20k.py'
+]
+custom_imports = dict(imports='datasets.crass_dataset')
+img_scale = (512, 512)
+data_preprocessor = dict(size=img_scale)
+optimizer = dict(lr=0.01)
+optim_wrapper = dict(optimizer=optimizer)
+model = dict(
+ data_preprocessor=data_preprocessor,
+ decode_head=dict(num_classes=2),
+ auxiliary_head=None,
+ test_cfg=dict(mode='whole', _delete_=True))
+vis_backends = None
+visualizer = dict(vis_backends=vis_backends)
diff --git a/projects/medical/2d_image/x_ray/crass/configs/fcn-unet-s5-d16_unet_1xb16-lr0.01-sigmoid-20k_crass-512x512.py b/projects/medical/2d_image/x_ray/crass/configs/fcn-unet-s5-d16_unet_1xb16-lr0.01-sigmoid-20k_crass-512x512.py
new file mode 100644
index 00000000000..0dde736bf76
--- /dev/null
+++ b/projects/medical/2d_image/x_ray/crass/configs/fcn-unet-s5-d16_unet_1xb16-lr0.01-sigmoid-20k_crass-512x512.py
@@ -0,0 +1,18 @@
+_base_ = [
+ 'mmseg::_base_/models/fcn_unet_s5-d16.py', './crass_512x512.py',
+ 'mmseg::_base_/default_runtime.py',
+ 'mmseg::_base_/schedules/schedule_20k.py'
+]
+custom_imports = dict(imports='datasets.crass_dataset')
+img_scale = (512, 512)
+data_preprocessor = dict(size=img_scale)
+optimizer = dict(lr=0.01)
+optim_wrapper = dict(optimizer=optimizer)
+model = dict(
+ data_preprocessor=data_preprocessor,
+ decode_head=dict(
+ num_classes=2, loss_decode=dict(use_sigmoid=True), out_channels=1),
+ auxiliary_head=None,
+ test_cfg=dict(mode='whole', _delete_=True))
+vis_backends = None
+visualizer = dict(vis_backends=vis_backends)
diff --git a/projects/medical/2d_image/x_ray/crass/datasets/crass_dataset.py b/projects/medical/2d_image/x_ray/crass/datasets/crass_dataset.py
new file mode 100644
index 00000000000..f6b5c5228bc
--- /dev/null
+++ b/projects/medical/2d_image/x_ray/crass/datasets/crass_dataset.py
@@ -0,0 +1,30 @@
+from mmseg.datasets import BaseSegDataset
+from mmseg.registry import DATASETS
+
+
+@DATASETS.register_module()
+class CRASSDataset(BaseSegDataset):
+ """CRASSDataset dataset.
+
+ In segmentation map annotation for CRASSDataset, 0 stands for background,
+ which is included in 2 categories. ``reduce_zero_label`` is fixed to
+ False. The ``img_suffix`` is fixed to '.png' and ``seg_map_suffix`` is
+ fixed to '.png'.
+ Args:
+ img_suffix (str): Suffix of images. Default: '.png'
+ seg_map_suffix (str): Suffix of segmentation maps. Default: '.png'
+ reduce_zero_label (bool): Whether to mark label zero as ignored.
+ Default to False..
+ """
+ METAINFO = dict(classes=('background', 'clavicles'))
+
+ def __init__(self,
+ img_suffix='.png',
+ seg_map_suffix='.png',
+ reduce_zero_label=False,
+ **kwargs) -> None:
+ super().__init__(
+ img_suffix=img_suffix,
+ seg_map_suffix=seg_map_suffix,
+ reduce_zero_label=reduce_zero_label,
+ **kwargs)
diff --git a/projects/medical/2d_image/x_ray/crass/tools/prepare_dataset.py b/projects/medical/2d_image/x_ray/crass/tools/prepare_dataset.py
new file mode 100644
index 00000000000..bbd5d8891d4
--- /dev/null
+++ b/projects/medical/2d_image/x_ray/crass/tools/prepare_dataset.py
@@ -0,0 +1,84 @@
+import glob
+import os
+
+import cv2
+import SimpleITK as sitk
+from PIL import Image
+
+root_path = 'data/'
+img_suffix = '.tif'
+seg_map_suffix = '.png'
+save_img_suffix = '.png'
+save_seg_map_suffix = '.png'
+
+src_img_train_dir = os.path.join(root_path, 'CRASS/data_train')
+src_mask_train_dir = os.path.join(root_path, 'CRASS/mask_mhd')
+src_img_test_dir = os.path.join(root_path, 'CRASS/data_test')
+
+tgt_img_train_dir = os.path.join(root_path, 'images/train/')
+tgt_mask_train_dir = os.path.join(root_path, 'masks/train/')
+tgt_img_test_dir = os.path.join(root_path, 'images/test/')
+os.system('mkdir -p ' + tgt_img_train_dir)
+os.system('mkdir -p ' + tgt_mask_train_dir)
+os.system('mkdir -p ' + tgt_img_test_dir)
+
+
+def filter_suffix_recursive(src_dir, suffix):
+ suffix = '.' + suffix if '.' not in suffix else suffix
+ file_paths = glob(
+ os.path.join(src_dir, '**', '*' + suffix), recursive=True)
+ file_names = [_.split('/')[-1] for _ in file_paths]
+ return sorted(file_paths), sorted(file_names)
+
+
+def read_single_array_from_med(path):
+ return sitk.GetArrayFromImage(sitk.ReadImage(path)).squeeze()
+
+
+def convert_meds_into_pngs(src_dir,
+ tgt_dir,
+ suffix='.dcm',
+ norm_min=0,
+ norm_max=255,
+ convert='RGB'):
+ if not os.path.exists(tgt_dir):
+ os.makedirs(tgt_dir)
+
+ src_paths, src_names = filter_suffix_recursive(src_dir, suffix=suffix)
+ num = len(src_paths)
+ for i, (src_name, src_path) in enumerate(zip(src_names, src_paths)):
+ tgt_name = src_name.replace(suffix, '.png')
+ tgt_path = os.path.join(tgt_dir, tgt_name)
+
+ img = read_single_array_from_med(src_path)
+ if norm_min is not None and norm_max is not None:
+ img = cv2.normalize(img, None, norm_min, norm_max, cv2.NORM_MINMAX,
+ cv2.CV_8U)
+ pil = Image.fromarray(img).convert(convert)
+ pil.save(tgt_path)
+ print(f'processed {i+1}/{num}.')
+
+
+convert_meds_into_pngs(
+ src_img_train_dir,
+ tgt_img_train_dir,
+ suffix='.mhd',
+ norm_min=0,
+ norm_max=255,
+ convert='RGB')
+
+convert_meds_into_pngs(
+ src_img_test_dir,
+ tgt_img_test_dir,
+ suffix='.mhd',
+ norm_min=0,
+ norm_max=255,
+ convert='RGB')
+
+convert_meds_into_pngs(
+ src_mask_train_dir,
+ tgt_mask_train_dir,
+ suffix='.mhd',
+ norm_min=0,
+ norm_max=1,
+ convert='L')
diff --git a/projects/pp_mobileseg/README.md b/projects/pp_mobileseg/README.md
new file mode 100644
index 00000000000..c9f9c128e74
--- /dev/null
+++ b/projects/pp_mobileseg/README.md
@@ -0,0 +1,123 @@
+# PP-MobileSeg: Exploring Transformer Blocks for Efficient Mobile Segmentation.
+
+## Reference
+
+> [PP-MobileSeg: Explore the Fast and Accurate Semantic Segmentation Model on Mobile Devices. ](https://arxiv.org/abs/2304.05152)
+
+## Introduction
+
+Official Repo
+
+Code Snippet
+
+## Abstract
+
+With the success of transformers in computer vision, several attempts have been made to adapt transformers to mobile devices. However, their performance is not satisfied for some real world applications. Therefore, we propose PP-MobileSeg, a SOTA semantic segmentation model for mobile devices.
+
+It is composed of three newly proposed parts, the strideformer backbone, the Aggregated Attention Module(AAM), and the Valid Interpolate Module(VIM):
+
+- With the four-stage MobileNetV3 block as the feature extractor, we manage to extract rich local features of different receptive fields with little parameter overhead. Also, we further efficiently empower features from the last two stages with the global view using strided sea attention.
+- To effectively fuse the features, we use AAM to filter the detail features with ensemble voting and add the semantic feature to it to enhance the semantic information to the most content.
+- At last, we use VIM to upsample the downsampled feature to the original resolution and significantly decrease latency in model inference stage. It only interpolates classes present in the final prediction which only takes around 10% in the ADE20K dataset. This is a common scenario for datasets with large classes. Therefore it significantly decreases the latency of the final upsample process which takes the greatest part of the model's overall latency.
+
+Extensive experiments show that PP-MobileSeg achieves a superior params-accuracy-latency tradeoff compared to other SOTA methods.
+
+
+
+
+ 
@@ -45,6 +46,12 @@ English | [简体中文](README_zh-CN.md)
+ 
+ 
- 
+
我们会在 OpenMMLab 社区为大家
diff --git a/configs/_base_/datasets/bdd100k.py b/configs/_base_/datasets/bdd100k.py
new file mode 100644
index 00000000000..24cec69bfeb
--- /dev/null
+++ b/configs/_base_/datasets/bdd100k.py
@@ -0,0 +1,70 @@
+# dataset settings
+dataset_type = 'BDD100KDataset'
+data_root = 'data/bdd100k/'
+
+crop_size = (512, 1024)
+train_pipeline = [
+ dict(type='LoadImageFromFile'),
+ dict(type='LoadAnnotations'),
+ dict(
+ type='RandomResize',
+ scale=(2048, 1024),
+ ratio_range=(0.5, 2.0),
+ keep_ratio=True),
+ dict(type='RandomCrop', crop_size=crop_size, cat_max_ratio=0.75),
+ dict(type='RandomFlip', prob=0.5),
+ dict(type='PhotoMetricDistortion'),
+ dict(type='PackSegInputs')
+]
+test_pipeline = [
+ dict(type='LoadImageFromFile'),
+ dict(type='Resize', scale=(2048, 1024), keep_ratio=True),
+ # add loading annotation after ``Resize`` because ground truth
+ # does not need to do resize data transform
+ dict(type='LoadAnnotations'),
+ dict(type='PackSegInputs')
+]
+img_ratios = [0.5, 0.75, 1.0, 1.25, 1.5, 1.75]
+tta_pipeline = [
+ dict(type='LoadImageFromFile', backend_args=None),
+ dict(
+ type='TestTimeAug',
+ transforms=[
+ [
+ dict(type='Resize', scale_factor=r, keep_ratio=True)
+ for r in img_ratios
+ ],
+ [
+ dict(type='RandomFlip', prob=0., direction='horizontal'),
+ dict(type='RandomFlip', prob=1., direction='horizontal')
+ ], [dict(type='LoadAnnotations')], [dict(type='PackSegInputs')]
+ ])
+]
+train_dataloader = dict(
+ batch_size=2,
+ num_workers=2,
+ persistent_workers=True,
+ sampler=dict(type='InfiniteSampler', shuffle=True),
+ dataset=dict(
+ type=dataset_type,
+ data_root=data_root,
+ data_prefix=dict(
+ img_path='images/10k/train',
+ seg_map_path='labels/sem_seg/masks/train'),
+ pipeline=train_pipeline))
+val_dataloader = dict(
+ batch_size=1,
+ num_workers=4,
+ persistent_workers=True,
+ sampler=dict(type='DefaultSampler', shuffle=False),
+ dataset=dict(
+ type=dataset_type,
+ data_root=data_root,
+ data_prefix=dict(
+ img_path='images/10k/val',
+ seg_map_path='labels/sem_seg/masks/val'),
+ pipeline=test_pipeline))
+test_dataloader = val_dataloader
+
+val_evaluator = dict(type='IoUMetric', iou_metrics=['mIoU'])
+test_evaluator = val_evaluator
diff --git a/configs/_base_/datasets/coco-stuff164k.py b/configs/_base_/datasets/coco-stuff164k.py
index baf633f9d6f..a9b9d90117a 100644
--- a/configs/_base_/datasets/coco-stuff164k.py
+++ b/configs/_base_/datasets/coco-stuff164k.py
@@ -48,7 +48,7 @@
type=dataset_type,
data_root=data_root,
data_prefix=dict(
- img_path='images/train2017', seg_map_path='annotations/val2017'),
+ img_path='images/train2017', seg_map_path='annotations/train2017'),
pipeline=train_pipeline))
val_dataloader = dict(
batch_size=1,
diff --git a/configs/_base_/datasets/levir_256x256.py b/configs/_base_/datasets/levir_256x256.py
new file mode 100644
index 00000000000..a2a69aa9e9c
--- /dev/null
+++ b/configs/_base_/datasets/levir_256x256.py
@@ -0,0 +1,59 @@
+# dataset settings
+dataset_type = 'LEVIRCDDataset'
+data_root = r'data/LEVIRCD'
+
+albu_train_transforms = [
+ dict(type='RandomBrightnessContrast', p=0.2),
+ dict(type='HorizontalFlip', p=0.5),
+ dict(type='VerticalFlip', p=0.5)
+]
+
+train_pipeline = [
+ dict(type='LoadMultipleRSImageFromFile'),
+ dict(type='LoadAnnotations'),
+ dict(type='Albu', transforms=albu_train_transforms),
+ dict(type='ConcatCDInput'),
+ dict(type='PackSegInputs')
+]
+test_pipeline = [
+ dict(type='LoadMultipleRSImageFromFile'),
+ dict(type='LoadAnnotations'),
+ dict(type='ConcatCDInput'),
+ dict(type='PackSegInputs')
+]
+
+tta_pipeline = [
+ dict(type='LoadMultipleRSImageFromFile'),
+ dict(
+ type='TestTimeAug',
+ transforms=[[dict(type='LoadAnnotations')],
+ [dict(type='ConcatCDInput')],
+ [dict(type='PackSegInputs')]])
+]
+train_dataloader = dict(
+ batch_size=4,
+ num_workers=4,
+ persistent_workers=True,
+ sampler=dict(type='InfiniteSampler', shuffle=True),
+ dataset=dict(
+ type=dataset_type,
+ data_root=data_root,
+ data_prefix=dict(
+ img_path='train/A',
+ img_path2='train/B',
+ seg_map_path='train/label'),
+ pipeline=train_pipeline))
+val_dataloader = dict(
+ batch_size=1,
+ num_workers=4,
+ persistent_workers=True,
+ sampler=dict(type='DefaultSampler', shuffle=False),
+ dataset=dict(
+ type=dataset_type,
+ data_root=data_root,
+ data_prefix=dict(
+ img_path='test/A', img_path2='test/B', seg_map_path='test/label'),
+ pipeline=test_pipeline))
+test_dataloader = val_dataloader
+val_evaluator = dict(type='IoUMetric', iou_metrics=['mIoU'])
+test_evaluator = val_evaluator
diff --git a/configs/_base_/datasets/nyu.py b/configs/_base_/datasets/nyu.py
new file mode 100644
index 00000000000..74d57c5fc50
--- /dev/null
+++ b/configs/_base_/datasets/nyu.py
@@ -0,0 +1,67 @@
+# dataset settings
+dataset_type = 'NYUDataset'
+data_root = 'data/nyu'
+
+train_pipeline = [
+ dict(type='LoadImageFromFile'),
+ dict(type='LoadDepthAnnotation', depth_rescale_factor=1e-3),
+ dict(type='RandomDepthMix', prob=0.25),
+ dict(type='RandomFlip', prob=0.5),
+ dict(type='RandomCrop', crop_size=(480, 480)),
+ dict(
+ type='Albu',
+ transforms=[
+ dict(type='RandomBrightnessContrast'),
+ dict(type='RandomGamma'),
+ dict(type='HueSaturationValue'),
+ ]),
+ dict(
+ type='PackSegInputs',
+ meta_keys=('img_path', 'depth_map_path', 'ori_shape', 'img_shape',
+ 'pad_shape', 'scale_factor', 'flip', 'flip_direction',
+ 'category_id')),
+]
+
+test_pipeline = [
+ dict(type='LoadImageFromFile'),
+ dict(type='Resize', scale=(2000, 480), keep_ratio=True),
+ dict(dict(type='LoadDepthAnnotation', depth_rescale_factor=1e-3)),
+ dict(
+ type='PackSegInputs',
+ meta_keys=('img_path', 'depth_map_path', 'ori_shape', 'img_shape',
+ 'pad_shape', 'scale_factor', 'flip', 'flip_direction',
+ 'category_id'))
+]
+
+train_dataloader = dict(
+ batch_size=8,
+ num_workers=8,
+ persistent_workers=True,
+ sampler=dict(type='InfiniteSampler', shuffle=True),
+ dataset=dict(
+ type=dataset_type,
+ data_root=data_root,
+ data_prefix=dict(
+ img_path='images/train', depth_map_path='annotations/train'),
+ pipeline=train_pipeline))
+
+val_dataloader = dict(
+ batch_size=1,
+ num_workers=4,
+ persistent_workers=True,
+ sampler=dict(type='DefaultSampler', shuffle=False),
+ dataset=dict(
+ type=dataset_type,
+ data_root=data_root,
+ test_mode=True,
+ data_prefix=dict(
+ img_path='images/test', depth_map_path='annotations/test'),
+ pipeline=test_pipeline))
+test_dataloader = val_dataloader
+
+val_evaluator = dict(
+ type='DepthMetric',
+ min_depth_eval=0.001,
+ max_depth_eval=10.0,
+ crop_type='nyu_crop')
+test_evaluator = val_evaluator
diff --git a/configs/_base_/datasets/nyu_512x512.py b/configs/_base_/datasets/nyu_512x512.py
new file mode 100644
index 00000000000..88e3878d334
--- /dev/null
+++ b/configs/_base_/datasets/nyu_512x512.py
@@ -0,0 +1,72 @@
+# dataset settings
+dataset_type = 'NYUDataset'
+data_root = 'data/nyu'
+
+train_pipeline = [
+ dict(type='LoadImageFromFile'),
+ dict(type='LoadDepthAnnotation', depth_rescale_factor=1e-3),
+ dict(type='RandomDepthMix', prob=0.25),
+ dict(type='RandomFlip', prob=0.5),
+ dict(
+ type='RandomResize',
+ scale=(768, 512),
+ ratio_range=(0.8, 1.5),
+ keep_ratio=True),
+ dict(type='RandomCrop', crop_size=(512, 512)),
+ dict(
+ type='Albu',
+ transforms=[
+ dict(type='RandomBrightnessContrast'),
+ dict(type='RandomGamma'),
+ dict(type='HueSaturationValue'),
+ ]),
+ dict(
+ type='PackSegInputs',
+ meta_keys=('img_path', 'depth_map_path', 'ori_shape', 'img_shape',
+ 'pad_shape', 'scale_factor', 'flip', 'flip_direction',
+ 'category_id')),
+]
+
+test_pipeline = [
+ dict(type='LoadImageFromFile'),
+ dict(type='Resize', scale=(2048, 512), keep_ratio=True),
+ dict(dict(type='LoadDepthAnnotation', depth_rescale_factor=1e-3)),
+ dict(
+ type='PackSegInputs',
+ meta_keys=('img_path', 'depth_map_path', 'ori_shape', 'img_shape',
+ 'pad_shape', 'scale_factor', 'flip', 'flip_direction',
+ 'category_id'))
+]
+
+train_dataloader = dict(
+ batch_size=8,
+ num_workers=8,
+ persistent_workers=True,
+ sampler=dict(type='InfiniteSampler', shuffle=True),
+ dataset=dict(
+ type=dataset_type,
+ data_root=data_root,
+ data_prefix=dict(
+ img_path='images/train', depth_map_path='annotations/train'),
+ pipeline=train_pipeline))
+
+val_dataloader = dict(
+ batch_size=1,
+ num_workers=4,
+ persistent_workers=True,
+ sampler=dict(type='DefaultSampler', shuffle=False),
+ dataset=dict(
+ type=dataset_type,
+ data_root=data_root,
+ test_mode=True,
+ data_prefix=dict(
+ img_path='images/test', depth_map_path='annotations/test'),
+ pipeline=test_pipeline))
+test_dataloader = val_dataloader
+
+val_evaluator = dict(
+ type='DepthMetric',
+ min_depth_eval=0.001,
+ max_depth_eval=10.0,
+ crop_type='nyu_crop')
+test_evaluator = val_evaluator
diff --git a/configs/_base_/models/fpn_poolformer_s12.py b/configs/_base_/models/fpn_poolformer_s12.py
index b6893f69774..086c804837a 100644
--- a/configs/_base_/models/fpn_poolformer_s12.py
+++ b/configs/_base_/models/fpn_poolformer_s12.py
@@ -1,10 +1,11 @@
# model settings
norm_cfg = dict(type='SyncBN', requires_grad=True)
checkpoint_file = 'https://download.openmmlab.com/mmclassification/v0/poolformer/poolformer-s12_3rdparty_32xb128_in1k_20220414-f8d83051.pth' # noqa
-# TODO: delete custom_imports after mmcls supports auto import
-# please install mmcls>=1.0
-# import mmcls.models to trigger register_module in mmcls
-custom_imports = dict(imports=['mmcls.models'], allow_failed_imports=False)
+# TODO: delete custom_imports after mmpretrain supports auto import
+# please install mmpretrain >= 1.0.0rc7
+# import mmpretrain.models to trigger register_module in mmpretrain
+custom_imports = dict(
+ imports=['mmpretrain.models'], allow_failed_imports=False)
data_preprocessor = dict(
type='SegDataPreProcessor',
mean=[123.675, 116.28, 103.53],
@@ -16,7 +17,7 @@
type='EncoderDecoder',
data_preprocessor=data_preprocessor,
backbone=dict(
- type='mmcls.PoolFormer',
+ type='mmpretrain.PoolFormer',
arch='s12',
init_cfg=dict(
type='Pretrained', checkpoint=checkpoint_file, prefix='backbone.'),
diff --git a/configs/_base_/models/san_vit-b16.py b/configs/_base_/models/san_vit-b16.py
new file mode 100644
index 00000000000..96ac41b8dad
--- /dev/null
+++ b/configs/_base_/models/san_vit-b16.py
@@ -0,0 +1,137 @@
+# model settings
+norm_cfg = dict(type='SyncBN', requires_grad=True)
+
+data_preprocessor = dict(
+ type='SegDataPreProcessor',
+ mean=[122.7709, 116.7460, 104.0937],
+ std=[68.5005, 66.6322, 70.3232],
+ bgr_to_rgb=True,
+ pad_val=0,
+ seg_pad_val=255,
+ size_divisor=640,
+ test_cfg=dict(size_divisor=32))
+
+num_classes = 171
+model = dict(
+ type='MultimodalEncoderDecoder',
+ data_preprocessor=data_preprocessor,
+ pretrained='pretrain/clip_vit_base_patch16_224.pth',
+ asymetric_input=True,
+ encoder_resolution=0.5,
+ image_encoder=dict(
+ type='VisionTransformer',
+ img_size=(224, 224),
+ patch_size=16,
+ patch_pad=0,
+ in_channels=3,
+ embed_dims=768,
+ num_layers=9,
+ num_heads=12,
+ mlp_ratio=4,
+ out_origin=True,
+ out_indices=(2, 5, 8),
+ qkv_bias=True,
+ drop_rate=0.0,
+ attn_drop_rate=0.0,
+ drop_path_rate=0.0,
+ with_cls_token=True,
+ output_cls_token=True,
+ patch_bias=False,
+ pre_norm=True,
+ norm_cfg=dict(type='LN', eps=1e-5),
+ act_cfg=dict(type='QuickGELU'),
+ norm_eval=False,
+ interpolate_mode='bicubic',
+ frozen_exclude=['pos_embed']),
+ text_encoder=dict(
+ type='CLIPTextEncoder',
+ dataset_name=None,
+ templates='vild',
+ embed_dims=512,
+ num_layers=12,
+ num_heads=8,
+ mlp_ratio=4,
+ output_dims=512,
+ cache_feature=True,
+ cat_bg=True,
+ norm_cfg=dict(type='LN', eps=1e-5)
+ ),
+ decode_head=dict(
+ type='SideAdapterCLIPHead',
+ num_classes=num_classes,
+ deep_supervision_idxs=[7],
+ san_cfg=dict(
+ in_channels=3,
+ clip_channels=768,
+ embed_dims=240,
+ patch_size=16,
+ patch_bias=True,
+ num_queries=100,
+ cfg_encoder=dict(
+ num_encode_layer=8,
+ num_heads=6,
+ mlp_ratio=4
+ ),
+ fusion_index=[0, 1, 2, 3],
+ cfg_decoder=dict(
+ num_heads=12,
+ num_layers=1,
+ embed_channels=256,
+ mlp_channels=256,
+ num_mlp=3,
+ rescale=True),
+ norm_cfg=dict(type='LN', eps=1e-6),
+ ),
+ maskgen_cfg=dict(
+ sos_token_format='cls_token',
+ sos_token_num=100,
+ cross_attn=False,
+ num_layers=3,
+ embed_dims=768,
+ num_heads=12,
+ mlp_ratio=4,
+ qkv_bias=True,
+ out_dims=512,
+ final_norm=True,
+ act_cfg=dict(type='QuickGELU'),
+ norm_cfg=dict(type='LN', eps=1e-5),
+ frozen_exclude=[]
+ ),
+ align_corners=False,
+ train_cfg=dict(
+ num_points=12544,
+ oversample_ratio=3.0,
+ importance_sample_ratio=0.75,
+ assigner=dict(
+ type='HungarianAssigner',
+ match_costs=[
+ dict(type='ClassificationCost', weight=2.0),
+ dict(
+ type='CrossEntropyLossCost',
+ weight=5.0,
+ use_sigmoid=True),
+ dict(
+ type='DiceCost',
+ weight=5.0,
+ pred_act=True,
+ eps=1.0)
+ ])),
+ loss_decode=[dict(type='CrossEntropyLoss',
+ loss_name='loss_cls_ce',
+ loss_weight=2.0,
+ class_weight=[1.0] * num_classes + [0.1]),
+ dict(type='CrossEntropyLoss',
+ use_sigmoid=True,
+ loss_name='loss_mask_ce',
+ loss_weight=5.0),
+ dict(type='DiceLoss',
+ ignore_index=None,
+ naive_dice=True,
+ eps=1,
+ loss_name='loss_mask_dice',
+ loss_weight=5.0)
+ ]),
+
+ # model training and testing settings
+ train_cfg=dict(),
+ test_cfg=dict(mode='whole')) # yapf: disable
diff --git a/configs/_base_/models/upernet_convnext.py b/configs/_base_/models/upernet_convnext.py
index 7595295871d..958994c91e6 100644
--- a/configs/_base_/models/upernet_convnext.py
+++ b/configs/_base_/models/upernet_convnext.py
@@ -1,5 +1,5 @@
norm_cfg = dict(type='SyncBN', requires_grad=True)
-custom_imports = dict(imports='mmcls.models', allow_failed_imports=False)
+custom_imports = dict(imports='mmpretrain.models', allow_failed_imports=False)
checkpoint_file = 'https://download.openmmlab.com/mmclassification/v0/convnext/downstream/convnext-base_3rdparty_32xb128-noema_in1k_20220301-2a0ee547.pth' # noqa
data_preprocessor = dict(
type='SegDataPreProcessor',
@@ -13,7 +13,7 @@
data_preprocessor=data_preprocessor,
pretrained=None,
backbone=dict(
- type='mmcls.ConvNeXt',
+ type='mmpretrain.ConvNeXt',
arch='base',
out_indices=[0, 1, 2, 3],
drop_path_rate=0.4,
diff --git a/configs/_base_/models/vpd_sd.py b/configs/_base_/models/vpd_sd.py
new file mode 100644
index 00000000000..87321e74f04
--- /dev/null
+++ b/configs/_base_/models/vpd_sd.py
@@ -0,0 +1,86 @@
+# model settings
+data_preprocessor = dict(
+ type='SegDataPreProcessor',
+ mean=[127.5, 127.5, 127.5],
+ std=[127.5, 127.5, 127.5],
+ bgr_to_rgb=True,
+ pad_val=0,
+ seg_pad_val=0)
+
+# adapted from stable-diffusion/configs/stable-diffusion/v1-inference.yaml
+stable_diffusion_cfg = dict(
+ base_learning_rate=0.0001,
+ target='ldm.models.diffusion.ddpm.LatentDiffusion',
+ checkpoint='https://download.openmmlab.com/mmsegmentation/v0.5/'
+ 'vpd/stable_diffusion_v1-5_pretrain_third_party.pth',
+ params=dict(
+ linear_start=0.00085,
+ linear_end=0.012,
+ num_timesteps_cond=1,
+ log_every_t=200,
+ timesteps=1000,
+ first_stage_key='jpg',
+ cond_stage_key='txt',
+ image_size=64,
+ channels=4,
+ cond_stage_trainable=False,
+ conditioning_key='crossattn',
+ monitor='val/loss_simple_ema',
+ scale_factor=0.18215,
+ use_ema=False,
+ scheduler_config=dict(
+ target='ldm.lr_scheduler.LambdaLinearScheduler',
+ params=dict(
+ warm_up_steps=[10000],
+ cycle_lengths=[10000000000000],
+ f_start=[1e-06],
+ f_max=[1.0],
+ f_min=[1.0])),
+ unet_config=dict(
+ target='ldm.modules.diffusionmodules.openaimodel.UNetModel',
+ params=dict(
+ image_size=32,
+ in_channels=4,
+ out_channels=4,
+ model_channels=320,
+ attention_resolutions=[4, 2, 1],
+ num_res_blocks=2,
+ channel_mult=[1, 2, 4, 4],
+ num_heads=8,
+ use_spatial_transformer=True,
+ transformer_depth=1,
+ context_dim=768,
+ use_checkpoint=True,
+ legacy=False)),
+ first_stage_config=dict(
+ target='ldm.models.autoencoder.AutoencoderKL',
+ params=dict(
+ embed_dim=4,
+ monitor='val/rec_loss',
+ ddconfig=dict(
+ double_z=True,
+ z_channels=4,
+ resolution=256,
+ in_channels=3,
+ out_ch=3,
+ ch=128,
+ ch_mult=[1, 2, 4, 4],
+ num_res_blocks=2,
+ attn_resolutions=[],
+ dropout=0.0),
+ lossconfig=dict(target='torch.nn.Identity'))),
+ cond_stage_config=dict(
+ target='ldm.modules.encoders.modules.AbstractEncoder')))
+
+model = dict(
+ type='DepthEstimator',
+ data_preprocessor=data_preprocessor,
+ backbone=dict(
+ type='VPD',
+ diffusion_cfg=stable_diffusion_cfg,
+ ),
+)
+
+# some of the parameters in stable-diffusion model will not be updated
+# during training
+find_unused_parameters = True
diff --git a/configs/_base_/schedules/schedule_25k.py b/configs/_base_/schedules/schedule_25k.py
new file mode 100644
index 00000000000..825e141ed12
--- /dev/null
+++ b/configs/_base_/schedules/schedule_25k.py
@@ -0,0 +1,28 @@
+# optimizer
+optimizer = dict(type='AdamW', lr=0.001, weight_decay=0.1)
+optim_wrapper = dict(type='OptimWrapper', optimizer=optimizer, clip_grad=None)
+# learning policy
+param_scheduler = [
+ dict(
+ type='LinearLR', start_factor=3e-2, begin=0, end=12000,
+ by_epoch=False),
+ dict(
+ type='PolyLRRatio',
+ eta_min_ratio=3e-2,
+ power=0.9,
+ begin=12000,
+ end=24000,
+ by_epoch=False),
+ dict(type='ConstantLR', by_epoch=False, factor=1, begin=24000, end=25000)
+]
+# training schedule for 25k
+train_cfg = dict(type='IterBasedTrainLoop', max_iters=25000, val_interval=1000)
+val_cfg = dict(type='ValLoop')
+test_cfg = dict(type='TestLoop')
+default_hooks = dict(
+ timer=dict(type='IterTimerHook'),
+ logger=dict(type='LoggerHook', interval=50, log_metric_by_epoch=False),
+ param_scheduler=dict(type='ParamSchedulerHook'),
+ checkpoint=dict(type='CheckpointHook', by_epoch=False, interval=2000),
+ sampler_seed=dict(type='DistSamplerSeedHook'),
+ visualization=dict(type='SegVisualizationHook'))
diff --git a/configs/ann/README.md b/configs/ann/README.md
index cbc1be2dc15..1281a9ee14f 100644
--- a/configs/ann/README.md
+++ b/configs/ann/README.md
@@ -26,34 +26,34 @@ The non-local module works as a particularly useful technique for semantic segme
### Cityscapes
-| Method | Backbone | Crop Size | Lr schd | Mem (GB) | Inf time (fps) | Device | mIoU | mIoU(ms+flip) | config | download |
-| ------ | -------- | --------- | ------: | -------- | -------------- | ------ | ----: | ------------: | --------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| ANN | R-50-D8 | 512x1024 | 40000 | 6 | 3.71 | V100 | 77.40 | 78.57 | [config](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/configs/ann/ann_r50-d8_4xb2-40k_cityscapes-512x1024.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/ann/ann_r50-d8_512x1024_40k_cityscapes/ann_r50-d8_512x1024_40k_cityscapes_20200605_095211-049fc292.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/ann/ann_r50-d8_512x1024_40k_cityscapes/ann_r50-d8_512x1024_40k_cityscapes_20200605_095211.log.json) |
-| ANN | R-101-D8 | 512x1024 | 40000 | 9.5 | 2.55 | V100 | 76.55 | 78.85 | [config](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/configs/ann/ann_r101-d8_4xb2-40k_cityscapes-512x1024.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/ann/ann_r101-d8_512x1024_40k_cityscapes/ann_r101-d8_512x1024_40k_cityscapes_20200605_095243-adf6eece.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/ann/ann_r101-d8_512x1024_40k_cityscapes/ann_r101-d8_512x1024_40k_cityscapes_20200605_095243.log.json) |
-| ANN | R-50-D8 | 769x769 | 40000 | 6.8 | 1.70 | V100 | 78.89 | 80.46 | [config](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/configs/ann/ann_r50-d8_4xb2-40k_cityscapes-769x769.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/ann/ann_r50-d8_769x769_40k_cityscapes/ann_r50-d8_769x769_40k_cityscapes_20200530_025712-2b46b04d.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/ann/ann_r50-d8_769x769_40k_cityscapes/ann_r50-d8_769x769_40k_cityscapes_20200530_025712.log.json) |
-| ANN | R-101-D8 | 769x769 | 40000 | 10.7 | 1.15 | V100 | 79.32 | 80.94 | [config](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/configs/ann/ann_r101-d8_4xb2-40k_cityscapes-769x769.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/ann/ann_r101-d8_769x769_40k_cityscapes/ann_r101-d8_769x769_40k_cityscapes_20200530_025720-059bff28.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/ann/ann_r101-d8_769x769_40k_cityscapes/ann_r101-d8_769x769_40k_cityscapes_20200530_025720.log.json) |
-| ANN | R-50-D8 | 512x1024 | 80000 | - | - | V100 | 77.34 | 78.65 | [config](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/configs/ann/ann_r50-d8_4xb2-80k_cityscapes-512x1024.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/ann/ann_r50-d8_512x1024_80k_cityscapes/ann_r50-d8_512x1024_80k_cityscapes_20200607_101911-5a9ad545.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/ann/ann_r50-d8_512x1024_80k_cityscapes/ann_r50-d8_512x1024_80k_cityscapes_20200607_101911.log.json) |
-| ANN | R-101-D8 | 512x1024 | 80000 | - | - | V100 | 77.14 | 78.81 | [config](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/configs/ann/ann_r101-d8_4xb2-80k_cityscapes-512x1024.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/ann/ann_r101-d8_512x1024_80k_cityscapes/ann_r101-d8_512x1024_80k_cityscapes_20200607_013728-aceccc6e.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/ann/ann_r101-d8_512x1024_80k_cityscapes/ann_r101-d8_512x1024_80k_cityscapes_20200607_013728.log.json) |
-| ANN | R-50-D8 | 769x769 | 80000 | - | - | V100 | 78.88 | 80.57 | [config](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/configs/ann/ann_r50-d8_4xb2-80k_cityscapes-769x769.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/ann/ann_r50-d8_769x769_80k_cityscapes/ann_r50-d8_769x769_80k_cityscapes_20200607_044426-cc7ff323.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/ann/ann_r50-d8_769x769_80k_cityscapes/ann_r50-d8_769x769_80k_cityscapes_20200607_044426.log.json) |
-| ANN | R-101-D8 | 769x769 | 80000 | - | - | V100 | 78.80 | 80.34 | [config](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/configs/ann/ann_r101-d8_4xb2-80k_cityscapes-769x769.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/ann/ann_r101-d8_769x769_80k_cityscapes/ann_r101-d8_769x769_80k_cityscapes_20200607_013713-a9d4be8d.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/ann/ann_r101-d8_769x769_80k_cityscapes/ann_r101-d8_769x769_80k_cityscapes_20200607_013713.log.json) |
+| Method | Backbone | Crop Size | Lr schd | Mem (GB) | Inf time (fps) | Device | mIoU | mIoU(ms+flip) | config | download |
+| ------ | -------- | --------- | ------: | -------- | -------------- | ------ | ----: | ------------: | ------------------------------------------------------------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| ANN | R-50-D8 | 512x1024 | 40000 | 6 | 3.71 | V100 | 77.40 | 78.57 | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/ann/ann_r50-d8_4xb2-40k_cityscapes-512x1024.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/ann/ann_r50-d8_512x1024_40k_cityscapes/ann_r50-d8_512x1024_40k_cityscapes_20200605_095211-049fc292.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/ann/ann_r50-d8_512x1024_40k_cityscapes/ann_r50-d8_512x1024_40k_cityscapes_20200605_095211.log.json) |
+| ANN | R-101-D8 | 512x1024 | 40000 | 9.5 | 2.55 | V100 | 76.55 | 78.85 | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/ann/ann_r101-d8_4xb2-40k_cityscapes-512x1024.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/ann/ann_r101-d8_512x1024_40k_cityscapes/ann_r101-d8_512x1024_40k_cityscapes_20200605_095243-adf6eece.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/ann/ann_r101-d8_512x1024_40k_cityscapes/ann_r101-d8_512x1024_40k_cityscapes_20200605_095243.log.json) |
+| ANN | R-50-D8 | 769x769 | 40000 | 6.8 | 1.70 | V100 | 78.89 | 80.46 | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/ann/ann_r50-d8_4xb2-40k_cityscapes-769x769.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/ann/ann_r50-d8_769x769_40k_cityscapes/ann_r50-d8_769x769_40k_cityscapes_20200530_025712-2b46b04d.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/ann/ann_r50-d8_769x769_40k_cityscapes/ann_r50-d8_769x769_40k_cityscapes_20200530_025712.log.json) |
+| ANN | R-101-D8 | 769x769 | 40000 | 10.7 | 1.15 | V100 | 79.32 | 80.94 | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/ann/ann_r101-d8_4xb2-40k_cityscapes-769x769.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/ann/ann_r101-d8_769x769_40k_cityscapes/ann_r101-d8_769x769_40k_cityscapes_20200530_025720-059bff28.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/ann/ann_r101-d8_769x769_40k_cityscapes/ann_r101-d8_769x769_40k_cityscapes_20200530_025720.log.json) |
+| ANN | R-50-D8 | 512x1024 | 80000 | - | - | V100 | 77.34 | 78.65 | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/ann/ann_r50-d8_4xb2-80k_cityscapes-512x1024.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/ann/ann_r50-d8_512x1024_80k_cityscapes/ann_r50-d8_512x1024_80k_cityscapes_20200607_101911-5a9ad545.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/ann/ann_r50-d8_512x1024_80k_cityscapes/ann_r50-d8_512x1024_80k_cityscapes_20200607_101911.log.json) |
+| ANN | R-101-D8 | 512x1024 | 80000 | - | - | V100 | 77.14 | 78.81 | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/ann/ann_r101-d8_4xb2-80k_cityscapes-512x1024.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/ann/ann_r101-d8_512x1024_80k_cityscapes/ann_r101-d8_512x1024_80k_cityscapes_20200607_013728-aceccc6e.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/ann/ann_r101-d8_512x1024_80k_cityscapes/ann_r101-d8_512x1024_80k_cityscapes_20200607_013728.log.json) |
+| ANN | R-50-D8 | 769x769 | 80000 | - | - | V100 | 78.88 | 80.57 | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/ann/ann_r50-d8_4xb2-80k_cityscapes-769x769.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/ann/ann_r50-d8_769x769_80k_cityscapes/ann_r50-d8_769x769_80k_cityscapes_20200607_044426-cc7ff323.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/ann/ann_r50-d8_769x769_80k_cityscapes/ann_r50-d8_769x769_80k_cityscapes_20200607_044426.log.json) |
+| ANN | R-101-D8 | 769x769 | 80000 | - | - | V100 | 78.80 | 80.34 | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/ann/ann_r101-d8_4xb2-80k_cityscapes-769x769.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/ann/ann_r101-d8_769x769_80k_cityscapes/ann_r101-d8_769x769_80k_cityscapes_20200607_013713-a9d4be8d.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/ann/ann_r101-d8_769x769_80k_cityscapes/ann_r101-d8_769x769_80k_cityscapes_20200607_013713.log.json) |
### ADE20K
-| Method | Backbone | Crop Size | Lr schd | Mem (GB) | Inf time (fps) | Device | mIoU | mIoU(ms+flip) | config | download |
-| ------ | -------- | --------- | ------: | -------- | -------------- | ------ | ----: | ------------: | ----------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| ANN | R-50-D8 | 512x512 | 80000 | 9.1 | 21.01 | V100 | 41.01 | 42.30 | [config](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/configs/ann/ann_r50-d8_4xb4-80k_ade20k-512x512.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/ann/ann_r50-d8_512x512_80k_ade20k/ann_r50-d8_512x512_80k_ade20k_20200615_014818-26f75e11.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/ann/ann_r50-d8_512x512_80k_ade20k/ann_r50-d8_512x512_80k_ade20k_20200615_014818.log.json) |
-| ANN | R-101-D8 | 512x512 | 80000 | 12.5 | 14.12 | V100 | 42.94 | 44.18 | [config](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/configs/ann/ann_r101-d8_4xb4-80k_ade20k-512x512.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/ann/ann_r101-d8_512x512_80k_ade20k/ann_r101-d8_512x512_80k_ade20k_20200615_014818-c0153543.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/ann/ann_r101-d8_512x512_80k_ade20k/ann_r101-d8_512x512_80k_ade20k_20200615_014818.log.json) |
-| ANN | R-50-D8 | 512x512 | 160000 | - | - | V100 | 41.74 | 42.62 | [config](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/configs/ann/ann_r50-d8_4xb4-160k_ade20k-512x512.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/ann/ann_r50-d8_512x512_160k_ade20k/ann_r50-d8_512x512_160k_ade20k_20200615_231733-892247bc.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/ann/ann_r50-d8_512x512_160k_ade20k/ann_r50-d8_512x512_160k_ade20k_20200615_231733.log.json) |
-| ANN | R-101-D8 | 512x512 | 160000 | - | - | V100 | 42.94 | 44.06 | [config](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/configs/ann/ann_r101-d8_4xb4-160k_ade20k-512x512.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/ann/ann_r101-d8_512x512_160k_ade20k/ann_r101-d8_512x512_160k_ade20k_20200615_231733-955eb1ec.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/ann/ann_r101-d8_512x512_160k_ade20k/ann_r101-d8_512x512_160k_ade20k_20200615_231733.log.json) |
+| Method | Backbone | Crop Size | Lr schd | Mem (GB) | Inf time (fps) | Device | mIoU | mIoU(ms+flip) | config | download |
+| ------ | -------- | --------- | ------: | -------- | -------------- | ------ | ----: | ------------: | -------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| ANN | R-50-D8 | 512x512 | 80000 | 9.1 | 21.01 | V100 | 41.01 | 42.30 | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/ann/ann_r50-d8_4xb4-80k_ade20k-512x512.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/ann/ann_r50-d8_512x512_80k_ade20k/ann_r50-d8_512x512_80k_ade20k_20200615_014818-26f75e11.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/ann/ann_r50-d8_512x512_80k_ade20k/ann_r50-d8_512x512_80k_ade20k_20200615_014818.log.json) |
+| ANN | R-101-D8 | 512x512 | 80000 | 12.5 | 14.12 | V100 | 42.94 | 44.18 | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/ann/ann_r101-d8_4xb4-80k_ade20k-512x512.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/ann/ann_r101-d8_512x512_80k_ade20k/ann_r101-d8_512x512_80k_ade20k_20200615_014818-c0153543.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/ann/ann_r101-d8_512x512_80k_ade20k/ann_r101-d8_512x512_80k_ade20k_20200615_014818.log.json) |
+| ANN | R-50-D8 | 512x512 | 160000 | - | - | V100 | 41.74 | 42.62 | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/ann/ann_r50-d8_4xb4-160k_ade20k-512x512.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/ann/ann_r50-d8_512x512_160k_ade20k/ann_r50-d8_512x512_160k_ade20k_20200615_231733-892247bc.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/ann/ann_r50-d8_512x512_160k_ade20k/ann_r50-d8_512x512_160k_ade20k_20200615_231733.log.json) |
+| ANN | R-101-D8 | 512x512 | 160000 | - | - | V100 | 42.94 | 44.06 | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/ann/ann_r101-d8_4xb4-160k_ade20k-512x512.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/ann/ann_r101-d8_512x512_160k_ade20k/ann_r101-d8_512x512_160k_ade20k_20200615_231733-955eb1ec.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/ann/ann_r101-d8_512x512_160k_ade20k/ann_r101-d8_512x512_160k_ade20k_20200615_231733.log.json) |
### Pascal VOC 2012 + Aug
-| Method | Backbone | Crop Size | Lr schd | Mem (GB) | Inf time (fps) | Device | mIoU | mIoU(ms+flip) | config | download |
-| ------ | -------- | --------- | ------: | -------- | -------------- | ------ | ----: | ------------: | ------------------------------------------------------------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| ANN | R-50-D8 | 512x512 | 20000 | 6 | 20.92 | V100 | 74.86 | 76.13 | [config](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/configs/ann/ann_r50-d8_4xb4-20k_voc12aug-512x512.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/ann/ann_r50-d8_512x512_20k_voc12aug/ann_r50-d8_512x512_20k_voc12aug_20200617_222246-dfcb1c62.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/ann/ann_r50-d8_512x512_20k_voc12aug/ann_r50-d8_512x512_20k_voc12aug_20200617_222246.log.json) |
-| ANN | R-101-D8 | 512x512 | 20000 | 9.5 | 13.94 | V100 | 77.47 | 78.70 | [config](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/configs/ann/ann_r101-d8_4xb4-20k_voc12aug-512x512.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/ann/ann_r101-d8_512x512_20k_voc12aug/ann_r101-d8_512x512_20k_voc12aug_20200617_222246-2fad0042.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/ann/ann_r101-d8_512x512_20k_voc12aug/ann_r101-d8_512x512_20k_voc12aug_20200617_222246.log.json) |
-| ANN | R-50-D8 | 512x512 | 40000 | - | - | V100 | 76.56 | 77.51 | [config](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/configs/ann/ann_r50-d8_4xb4-40k_voc12aug-512x512.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/ann/ann_r50-d8_512x512_40k_voc12aug/ann_r50-d8_512x512_40k_voc12aug_20200613_231314-b5dac322.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/ann/ann_r50-d8_512x512_40k_voc12aug/ann_r50-d8_512x512_40k_voc12aug_20200613_231314.log.json) |
-| ANN | R-101-D8 | 512x512 | 40000 | - | - | V100 | 76.70 | 78.06 | [config](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/configs/ann/ann_r101-d8_4xb4-40k_voc12aug-512x512.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/ann/ann_r101-d8_512x512_40k_voc12aug/ann_r101-d8_512x512_40k_voc12aug_20200613_231314-bd205bbe.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/ann/ann_r101-d8_512x512_40k_voc12aug/ann_r101-d8_512x512_40k_voc12aug_20200613_231314.log.json) |
+| Method | Backbone | Crop Size | Lr schd | Mem (GB) | Inf time (fps) | Device | mIoU | mIoU(ms+flip) | config | download |
+| ------ | -------- | --------- | ------: | -------- | -------------- | ------ | ----: | ------------: | --------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| ANN | R-50-D8 | 512x512 | 20000 | 6 | 20.92 | V100 | 74.86 | 76.13 | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/ann/ann_r50-d8_4xb4-20k_voc12aug-512x512.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/ann/ann_r50-d8_512x512_20k_voc12aug/ann_r50-d8_512x512_20k_voc12aug_20200617_222246-dfcb1c62.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/ann/ann_r50-d8_512x512_20k_voc12aug/ann_r50-d8_512x512_20k_voc12aug_20200617_222246.log.json) |
+| ANN | R-101-D8 | 512x512 | 20000 | 9.5 | 13.94 | V100 | 77.47 | 78.70 | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/ann/ann_r101-d8_4xb4-20k_voc12aug-512x512.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/ann/ann_r101-d8_512x512_20k_voc12aug/ann_r101-d8_512x512_20k_voc12aug_20200617_222246-2fad0042.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/ann/ann_r101-d8_512x512_20k_voc12aug/ann_r101-d8_512x512_20k_voc12aug_20200617_222246.log.json) |
+| ANN | R-50-D8 | 512x512 | 40000 | - | - | V100 | 76.56 | 77.51 | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/ann/ann_r50-d8_4xb4-40k_voc12aug-512x512.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/ann/ann_r50-d8_512x512_40k_voc12aug/ann_r50-d8_512x512_40k_voc12aug_20200613_231314-b5dac322.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/ann/ann_r50-d8_512x512_40k_voc12aug/ann_r50-d8_512x512_40k_voc12aug_20200613_231314.log.json) |
+| ANN | R-101-D8 | 512x512 | 40000 | - | - | V100 | 76.70 | 78.06 | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/ann/ann_r101-d8_4xb4-40k_voc12aug-512x512.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/ann/ann_r101-d8_512x512_40k_voc12aug/ann_r101-d8_512x512_40k_voc12aug_20200613_231314-bd205bbe.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/ann/ann_r101-d8_512x512_40k_voc12aug/ann_r101-d8_512x512_40k_voc12aug_20200613_231314.log.json) |
## Citation
diff --git a/configs/apcnet/README.md b/configs/apcnet/README.md
index 0d7f1d66149..9104f3c87f5 100644
--- a/configs/apcnet/README.md
+++ b/configs/apcnet/README.md
@@ -26,25 +26,25 @@ Recent studies witnessed that context features can significantly improve the per
### Cityscapes
-| Method | Backbone | Crop Size | Lr schd | Mem (GB) | Inf time (fps) | Device | mIoU | mIoU(ms+flip) | config | download |
-| ------ | -------- | --------- | ------: | -------- | -------------- | ------ | ----: | ------------: | --------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| APCNet | R-50-D8 | 512x1024 | 40000 | 7.7 | 3.57 | V100 | 78.02 | 79.26 | [config](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/configs/apcnet/apcnet_r50-d8_4xb2-40k_cityscapes-512x1024.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/apcnet/apcnet_r50-d8_512x1024_40k_cityscapes/apcnet_r50-d8_512x1024_40k_cityscapes_20201214_115717-5e88fa33.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/apcnet/apcnet_r50-d8_512x1024_40k_cityscapes/apcnet_r50-d8_512x1024_40k_cityscapes-20201214_115717.log.json) |
-| APCNet | R-101-D8 | 512x1024 | 40000 | 11.2 | 2.15 | V100 | 79.08 | 80.34 | [config](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/configs/apcnet/apcnet_r101-d8_4xb2-40k_cityscapes-512x1024.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/apcnet/apcnet_r101-d8_512x1024_40k_cityscapes/apcnet_r101-d8_512x1024_40k_cityscapes_20201214_115716-abc9d111.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/apcnet/apcnet_r101-d8_512x1024_40k_cityscapes/apcnet_r101-d8_512x1024_40k_cityscapes-20201214_115716.log.json) |
-| APCNet | R-50-D8 | 769x769 | 40000 | 8.7 | 1.52 | V100 | 77.89 | 79.75 | [config](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/configs/apcnet/apcnet_r50-d8_4xb2-40k_cityscapes-769x769.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/apcnet/apcnet_r50-d8_769x769_40k_cityscapes/apcnet_r50-d8_769x769_40k_cityscapes_20201214_115717-2a2628d7.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/apcnet/apcnet_r50-d8_769x769_40k_cityscapes/apcnet_r50-d8_769x769_40k_cityscapes-20201214_115717.log.json) |
-| APCNet | R-101-D8 | 769x769 | 40000 | 12.7 | 1.03 | V100 | 77.96 | 79.24 | [config](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/configs/apcnet/apcnet_r101-d8_4xb2-40k_cityscapes-769x769.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/apcnet/apcnet_r101-d8_769x769_40k_cityscapes/apcnet_r101-d8_769x769_40k_cityscapes_20201214_115718-b650de90.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/apcnet/apcnet_r101-d8_769x769_40k_cityscapes/apcnet_r101-d8_769x769_40k_cityscapes-20201214_115718.log.json) |
-| APCNet | R-50-D8 | 512x1024 | 80000 | - | - | V100 | 78.96 | 79.94 | [config](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/configs/apcnet/apcnet_r50-d8_4xb2-80k_cityscapes-512x1024.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/apcnet/apcnet_r50-d8_512x1024_80k_cityscapes/apcnet_r50-d8_512x1024_80k_cityscapes_20201214_115716-987f51e3.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/apcnet/apcnet_r50-d8_512x1024_80k_cityscapes/apcnet_r50-d8_512x1024_80k_cityscapes-20201214_115716.log.json) |
-| APCNet | R-101-D8 | 512x1024 | 80000 | - | - | V100 | 79.64 | 80.61 | [config](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/configs/apcnet/apcnet_r101-d8_4xb2-80k_cityscapes-512x1024.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/apcnet/apcnet_r101-d8_512x1024_80k_cityscapes/apcnet_r101-d8_512x1024_80k_cityscapes_20201214_115705-b1ff208a.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/apcnet/apcnet_r101-d8_512x1024_80k_cityscapes/apcnet_r101-d8_512x1024_80k_cityscapes-20201214_115705.log.json) |
-| APCNet | R-50-D8 | 769x769 | 80000 | - | - | V100 | 78.79 | 80.35 | [config](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/configs/apcnet/apcnet_r50-d8_4xb2-80k_cityscapes-769x769.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/apcnet/apcnet_r50-d8_769x769_80k_cityscapes/apcnet_r50-d8_769x769_80k_cityscapes_20201214_115718-7ea9fa12.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/apcnet/apcnet_r50-d8_769x769_80k_cityscapes/apcnet_r50-d8_769x769_80k_cityscapes-20201214_115718.log.json) |
-| APCNet | R-101-D8 | 769x769 | 80000 | - | - | V100 | 78.45 | 79.91 | [config](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/configs/apcnet/apcnet_r101-d8_4xb2-80k_cityscapes-769x769.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/apcnet/apcnet_r101-d8_769x769_80k_cityscapes/apcnet_r101-d8_769x769_80k_cityscapes_20201214_115716-a7fbc2ab.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/apcnet/apcnet_r101-d8_769x769_80k_cityscapes/apcnet_r101-d8_769x769_80k_cityscapes-20201214_115716.log.json) |
+| Method | Backbone | Crop Size | Lr schd | Mem (GB) | Inf time (fps) | Device | mIoU | mIoU(ms+flip) | config | download |
+| ------ | -------- | --------- | ------: | -------- | -------------- | ------ | ----: | ------------: | ------------------------------------------------------------------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| APCNet | R-50-D8 | 512x1024 | 40000 | 7.7 | 3.57 | V100 | 78.02 | 79.26 | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/apcnet/apcnet_r50-d8_4xb2-40k_cityscapes-512x1024.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/apcnet/apcnet_r50-d8_512x1024_40k_cityscapes/apcnet_r50-d8_512x1024_40k_cityscapes_20201214_115717-5e88fa33.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/apcnet/apcnet_r50-d8_512x1024_40k_cityscapes/apcnet_r50-d8_512x1024_40k_cityscapes-20201214_115717.log.json) |
+| APCNet | R-101-D8 | 512x1024 | 40000 | 11.2 | 2.15 | V100 | 79.08 | 80.34 | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/apcnet/apcnet_r101-d8_4xb2-40k_cityscapes-512x1024.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/apcnet/apcnet_r101-d8_512x1024_40k_cityscapes/apcnet_r101-d8_512x1024_40k_cityscapes_20201214_115716-abc9d111.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/apcnet/apcnet_r101-d8_512x1024_40k_cityscapes/apcnet_r101-d8_512x1024_40k_cityscapes-20201214_115716.log.json) |
+| APCNet | R-50-D8 | 769x769 | 40000 | 8.7 | 1.52 | V100 | 77.89 | 79.75 | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/apcnet/apcnet_r50-d8_4xb2-40k_cityscapes-769x769.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/apcnet/apcnet_r50-d8_769x769_40k_cityscapes/apcnet_r50-d8_769x769_40k_cityscapes_20201214_115717-2a2628d7.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/apcnet/apcnet_r50-d8_769x769_40k_cityscapes/apcnet_r50-d8_769x769_40k_cityscapes-20201214_115717.log.json) |
+| APCNet | R-101-D8 | 769x769 | 40000 | 12.7 | 1.03 | V100 | 77.96 | 79.24 | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/apcnet/apcnet_r101-d8_4xb2-40k_cityscapes-769x769.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/apcnet/apcnet_r101-d8_769x769_40k_cityscapes/apcnet_r101-d8_769x769_40k_cityscapes_20201214_115718-b650de90.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/apcnet/apcnet_r101-d8_769x769_40k_cityscapes/apcnet_r101-d8_769x769_40k_cityscapes-20201214_115718.log.json) |
+| APCNet | R-50-D8 | 512x1024 | 80000 | - | - | V100 | 78.96 | 79.94 | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/apcnet/apcnet_r50-d8_4xb2-80k_cityscapes-512x1024.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/apcnet/apcnet_r50-d8_512x1024_80k_cityscapes/apcnet_r50-d8_512x1024_80k_cityscapes_20201214_115716-987f51e3.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/apcnet/apcnet_r50-d8_512x1024_80k_cityscapes/apcnet_r50-d8_512x1024_80k_cityscapes-20201214_115716.log.json) |
+| APCNet | R-101-D8 | 512x1024 | 80000 | - | - | V100 | 79.64 | 80.61 | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/apcnet/apcnet_r101-d8_4xb2-80k_cityscapes-512x1024.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/apcnet/apcnet_r101-d8_512x1024_80k_cityscapes/apcnet_r101-d8_512x1024_80k_cityscapes_20201214_115705-b1ff208a.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/apcnet/apcnet_r101-d8_512x1024_80k_cityscapes/apcnet_r101-d8_512x1024_80k_cityscapes-20201214_115705.log.json) |
+| APCNet | R-50-D8 | 769x769 | 80000 | - | - | V100 | 78.79 | 80.35 | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/apcnet/apcnet_r50-d8_4xb2-80k_cityscapes-769x769.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/apcnet/apcnet_r50-d8_769x769_80k_cityscapes/apcnet_r50-d8_769x769_80k_cityscapes_20201214_115718-7ea9fa12.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/apcnet/apcnet_r50-d8_769x769_80k_cityscapes/apcnet_r50-d8_769x769_80k_cityscapes-20201214_115718.log.json) |
+| APCNet | R-101-D8 | 769x769 | 80000 | - | - | V100 | 78.45 | 79.91 | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/apcnet/apcnet_r101-d8_4xb2-80k_cityscapes-769x769.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/apcnet/apcnet_r101-d8_769x769_80k_cityscapes/apcnet_r101-d8_769x769_80k_cityscapes_20201214_115716-a7fbc2ab.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/apcnet/apcnet_r101-d8_769x769_80k_cityscapes/apcnet_r101-d8_769x769_80k_cityscapes-20201214_115716.log.json) |
### ADE20K
-| Method | Backbone | Crop Size | Lr schd | Mem (GB) | Inf time (fps) | Device | mIoU | mIoU(ms+flip) | config | download |
-| ------ | -------- | --------- | ------: | -------- | -------------- | ------ | ----: | ------------: | ----------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| APCNet | R-50-D8 | 512x512 | 80000 | 10.1 | 19.61 | V100 | 42.20 | 43.30 | [config](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/configs/apcnet/apcnet_r50-d8_4xb4-80k_ade20k-512x512.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/apcnet/apcnet_r50-d8_512x512_80k_ade20k/apcnet_r50-d8_512x512_80k_ade20k_20201214_115705-a8626293.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/apcnet/apcnet_r50-d8_512x512_80k_ade20k/apcnet_r50-d8_512x512_80k_ade20k-20201214_115705.log.json) |
-| APCNet | R-101-D8 | 512x512 | 80000 | 13.6 | 13.10 | V100 | 45.54 | 46.65 | [config](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/configs/apcnet/apcnet_r101-d8_4xb4-80k_ade20k-512x512.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/apcnet/apcnet_r101-d8_512x512_80k_ade20k/apcnet_r101-d8_512x512_80k_ade20k_20201214_115704-c656c3fb.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/apcnet/apcnet_r101-d8_512x512_80k_ade20k/apcnet_r101-d8_512x512_80k_ade20k-20201214_115704.log.json) |
-| APCNet | R-50-D8 | 512x512 | 160000 | - | - | V100 | 43.40 | 43.94 | [config](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/configs/apcnet/apcnet_r50-d8_4xb4-160k_ade20k-512x512.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/apcnet/apcnet_r50-d8_512x512_160k_ade20k/apcnet_r50-d8_512x512_160k_ade20k_20201214_115706-25fb92c2.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/apcnet/apcnet_r50-d8_512x512_160k_ade20k/apcnet_r50-d8_512x512_160k_ade20k-20201214_115706.log.json) |
-| APCNet | R-101-D8 | 512x512 | 160000 | - | - | V100 | 45.41 | 46.63 | [config](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/configs/apcnet/apcnet_r101-d8_4xb4-160k_ade20k-512x512.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/apcnet/apcnet_r101-d8_512x512_160k_ade20k/apcnet_r101-d8_512x512_160k_ade20k_20201214_115705-73f9a8d7.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/apcnet/apcnet_r101-d8_512x512_160k_ade20k/apcnet_r101-d8_512x512_160k_ade20k-20201214_115705.log.json) |
+| Method | Backbone | Crop Size | Lr schd | Mem (GB) | Inf time (fps) | Device | mIoU | mIoU(ms+flip) | config | download |
+| ------ | -------- | --------- | ------: | -------- | -------------- | ------ | ----: | ------------: | -------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| APCNet | R-50-D8 | 512x512 | 80000 | 10.1 | 19.61 | V100 | 42.20 | 43.30 | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/apcnet/apcnet_r50-d8_4xb4-80k_ade20k-512x512.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/apcnet/apcnet_r50-d8_512x512_80k_ade20k/apcnet_r50-d8_512x512_80k_ade20k_20201214_115705-a8626293.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/apcnet/apcnet_r50-d8_512x512_80k_ade20k/apcnet_r50-d8_512x512_80k_ade20k-20201214_115705.log.json) |
+| APCNet | R-101-D8 | 512x512 | 80000 | 13.6 | 13.10 | V100 | 45.54 | 46.65 | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/apcnet/apcnet_r101-d8_4xb4-80k_ade20k-512x512.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/apcnet/apcnet_r101-d8_512x512_80k_ade20k/apcnet_r101-d8_512x512_80k_ade20k_20201214_115704-c656c3fb.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/apcnet/apcnet_r101-d8_512x512_80k_ade20k/apcnet_r101-d8_512x512_80k_ade20k-20201214_115704.log.json) |
+| APCNet | R-50-D8 | 512x512 | 160000 | - | - | V100 | 43.40 | 43.94 | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/apcnet/apcnet_r50-d8_4xb4-160k_ade20k-512x512.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/apcnet/apcnet_r50-d8_512x512_160k_ade20k/apcnet_r50-d8_512x512_160k_ade20k_20201214_115706-25fb92c2.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/apcnet/apcnet_r50-d8_512x512_160k_ade20k/apcnet_r50-d8_512x512_160k_ade20k-20201214_115706.log.json) |
+| APCNet | R-101-D8 | 512x512 | 160000 | - | - | V100 | 45.41 | 46.63 | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/apcnet/apcnet_r101-d8_4xb4-160k_ade20k-512x512.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/apcnet/apcnet_r101-d8_512x512_160k_ade20k/apcnet_r101-d8_512x512_160k_ade20k_20201214_115705-73f9a8d7.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/apcnet/apcnet_r101-d8_512x512_160k_ade20k/apcnet_r101-d8_512x512_160k_ade20k-20201214_115705.log.json) |
## Citation
diff --git a/configs/beit/README.md b/configs/beit/README.md
index 8e34e294101..b005c88c501 100644
--- a/configs/beit/README.md
+++ b/configs/beit/README.md
@@ -67,10 +67,10 @@ upernet_beit-large_fp16_8x1_640x640_160k_ade20k-8fc0dd5d.pth $GPUS --eval mIoU
### ADE20K
-| Method | Backbone | Crop Size | pretrain | pretrain img size | Batch Size | Lr schd | Mem (GB) | Inf time (fps) | Device | mIoU | mIoU(ms+flip) | config | download |
-| ------- | -------- | --------- | ------------ | ----------------- | ---------- | ------- | -------- | -------------- | ------ | ----- | ------------: | ----------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| UPerNet | BEiT-B | 640x640 | ImageNet-22K | 224x224 | 16 | 160000 | 15.88 | 2.00 | V100 | 53.08 | 53.84 | [config](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/configs/beit/beit-base_upernet_8xb2-160k_ade20k-640x640.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/beit/upernet_beit-base_8x2_640x640_160k_ade20k/upernet_beit-base_8x2_640x640_160k_ade20k-eead221d.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/beit/upernet_beit-base_8x2_640x640_160k_ade20k/upernet_beit-base_8x2_640x640_160k_ade20k.log.json) |
-| UPerNet | BEiT-L | 640x640 | ImageNet-22K | 224x224 | 8 | 320000 | 22.64 | 0.96 | V100 | 56.33 | 56.84 | [config](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/configs/beit/beit-large_upernet_8xb1-amp-160k_ade20k-640x640.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/beit/upernet_beit-large_fp16_8x1_640x640_160k_ade20k/upernet_beit-large_fp16_8x1_640x640_160k_ade20k-8fc0dd5d.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/beit/upernet_beit-large_fp16_8x1_640x640_160k_ade20k/upernet_beit-large_fp16_8x1_640x640_160k_ade20k.log.json) |
+| Method | Backbone | Crop Size | pretrain | pretrain img size | Batch Size | Lr schd | Mem (GB) | Inf time (fps) | Device | mIoU | mIoU(ms+flip) | config | download |
+| ------- | -------- | --------- | ------------ | ----------------- | ---------- | ------- | -------- | -------------- | ------ | ----- | ------------: | -------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| UPerNet | BEiT-B | 640x640 | ImageNet-22K | 224x224 | 16 | 160000 | 15.88 | 2.00 | V100 | 53.08 | 53.84 | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/beit/beit-base_upernet_8xb2-160k_ade20k-640x640.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/beit/upernet_beit-base_8x2_640x640_160k_ade20k/upernet_beit-base_8x2_640x640_160k_ade20k-eead221d.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/beit/upernet_beit-base_8x2_640x640_160k_ade20k/upernet_beit-base_8x2_640x640_160k_ade20k.log.json) |
+| UPerNet | BEiT-L | 640x640 | ImageNet-22K | 224x224 | 8 | 320000 | 22.64 | 0.96 | V100 | 56.33 | 56.84 | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/beit/beit-large_upernet_8xb1-amp-160k_ade20k-640x640.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/beit/upernet_beit-large_fp16_8x1_640x640_160k_ade20k/upernet_beit-large_fp16_8x1_640x640_160k_ade20k-8fc0dd5d.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/beit/upernet_beit-large_fp16_8x1_640x640_160k_ade20k/upernet_beit-large_fp16_8x1_640x640_160k_ade20k.log.json) |
## Citation
diff --git a/configs/bisenetv1/README.md b/configs/bisenetv1/README.md
index c895fdfe63f..a5058957f0c 100644
--- a/configs/bisenetv1/README.md
+++ b/configs/bisenetv1/README.md
@@ -26,24 +26,24 @@ Semantic segmentation requires both rich spatial information and sizeable recept
### Cityscapes
-| Method | Backbone | Crop Size | Lr schd | Mem (GB) | Inf time (fps) | Device | mIoU | mIoU(ms+flip) | config | download |
-| --------- | ---------------------- | --------- | ------: | -------- | -------------- | ------ | ----: | ------------- | -------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| BiSeNetV1 | R-18-D32 (No Pretrain) | 1024x1024 | 160000 | 5.69 | 31.77 | V100 | 74.44 | 77.05 | [config](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/configs/bisenetv1/bisenetv1_r18-d32_4xb4-160k_cityscapes-1024x1024.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/bisenetv1/bisenetv1_r18-d32_4x4_1024x1024_160k_cityscapes/bisenetv1_r18-d32_4x4_1024x1024_160k_cityscapes_20210922_172239-c55e78e2.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/bisenetv1/bisenetv1_r18-d32_4x4_1024x1024_160k_cityscapes/bisenetv1_r18-d32_4x4_1024x1024_160k_cityscapes_20210922_172239.log.json) |
-| BiSeNetV1 | R-18-D32 | 1024x1024 | 160000 | 5.69 | 31.77 | V100 | 74.37 | 76.91 | [config](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/configs/bisenetv1/bisenetv1_r18-d32-in1k-pre_4xb4-160k_cityscapes-1024x1024.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/bisenetv1/bisenetv1_r18-d32_in1k-pre_4x4_1024x1024_160k_cityscapes/bisenetv1_r18-d32_in1k-pre_4x4_1024x1024_160k_cityscapes_20210905_220251-8ba80eff.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/bisenetv1/bisenetv1_r18-d32_in1k-pre_4x4_1024x1024_160k_cityscapes/bisenetv1_r18-d32_in1k-pre_4x4_1024x1024_160k_cityscapes_20210905_220251.log.json) |
-| BiSeNetV1 | R-18-D32 (4x8) | 1024x1024 | 160000 | 11.17 | 31.77 | V100 | 75.16 | 77.24 | [config](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/configs/bisenetv1/bisenetv1_r18-d32-in1k-pre_4xb8-160k_cityscapes-1024x1024.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/bisenetv1/bisenetv1_r18-d32_in1k-pre_4x8_1024x1024_160k_cityscapes/bisenetv1_r18-d32_in1k-pre_4x8_1024x1024_160k_cityscapes_20210905_220322-bb8db75f.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/bisenetv1/bisenetv1_r18-d32_in1k-pre_4x8_1024x1024_160k_cityscapes/bisenetv1_r18-d32_in1k-pre_4x8_1024x1024_160k_cityscapes_20210905_220322.log.json) |
-| BiSeNetV1 | R-50-D32 (No Pretrain) | 1024x1024 | 160000 | 15.39 | 7.71 | V100 | 76.92 | 78.87 | [config](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/configs/bisenetv1/bisenetv1_r50-d32_4xb4-160k_cityscapes-1024x1024.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/bisenetv1/bisenetv1_r50-d32_4x4_1024x1024_160k_cityscapes/bisenetv1_r50-d32_4x4_1024x1024_160k_cityscapes_20210923_222639-7b28a2a6.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/bisenetv1/bisenetv1_r50-d32_4x4_1024x1024_160k_cityscapes/bisenetv1_r50-d32_4x4_1024x1024_160k_cityscapes_20210923_222639.log.json) |
-| BiSeNetV1 | R-50-D32 | 1024x1024 | 160000 | 15.39 | 7.71 | V100 | 77.68 | 79.57 | [config](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/configs/bisenetv1/bisenetv1_r50-d32-in1k-pre_4xb4-160k_cityscapes-1024x1024.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/bisenetv1/bisenetv1_r50-d32_in1k-pre_4x4_1024x1024_160k_cityscapes/bisenetv1_r50-d32_in1k-pre_4x4_1024x1024_160k_cityscapes_20210917_234628-8b304447.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/bisenetv1/bisenetv1_r50-d32_in1k-pre_4x4_1024x1024_160k_cityscapes/bisenetv1_r50-d32_in1k-pre_4x4_1024x1024_160k_cityscapes_20210917_234628.log.json) |
+| Method | Backbone | Crop Size | Lr schd | Mem (GB) | Inf time (fps) | Device | mIoU | mIoU(ms+flip) | config | download |
+| --------- | ---------------------- | --------- | ------: | -------- | -------------- | ------ | ----: | ------------- | ----------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| BiSeNetV1 | R-18-D32 (No Pretrain) | 1024x1024 | 160000 | 5.69 | 31.77 | V100 | 74.44 | 77.05 | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/bisenetv1/bisenetv1_r18-d32_4xb4-160k_cityscapes-1024x1024.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/bisenetv1/bisenetv1_r18-d32_4x4_1024x1024_160k_cityscapes/bisenetv1_r18-d32_4x4_1024x1024_160k_cityscapes_20210922_172239-c55e78e2.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/bisenetv1/bisenetv1_r18-d32_4x4_1024x1024_160k_cityscapes/bisenetv1_r18-d32_4x4_1024x1024_160k_cityscapes_20210922_172239.log.json) |
+| BiSeNetV1 | R-18-D32 | 1024x1024 | 160000 | 5.69 | 31.77 | V100 | 74.37 | 76.91 | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/bisenetv1/bisenetv1_r18-d32-in1k-pre_4xb4-160k_cityscapes-1024x1024.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/bisenetv1/bisenetv1_r18-d32_in1k-pre_4x4_1024x1024_160k_cityscapes/bisenetv1_r18-d32_in1k-pre_4x4_1024x1024_160k_cityscapes_20210905_220251-8ba80eff.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/bisenetv1/bisenetv1_r18-d32_in1k-pre_4x4_1024x1024_160k_cityscapes/bisenetv1_r18-d32_in1k-pre_4x4_1024x1024_160k_cityscapes_20210905_220251.log.json) |
+| BiSeNetV1 | R-18-D32 (4x8) | 1024x1024 | 160000 | 11.17 | 31.77 | V100 | 75.16 | 77.24 | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/bisenetv1/bisenetv1_r18-d32-in1k-pre_4xb8-160k_cityscapes-1024x1024.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/bisenetv1/bisenetv1_r18-d32_in1k-pre_4x8_1024x1024_160k_cityscapes/bisenetv1_r18-d32_in1k-pre_4x8_1024x1024_160k_cityscapes_20210905_220322-bb8db75f.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/bisenetv1/bisenetv1_r18-d32_in1k-pre_4x8_1024x1024_160k_cityscapes/bisenetv1_r18-d32_in1k-pre_4x8_1024x1024_160k_cityscapes_20210905_220322.log.json) |
+| BiSeNetV1 | R-50-D32 (No Pretrain) | 1024x1024 | 160000 | 15.39 | 7.71 | V100 | 76.92 | 78.87 | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/bisenetv1/bisenetv1_r50-d32_4xb4-160k_cityscapes-1024x1024.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/bisenetv1/bisenetv1_r50-d32_4x4_1024x1024_160k_cityscapes/bisenetv1_r50-d32_4x4_1024x1024_160k_cityscapes_20210923_222639-7b28a2a6.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/bisenetv1/bisenetv1_r50-d32_4x4_1024x1024_160k_cityscapes/bisenetv1_r50-d32_4x4_1024x1024_160k_cityscapes_20210923_222639.log.json) |
+| BiSeNetV1 | R-50-D32 | 1024x1024 | 160000 | 15.39 | 7.71 | V100 | 77.68 | 79.57 | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/bisenetv1/bisenetv1_r50-d32-in1k-pre_4xb4-160k_cityscapes-1024x1024.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/bisenetv1/bisenetv1_r50-d32_in1k-pre_4x4_1024x1024_160k_cityscapes/bisenetv1_r50-d32_in1k-pre_4x4_1024x1024_160k_cityscapes_20210917_234628-8b304447.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/bisenetv1/bisenetv1_r50-d32_in1k-pre_4x4_1024x1024_160k_cityscapes/bisenetv1_r50-d32_in1k-pre_4x4_1024x1024_160k_cityscapes_20210917_234628.log.json) |
### COCO-Stuff 164k
-| Method | Backbone | Crop Size | Lr schd | Mem (GB) | Inf time (fps) | Device | mIoU | mIoU(ms+flip) | config | download |
-| --------- | ----------------------- | --------- | ------: | -------- | -------------- | ------ | ----: | ------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
-| BiSeNetV1 | R-18-D32 (No Pretrain) | 512x512 | 160000 | - | - | V100 | 25.45 | 26.15 | [config](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/configs/bisenetv1/bisenetv1_r18-d32_4xb4-160k_coco-stuff164k-512x512.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/bisenetv1/bisenetv1_r18-d32_lr5e-3_4x4_512x512_160k_coco-stuff164k/bisenetv1_r18-d32_lr5e-3_4x4_512x512_160k_coco-stuff164k_20211022_054328-046aa2f2.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/bisenetv1/bisenetv1_r18-d32_lr5e-3_4x4_512x512_160k_coco-stuff164k/bisenetv1_r18-d32_lr5e-3_4x4_512x512_160k_coco-stuff164k_20211022_054328.log.json) |
-| BiSeNetV1 | R-18-D32 | 512x512 | 160000 | 6.33 | 74.24 | V100 | 28.55 | 29.26 | [config](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/configs/bisenetv1/bisenetv1_r18-d32-in1k-pre_4xb4-160k_coco-stuff164k-512x512.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/bisenetv1/bisenetv1_r18-d32_in1k-pre_lr5e-3_4x4_512x512_160k_coco-stuff164k/bisenetv1_r18-d32_in1k-pre_lr5e-3_4x4_512x512_160k_coco-stuff164k_20211023_013100-f700dbf7.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/bisenetv1/bisenetv1_r18-d32_in1k-pre_lr5e-3_4x4_512x512_160k_coco-stuff164k/bisenetv1_r18-d32_in1k-pre_lr5e-3_4x4_512x512_160k_coco-stuff164k_20211023_013100.log.json) |
-| BiSeNetV1 | R-50-D32 (No Pretrain) | 512x512 | 160000 | - | - | V100 | 29.82 | 30.33 | [config](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/configs/bisenetv1/bisenetv1_r50-d32_4xb4-160k_coco-stuff164k-512x512.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/bisenetv1/bisenetv1_r50-d32_lr5e-3_4x4_512x512_160k_coco-stuff164k/bisenetv1_r50-d32_lr5e-3_4x4_512x512_160k_coco-stuff164k_20211101_040616-d2bb0df4.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/bisenetv1/bisenetv1_r50-d32_lr5e-3_4x4_512x512_160k_coco-stuff164k/bisenetv1_r50-d32_lr5e-3_4x4_512x512_160k_coco-stuff164k_20211101_040616.log.json) |
-| BiSeNetV1 | R-50-D32 | 512x512 | 160000 | 9.28 | 32.60 | V100 | 34.88 | 35.37 | [config](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/configs/bisenetv1/bisenetv1_r50-d32-in1k-pre_4xb4-160k_coco-stuff164k-512x512.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/bisenetv1/bisenetv1_r50-d32_in1k-pre_lr5e-3_4x4_512x512_160k_coco-stuff164k/bisenetv1_r50-d32_in1k-pre_lr5e-3_4x4_512x512_160k_coco-stuff164k_20211101_181932-66747911.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/bisenetv1/bisenetv1_r50-d32_in1k-pre_lr5e-3_4x4_512x512_160k_coco-stuff164k/bisenetv1_r50-d32_in1k-pre_lr5e-3_4x4_512x512_160k_coco-stuff164k_20211101_181932.log.json) |
-| BiSeNetV1 | R-101-D32 (No Pretrain) | 512x512 | 160000 | - | - | V100 | 31.14 | 31.76 | [config](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/configs/bisenetv1/bisenetv1_r50-d32-in1k-pre_4xb4-160k_coco-stuff164k-512x512.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/bisenetv1/bisenetv1_r101-d32_lr5e-3_4x4_512x512_160k_coco-stuff164k/bisenetv1_r101-d32_lr5e-3_4x4_512x512_160k_coco-stuff164k_20211102_164147-c6b32c3b.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/bisenetv1/bisenetv1_r101-d32_lr5e-3_4x4_512x512_160k_coco-stuff164k/bisenetv1_r101-d32_lr5e-3_4x4_512x512_160k_coco-stuff164k_20211102_164147.log.json) |
-| BiSeNetV1 | R-101-D32 | 512x512 | 160000 | 10.36 | 25.25 | V100 | 37.38 | 37.99 | [config](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/configs/bisenetv1/bisenetv1_r101-d32-in1k-pre_4xb4-160k_coco-stuff164k-512x512.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/bisenetv1/bisenetv1_r101-d32_in1k-pre_lr5e-3_4x4_512x512_160k_coco-stuff164k/bisenetv1_r101-d32_in1k-pre_lr5e-3_4x4_512x512_160k_coco-stuff164k_20211101_225220-28c8f092.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/bisenetv1/bisenetv1_r101-d32_in1k-pre_lr5e-3_4x4_512x512_160k_coco-stuff164k/bisenetv1_r101-d32_in1k-pre_lr5e-3_4x4_512x512_160k_coco-stuff164k_20211101_225220.log.json) |
+| Method | Backbone | Crop Size | Lr schd | Mem (GB) | Inf time (fps) | Device | mIoU | mIoU(ms+flip) | config | download |
+| --------- | ----------------------- | --------- | ------: | -------- | -------------- | ------ | ----: | ------------- | -------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
+| BiSeNetV1 | R-18-D32 (No Pretrain) | 512x512 | 160000 | - | - | V100 | 25.45 | 26.15 | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/bisenetv1/bisenetv1_r18-d32_4xb4-160k_coco-stuff164k-512x512.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/bisenetv1/bisenetv1_r18-d32_lr5e-3_4x4_512x512_160k_coco-stuff164k/bisenetv1_r18-d32_lr5e-3_4x4_512x512_160k_coco-stuff164k_20211022_054328-046aa2f2.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/bisenetv1/bisenetv1_r18-d32_lr5e-3_4x4_512x512_160k_coco-stuff164k/bisenetv1_r18-d32_lr5e-3_4x4_512x512_160k_coco-stuff164k_20211022_054328.log.json) |
+| BiSeNetV1 | R-18-D32 | 512x512 | 160000 | 6.33 | 74.24 | V100 | 28.55 | 29.26 | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/bisenetv1/bisenetv1_r18-d32-in1k-pre_4xb4-160k_coco-stuff164k-512x512.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/bisenetv1/bisenetv1_r18-d32_in1k-pre_lr5e-3_4x4_512x512_160k_coco-stuff164k/bisenetv1_r18-d32_in1k-pre_lr5e-3_4x4_512x512_160k_coco-stuff164k_20211023_013100-f700dbf7.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/bisenetv1/bisenetv1_r18-d32_in1k-pre_lr5e-3_4x4_512x512_160k_coco-stuff164k/bisenetv1_r18-d32_in1k-pre_lr5e-3_4x4_512x512_160k_coco-stuff164k_20211023_013100.log.json) |
+| BiSeNetV1 | R-50-D32 (No Pretrain) | 512x512 | 160000 | - | - | V100 | 29.82 | 30.33 | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/bisenetv1/bisenetv1_r50-d32_4xb4-160k_coco-stuff164k-512x512.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/bisenetv1/bisenetv1_r50-d32_lr5e-3_4x4_512x512_160k_coco-stuff164k/bisenetv1_r50-d32_lr5e-3_4x4_512x512_160k_coco-stuff164k_20211101_040616-d2bb0df4.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/bisenetv1/bisenetv1_r50-d32_lr5e-3_4x4_512x512_160k_coco-stuff164k/bisenetv1_r50-d32_lr5e-3_4x4_512x512_160k_coco-stuff164k_20211101_040616.log.json) |
+| BiSeNetV1 | R-50-D32 | 512x512 | 160000 | 9.28 | 32.60 | V100 | 34.88 | 35.37 | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/bisenetv1/bisenetv1_r50-d32-in1k-pre_4xb4-160k_coco-stuff164k-512x512.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/bisenetv1/bisenetv1_r50-d32_in1k-pre_lr5e-3_4x4_512x512_160k_coco-stuff164k/bisenetv1_r50-d32_in1k-pre_lr5e-3_4x4_512x512_160k_coco-stuff164k_20211101_181932-66747911.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/bisenetv1/bisenetv1_r50-d32_in1k-pre_lr5e-3_4x4_512x512_160k_coco-stuff164k/bisenetv1_r50-d32_in1k-pre_lr5e-3_4x4_512x512_160k_coco-stuff164k_20211101_181932.log.json) |
+| BiSeNetV1 | R-101-D32 (No Pretrain) | 512x512 | 160000 | - | - | V100 | 31.14 | 31.76 | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/bisenetv1/bisenetv1_r50-d32-in1k-pre_4xb4-160k_coco-stuff164k-512x512.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/bisenetv1/bisenetv1_r101-d32_lr5e-3_4x4_512x512_160k_coco-stuff164k/bisenetv1_r101-d32_lr5e-3_4x4_512x512_160k_coco-stuff164k_20211102_164147-c6b32c3b.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/bisenetv1/bisenetv1_r101-d32_lr5e-3_4x4_512x512_160k_coco-stuff164k/bisenetv1_r101-d32_lr5e-3_4x4_512x512_160k_coco-stuff164k_20211102_164147.log.json) |
+| BiSeNetV1 | R-101-D32 | 512x512 | 160000 | 10.36 | 25.25 | V100 | 37.38 | 37.99 | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/bisenetv1/bisenetv1_r101-d32-in1k-pre_4xb4-160k_coco-stuff164k-512x512.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/bisenetv1/bisenetv1_r101-d32_in1k-pre_lr5e-3_4x4_512x512_160k_coco-stuff164k/bisenetv1_r101-d32_in1k-pre_lr5e-3_4x4_512x512_160k_coco-stuff164k_20211101_225220-28c8f092.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/bisenetv1/bisenetv1_r101-d32_in1k-pre_lr5e-3_4x4_512x512_160k_coco-stuff164k/bisenetv1_r101-d32_in1k-pre_lr5e-3_4x4_512x512_160k_coco-stuff164k_20211101_225220.log.json) |
Note:
diff --git a/configs/bisenetv2/README.md b/configs/bisenetv2/README.md
index 1f44c01f36a..a5871dfeb98 100644
--- a/configs/bisenetv2/README.md
+++ b/configs/bisenetv2/README.md
@@ -26,12 +26,12 @@ The low-level details and high-level semantics are both essential to the semanti
### Cityscapes
-| Method | Backbone | Crop Size | Lr schd | Mem (GB) | Inf time (fps) | Device | mIoU | mIoU(ms+flip) | config | download |
-| --------- | ---------------- | --------- | ------: | -------- | -------------- | ------ | ----: | ------------: | ------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
-| BiSeNetV2 | BiSeNetV2 | 1024x1024 | 160000 | 7.64 | 31.77 | V100 | 73.21 | 75.74 | [config](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/configs/bisenetv2/bisenetv2_fcn_4xb4-160k_cityscapes-1024x1024.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/bisenetv2/bisenetv2_fcn_4x4_1024x1024_160k_cityscapes/bisenetv2_fcn_4x4_1024x1024_160k_cityscapes_20210902_015551-bcf10f09.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/bisenetv2/bisenetv2_fcn_4x4_1024x1024_160k_cityscapes/bisenetv2_fcn_4x4_1024x1024_160k_cityscapes_20210902_015551.log.json) |
-| BiSeNetV2 | BiSeNetV2 (OHEM) | 1024x1024 | 160000 | 7.64 | - | V100 | 73.57 | 75.80 | [config](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/configs/bisenetv2/bisenetv2_fcn_4xb4-ohem-160k_cityscapes-1024x1024.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/bisenetv2/bisenetv2_fcn_ohem_4x4_1024x1024_160k_cityscapes/bisenetv2_fcn_ohem_4x4_1024x1024_160k_cityscapes_20210902_112947-5f8103b4.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/bisenetv2/bisenetv2_fcn_ohem_4x4_1024x1024_160k_cityscapes/bisenetv2_fcn_ohem_4x4_1024x1024_160k_cityscapes_20210902_112947.log.json) |
-| BiSeNetV2 | BiSeNetV2 (4x8) | 1024x1024 | 160000 | 15.05 | - | V100 | 75.76 | 77.79 | [config](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/configs/bisenetv2/bisenetv2_fcn_4xb8-160k_cityscapes-1024x1024.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/bisenetv2/bisenetv2_fcn_4x8_1024x1024_160k_cityscapes/bisenetv2_fcn_4x8_1024x1024_160k_cityscapes_20210903_000032-e1a2eed6.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/bisenetv2/bisenetv2_fcn_4x8_1024x1024_160k_cityscapes/bisenetv2_fcn_4x8_1024x1024_160k_cityscapes_20210903_000032.log.json) |
-| BiSeNetV2 | BiSeNetV2 (FP16) | 1024x1024 | 160000 | 5.77 | 36.65 | V100 | 73.07 | 75.13 | [config](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/configs/bisenetv2/bisenetv2_fcn_4xb4-amp-160k_cityscapes-1024x1024.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/bisenetv2/bisenetv2_fcn_fp16_4x4_1024x1024_160k_cityscapes/bisenetv2_fcn_fp16_4x4_1024x1024_160k_cityscapes_20210902_045942-b979777b.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/bisenetv2/bisenetv2_fcn_fp16_4x4_1024x1024_160k_cityscapes/bisenetv2_fcn_fp16_4x4_1024x1024_160k_cityscapes_20210902_045942.log.json) |
+| Method | Backbone | Crop Size | Lr schd | Mem (GB) | Inf time (fps) | Device | mIoU | mIoU(ms+flip) | config | download |
+| --------- | ---------------- | --------- | ------: | -------- | -------------- | ------ | ----: | ------------: | --------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
+| BiSeNetV2 | BiSeNetV2 | 1024x1024 | 160000 | 7.64 | 31.77 | V100 | 73.21 | 75.74 | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/bisenetv2/bisenetv2_fcn_4xb4-160k_cityscapes-1024x1024.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/bisenetv2/bisenetv2_fcn_4x4_1024x1024_160k_cityscapes/bisenetv2_fcn_4x4_1024x1024_160k_cityscapes_20210902_015551-bcf10f09.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/bisenetv2/bisenetv2_fcn_4x4_1024x1024_160k_cityscapes/bisenetv2_fcn_4x4_1024x1024_160k_cityscapes_20210902_015551.log.json) |
+| BiSeNetV2 | BiSeNetV2 (OHEM) | 1024x1024 | 160000 | 7.64 | - | V100 | 73.57 | 75.80 | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/bisenetv2/bisenetv2_fcn_4xb4-ohem-160k_cityscapes-1024x1024.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/bisenetv2/bisenetv2_fcn_ohem_4x4_1024x1024_160k_cityscapes/bisenetv2_fcn_ohem_4x4_1024x1024_160k_cityscapes_20210902_112947-5f8103b4.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/bisenetv2/bisenetv2_fcn_ohem_4x4_1024x1024_160k_cityscapes/bisenetv2_fcn_ohem_4x4_1024x1024_160k_cityscapes_20210902_112947.log.json) |
+| BiSeNetV2 | BiSeNetV2 (4x8) | 1024x1024 | 160000 | 15.05 | - | V100 | 75.76 | 77.79 | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/bisenetv2/bisenetv2_fcn_4xb8-160k_cityscapes-1024x1024.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/bisenetv2/bisenetv2_fcn_4x8_1024x1024_160k_cityscapes/bisenetv2_fcn_4x8_1024x1024_160k_cityscapes_20210903_000032-e1a2eed6.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/bisenetv2/bisenetv2_fcn_4x8_1024x1024_160k_cityscapes/bisenetv2_fcn_4x8_1024x1024_160k_cityscapes_20210903_000032.log.json) |
+| BiSeNetV2 | BiSeNetV2 (FP16) | 1024x1024 | 160000 | 5.77 | 36.65 | V100 | 73.07 | 75.13 | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/bisenetv2/bisenetv2_fcn_4xb4-amp-160k_cityscapes-1024x1024.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/bisenetv2/bisenetv2_fcn_fp16_4x4_1024x1024_160k_cityscapes/bisenetv2_fcn_fp16_4x4_1024x1024_160k_cityscapes_20210902_045942-b979777b.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/bisenetv2/bisenetv2_fcn_fp16_4x4_1024x1024_160k_cityscapes/bisenetv2_fcn_fp16_4x4_1024x1024_160k_cityscapes_20210902_045942.log.json) |
Note:
diff --git a/configs/ccnet/README.md b/configs/ccnet/README.md
index c7a0163495d..64dd5f0298a 100644
--- a/configs/ccnet/README.md
+++ b/configs/ccnet/README.md
@@ -26,34 +26,34 @@ Contextual information is vital in visual understanding problems, such as semant
### Cityscapes
-| Method | Backbone | Crop Size | Lr schd | Mem (GB) | Inf time (fps) | Device | mIoU | mIoU(ms+flip) | config | download |
-| ------ | -------- | --------- | ------: | -------- | -------------- | ------ | ----: | ------------: | ------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| CCNet | R-50-D8 | 512x1024 | 40000 | 6 | 3.32 | V100 | 77.76 | 78.87 | [config](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/configs/ccnet/ccnet_r50-d8_4xb2-40k_cityscapes-512x1024.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/ccnet/ccnet_r50-d8_512x1024_40k_cityscapes/ccnet_r50-d8_512x1024_40k_cityscapes_20200616_142517-4123f401.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/ccnet/ccnet_r50-d8_512x1024_40k_cityscapes/ccnet_r50-d8_512x1024_40k_cityscapes_20200616_142517.log.json) |
-| CCNet | R-101-D8 | 512x1024 | 40000 | 9.5 | 2.31 | V100 | 76.35 | 78.19 | [config](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/configs/ccnet/ccnet_r101-d8_4xb2-40k_cityscapes-512x1024.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/ccnet/ccnet_r101-d8_512x1024_40k_cityscapes/ccnet_r101-d8_512x1024_40k_cityscapes_20200616_142540-a3b84ba6.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/ccnet/ccnet_r101-d8_512x1024_40k_cityscapes/ccnet_r101-d8_512x1024_40k_cityscapes_20200616_142540.log.json) |
-| CCNet | R-50-D8 | 769x769 | 40000 | 6.8 | 1.43 | V100 | 78.46 | 79.93 | [config](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/configs/ccnet/ccnet_r50-d8_4xb2-40k_cityscapes-769x769.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/ccnet/ccnet_r50-d8_769x769_40k_cityscapes/ccnet_r50-d8_769x769_40k_cityscapes_20200616_145125-76d11884.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/ccnet/ccnet_r50-d8_769x769_40k_cityscapes/ccnet_r50-d8_769x769_40k_cityscapes_20200616_145125.log.json) |
-| CCNet | R-101-D8 | 769x769 | 40000 | 10.7 | 1.01 | V100 | 76.94 | 78.62 | [config](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/configs/ccnet/ccnet_r101-d8_4xb2-40k_cityscapes-769x769.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/ccnet/ccnet_r101-d8_769x769_40k_cityscapes/ccnet_r101-d8_769x769_40k_cityscapes_20200617_101428-4f57c8d0.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/ccnet/ccnet_r101-d8_769x769_40k_cityscapes/ccnet_r101-d8_769x769_40k_cityscapes_20200617_101428.log.json) |
-| CCNet | R-50-D8 | 512x1024 | 80000 | - | - | V100 | 79.03 | 80.16 | [config](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/configs/ccnet/ccnet_r50-d8_4xb2-80k_cityscapes-512x1024.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/ccnet/ccnet_r50-d8_512x1024_80k_cityscapes/ccnet_r50-d8_512x1024_80k_cityscapes_20200617_010421-869a3423.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/ccnet/ccnet_r50-d8_512x1024_80k_cityscapes/ccnet_r50-d8_512x1024_80k_cityscapes_20200617_010421.log.json) |
-| CCNet | R-101-D8 | 512x1024 | 80000 | - | - | V100 | 78.87 | 79.90 | [config](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/configs/ccnet/ccnet_r101-d8_4xb2-80k_cityscapes-512x1024.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/ccnet/ccnet_r101-d8_512x1024_80k_cityscapes/ccnet_r101-d8_512x1024_80k_cityscapes_20200617_203935-ffae8917.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/ccnet/ccnet_r101-d8_512x1024_80k_cityscapes/ccnet_r101-d8_512x1024_80k_cityscapes_20200617_203935.log.json) |
-| CCNet | R-50-D8 | 769x769 | 80000 | - | - | V100 | 79.29 | 81.08 | [config](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/configs/ccnet/ccnet_r50-d8_4xb2-80k_cityscapes-769x769.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/ccnet/ccnet_r50-d8_769x769_80k_cityscapes/ccnet_r50-d8_769x769_80k_cityscapes_20200617_010421-73eed8ca.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/ccnet/ccnet_r50-d8_769x769_80k_cityscapes/ccnet_r50-d8_769x769_80k_cityscapes_20200617_010421.log.json) |
-| CCNet | R-101-D8 | 769x769 | 80000 | - | - | V100 | 79.45 | 80.66 | [config](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/configs/ccnet/ccnet_r101-d8_4xb2-80k_cityscapes-769x769.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/ccnet/ccnet_r101-d8_769x769_80k_cityscapes/ccnet_r101-d8_769x769_80k_cityscapes_20200618_011502-ad3cd481.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/ccnet/ccnet_r101-d8_769x769_80k_cityscapes/ccnet_r101-d8_769x769_80k_cityscapes_20200618_011502.log.json) |
+| Method | Backbone | Crop Size | Lr schd | Mem (GB) | Inf time (fps) | Device | mIoU | mIoU(ms+flip) | config | download |
+| ------ | -------- | --------- | ------: | -------- | -------------- | ------ | ----: | ------------: | ---------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| CCNet | R-50-D8 | 512x1024 | 40000 | 6 | 3.32 | V100 | 77.76 | 78.87 | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/ccnet/ccnet_r50-d8_4xb2-40k_cityscapes-512x1024.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/ccnet/ccnet_r50-d8_512x1024_40k_cityscapes/ccnet_r50-d8_512x1024_40k_cityscapes_20200616_142517-4123f401.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/ccnet/ccnet_r50-d8_512x1024_40k_cityscapes/ccnet_r50-d8_512x1024_40k_cityscapes_20200616_142517.log.json) |
+| CCNet | R-101-D8 | 512x1024 | 40000 | 9.5 | 2.31 | V100 | 76.35 | 78.19 | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/ccnet/ccnet_r101-d8_4xb2-40k_cityscapes-512x1024.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/ccnet/ccnet_r101-d8_512x1024_40k_cityscapes/ccnet_r101-d8_512x1024_40k_cityscapes_20200616_142540-a3b84ba6.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/ccnet/ccnet_r101-d8_512x1024_40k_cityscapes/ccnet_r101-d8_512x1024_40k_cityscapes_20200616_142540.log.json) |
+| CCNet | R-50-D8 | 769x769 | 40000 | 6.8 | 1.43 | V100 | 78.46 | 79.93 | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/ccnet/ccnet_r50-d8_4xb2-40k_cityscapes-769x769.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/ccnet/ccnet_r50-d8_769x769_40k_cityscapes/ccnet_r50-d8_769x769_40k_cityscapes_20200616_145125-76d11884.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/ccnet/ccnet_r50-d8_769x769_40k_cityscapes/ccnet_r50-d8_769x769_40k_cityscapes_20200616_145125.log.json) |
+| CCNet | R-101-D8 | 769x769 | 40000 | 10.7 | 1.01 | V100 | 76.94 | 78.62 | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/ccnet/ccnet_r101-d8_4xb2-40k_cityscapes-769x769.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/ccnet/ccnet_r101-d8_769x769_40k_cityscapes/ccnet_r101-d8_769x769_40k_cityscapes_20200617_101428-4f57c8d0.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/ccnet/ccnet_r101-d8_769x769_40k_cityscapes/ccnet_r101-d8_769x769_40k_cityscapes_20200617_101428.log.json) |
+| CCNet | R-50-D8 | 512x1024 | 80000 | - | - | V100 | 79.03 | 80.16 | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/ccnet/ccnet_r50-d8_4xb2-80k_cityscapes-512x1024.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/ccnet/ccnet_r50-d8_512x1024_80k_cityscapes/ccnet_r50-d8_512x1024_80k_cityscapes_20200617_010421-869a3423.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/ccnet/ccnet_r50-d8_512x1024_80k_cityscapes/ccnet_r50-d8_512x1024_80k_cityscapes_20200617_010421.log.json) |
+| CCNet | R-101-D8 | 512x1024 | 80000 | - | - | V100 | 78.87 | 79.90 | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/ccnet/ccnet_r101-d8_4xb2-80k_cityscapes-512x1024.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/ccnet/ccnet_r101-d8_512x1024_80k_cityscapes/ccnet_r101-d8_512x1024_80k_cityscapes_20200617_203935-ffae8917.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/ccnet/ccnet_r101-d8_512x1024_80k_cityscapes/ccnet_r101-d8_512x1024_80k_cityscapes_20200617_203935.log.json) |
+| CCNet | R-50-D8 | 769x769 | 80000 | - | - | V100 | 79.29 | 81.08 | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/ccnet/ccnet_r50-d8_4xb2-80k_cityscapes-769x769.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/ccnet/ccnet_r50-d8_769x769_80k_cityscapes/ccnet_r50-d8_769x769_80k_cityscapes_20200617_010421-73eed8ca.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/ccnet/ccnet_r50-d8_769x769_80k_cityscapes/ccnet_r50-d8_769x769_80k_cityscapes_20200617_010421.log.json) |
+| CCNet | R-101-D8 | 769x769 | 80000 | - | - | V100 | 79.45 | 80.66 | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/ccnet/ccnet_r101-d8_4xb2-80k_cityscapes-769x769.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/ccnet/ccnet_r101-d8_769x769_80k_cityscapes/ccnet_r101-d8_769x769_80k_cityscapes_20200618_011502-ad3cd481.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/ccnet/ccnet_r101-d8_769x769_80k_cityscapes/ccnet_r101-d8_769x769_80k_cityscapes_20200618_011502.log.json) |
### ADE20K
-| Method | Backbone | Crop Size | Lr schd | Mem (GB) | Inf time (fps) | Device | mIoU | mIoU(ms+flip) | config | download |
-| ------ | -------- | --------- | ------: | -------- | -------------- | ------ | ----: | ------------: | --------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| CCNet | R-50-D8 | 512x512 | 80000 | 8.8 | 20.89 | V100 | 41.78 | 42.98 | [config](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/configs/ccnet/ccnet_r50-d8_4xb4-80k_ade20k-512x512.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/ccnet/ccnet_r50-d8_512x512_80k_ade20k/ccnet_r50-d8_512x512_80k_ade20k_20200615_014848-aa37f61e.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/ccnet/ccnet_r50-d8_512x512_80k_ade20k/ccnet_r50-d8_512x512_80k_ade20k_20200615_014848.log.json) |
-| CCNet | R-101-D8 | 512x512 | 80000 | 12.2 | 14.11 | V100 | 43.97 | 45.13 | [config](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/configs/ccnet/ccnet_r101-d8_4xb4-80k_ade20k-512x512.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/ccnet/ccnet_r101-d8_512x512_80k_ade20k/ccnet_r101-d8_512x512_80k_ade20k_20200615_014848-1f4929a3.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/ccnet/ccnet_r101-d8_512x512_80k_ade20k/ccnet_r101-d8_512x512_80k_ade20k_20200615_014848.log.json) |
-| CCNet | R-50-D8 | 512x512 | 160000 | - | - | V100 | 42.08 | 43.13 | [config](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/configs/ccnet/ccnet_r50-d8_4xb4-160k_ade20k-512x512.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/ccnet/ccnet_r50-d8_512x512_160k_ade20k/ccnet_r50-d8_512x512_160k_ade20k_20200616_084435-7c97193b.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/ccnet/ccnet_r50-d8_512x512_160k_ade20k/ccnet_r50-d8_512x512_160k_ade20k_20200616_084435.log.json) |
-| CCNet | R-101-D8 | 512x512 | 160000 | - | - | V100 | 43.71 | 45.04 | [config](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/configs/ccnet/ccnet_r101-d8_4xb4-160k_ade20k-512x512.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/ccnet/ccnet_r101-d8_512x512_160k_ade20k/ccnet_r101-d8_512x512_160k_ade20k_20200616_000644-e849e007.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/ccnet/ccnet_r101-d8_512x512_160k_ade20k/ccnet_r101-d8_512x512_160k_ade20k_20200616_000644.log.json) |
+| Method | Backbone | Crop Size | Lr schd | Mem (GB) | Inf time (fps) | Device | mIoU | mIoU(ms+flip) | config | download |
+| ------ | -------- | --------- | ------: | -------- | -------------- | ------ | ----: | ------------: | ------------------------------------------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| CCNet | R-50-D8 | 512x512 | 80000 | 8.8 | 20.89 | V100 | 41.78 | 42.98 | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/ccnet/ccnet_r50-d8_4xb4-80k_ade20k-512x512.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/ccnet/ccnet_r50-d8_512x512_80k_ade20k/ccnet_r50-d8_512x512_80k_ade20k_20200615_014848-aa37f61e.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/ccnet/ccnet_r50-d8_512x512_80k_ade20k/ccnet_r50-d8_512x512_80k_ade20k_20200615_014848.log.json) |
+| CCNet | R-101-D8 | 512x512 | 80000 | 12.2 | 14.11 | V100 | 43.97 | 45.13 | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/ccnet/ccnet_r101-d8_4xb4-80k_ade20k-512x512.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/ccnet/ccnet_r101-d8_512x512_80k_ade20k/ccnet_r101-d8_512x512_80k_ade20k_20200615_014848-1f4929a3.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/ccnet/ccnet_r101-d8_512x512_80k_ade20k/ccnet_r101-d8_512x512_80k_ade20k_20200615_014848.log.json) |
+| CCNet | R-50-D8 | 512x512 | 160000 | - | - | V100 | 42.08 | 43.13 | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/ccnet/ccnet_r50-d8_4xb4-160k_ade20k-512x512.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/ccnet/ccnet_r50-d8_512x512_160k_ade20k/ccnet_r50-d8_512x512_160k_ade20k_20200616_084435-7c97193b.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/ccnet/ccnet_r50-d8_512x512_160k_ade20k/ccnet_r50-d8_512x512_160k_ade20k_20200616_084435.log.json) |
+| CCNet | R-101-D8 | 512x512 | 160000 | - | - | V100 | 43.71 | 45.04 | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/ccnet/ccnet_r101-d8_4xb4-160k_ade20k-512x512.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/ccnet/ccnet_r101-d8_512x512_160k_ade20k/ccnet_r101-d8_512x512_160k_ade20k_20200616_000644-e849e007.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/ccnet/ccnet_r101-d8_512x512_160k_ade20k/ccnet_r101-d8_512x512_160k_ade20k_20200616_000644.log.json) |
### Pascal VOC 2012 + Aug
-| Method | Backbone | Crop Size | Lr schd | Mem (GB) | Inf time (fps) | Device | mIoU | mIoU(ms+flip) | config | download |
-| ------ | -------- | --------- | ------: | -------- | -------------- | ------ | ----: | ------------: | ---------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| CCNet | R-50-D8 | 512x512 | 20000 | 6 | 20.45 | V100 | 76.17 | 77.51 | [config](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/configs/ccnet/ccnet_r50-d8_4xb4-20k_voc12aug-512x512.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/ccnet/ccnet_r50-d8_512x512_20k_voc12aug/ccnet_r50-d8_512x512_20k_voc12aug_20200617_193212-fad81784.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/ccnet/ccnet_r50-d8_512x512_20k_voc12aug/ccnet_r50-d8_512x512_20k_voc12aug_20200617_193212.log.json) |
-| CCNet | R-101-D8 | 512x512 | 20000 | 9.5 | 13.64 | V100 | 77.27 | 79.02 | [config](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/configs/ccnet/ccnet_r101-d8_4xb4-20k_voc12aug-512x512.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/ccnet/ccnet_r101-d8_512x512_20k_voc12aug/ccnet_r101-d8_512x512_20k_voc12aug_20200617_193212-0007b61d.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/ccnet/ccnet_r101-d8_512x512_20k_voc12aug/ccnet_r101-d8_512x512_20k_voc12aug_20200617_193212.log.json) |
-| CCNet | R-50-D8 | 512x512 | 40000 | - | - | V100 | 75.96 | 77.04 | [config](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/configs/ccnet/ccnet_r50-d8_4xb4-40k_voc12aug-512x512.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/ccnet/ccnet_r50-d8_512x512_40k_voc12aug/ccnet_r50-d8_512x512_40k_voc12aug_20200613_232127-c2a15f02.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/ccnet/ccnet_r50-d8_512x512_40k_voc12aug/ccnet_r50-d8_512x512_40k_voc12aug_20200613_232127.log.json) |
-| CCNet | R-101-D8 | 512x512 | 40000 | - | - | V100 | 77.87 | 78.90 | [config](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/configs/ccnet/ccnet_r101-d8_4xb4-40k_voc12aug-512x512.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/ccnet/ccnet_r101-d8_512x512_40k_voc12aug/ccnet_r101-d8_512x512_40k_voc12aug_20200613_232127-c30da577.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/ccnet/ccnet_r101-d8_512x512_40k_voc12aug/ccnet_r101-d8_512x512_40k_voc12aug_20200613_232127.log.json) |
+| Method | Backbone | Crop Size | Lr schd | Mem (GB) | Inf time (fps) | Device | mIoU | mIoU(ms+flip) | config | download |
+| ------ | -------- | --------- | ------: | -------- | -------------- | ------ | ----: | ------------: | ------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| CCNet | R-50-D8 | 512x512 | 20000 | 6 | 20.45 | V100 | 76.17 | 77.51 | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/ccnet/ccnet_r50-d8_4xb4-20k_voc12aug-512x512.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/ccnet/ccnet_r50-d8_512x512_20k_voc12aug/ccnet_r50-d8_512x512_20k_voc12aug_20200617_193212-fad81784.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/ccnet/ccnet_r50-d8_512x512_20k_voc12aug/ccnet_r50-d8_512x512_20k_voc12aug_20200617_193212.log.json) |
+| CCNet | R-101-D8 | 512x512 | 20000 | 9.5 | 13.64 | V100 | 77.27 | 79.02 | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/ccnet/ccnet_r101-d8_4xb4-20k_voc12aug-512x512.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/ccnet/ccnet_r101-d8_512x512_20k_voc12aug/ccnet_r101-d8_512x512_20k_voc12aug_20200617_193212-0007b61d.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/ccnet/ccnet_r101-d8_512x512_20k_voc12aug/ccnet_r101-d8_512x512_20k_voc12aug_20200617_193212.log.json) |
+| CCNet | R-50-D8 | 512x512 | 40000 | - | - | V100 | 75.96 | 77.04 | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/ccnet/ccnet_r50-d8_4xb4-40k_voc12aug-512x512.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/ccnet/ccnet_r50-d8_512x512_40k_voc12aug/ccnet_r50-d8_512x512_40k_voc12aug_20200613_232127-c2a15f02.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/ccnet/ccnet_r50-d8_512x512_40k_voc12aug/ccnet_r50-d8_512x512_40k_voc12aug_20200613_232127.log.json) |
+| CCNet | R-101-D8 | 512x512 | 40000 | - | - | V100 | 77.87 | 78.90 | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/ccnet/ccnet_r101-d8_4xb4-40k_voc12aug-512x512.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/ccnet/ccnet_r101-d8_512x512_40k_voc12aug/ccnet_r101-d8_512x512_40k_voc12aug_20200613_232127-c30da577.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/ccnet/ccnet_r101-d8_512x512_40k_voc12aug/ccnet_r101-d8_512x512_40k_voc12aug_20200613_232127.log.json) |
## Citation
diff --git a/configs/cgnet/README.md b/configs/cgnet/README.md
index 60cde582918..96c9fcf515c 100644
--- a/configs/cgnet/README.md
+++ b/configs/cgnet/README.md
@@ -26,10 +26,10 @@ The demand of applying semantic segmentation model on mobile devices has been in
### Cityscapes
-| Method | Backbone | Crop Size | Lr schd | Mem (GB) | Inf time (fps) | Device | mIoU | mIoU(ms+flip) | config | download |
-| ------ | -------- | --------- | ------: | -------- | -------------- | ------ | ----: | ------------: | --------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
-| CGNet | M3N21 | 680x680 | 60000 | 7.5 | 30.51 | V100 | 65.63 | 68.04 | [config](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/configs/cgnet/cgnet_fcn_4xb4-60k_cityscapes-680x680.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/cgnet/cgnet_680x680_60k_cityscapes/cgnet_680x680_60k_cityscapes_20201101_110253-4c0b2f2d.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/cgnet/cgnet_680x680_60k_cityscapes/cgnet_680x680_60k_cityscapes-20201101_110253.log.json) |
-| CGNet | M3N21 | 512x1024 | 60000 | 8.3 | 31.14 | V100 | 68.27 | 70.33 | [config](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/configs/cgnet/cgnet_fcn_4xb8-60k_cityscapes-512x1024.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/cgnet/cgnet_512x1024_60k_cityscapes/cgnet_512x1024_60k_cityscapes_20201101_110254-124ea03b.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/cgnet/cgnet_512x1024_60k_cityscapes/cgnet_512x1024_60k_cityscapes-20201101_110254.log.json) |
+| Method | Backbone | Crop Size | Lr schd | Mem (GB) | Inf time (fps) | Device | mIoU | mIoU(ms+flip) | config | download |
+| ------ | -------- | --------- | ------: | -------- | -------------- | ------ | ----: | ------------: | ------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
+| CGNet | M3N21 | 680x680 | 60000 | 7.5 | 30.51 | V100 | 65.63 | 68.04 | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/cgnet/cgnet_fcn_4xb4-60k_cityscapes-680x680.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/cgnet/cgnet_680x680_60k_cityscapes/cgnet_680x680_60k_cityscapes_20201101_110253-4c0b2f2d.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/cgnet/cgnet_680x680_60k_cityscapes/cgnet_680x680_60k_cityscapes-20201101_110253.log.json) |
+| CGNet | M3N21 | 512x1024 | 60000 | 8.3 | 31.14 | V100 | 68.27 | 70.33 | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/cgnet/cgnet_fcn_4xb8-60k_cityscapes-512x1024.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/cgnet/cgnet_512x1024_60k_cityscapes/cgnet_512x1024_60k_cityscapes_20201101_110254-124ea03b.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/cgnet/cgnet_512x1024_60k_cityscapes/cgnet_512x1024_60k_cityscapes-20201101_110254.log.json) |
## Citation
diff --git a/configs/convnext/README.md b/configs/convnext/README.md
index 91339377c01..d78fe6ee1bb 100644
--- a/configs/convnext/README.md
+++ b/configs/convnext/README.md
@@ -27,7 +27,7 @@ The "Roaring 20s" of visual recognition began with the introduction of Vision Tr
- ConvNeXt backbone needs to install [MMClassification](https://github.com/open-mmlab/mmclassification) first, which has abundant backbones for downstream tasks.
```shell
-pip install mmcls>=0.20.1
+pip install mmpretrain>=1.0.0rc7
```
### Pre-trained Models
@@ -49,14 +49,14 @@ The pre-trained models on ImageNet-1k or ImageNet-21k are used to fine-tune on t
### ADE20K
-| Method | Backbone | Crop Size | Lr schd | Mem (GB) | Inf time (fps) | Device | mIoU | mIoU(ms+flip) | config | download |
-| ------- | ----------- | --------- | ------- | -------- | -------------- | ------ | ----- | ------------- | -------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| UPerNet | ConvNeXt-T | 512x512 | 160000 | 4.23 | 19.90 | V100 | 46.11 | 46.62 | [config](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/configs/convnext/convnext-tiny_upernet_8xb2-amp-160k_ade20k-512x512.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/convnext/upernet_convnext_tiny_fp16_512x512_160k_ade20k/upernet_convnext_tiny_fp16_512x512_160k_ade20k_20220227_124553-cad485de.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/convnext/upernet_convnext_tiny_fp16_512x512_160k_ade20k/upernet_convnext_tiny_fp16_512x512_160k_ade20k_20220227_124553.log.json) |
-| UPerNet | ConvNeXt-S | 512x512 | 160000 | 5.16 | 15.18 | V100 | 48.56 | 49.02 | [config](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/configs/convnext/convnext-small_upernet_8xb2-amp-160k_ade20k-512x512.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/convnext/upernet_convnext_small_fp16_512x512_160k_ade20k/upernet_convnext_small_fp16_512x512_160k_ade20k_20220227_131208-1b1e394f.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/convnext/upernet_convnext_small_fp16_512x512_160k_ade20k/upernet_convnext_small_fp16_512x512_160k_ade20k_20220227_131208.log.json) |
-| UPerNet | ConvNeXt-B | 512x512 | 160000 | 6.33 | 14.41 | V100 | 48.71 | 49.54 | [config](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/configs/convnext/convnext-base_upernet_8xb2-amp-160k_ade20k-512x512.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/convnext/upernet_convnext_base_fp16_512x512_160k_ade20k/upernet_convnext_base_fp16_512x512_160k_ade20k_20220227_181227-02a24fc6.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/convnext/upernet_convnext_base_fp16_512x512_160k_ade20k/upernet_convnext_base_fp16_512x512_160k_ade20k_20220227_181227.log.json) |
-| UPerNet | ConvNeXt-B | 640x640 | 160000 | 8.53 | 10.88 | V100 | 52.13 | 52.66 | [config](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/configs/convnext/convnext-base_upernet_8xb2-amp-160k_ade20k-640x640.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/convnext/upernet_convnext_base_fp16_640x640_160k_ade20k/upernet_convnext_base_fp16_640x640_160k_ade20k_20220227_182859-9280e39b.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/convnext/upernet_convnext_base_fp16_640x640_160k_ade20k/upernet_convnext_base_fp16_640x640_160k_ade20k_20220227_182859.log.json) |
-| UPerNet | ConvNeXt-L | 640x640 | 160000 | 12.08 | 7.69 | V100 | 53.16 | 53.38 | [config](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/configs/convnext/convnext-large_upernet_8xb2-amp-160k_ade20k-640x640.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/convnext/upernet_convnext_large_fp16_640x640_160k_ade20k/upernet_convnext_large_fp16_640x640_160k_ade20k_20220226_040532-e57aa54d.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/convnext/upernet_convnext_large_fp16_640x640_160k_ade20k/upernet_convnext_large_fp16_640x640_160k_ade20k_20220226_040532.log.json) |
-| UPerNet | ConvNeXt-XL | 640x640 | 160000 | 26.16\* | 6.33 | V100 | 53.58 | 54.11 | [config](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/configs/convnext/convnext-xlarge_upernet_8xb2-amp-160k_ade20k-640x640.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/convnext/upernet_convnext_xlarge_fp16_640x640_160k_ade20k/upernet_convnext_xlarge_fp16_640x640_160k_ade20k_20220226_080344-95fc38c2.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/convnext/upernet_convnext_xlarge_fp16_640x640_160k_ade20k/upernet_convnext_xlarge_fp16_640x640_160k_ade20k_20220226_080344.log.json) |
+| Method | Backbone | Crop Size | Lr schd | Mem (GB) | Inf time (fps) | Device | mIoU | mIoU(ms+flip) | config | download |
+| ------- | ----------- | --------- | ------- | -------- | -------------- | ------ | ----- | ------------- | ----------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| UPerNet | ConvNeXt-T | 512x512 | 160000 | 4.23 | 19.90 | V100 | 46.11 | 46.62 | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/convnext/convnext-tiny_upernet_8xb2-amp-160k_ade20k-512x512.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/convnext/upernet_convnext_tiny_fp16_512x512_160k_ade20k/upernet_convnext_tiny_fp16_512x512_160k_ade20k_20220227_124553-cad485de.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/convnext/upernet_convnext_tiny_fp16_512x512_160k_ade20k/upernet_convnext_tiny_fp16_512x512_160k_ade20k_20220227_124553.log.json) |
+| UPerNet | ConvNeXt-S | 512x512 | 160000 | 5.16 | 15.18 | V100 | 48.56 | 49.02 | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/convnext/convnext-small_upernet_8xb2-amp-160k_ade20k-512x512.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/convnext/upernet_convnext_small_fp16_512x512_160k_ade20k/upernet_convnext_small_fp16_512x512_160k_ade20k_20220227_131208-1b1e394f.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/convnext/upernet_convnext_small_fp16_512x512_160k_ade20k/upernet_convnext_small_fp16_512x512_160k_ade20k_20220227_131208.log.json) |
+| UPerNet | ConvNeXt-B | 512x512 | 160000 | 6.33 | 14.41 | V100 | 48.71 | 49.54 | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/convnext/convnext-base_upernet_8xb2-amp-160k_ade20k-512x512.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/convnext/upernet_convnext_base_fp16_512x512_160k_ade20k/upernet_convnext_base_fp16_512x512_160k_ade20k_20220227_181227-02a24fc6.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/convnext/upernet_convnext_base_fp16_512x512_160k_ade20k/upernet_convnext_base_fp16_512x512_160k_ade20k_20220227_181227.log.json) |
+| UPerNet | ConvNeXt-B | 640x640 | 160000 | 8.53 | 10.88 | V100 | 52.13 | 52.66 | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/convnext/convnext-base_upernet_8xb2-amp-160k_ade20k-640x640.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/convnext/upernet_convnext_base_fp16_640x640_160k_ade20k/upernet_convnext_base_fp16_640x640_160k_ade20k_20220227_182859-9280e39b.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/convnext/upernet_convnext_base_fp16_640x640_160k_ade20k/upernet_convnext_base_fp16_640x640_160k_ade20k_20220227_182859.log.json) |
+| UPerNet | ConvNeXt-L | 640x640 | 160000 | 12.08 | 7.69 | V100 | 53.16 | 53.38 | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/convnext/convnext-large_upernet_8xb2-amp-160k_ade20k-640x640.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/convnext/upernet_convnext_large_fp16_640x640_160k_ade20k/upernet_convnext_large_fp16_640x640_160k_ade20k_20220226_040532-e57aa54d.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/convnext/upernet_convnext_large_fp16_640x640_160k_ade20k/upernet_convnext_large_fp16_640x640_160k_ade20k_20220226_040532.log.json) |
+| UPerNet | ConvNeXt-XL | 640x640 | 160000 | 26.16\* | 6.33 | V100 | 53.58 | 54.11 | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/convnext/convnext-xlarge_upernet_8xb2-amp-160k_ade20k-640x640.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/convnext/upernet_convnext_xlarge_fp16_640x640_160k_ade20k/upernet_convnext_xlarge_fp16_640x640_160k_ade20k_20220226_080344-95fc38c2.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/convnext/upernet_convnext_xlarge_fp16_640x640_160k_ade20k/upernet_convnext_xlarge_fp16_640x640_160k_ade20k_20220226_080344.log.json) |
Note:
diff --git a/configs/convnext/convnext-base_upernet_8xb2-amp-160k_ade20k-640x640.py b/configs/convnext/convnext-base_upernet_8xb2-amp-160k_ade20k-640x640.py
index a743e9322ac..06a86431442 100644
--- a/configs/convnext/convnext-base_upernet_8xb2-amp-160k_ade20k-640x640.py
+++ b/configs/convnext/convnext-base_upernet_8xb2-amp-160k_ade20k-640x640.py
@@ -9,7 +9,7 @@
model = dict(
data_preprocessor=data_preprocessor,
backbone=dict(
- type='mmcls.ConvNeXt',
+ type='mmpretrain.ConvNeXt',
arch='base',
out_indices=[0, 1, 2, 3],
drop_path_rate=0.4,
diff --git a/configs/convnext/convnext-large_upernet_8xb2-amp-160k_ade20k-640x640.py b/configs/convnext/convnext-large_upernet_8xb2-amp-160k_ade20k-640x640.py
index 6d94989ee1e..2956e86f042 100644
--- a/configs/convnext/convnext-large_upernet_8xb2-amp-160k_ade20k-640x640.py
+++ b/configs/convnext/convnext-large_upernet_8xb2-amp-160k_ade20k-640x640.py
@@ -9,7 +9,7 @@
model = dict(
data_preprocessor=data_preprocessor,
backbone=dict(
- type='mmcls.ConvNeXt',
+ type='mmpretrain.ConvNeXt',
arch='large',
out_indices=[0, 1, 2, 3],
drop_path_rate=0.4,
diff --git a/configs/convnext/convnext-small_upernet_8xb2-amp-160k_ade20k-512x512.py b/configs/convnext/convnext-small_upernet_8xb2-amp-160k_ade20k-512x512.py
index 3cbf09902dc..dbe45f10e0b 100644
--- a/configs/convnext/convnext-small_upernet_8xb2-amp-160k_ade20k-512x512.py
+++ b/configs/convnext/convnext-small_upernet_8xb2-amp-160k_ade20k-512x512.py
@@ -8,7 +8,7 @@
model = dict(
data_preprocessor=data_preprocessor,
backbone=dict(
- type='mmcls.ConvNeXt',
+ type='mmpretrain.ConvNeXt',
arch='small',
out_indices=[0, 1, 2, 3],
drop_path_rate=0.3,
diff --git a/configs/convnext/convnext-tiny_upernet_8xb2-amp-160k_ade20k-512x512.py b/configs/convnext/convnext-tiny_upernet_8xb2-amp-160k_ade20k-512x512.py
index 9d4968df600..d2e545a76d0 100644
--- a/configs/convnext/convnext-tiny_upernet_8xb2-amp-160k_ade20k-512x512.py
+++ b/configs/convnext/convnext-tiny_upernet_8xb2-amp-160k_ade20k-512x512.py
@@ -8,7 +8,7 @@
model = dict(
data_preprocessor=data_preprocessor,
backbone=dict(
- type='mmcls.ConvNeXt',
+ type='mmpretrain.ConvNeXt',
arch='tiny',
out_indices=[0, 1, 2, 3],
drop_path_rate=0.4,
diff --git a/configs/convnext/convnext-xlarge_upernet_8xb2-amp-160k_ade20k-640x640.py b/configs/convnext/convnext-xlarge_upernet_8xb2-amp-160k_ade20k-640x640.py
index 749391cac12..dfad7345215 100644
--- a/configs/convnext/convnext-xlarge_upernet_8xb2-amp-160k_ade20k-640x640.py
+++ b/configs/convnext/convnext-xlarge_upernet_8xb2-amp-160k_ade20k-640x640.py
@@ -9,7 +9,7 @@
model = dict(
data_preprocessor=data_preprocessor,
backbone=dict(
- type='mmcls.ConvNeXt',
+ type='mmpretrain.ConvNeXt',
arch='xlarge',
out_indices=[0, 1, 2, 3],
drop_path_rate=0.4,
diff --git a/configs/danet/README.md b/configs/danet/README.md
index b5841e23be1..90194f3073e 100644
--- a/configs/danet/README.md
+++ b/configs/danet/README.md
@@ -26,34 +26,34 @@ In this paper, we address the scene segmentation task by capturing rich contextu
### Cityscapes
-| Method | Backbone | Crop Size | Lr schd | Mem (GB) | Inf time (fps) | Device | mIoU | mIoU(ms+flip) | config | download |
-| ------ | -------- | --------- | ------: | -------- | -------------- | ------ | ----: | ------------- | ------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| DANet | R-50-D8 | 512x1024 | 40000 | 7.4 | 2.66 | V100 | 78.74 | - | [config](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/configs/danet/danet_r50-d8_4xb2-40k_cityscapes-512x1024.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/danet/danet_r50-d8_512x1024_40k_cityscapes/danet_r50-d8_512x1024_40k_cityscapes_20200605_191324-c0dbfa5f.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/danet/danet_r50-d8_512x1024_40k_cityscapes/danet_r50-d8_512x1024_40k_cityscapes_20200605_191324.log.json) |
-| DANet | R-101-D8 | 512x1024 | 40000 | 10.9 | 1.99 | V100 | 80.52 | - | [config](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/configs/danet/danet_r101-d8_4xb2-40k_cityscapes-512x1024.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/danet/danet_r101-d8_512x1024_40k_cityscapes/danet_r101-d8_512x1024_40k_cityscapes_20200605_200831-c57a7157.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/danet/danet_r101-d8_512x1024_40k_cityscapes/danet_r101-d8_512x1024_40k_cityscapes_20200605_200831.log.json) |
-| DANet | R-50-D8 | 769x769 | 40000 | 8.8 | 1.56 | V100 | 78.88 | 80.62 | [config](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/configs/danet/danet_r50-d8_4xb2-40k_cityscapes-769x769.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/danet/danet_r50-d8_769x769_40k_cityscapes/danet_r50-d8_769x769_40k_cityscapes_20200530_025703-76681c60.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/danet/danet_r50-d8_769x769_40k_cityscapes/danet_r50-d8_769x769_40k_cityscapes_20200530_025703.log.json) |
-| DANet | R-101-D8 | 769x769 | 40000 | 12.8 | 1.07 | V100 | 79.88 | 81.47 | [config](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/configs/danet/danet_r101-d8_4xb2-40k_cityscapes-769x769.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/danet/danet_r101-d8_769x769_40k_cityscapes/danet_r101-d8_769x769_40k_cityscapes_20200530_025717-dcb7fd4e.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/danet/danet_r101-d8_769x769_40k_cityscapes/danet_r101-d8_769x769_40k_cityscapes_20200530_025717.log.json) |
-| DANet | R-50-D8 | 512x1024 | 80000 | - | - | V100 | 79.34 | - | [config](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/configs/danet/danet_r50-d8_4xb2-80k_cityscapes-512x1024.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/danet/danet_r50-d8_512x1024_80k_cityscapes/danet_r50-d8_512x1024_80k_cityscapes_20200607_133029-2bfa2293.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/danet/danet_r50-d8_512x1024_80k_cityscapes/danet_r50-d8_512x1024_80k_cityscapes_20200607_133029.log.json) |
-| DANet | R-101-D8 | 512x1024 | 80000 | - | - | V100 | 80.41 | - | [config](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/configs/danet/danet_r101-d8_4xb2-80k_cityscapes-512x1024.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/danet/danet_r101-d8_512x1024_80k_cityscapes/danet_r101-d8_512x1024_80k_cityscapes_20200607_132918-955e6350.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/danet/danet_r101-d8_512x1024_80k_cityscapes/danet_r101-d8_512x1024_80k_cityscapes_20200607_132918.log.json) |
-| DANet | R-50-D8 | 769x769 | 80000 | - | - | V100 | 79.27 | 80.96 | [config](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/configs/danet/danet_r50-d8_4xb2-80k_cityscapes-769x769.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/danet/danet_r50-d8_769x769_80k_cityscapes/danet_r50-d8_769x769_80k_cityscapes_20200607_132954-495689b4.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/danet/danet_r50-d8_769x769_80k_cityscapes/danet_r50-d8_769x769_80k_cityscapes_20200607_132954.log.json) |
-| DANet | R-101-D8 | 769x769 | 80000 | - | - | V100 | 80.47 | 82.02 | [config](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/configs/danet/danet_r101-d8_4xb2-80k_cityscapes-769x769.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/danet/danet_r101-d8_769x769_80k_cityscapes/danet_r101-d8_769x769_80k_cityscapes_20200607_132918-f3a929e7.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/danet/danet_r101-d8_769x769_80k_cityscapes/danet_r101-d8_769x769_80k_cityscapes_20200607_132918.log.json) |
+| Method | Backbone | Crop Size | Lr schd | Mem (GB) | Inf time (fps) | Device | mIoU | mIoU(ms+flip) | config | download |
+| ------ | -------- | --------- | ------: | -------- | -------------- | ------ | ----: | ------------- | ---------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| DANet | R-50-D8 | 512x1024 | 40000 | 7.4 | 2.66 | V100 | 78.74 | - | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/danet/danet_r50-d8_4xb2-40k_cityscapes-512x1024.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/danet/danet_r50-d8_512x1024_40k_cityscapes/danet_r50-d8_512x1024_40k_cityscapes_20200605_191324-c0dbfa5f.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/danet/danet_r50-d8_512x1024_40k_cityscapes/danet_r50-d8_512x1024_40k_cityscapes_20200605_191324.log.json) |
+| DANet | R-101-D8 | 512x1024 | 40000 | 10.9 | 1.99 | V100 | 80.52 | - | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/danet/danet_r101-d8_4xb2-40k_cityscapes-512x1024.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/danet/danet_r101-d8_512x1024_40k_cityscapes/danet_r101-d8_512x1024_40k_cityscapes_20200605_200831-c57a7157.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/danet/danet_r101-d8_512x1024_40k_cityscapes/danet_r101-d8_512x1024_40k_cityscapes_20200605_200831.log.json) |
+| DANet | R-50-D8 | 769x769 | 40000 | 8.8 | 1.56 | V100 | 78.88 | 80.62 | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/danet/danet_r50-d8_4xb2-40k_cityscapes-769x769.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/danet/danet_r50-d8_769x769_40k_cityscapes/danet_r50-d8_769x769_40k_cityscapes_20200530_025703-76681c60.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/danet/danet_r50-d8_769x769_40k_cityscapes/danet_r50-d8_769x769_40k_cityscapes_20200530_025703.log.json) |
+| DANet | R-101-D8 | 769x769 | 40000 | 12.8 | 1.07 | V100 | 79.88 | 81.47 | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/danet/danet_r101-d8_4xb2-40k_cityscapes-769x769.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/danet/danet_r101-d8_769x769_40k_cityscapes/danet_r101-d8_769x769_40k_cityscapes_20200530_025717-dcb7fd4e.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/danet/danet_r101-d8_769x769_40k_cityscapes/danet_r101-d8_769x769_40k_cityscapes_20200530_025717.log.json) |
+| DANet | R-50-D8 | 512x1024 | 80000 | - | - | V100 | 79.34 | - | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/danet/danet_r50-d8_4xb2-80k_cityscapes-512x1024.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/danet/danet_r50-d8_512x1024_80k_cityscapes/danet_r50-d8_512x1024_80k_cityscapes_20200607_133029-2bfa2293.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/danet/danet_r50-d8_512x1024_80k_cityscapes/danet_r50-d8_512x1024_80k_cityscapes_20200607_133029.log.json) |
+| DANet | R-101-D8 | 512x1024 | 80000 | - | - | V100 | 80.41 | - | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/danet/danet_r101-d8_4xb2-80k_cityscapes-512x1024.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/danet/danet_r101-d8_512x1024_80k_cityscapes/danet_r101-d8_512x1024_80k_cityscapes_20200607_132918-955e6350.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/danet/danet_r101-d8_512x1024_80k_cityscapes/danet_r101-d8_512x1024_80k_cityscapes_20200607_132918.log.json) |
+| DANet | R-50-D8 | 769x769 | 80000 | - | - | V100 | 79.27 | 80.96 | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/danet/danet_r50-d8_4xb2-80k_cityscapes-769x769.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/danet/danet_r50-d8_769x769_80k_cityscapes/danet_r50-d8_769x769_80k_cityscapes_20200607_132954-495689b4.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/danet/danet_r50-d8_769x769_80k_cityscapes/danet_r50-d8_769x769_80k_cityscapes_20200607_132954.log.json) |
+| DANet | R-101-D8 | 769x769 | 80000 | - | - | V100 | 80.47 | 82.02 | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/danet/danet_r101-d8_4xb2-80k_cityscapes-769x769.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/danet/danet_r101-d8_769x769_80k_cityscapes/danet_r101-d8_769x769_80k_cityscapes_20200607_132918-f3a929e7.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/danet/danet_r101-d8_769x769_80k_cityscapes/danet_r101-d8_769x769_80k_cityscapes_20200607_132918.log.json) |
### ADE20K
-| Method | Backbone | Crop Size | Lr schd | Mem (GB) | Inf time (fps) | Device | mIoU | mIoU(ms+flip) | config | download |
-| ------ | -------- | --------- | ------: | -------- | -------------- | ------ | ----: | ------------: | --------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| DANet | R-50-D8 | 512x512 | 80000 | 11.5 | 21.20 | V100 | 41.66 | 42.90 | [config](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/configs/danet/danet_r50-d8_4xb4-80k_ade20k-512x512.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/danet/danet_r50-d8_512x512_80k_ade20k/danet_r50-d8_512x512_80k_ade20k_20200615_015125-edb18e08.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/danet/danet_r50-d8_512x512_80k_ade20k/danet_r50-d8_512x512_80k_ade20k_20200615_015125.log.json) |
-| DANet | R-101-D8 | 512x512 | 80000 | 15 | 14.18 | V100 | 43.64 | 45.19 | [config](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/configs/danet/danet_r101-d8_4xb4-80k_ade20k-512x512.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/danet/danet_r101-d8_512x512_80k_ade20k/danet_r101-d8_512x512_80k_ade20k_20200615_015126-d0357c73.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/danet/danet_r101-d8_512x512_80k_ade20k/danet_r101-d8_512x512_80k_ade20k_20200615_015126.log.json) |
-| DANet | R-50-D8 | 512x512 | 160000 | - | - | V100 | 42.45 | 43.25 | [config](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/configs/danet/danet_r50-d8_4xb4-160k_ade20k-512x512.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/danet/danet_r50-d8_512x512_160k_ade20k/danet_r50-d8_512x512_160k_ade20k_20200616_082340-9cb35dcd.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/danet/danet_r50-d8_512x512_160k_ade20k/danet_r50-d8_512x512_160k_ade20k_20200616_082340.log.json) |
-| DANet | R-101-D8 | 512x512 | 160000 | - | - | V100 | 44.17 | 45.02 | [config](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/configs/danet/danet_r101-d8_4xb4-160k_ade20k-512x512.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/danet/danet_r101-d8_512x512_160k_ade20k/danet_r101-d8_512x512_160k_ade20k_20200616_082348-23bf12f9.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/danet/danet_r101-d8_512x512_160k_ade20k/danet_r101-d8_512x512_160k_ade20k_20200616_082348.log.json) |
+| Method | Backbone | Crop Size | Lr schd | Mem (GB) | Inf time (fps) | Device | mIoU | mIoU(ms+flip) | config | download |
+| ------ | -------- | --------- | ------: | -------- | -------------- | ------ | ----: | ------------: | ------------------------------------------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| DANet | R-50-D8 | 512x512 | 80000 | 11.5 | 21.20 | V100 | 41.66 | 42.90 | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/danet/danet_r50-d8_4xb4-80k_ade20k-512x512.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/danet/danet_r50-d8_512x512_80k_ade20k/danet_r50-d8_512x512_80k_ade20k_20200615_015125-edb18e08.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/danet/danet_r50-d8_512x512_80k_ade20k/danet_r50-d8_512x512_80k_ade20k_20200615_015125.log.json) |
+| DANet | R-101-D8 | 512x512 | 80000 | 15 | 14.18 | V100 | 43.64 | 45.19 | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/danet/danet_r101-d8_4xb4-80k_ade20k-512x512.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/danet/danet_r101-d8_512x512_80k_ade20k/danet_r101-d8_512x512_80k_ade20k_20200615_015126-d0357c73.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/danet/danet_r101-d8_512x512_80k_ade20k/danet_r101-d8_512x512_80k_ade20k_20200615_015126.log.json) |
+| DANet | R-50-D8 | 512x512 | 160000 | - | - | V100 | 42.45 | 43.25 | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/danet/danet_r50-d8_4xb4-160k_ade20k-512x512.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/danet/danet_r50-d8_512x512_160k_ade20k/danet_r50-d8_512x512_160k_ade20k_20200616_082340-9cb35dcd.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/danet/danet_r50-d8_512x512_160k_ade20k/danet_r50-d8_512x512_160k_ade20k_20200616_082340.log.json) |
+| DANet | R-101-D8 | 512x512 | 160000 | - | - | V100 | 44.17 | 45.02 | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/danet/danet_r101-d8_4xb4-160k_ade20k-512x512.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/danet/danet_r101-d8_512x512_160k_ade20k/danet_r101-d8_512x512_160k_ade20k_20200616_082348-23bf12f9.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/danet/danet_r101-d8_512x512_160k_ade20k/danet_r101-d8_512x512_160k_ade20k_20200616_082348.log.json) |
### Pascal VOC 2012 + Aug
-| Method | Backbone | Crop Size | Lr schd | Mem (GB) | Inf time (fps) | Device | mIoU | mIoU(ms+flip) | config | download |
-| ------ | -------- | --------- | ------: | -------- | -------------- | ------ | ----: | ------------: | ---------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| DANet | R-50-D8 | 512x512 | 20000 | 6.5 | 20.94 | V100 | 74.45 | 75.69 | [config](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/configs/danet/danet_r50-d8_4xb4-20k_voc12aug-512x512.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/danet/danet_r50-d8_512x512_20k_voc12aug/danet_r50-d8_512x512_20k_voc12aug_20200618_070026-9e9e3ab3.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/danet/danet_r50-d8_512x512_20k_voc12aug/danet_r50-d8_512x512_20k_voc12aug_20200618_070026.log.json) |
-| DANet | R-101-D8 | 512x512 | 20000 | 9.9 | 13.76 | V100 | 76.02 | 77.23 | [config](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/configs/danet/danet_r101-d8_4xb4-20k_voc12aug-512x512.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/danet/danet_r101-d8_512x512_20k_voc12aug/danet_r101-d8_512x512_20k_voc12aug_20200618_070026-d48d23b2.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/danet/danet_r101-d8_512x512_20k_voc12aug/danet_r101-d8_512x512_20k_voc12aug_20200618_070026.log.json) |
-| DANet | R-50-D8 | 512x512 | 40000 | - | - | V100 | 76.37 | 77.29 | [config](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/configs/danet/danet_r50-d8_4xb4-40k_voc12aug-512x512.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/danet/danet_r50-d8_512x512_40k_voc12aug/danet_r50-d8_512x512_40k_voc12aug_20200613_235526-426e3a64.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/danet/danet_r50-d8_512x512_40k_voc12aug/danet_r50-d8_512x512_40k_voc12aug_20200613_235526.log.json) |
-| DANet | R-101-D8 | 512x512 | 40000 | - | - | V100 | 76.51 | 77.32 | [config](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/configs/danet/danet_r101-d8_4xb4-40k_voc12aug-512x512.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/danet/danet_r101-d8_512x512_40k_voc12aug/danet_r101-d8_512x512_40k_voc12aug_20200613_223031-788e232a.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/danet/danet_r101-d8_512x512_40k_voc12aug/danet_r101-d8_512x512_40k_voc12aug_20200613_223031.log.json) |
+| Method | Backbone | Crop Size | Lr schd | Mem (GB) | Inf time (fps) | Device | mIoU | mIoU(ms+flip) | config | download |
+| ------ | -------- | --------- | ------: | -------- | -------------- | ------ | ----: | ------------: | ------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| DANet | R-50-D8 | 512x512 | 20000 | 6.5 | 20.94 | V100 | 74.45 | 75.69 | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/danet/danet_r50-d8_4xb4-20k_voc12aug-512x512.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/danet/danet_r50-d8_512x512_20k_voc12aug/danet_r50-d8_512x512_20k_voc12aug_20200618_070026-9e9e3ab3.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/danet/danet_r50-d8_512x512_20k_voc12aug/danet_r50-d8_512x512_20k_voc12aug_20200618_070026.log.json) |
+| DANet | R-101-D8 | 512x512 | 20000 | 9.9 | 13.76 | V100 | 76.02 | 77.23 | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/danet/danet_r101-d8_4xb4-20k_voc12aug-512x512.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/danet/danet_r101-d8_512x512_20k_voc12aug/danet_r101-d8_512x512_20k_voc12aug_20200618_070026-d48d23b2.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/danet/danet_r101-d8_512x512_20k_voc12aug/danet_r101-d8_512x512_20k_voc12aug_20200618_070026.log.json) |
+| DANet | R-50-D8 | 512x512 | 40000 | - | - | V100 | 76.37 | 77.29 | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/danet/danet_r50-d8_4xb4-40k_voc12aug-512x512.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/danet/danet_r50-d8_512x512_40k_voc12aug/danet_r50-d8_512x512_40k_voc12aug_20200613_235526-426e3a64.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/danet/danet_r50-d8_512x512_40k_voc12aug/danet_r50-d8_512x512_40k_voc12aug_20200613_235526.log.json) |
+| DANet | R-101-D8 | 512x512 | 40000 | - | - | V100 | 76.51 | 77.32 | [config](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/danet/danet_r101-d8_4xb4-40k_voc12aug-512x512.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/danet/danet_r101-d8_512x512_40k_voc12aug/danet_r101-d8_512x512_40k_voc12aug_20200613_223031-788e232a.pth) \| [log](https://download.openmmlab.com/mmsegmentation/v0.5/danet/danet_r101-d8_512x512_40k_voc12aug/danet_r101-d8_512x512_40k_voc12aug_20200613_223031.log.json) |
## Citation
diff --git a/configs/ddrnet/README.md b/configs/ddrnet/README.md
new file mode 100644
index 00000000000..ccbfcdff359
--- /dev/null
+++ b/configs/ddrnet/README.md
@@ -0,0 +1,46 @@
+# DDRNet
+
+> [Deep Dual-resolution Networks for Real-time and Accurate Semantic Segmentation of Road Scenes](http://arxiv.org/abs/2101.06085)
+
+## Introduction
+
+
+
+Official Repo
+
+## Abstract
+
+
+
+Semantic segmentation is a key technology for autonomous vehicles to understand the surrounding scenes. The appealing performances of contemporary models usually come at the expense of heavy computations and lengthy inference time, which is intolerable for self-driving. Using light-weight architectures (encoder-decoder or two-pathway) or reasoning on low-resolution images, recent methods realize very fast scene parsing, even running at more than 100 FPS on a single 1080Ti GPU. However, there is still a significant gap in performance between these real-time methods and the models based on dilation backbones. To tackle this problem, we proposed a family of efficient backbones specially designed for real-time semantic segmentation. The proposed deep dual-resolution networks (DDRNets) are composed of two deep branches between which multiple bilateral fusions are performed. Additionally, we design a new contextual information extractor named Deep Aggregation Pyramid Pooling Module (DAPPM) to enlarge effective receptive fields and fuse multi-scale context based on low-resolution feature maps. Our method achieves a new state-of-the-art trade-off between accuracy and speed on both Cityscapes and CamVid dataset. In particular, on a single 2080Ti GPU, DDRNet-23-slim yields 77.4% mIoU at 102 FPS on Cityscapes test set and 74.7% mIoU at 230 FPS on CamVid test set. With widely used test augmentation, our method is superior to most state-of-the-art models and requires much less computation. Codes and trained models are available online.
+
+
+
+
+
+![\"Open](\"https://colab.research.google.com/assets/colab-badge.svg\"){kind=icon}
"
+ "