-
Notifications
You must be signed in to change notification settings - Fork 2.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
C #3620
base: main
Are you sure you want to change the base?
C #3620
Conversation
* Bump 1.0.0rc2 * typo
Merge MMSegmentation 1.x development branch dev-1.x to main branch 1.x for v1.0.0rc2
* [Doc]Translate the 1_config.md and modify a wrong statement in 1_config.md * Translate the 1_config.md and modify a wrong statement in 1_config.md * Modify some expressions * Apply suggestions from code review
…orial (2355) doc modify part of content changed parts of content modified Update docs/zh_cn/user_guides/4_train_test.md Co-authored-by: 谢昕辰 <[email protected]>
…en-mmlab#2397) * CodeCamp open-mmlab#1562 [Doc] update `overview.md` * Update overview.md * Update docs/zh_cn/overview.md Co-authored-by: 谢昕辰 <[email protected]> * Update docs/zh_cn/overview.md Co-authored-by: 谢昕辰 <[email protected]>
* Add torch1.13 in CI * use mim install mm packages * install all requirements * install wheel * add ref
* [Doc] Add ZN datasets.md in dev-1.x * fix typo * fix * fix
…ted.md (open-mmlab#2417) * DOC Update docs/zh_cn/get_started.md Co-authored-by: 谢昕辰 <[email protected]>
…n-mmlab#2412) add example project add ci ignore add version limits
* draft * update loss * update * add runner * add steps * update
) * [Feature] Add Biomedical 3D array random crop transform * fix lint * fix gen crop bbox * fix gen crop bbox * docstring * typo Co-authored-by: MeowZheng <[email protected]>
…en-mmlab#2341) * [Doc] Change some content from customize_runtime to engine ZN doc * fix comments * add customize runtime setting zn doc * move optimizer content into one section * fix * fix * fix * fix
* tta init * use mmcv transform * test city * add multiscale * fix merge * add softmax to post process * add ut * add tta pipeline to other datasets * remove softmax * add encoder_decoder_tta ut * add encoder_decoder_tta ut * rename * rename file * rename config * rm aug_test * move flip to post process * fix channel
…ET_Mask2FormerHead` (open-mmlab#2448) ## Motivation Calling `mmseg.utils.register_all_modules` will import `MaskFormerHead` and `Mask2FormerHead`, it will crash if mmdet is not installed as `None` cannot be initialized. ## Modification - Modify `MMDET_MaskFormerHead=BaseModule` and `MMDET_Mask2FormerHead = BaseModule` when cannot import from mmdet
## Motivation The conflict is caused by: The user requested torch==1.12.1+cpu torchvision 0.13.0+cpu depends on torch==1.12.0 ## Modification modify the torch==1.12.0+cpu
## Motivation To release 1.0.0rc3 ## Modification 1. Modify mmseg version 2. Add change log 3. Modify README 4. Modify faq 5. Revise docker file
## Motivation implement nnUNet-style Gaussian Noise and Blur
Add the random gamma correction transform for biomedical images, which follows the design of the nnUNet.
## Motivation Add the 3d pad transform for biomedical images, which follows the design of the nnUNet.
## Motivation Support ISNet. paper link: [ISNet: Integrate Image-Level and Semantic-Level Context for Semantic Segmentation](https://openaccess.thecvf.com/content/ICCV2021/papers/Jin_ISNet_Integrate_Image-Level_and_Semantic-Level_Context_for_Semantic_Segmentation_ICCV_2021_paper.pdf) ## Modification Add ISNet decoder head. Add ISNet config.
…augmentation in dev-1.x. (open-mmlab#2432) ## Motivation Add Synapse dataset in MMSegmentation. Old PR: open-mmlab#2372.
## Motivation As title. ## Modification - projects/isnet/README.md
## Motivation as title ## Modification 1. .circleci/test.yml 2. .github/workflows/merge_stage_test.yml
…mmlab#2469) ## Motivation Fix incorrect `img_shape` value assignment. ## Modification - mmseg/datasets/transforms/transforms.py
## Motivation - Add Chinese version of Synapse dataset preparation. - Modify all `,` and `。` to `,` and `.` in `docs/zh_cn/user_guides/2_dataset_prepare.md`.
## Motivation The motivation of this PR is to add `gt_edge_map` field to support boundary loss. ## Modification - GenerateEdge Modify `gt_edge` field to `gt_edge_map`. - PackSegInputs Add `gt_edge_map` to data_sample. - stack_batch Pad `gt_edge_map` to max_shape. ## BC-breaking (Optional) No ## Use cases (Optional) Reference `GenerateEdge`.
…-mmlab#2444) ## Support `HieraSeg` interface on `cityscapes` ## Motivation Support `HieraSeg` interface on cityscapes dataset Paper link : https://ieeexplore.ieee.org/document/9878466/ ``` @Article{li2022deep, title={Deep Hierarchical Semantic Segmentation}, author={Li, Liulei and Zhou, Tianfei and Wang, Wenguan and Li, Jianwu and Yang, Yi}, journal={CVPR}, year={2022} } ``` ## Modification Add `HieraSeg_Projects` on `projects/` Add `sep_aspp_contrast_head` decoder head. Add `HieraSeg` config. Add `hiera_loss`, `hiera_triplet_loss_cityscape`, `tree_triplet_loss`
Thanks for your contribution and we appreciate it a lot. The following instructions would make your pull request more healthy and more easily get feedback. If you do not understand some items, don't worry, just make the pull request and seek help from maintainers. ## Motivation Support inference and visualization of VPD ## Modification 1. add a new VPD model that does not generate black border in predictions 2. update `SegLocalVisualizer` to support depth visualization 3. update `MMSegInferencer` to support save predictions of depth estimation in method `postprocess` ## BC-breaking (Optional) Does the modification introduce changes that break the backward-compatibility of the downstream repos? If so, please describe how it breaks the compatibility and how the downstream projects should modify their code to keep compatibility with this PR. ## Use cases (Optional) Run inference with VPD using the this command ```sh python demo/image_demo_with_inferencer.py demo/classroom__rgb_00283.jpg vpd_depth --out-dir vis_results ``` The following image will be saved under `vis_results/vis` ![classroom__rgb_00283](https://github.com/open-mmlab/mmsegmentation/assets/26127467/051e8c4b-8f92-495f-8c3e-f249aac888e3) ## Checklist 1. Pre-commit or other linting tools are used to fix the potential lint issues. 4. The modification is covered by complete unit tests. If not, please add more unit test to ensure the correctness. 5. If the modification has potential influence on downstream projects, this PR should be tested with downstream projects, like MMDet or MMDet3D. 6. The documentation has been modified accordingly, like docstring or example tutorials.
open-mmlab#3324) Thanks for your contribution and we appreciate it a lot. The following instructions would make your pull request more healthy and more easily get feedback. If you do not understand some items, don't worry, just make the pull request and seek help from maintainers. ## Motivation Use the pytorch-grad-cam tool to visualize Class Activation Maps (CAM). ## Modification Use the pytorch-grad-cam tool to visualize Class Activation Maps (CAM). requirement: pip install grad-cam run commad: python tools/analysis_tools/visualization_cam.py ## BC-breaking (Optional) Does the modification introduce changes that break the backward-compatibility of the downstream repos? If so, please describe how it breaks the compatibility and how the downstream projects should modify their code to keep compatibility with this PR. ## Use cases (Optional) If this PR introduces a new feature, it is better to list some use cases here, and update the documentation. ## Checklist 1. Pre-commit or other linting tools are used to fix the potential lint issues. 2. The modification is covered by complete unit tests. If not, please add more unit test to ensure the correctness. 3. The documentation has been modified accordingly, like docstring or example tutorials.
## Motivation Support SAN for Open-Vocabulary Semantic Segmentation Paper: [Side Adapter Network for Open-Vocabulary Semantic Segmentation](https://arxiv.org/abs/2302.12242) official Code: [SAN](https://github.com/MendelXu/SAN) ## Modification - Added the parameters of backbone vit for implementing the image encoder of CLIP. - Added text encoder code. - Added segmentor multimodel encoder-decoder code for open-vocabulary semantic segmentation. - Added SideAdapterNetwork decode head code. - Added config files for train and inference. - Added tools for converting pretrained models. - Added loss implementation for mask classification model, such as SAN, Maskformer and remove dependency on mmdetection. - Added test units for text encoder, multimodel encoder-decoder, san decode head and hungarian_assigner. ## Use cases ### Convert Models **pretrained SAN model** The official pretrained model can be downloaded from [san_clip_vit_b_16.pth](https://huggingface.co/Mendel192/san/blob/main/san_vit_b_16.pth) and [san_clip_vit_large_14.pth](https://huggingface.co/Mendel192/san/blob/main/san_vit_large_14.pth). Use tools/model_converters/san2mmseg.py to convert offcial model into mmseg style. `python tools/model_converters/san2mmseg.py <MODEL_PATH> <OUTPUT_PATH>` **pretrained CLIP model** Use the CLIP model provided by openai to train SAN. The CLIP model can be download from [ViT-B-16.pt](https://openaipublic.azureedge.net/clip/models/5806e77cd80f8b59890b7e101eabd078d9fb84e6937f9e85e4ecb61988df416f/ViT-B-16.pt) and [ViT-L-14-336px.pt](https://openaipublic.azureedge.net/clip/models/3035c92b350959924f9f00213499208652fc7ea050643e8b385c2dac08641f02/ViT-L-14-336px.pt). Use tools/model_converters/clip2mmseg.py to convert model into mmseg style. `python tools/model_converters/clip2mmseg.py <MODEL_PATH> <OUTPUT_PATH>` ### Inference test san_vit-base-16 model on coco-stuff164k dataset `python tools/test.py ./configs/san/san-vit-b16_coco-stuff164k-640x640.py <TRAINED_MODEL_PATH>` ### Train test san_vit-base-16 model on coco-stuff164k dataset `python tools/train.py ./configs/san/san-vit-b16_coco-stuff164k-640x640.py --cfg-options model.pretrained=<PRETRAINED_MODEL_PATH>` ## Comparision Results ### Train on COCO-Stuff164k | | | mIoU | mAcc | pAcc | | --------------- | ----- | ----- | ----- | ----- | | san-vit-base16 | official | 41.93 | 56.73 | 67.69 | | | mmseg | 41.93 | 56.84 | 67.84 | | san-vit-large14 | official | 45.57 | 59.52 | 69.76 | | | mmseg | 45.78 | 59.61 | 69.21 | ### Evaluate on Pascal Context | | | mIoU | mAcc | pAcc | | --------------- | ----- | ----- | ----- | ----- | | san-vit-base16 | official | 54.05 | 72.96 | 77.77 | | | mmseg | 54.04 | 73.74 | 77.71 | | san-vit-large14 | official | 57.53 | 77.56 | 78.89 | | | mmseg | 56.89 | 76.96 | 78.74 | ### Evaluate on Voc12Aug | | | mIoU | mAcc | pAcc | | --------------- | ----- | ----- | ----- | ----- | | san-vit-base16 | official | 93.86 | 96.61 | 97.11 | | | mmseg | 94.58 | 97.01 | 97.38 | | san-vit-large14 | official | 95.17 | 97.61 | 97.63 | | | mmseg | 95.58 | 97.75 | 97.79 | --------- Co-authored-by: CastleDream <[email protected]> Co-authored-by: yeedrag <[email protected]> Co-authored-by: Yang-ChangHui <[email protected]> Co-authored-by: Xu CAO <[email protected]> Co-authored-by: xiexinch <[email protected]> Co-authored-by: 小飞猪 <[email protected]>
…or NVIDIA Jetson (open-mmlab#3372) Fine tune ONNX Models (MMSegemetation) Inference for NVIDIA Jetson
## Motivation open-mmlab#3383 ## Modification - Add bpe_simple_vocab_16e6.txt.gz to `MANIFEST.in`
## Motivation open-mmlab#3384 ## Modification - mmseg/apis/inference.py
Thanks for your contribution and we appreciate it a lot. The following instructions would make your pull request more healthy and more easily get feedback. If you do not understand some items, don't worry, just make the pull request and seek help from maintainers. ## Motivation Fixes open-mmlab#3412 ## Modification We just need to replace tensor creation using torch.stack() instead of torch.tensor(). ## BC-breaking (Optional) Does the modification introduce changes that break the backward-compatibility of the downstream repos? If so, please describe how it breaks the compatibility and how the downstream projects should modify their code to keep compatibility with this PR. ## Use cases (Optional) If this PR introduces a new feature, it is better to list some use cases here, and update the documentation. ## Checklist 1. Pre-commit or other linting tools are used to fix the potential lint issues. 2. The modification is covered by complete unit tests. If not, please add more unit test to ensure the correctness. 3. If the modification has potential influence on downstream projects, this PR should be tested with downstream projects, like MMDet or MMDet3D. 4. The documentation has been modified accordingly, like docstring or example tutorials.
Thanks for your contribution and we appreciate it a lot. The following instructions would make your pull request more healthy and more easily get feedback. If you do not understand some items, don't worry, just make the pull request and seek help from maintainers. ## Motivation Current Visualization Hook can only get instances of `SegLocalVisualizer`. This makes impossible to use any other custom implementation. ## Modification This PR just allows to instantiate a different visualizer (following mmdetection implementation): https://github.com/open-mmlab/mmdetection/blob/main/mmdet/engine/hooks/visualization_hook.py#L58
## Motivation When using `test_cfg` for `data_preprocessor`, `predict_by_feat` resizes to the original size, not the padded size. ``` data_preprocessor = dict( type="SegDataPreProcessor", #type="SegDataPreProcessorWithPad", mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], bgr_to_rgb=True, pad_val=0, seg_pad_val=255, test_cfg=dict(size=(128, 128))) ``` Refar to: https://github.com/open-mmlab/mmsegmentation/blob/main/mmseg/models/decode_heads/san_head.py#L589-L592 ## Checklist 1. Pre-commit or other linting tools are used to fix the potential lint issues. 2. The modification is covered by complete unit tests. If not, please add more unit test to ensure the correctness. 3. If the modification has potential influence on downstream projects, this PR should be tested with downstream projects, like MMDet or MMDet3D. 4. The documentation has been modified accordingly, like docstring or example tutorials.
…t labels (open-mmlab#3466) Thanks for your contribution and we appreciate it a lot. The following instructions would make your pull request more healthy and more easily get feedback. If you do not understand some items, don't worry, just make the pull request and seek help from maintainers. ## Motivation It is difficult to visualize without "labels" when using the inferencer. - While using the `MMSegInferencer`, the visualized prediction contains labels on the mask, but it is difficult to pass `withLabels=False` without rewriting the config (which is harder to do when you initialize the inferencer with a model name rather than the config). - I thought it would be easier to just pass `withLabels=False` to `inferencer.__call__()` since you can also pass `opacity` and other parameters anyway. ## Modification Please briefly describe what modification is made in this PR. - Added `with_labels` to `visualize_kwargs` inside `MMSegInferencer`. - Modified to `visualize()` function. ## BC-breaking (Optional) Does the modification introduce changes that break the backward-compatibility of the downstream repos? If so, please describe how it breaks the compatibility and how the downstream projects should modify their code to keep compatibility with this PR. ## Use cases (Optional) If this PR introduces a new feature, it is better to list some use cases here, and update the documentation. ## Checklist 1. Pre-commit or other linting tools are used to fix the potential lint issues. 2. The modification is covered by complete unit tests. If not, please add more unit test to ensure the correctness. 3. If the modification has potential influence on downstream projects, this PR should be tested with downstream projects, like MMDet or MMDet3D. 4. The documentation has been modified accordingly, like docstring or example tutorials. --------- Co-authored-by: xiexinch <[email protected]>
|
Thanks for your contribution and we appreciate it a lot. The following instructions would make your pull request more healthy and more easily get feedback. If you do not understand some items, don't worry, just make the pull request and seek help from maintainers.
Motivation
Please describe the motivation of this PR and the goal you want to achieve through this PR.
Modification
Please briefly describe what modification is made in this PR.
BC-breaking (Optional)
Does the modification introduce changes that break the backward-compatibility of the downstream repos?
If so, please describe how it breaks the compatibility and how the downstream projects should modify their code to keep compatibility with this PR.
Use cases (Optional)
If this PR introduces a new feature, it is better to list some use cases here, and update the documentation.
Checklist