ERROR:root:Error in data_generator: Mismatch between number of class IDs (10) and number of masks (11) and generator_output = next(output_generator) StopIteration

[ZaidArman]

Hi,

I am trying to train this model for 3 days with the dataset I have. I am using TF 1.3, keras2.0.8, python 3.4 (the recommended versions)

My dataset consists of plants (species) and I have total 8 + 1 (classes + background). I've annotated the images of the plants using VGG Image annotator and then download the json file individually for training and testing. Am I doing something wrong with the annotation, perhaps? You can check the .json file here.

I am facing 2 error:
1:-
`
Starting at epoch 0. LR=0.001

Checkpoint Path: /Users/apple/Documents/Custom_MaskRCNN/logs/object20240703T2142/mask_rcnn_object_{epoch:04d}.h5
Selecting layers to train
fpn_c5p5 (Conv2D)
fpn_c4p4 (Conv2D)
fpn_c3p3 (Conv2D)
fpn_c2p2 (Conv2D)
fpn_p5 (Conv2D)
fpn_p2 (Conv2D)
fpn_p3 (Conv2D)
fpn_p4 (Conv2D)
In model: rpn_model
rpn_conv_shared (Conv2D)
rpn_class_raw (Conv2D)
rpn_bbox_pred (Conv2D)
mrcnn_mask_conv1 (TimeDistributed)
mrcnn_mask_bn1 (TimeDistributed)
mrcnn_mask_conv2 (TimeDistributed)
mrcnn_mask_bn2 (TimeDistributed)
mrcnn_class_conv1 (TimeDistributed)
mrcnn_class_bn1 (TimeDistributed)
mrcnn_mask_conv3 (TimeDistributed)
mrcnn_mask_bn3 (TimeDistributed)
mrcnn_class_conv2 (TimeDistributed)
mrcnn_class_bn2 (TimeDistributed)
mrcnn_mask_conv4 (TimeDistributed)
mrcnn_mask_bn4 (TimeDistributed)
mrcnn_bbox_fc (TimeDistributed)
mrcnn_mask_deconv (TimeDistributed)
mrcnn_class_logits (TimeDistributed)
mrcnn_mask (TimeDistributed)
Epoch 1/2
ERROR:root:Error in data_generator: Mismatch between number of class IDs (10) and number of masks (11)
ERROR:root:Error in data_generator: Mismatch between number of class IDs (10) and number of masks (11)
ERROR:root:Error in data_generator: Mismatch between number of class IDs (10) and number of masks (11)
ERROR:root:Error in data_generator: Mismatch between number of class IDs (10) and number of masks (11)
ERROR:root:Error in data_generator: Mismatch between number of class IDs (10) and number of masks (11)
ERROR:root:Error in data_generator: Mismatch between number of class IDs (10) and number of masks (11)
ERROR:root:Error in data_generator: Mismatch between number of class IDs (10) and number of masks (11)
ERROR:root:Error in data_generator: Mismatch between number of class IDs (10) and number of masks (11)
ERROR:root:Error in data_generator: Mismatch between number of class IDs (10) and number of masks (11)
ERROR:root:Error in data_generator: Mismatch between number of class IDs (10) and number of masks (11)
ERROR:root:Error in data_generator: Mismatch between number of class IDs (10) and number of masks (11)
ERROR:root:Error in data_generator: Mismatch between number of class IDs (10) and number of masks (11)
ERROR:root:Error in data_generator: Mismatch between number of class IDs (10) and number of masks (11)
ERROR:root:Error in data_generator: Mismatch between number of class IDs (10) and number of masks (11)
**2 error:-** Traceback (most recent call last):
File "custom.py", line 514, in
train(model)
File "custom.py", line 345, in train
augmentation=augmentation)
File "/Users/apple/Documents/Custom_MaskRCNN/mrcnn/model.py", line 2294, in train
use_multiprocessing=True,
File "/Users/apple/Documents/Custom_MaskRCNN/myenv345/lib/python3.6/site-packages/keras/legacy/interfaces.py", line 87, in wrapper
return func(*args, **kwargs)
File "/Users/apple/Documents/Custom_MaskRCNN/myenv345/lib/python3.6/site-packages/keras/engine/training.py", line 2064, in fit_generator
use_multiprocessing=use_multiprocessing)
File "/Users/apple/Documents/Custom_MaskRCNN/myenv345/lib/python3.6/site-packages/keras/legacy/interfaces.py", line 87, in wrapper
return func(*args, **kwargs)
File "/Users/apple/Documents/Custom_MaskRCNN/myenv345/lib/python3.6/site-packages/keras/engine/training.py", line 2160, in evaluate_generator
generator_output = next(output_generator)
StopIteration`

and here is my custom code
My Code:
`
import os
import sys
import json
import datetime
import numpy as np
import skimage.draw
import imgaug

import warnings
warnings.filterwarnings("ignore")

Root directory of the project

ROOT_DIR = os.path.abspath("../../")

Import Mask RCNN

sys.path.append(ROOT_DIR) # To find local version of the library
from mrcnn.config import Config
from mrcnn import model as modellib, utils

Path to trained weights file

COCO_WEIGHTS_PATH = os.path.join(ROOT_DIR, "mask_rcnn_coco.h5")

Directory to save logs and model checkpoints, if not provided

through the command line argument --logs

DEFAULT_LOGS_DIR = os.path.join(ROOT_DIR, "logs")

############################################################

Configurations

############################################################

class CustomConfig(Config):
NAME = "object"
IMAGES_PER_GPU = 2
NUM_CLASSES = 1 + 8 # Background + number of classes (Here, 2)
STEPS_PER_EPOCH = 100
DETECTION_MIN_CONFIDENCE = 0.9

############################################################

Dataset

############################################################
class CustomDataset(utils.Dataset):

def load_custom(self, dataset_dir, subset):
    self.add_class("object", 1, "Acacia nilotica L")
    self.add_class("object", 2, "Arundon Donax L")
    self.add_class("object", 3, "Canotia holacantha")
    self.add_class("object", 4, "Dodonaea viscosa")
    self.add_class("object", 5, "Kambilai")
    self.add_class("object", 6, "Sageretia thea var")
    self.add_class("object", 7, "Withania coagulans Dunal")
    self.add_class("object", 8, "Ziziphus nummularia")

    assert subset in ["train", "val"]
    dataset_dir = os.path.join(dataset_dir, subset)

    annotations1 = json.load(open(os.path.join(dataset_dir, "annotations.json")))        
    annotations = list(annotations1.values())  # don't need the dict keys

    annotations = [a for a in annotations if a['regions']]

    for a in annotations:
        polygons = [r['shape_attributes'] for r in a['regions']]
        objects = [s['region_attributes'].get('masskrcnn_dataset', None) for s in a['regions']]
        print("objects:", objects)

        objects = [obj for obj in objects if obj is not None]

        if not objects:
            continue

        name_dict = {
            "Acacia nilotica L": 1,
            "Arundon Donax L": 2,
            "Canotia holacantha": 3,
            "Dodonaea viscosa ": 4,
            "Kambilai": 5,
            "Sageretia thea var ": 6,
            "Withania coagulans Dunal ": 7,
            "Ziziphus nummularia ": 8
        }

        num_ids = [name_dict[a] for a in objects]
        print("num_ids", num_ids)

        image_path = os.path.join(dataset_dir, a['filename'])
        image = skimage.io.imread(image_path)
        height, width = image.shape[:2]

        self.add_image(
            "object",
            image_id=a['filename'],
            path=image_path,
            width=width, height=height,
            polygons=polygons,
            num_ids=num_ids
        )

def load_mask(self, image_id):
    image_info = self.image_info[image_id]
    if image_info["source"] != "object":
        return super(self.__class__, self).load_mask(image_id)

    info = self.image_info[image_id]
    if info["source"] != "object":
        return super(self.__class__, self).load_mask(image_id)

    num_ids = info['num_ids']
    mask = np.zeros([info["height"], info["width"], len(info["polygons"])],
                    dtype=np.uint8)
    for i, p in enumerate(info["polygons"]):
        rr, cc = skimage.draw.polygon(p['all_points_y'], p['all_points_x'])
        mask[rr, cc, i] = 1

    num_ids = np.array(num_ids, dtype=np.int32)
    return mask, num_ids

def image_reference(self, image_id):
    info = self.image_info[image_id]
    if info["source"] == "object":
        return info["path"]
    else:
        super(self.__class__, self).image_reference(image_id)

def train(model):
"""Train the model."""
dataset_train = CustomDataset()
dataset_train.load_custom(args.dataset, "train")
dataset_train.prepare()

dataset_val = CustomDataset()
dataset_val.load_custom(args.dataset, "val")
dataset_val.prepare()

augmentation = imgaug.augmenters.Sometimes(0.5, [
    imgaug.augmenters.Fliplr(0.5),
    imgaug.augmenters.Flipud(0.5),
    imgaug.augmenters.GaussianBlur(sigma=(0.0, 5.0))
])

print("\n Training network heads \n")
model.train(dataset_train, dataset_val,
            learning_rate=config.LEARNING_RATE,
            epochs=10,
            layers='heads',
            augmentation=augmentation)

def color_splash(image, mask):
gray = skimage.color.gray2rgb(skimage.color.rgb2gray(image)) * 255
if mask.shape[-1] > 0:
mask = (np.sum(mask, -1, keepdims=True) >= 1)
splash = np.where(mask, image, gray).astype(np.uint8)
else:
splash = gray.astype(np.uint8)
return splash

def detect_and_color_splash(model, image_path=None, video_path=None):
if image_path:
print("Running on {}".format(args.image))
image = skimage.io.imread(args.image)
r = model.detect([image], verbose=1)[0]
splash = color_splash(image, r['masks'])
file_name = "splash_{:%Y%m%dT%H%M%S}.png".format(datetime.datetime.now())
skimage.io.imsave(file_name, splash)
elif video_path:
import cv2
vcapture = cv2.VideoCapture(video_path)
width = int(vcapture.get(cv2.CAP_PROP_FRAME_WIDTH))
height = int(vcapture.get(cv2.CAP_PROP_FRAME_HEIGHT))
fps = vcapture.get(cv2.CAP_PROP_FPS)
file_name = "splash_{:%Y%m%dT%H%M%S}.avi".format(datetime.datetime.now())
vwriter = cv2.VideoWriter(file_name,
cv2.VideoWriter_fourcc(*'MJPG'),
fps, (width, height))

    count = 0
    success = True
    while success:
        print("frame: ", count)
        success, image = vcapture.read()
        if success:
            image = image[..., ::-1]
            r = model.detect([image], verbose=0)[0]
            splash = color_splash(image, r['masks'])
            splash = splash[..., ::-1]
            vwriter.write(splash)
            count += 1
    vwriter.release()
print("Saved to ", file_name)

############################################################

Training

############################################################

if name == 'main':
import argparse

# Parse command line arguments
parser = argparse.ArgumentParser(
    description='Train Mask R-CNN to detect custom objects.')
parser.add_argument("command",
                    metavar="<command>",
                    help="'train' or 'splash'")
parser.add_argument('--dataset', required=False,
                    metavar="/path/to/custom/dataset/",
                    help='Directory of the custom dataset')
parser.add_argument('--weights', required=True,
                    metavar="/path/to/weights.h5",
                    help="Path to weights .h5 file or 'coco'")
parser.add_argument('--logs', required=False,
                    default=DEFAULT_LOGS_DIR,
                    metavar="/path/to/logs/",
                    help='Logs and checkpoints directory (default=logs/)')
parser.add_argument('--image', required=False,
                    metavar="path or URL to image",
                    help='Image to apply the color splash effect on')
parser.add_argument('--video', required=False,
                    metavar="path or URL to video",
                    help='Video to apply the color splash effect on')
args = parser.parse_args()

# Validate arguments
if args.command == "train":
    assert args.dataset, "Argument --dataset is required for training"
elif args.command == "splash":
    assert args.image or args.video,\
           "Provide --image or --video to apply color splash"

print("Weights: ", args.weights)
print("Dataset: ", args.dataset)
print("Logs: ", args.logs)

# Configurations
if args.command == "train":
    config = CustomConfig()
else:
    class InferenceConfig(CustomConfig):
        # Set batch size to 1 since we'll be running inference on
        # one image at a time. Batch size = GPU_COUNT * IMAGES_PER_GPU
        GPU_COUNT = 1
        IMAGES_PER_GPU = 1
    config = InferenceConfig()
config.display()

# Create model
if args.command == "train":
    model = modellib.MaskRCNN(mode="training", config=config,
                              model_dir=args.logs)
else:
    model = modellib.MaskRCNN(mode="inference", config=config,
                              model_dir=args.logs)

# Select weights file to load
if args.weights.lower() == "new":	
    print("weight path entered")	
    print(NEW_WEIGHTS_PATH)	
    weights_path = NEW_WEIGHTS_PATH
if args.weights.lower() == "coco":
    weights_path = COCO_WEIGHTS_PATH
    # Download weights file
    if not os.path.exists(weights_path):
        utils.download_trained_weights(weights_path)
elif args.weights.lower() == "last":
    # Find last trained weights
    weights_path = model.find_last()
elif args.weights.lower() == "imagenet":
    # Start from ImageNet trained weights
    weights_path = model.get_imagenet_weights()
else:
    weights_path = args.weights

# Load weights
print("Loading weights ", weights_path)
if args.weights.lower() == "coco":
    # Exclude the last layers because they require a matching
    # number of classes
    model.load_weights(weights_path, by_name=True, exclude=[
        "mrcnn_class_logits", "mrcnn_bbox_fc",
        "mrcnn_bbox", "mrcnn_mask"])
else:
    model.load_weights(weights_path, by_name=True)

# Train or evaluate
if args.command == "train":
    train(model)
elif args.command == "splash":
    detect_and_color_splash(model, image_path=args.image,
                            video_path=args.video)
else:
    print("'{}' is not recognized. "
          "Use 'train' or 'splash'".format(args.command))`