Input image size

[marthajoddrell]

Branch

main branch (mmpretrain version)

Describe the bug

I'm trying to change the input size of the mmpretrain MAE https://mmpretrain.readthedocs.io/en/latest/papers/mae.html

The current code takes an input (224x224). I need a larger input size of (998x479) but I don't know how to update this in the code. When I change the input size, I get the error below:

RuntimeError: The size of tensor a (1890) must match the size of tensor b (196) at non-singleton dimension 1

##Code File 1:

base = [
'/home/Documents/MAENEW/segmentation/mae_vit-base-p16.py',
'/home/Documents/MAENEW/segmentation/imagenet_bs512_mae.py',
'/home/Documents/MAENEW/segmentation/default_runtime.py',
]

optim_wrapper = dict(
type='AmpOptimWrapper',
loss_scale='dynamic',
optimizer=dict(
type='AdamW',
lr=1.5e-4 * 4096 / 256,
betas=(0.9, 0.95),
weight_decay=0.05),
paramwise_cfg=dict(
custom_keys={
'ln': dict(decay_mult=0.0),
'bias': dict(decay_mult=0.0),
'pos_embed': dict(decay_mult=0.),
'mask_token': dict(decay_mult=0.),
'cls_token': dict(decay_mult=0.)
}))

param_scheduler = [
dict(
type='LinearLR',
start_factor=0.0001,
by_epoch=True,
begin=0,
end=40,
convert_to_iter_based=True),
dict(
type='CosineAnnealingLR',
T_max=260,
by_epoch=True,
begin=40,
end=300,
convert_to_iter_based=True)
]

train_cfg = dict(type='EpochBasedTrainLoop', max_epochs=300)
default_hooks = dict(
checkpoint=dict(type='CheckpointHook', interval=1, max_keep_ckpts=3))

randomness = dict(seed=0, diff_rank_seed=True)
resume = True
auto_scale_lr = dict(base_batch_size=4096)

---

##Code file 2

dataset_type = 'CustomDataset'
data_root = '/home/Documents/MAENEW/data/'
data_preprocessor = dict(
type='SelfSupDataPreprocessor',
mean=[123.675, 116.28, 103.53],
std=[58.395, 57.12, 57.375],
to_rgb=True)

train_pipeline = [
dict(type='LoadImageFromFile'),
dict(
type='RandomResizedCrop',
scale=(998,479),
crop_ratio_range=(0.2, 1.0),
backend='pillow',
interpolation='bicubic'),
dict(type='PackInputs')
]

train_dataloader = dict(
batch_size=100,
num_workers=8,
persistent_workers=True,
sampler=dict(type='DefaultSampler', shuffle=True),
collate_fn=dict(type='default_collate'),
dataset=dict(
type='CustomDataset',
data_root='/home/Documents/MAENEW/data/',
pipeline=train_pipeline))

---

##Code file 3

model = dict(
type='MAE',
backbone=dict(type='MAEViT', arch='b', patch_size=16, mask_ratio=0.75),
neck=dict(
type='MAEPretrainDecoder',
patch_size=16,
in_chans=3,
embed_dim=768,
decoder_embed_dim=512,
decoder_depth=8,
decoder_num_heads=16,
mlp_ratio=4.,
),
head=dict(
type='MAEPretrainHead',
norm_pix=True,
patch_size=16,
loss=dict(type='PixelReconstructionLoss', criterion='L2')),
init_cfg=[
dict(type='Xavier', layer='Linear', distribution='uniform'),
dict(type='Constant', layer='LayerNorm', val=1.0, bias=0.0)
])