FEDFormer block error

[Nutingnon]

I was running experiment with FEDFormer, however, I met error in the FourierCorrelation.py at this line:

            out_ft[:, :, :, wi] = self.compl_mul1d("bhi,hio->bho", x_ft[:, :, :, i],
                                                   torch.complex(self.weights1, self.weights2)[:, :, :, wi])

The error message is:
RuntimeError: einsum(): subscript h has size 8 for operand 1 which does not broadcast with previously seen size 4

I print the following information before the above code as this:

print(x_ft[:, :, :, i].size(), self.weights1.shape, self.weights2.shape, torch.complex(self.weights1, self.weights2)[:, :, :, wi].size())

The result is:
torch.Size([128, 4, 4]) torch.Size([8, 2, 2, 32]) torch.Size([8, 2, 2, 32]) torch.Size([8, 2, 2])

Then, I further double check the tensor information in the compl_mul1d function at the following lines

        if x_flag or w_flag:
            print("here: x size:", x.real.size(), weights.imag.size())

            return torch.complex(torch.einsum(order, x.real, weights.real) - torch.einsum(order, x.imag, weights.imag),
                                 torch.einsum(order, x.real, weights.imag) + torch.einsum(order, x.imag, weights.real))

and I get :
here: x size: torch.Size([128, 4, 4]) torch.Size([8, 2, 2])

I guess there is a bug in the implementation of FourierCorrelation.py file, I can successfully run my training data through other models such as AutoFormer, iTransformer, Informer, Mamba through your implementations.

Thank you for your great work and look forward to your answer.

[Musongwhk]

Hi. We do not find this issue during our experiment. Could you please provide us with the dataset and script you used to help us fix the bug?

[Nutingnon]

Sure, can you please give me an email address?

[Musongwhk]

[email protected]
This is my email address. Thank you for your issue and discussion.