求助：在执行 loss.backwad() 时发生 RuntimeError，添加functional.reset_net后仍没有解决

[Ly0910]

感谢作者提出了这个框架！
我想构建一个基于Spikingjelly的SAC框架，SNN模型定义为：
self.network1 = nn.Sequential( layer.Linear(num_inputs + num_actions, hidden_dim), neuron.IFNode(), layer.Linear(hidden_dim, hidden_dim), neuron.IFNode(), layer.Linear(hidden_dim, 1) )
在运行时报错：
spikingjelly Trying to backward through the graph a second time, but the buffers have already been freed. Specify retain_graph=True when calling backward the first time.
参考了这个链接在SAC的update_parameters函数中增加了functional.reset_net函数，但是还是没有解决这个问题：
`def update_parameters(self, memory: Any,batch_size: int,updates: Any) -> Tuple[float, float,float, float, float]:
state_batch, action_batch, reward_batch, next_state_batch, mask_batch = memory.sample(batch_size=batch_size)
state_batch = torch.FloatTensor(state_batch).to(self.device)
next_state_batch = torch.FloatTensor(next_state_batch).to(self.device)
action_batch = torch.FloatTensor(action_batch).to(self.device)
reward_batch = torch.FloatTensor(reward_batch).to(self.device).unsqueeze(1)
mask_batch = torch.FloatTensor(mask_batch).to(self.device).unsqueeze(1)

    with torch.no_grad():
        next_state_action, next_state_log_pi, _ = self.policy.sample(next_state_batch)
        qf1_next_target, qf2_next_target = self.critic_target(next_state_batch,
                                                              next_state_action)
        min_qf_next_target = torch.min(qf1_next_target,
                                       qf2_next_target)- self.alpha * next_state_log_pi
        next_q_value = reward_batch + mask_batch * self.gamma * (min_qf_next_target)
    qf1, qf2 = self.critic(state_batch, action_batch)


    qf1_loss = F.mse_loss(qf1, next_q_value)
    qf2_loss = F.mse_loss(qf2, next_q_value)

    pi, log_pi, _ = self.policy.sample(state_batch)

    qf1_pi, qf2_pi = self.critic(state_batch, pi)
    min_qf_pi = torch.min(qf1_pi, qf2_pi)

    policy_loss = ((self.alpha * log_pi) - min_qf_pi).mean()

    critic_loss = min(qf1_loss,qf2_loss)

    self.critic_optim.zero_grad()
    qf1_loss.backward()
    self.critic_optim.step()
    # ### 增加reset_net以解决retain_graph报错的问题
    functional.reset_net(self.critic)
    functional.reset_net(self.critic_target)

    self.critic_optim.zero_grad()
    qf2_loss.backward()
    self.critic_optim.step()
    ## 增加reset_net以解决retain_graph报错的问题
    functional.reset_net(self.critic)
    functional.reset_net(self.critic_target)

    self.policy_optim.zero_grad()
    policy_loss.backward()
    self.policy_optim.step()
    # ### 增加reset_net解决retain_graph的问题
    functional.reset_net(self.critic)
    functional.reset_net(self.critic_target)
    torch.cuda.empty_cache()

    if self.automatic_entropy_tuning:
        alpha_loss = -(self.log_alpha * (log_pi
                                         + self.target_entropy).detach()).mean()

        self.alpha_optim.zero_grad()
        alpha_loss.backward()
        self.alpha_optim.step()

        self.alpha = self.log_alpha.exp()
        alpha_tlogs = self.alpha.clone()
    else:
        alpha_loss = torch.tensor(0.).to(self.device)
        alpha_tlogs = torch.tensor(self.alpha)


    if updates % self.target_update_interval == 0:
        soft_update(self.critic_target, self.critic, self.tau)

    return critic_loss.item(), qf2_loss.item(), policy_loss.item(), alpha_loss.item(), alpha_tlogs.item()`

卡住很久了，求大神指点functional.reset_net加的错在哪里！感谢！！