You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
MMEngine provides the wrapper AmpOptimWrapper for auto-mixing precision training, just set type='AmpOptimWrapper' in optim_wrapper to enable auto-mixing precision training, no other code changes are needed.
The example code builds a Runner and sets an AmpOptimWrapper.
However, accordings to this documentation of the Optim_Wrapper, optim_context has to be called in order to activate the mixed precision training for the model.
I checked by putting a breakpoint into the optim_context method when using a Runner and found that the optim_context method is never called when using the Runner. And also there does not seem to be any performance change as well.
So my conclusion is that either
The documentation here is wrong that optim_context has to be called in order for mixed precision to work and autocast was simply not able to have any impact on my model or
The documentation here is wrong that "no other code changes are needed" or
There is a bug in the Runner class which does not factor in how the AmpOptimWrapper has to be used.
Suggest a potential alternative/fix
Either change the Runner to call the optim_context method if an AmpOptimWrapper is specified or further specify here what additional steps are needed in order to make the AmpOptimWrapper work with the Runner class.
The text was updated successfully, but these errors were encountered:
📚 The doc issue
According to this documentation:
The example code builds a Runner and sets an AmpOptimWrapper.
However, accordings to this documentation of the Optim_Wrapper,
optim_context
has to be called in order to activate the mixed precision training for the model.I checked by putting a breakpoint into the
optim_context
method when using a Runner and found that theoptim_context
method is never called when using the Runner. And also there does not seem to be any performance change as well.So my conclusion is that either
optim_context
has to be called in order for mixed precision to work and autocast was simply not able to have any impact on my model orSuggest a potential alternative/fix
Either change the Runner to call the optim_context method if an AmpOptimWrapper is specified or further specify here what additional steps are needed in order to make the AmpOptimWrapper work with the Runner class.
The text was updated successfully, but these errors were encountered: