Beauty sleep
LM with a revisor block learning to predict the next embedding weights.
Attempting to learn what it will learn with the effect of improving representations.
Turns out this is more of a framework for now for training :)
Lots of it was borrowed from https://github.com/HomebrewNLP/HomebrewNLP
python -m pip install -r requirements.txt
python3 main.py train --config_path configs/small.yaml