How to fine tune LLaMA 3 in Google Colab (Pro)?

[yukiarimo]

I have a JSONL dataset like this:

{"text": "This is raw text in 2048 tokens I want to feed in"},
{"text": "This is next line, tokens are also 2048"}

It would be nice to fine-tune in 4, 8, or 16-bit LoRA and then just merge as before!

[ishandhanani]

Hi @yukiarimo - thanks for opening this issue! Creating a lighter version of the llama3 finetune is on our roadmap. It would be extremely helpful if you were able to edit the notebook and add a quantization configuration which would work on Colab's T4 GPUs!

Let me know!