Meta-Llama-3.1-70B-Instruct does not appear to have a file named config.json

[jcruzer2012]

I submitted a request for access and obtained a key from the following URL: https://llama.meta.com/llama-downloads/

Instructions refer to download refer to this link : https://github.com/meta-llama/llama-models/blob/main/README.md

I replicated the download.sh on my system.

i ran ./download.sh

It asked the questions of which model i wanted, i selected the Meta-Llama-3.1-70B-Instruct which resulted in:

~/Meta-Llama-3.1-70B-Instruct$ ls
consolidated.00.pth  consolidated.02.pth  consolidated.04.pth  consolidated.06.pth  download.sh  tokenizer.model
consolidated.01.pth  consolidated.03.pth  consolidated.05.pth  consolidated.07.pth  params.json

In Juptyer Notebook I preformed the following Python Syntax:

from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained("Meta-Llama-3.1-70B-Instruct")
model = AutoModelForCausalLM.from_pretrained("Meta-Llama-3.1-70B-Instruct")

resulting in an error:

OSError: Meta-Llama-3.1-70B-Instruct does not appear to have a file named config.json. Checkout 'https://huggingface.co/Meta-Llama-3.1-70B-Instruct/tree/None' for available files.

I inspected the download.sh and it does not call for a config.json for the Llama-3.1-70B-Instruct? Maybe this is the cause of the error, I am do not know the file structure so i did not want to modify. It also appears that the config file exists on the hugging face site, however i am unsure how to gain access to the model their vs GitHub?

Regardless primary issues is the model wants a config.json.

[EmanuelaBoros]

I think maybe you should load it directly from HuggingFace, instead of manually downloading it (you will see the config there).

from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained("meta-llama/Meta-Llama-3.1-70B-Instruct")
model = AutoModelForCausalLM.from_pretrained("meta-llama/Meta-Llama-3.1-70B-Instruct")

[jcruzer2012]

@EmanuelaBoros This was the fix, it was simply a matter of reading the model card where, at the bottom was the request for access. Thank you!!