You need to download LLMs onto your local system using the Hugging Face transformers Python package. The transformers package provides an interface to download models from the Hugging Face model hub.
Compress the content of the LLM directory by omitting the top directory.
You can download Hugging Face LLMs in either native format or streamlined format. The APPLY Python script that loads the language model must match the format.
Hugging Face - Native Format
This approach creates a directory named models--<model_owner>-<model_name> and downloads the model to the location under the default Hugging Face cache directory.
Use transformers Python package from_pretrained method to save the model by specifying the model in Hugging Face native format, that is, model_owner/model_path. In the following example, Helsinki-NLP is the model owner and opus-mt-en-fr is the model name.
# Use transformers to load Helsinki-NLP/opus-mt-en-fr LLM
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
tokenizer = AutoTokenizer.from_pretrained('Helsinki-NLP/opus-mt-en-fr')
model = AutoModelForSeq2SeqLM.from_pretrained('Helsinki-NLP/opus-mt-en-fr')
# On a linux machine the default Hugging Face cache directory is ~/.cache/huggingface/hub
# The model will be saved as ~/.cache/huggingface/hub/models--Helsinki-NLP--opus-mt-en-fr
Compress the saved model directory into zip file and upload it to user environment using install_model API.
Once the model is installed using the install_model API, APPLY Python scripts can load the model using Hugging Face native format, that is, model_owner/model_name .
# Load the model from cached location
model = AutoModelForSeq2SeqLM.from_pretrained("Helsinki-NLP/opus-mt-en-fr")
# Load the model from cached location
tokenizer = AutoTokenizer.from_pretrained("Helsinki-NLP/opus-mt-en-fr")
Hugging Face - Streamlined Format
You can download and save Hugging Face LLMs to your preferred directory in streamlined format by using transformers package save_pretrained method.
This example saves Helsinki-NLP/opus-mt-en-fr model into opus-mt-en-fr-local directory in the local system working directory.
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
tokenizer = AutoTokenizer.from_pretrained('Helsinki-NLP/opus-mt-en-fr')
model = AutoModelForSeq2SeqLM.from_pretrained('Helsinki-NLP/opus-mt-en-fr')
# Save model under directory name 'opus-mt-en-fr-local'
tokenizer.save_pretrained("./opus-mt-en-fr-local")
model.save_pretrained("./opus-mt-en-fr-local")
Compress the saved model directory into zip file and upload it to user environment using the install_model API.
If your model is installed using the install_model API, then the APPLY Python scripts must use either OPENAF_MODELS_DIR or ./models to prepend the model name to generate the model_path as follows:
model_path = os.environ.get('OPENAF_MODELS_DIR') + '/opus-mt-en-fr-local'
or
model_path = './models/opus-mt-en-fr-local'
Use the following syntax in your APPLY Python script to load it into memory.
# Either use `OPENAF_MODELS_DIR` or `./models` to append model name to generate model_path
# model_path = './models/opus-mt-en-fr-local'
model_path = os.environ.get('OPENAF_MODELS_DIR') + '/opus-mt-en-fr-local'
tokenizer = AutoTokenizer.from_pretrained(model_path)
model = AutoModelForSeq2SeqLM.from_pretrained(model_path)