Artificial Intelligence

Large Language Models

ARC offers Large Language Models (LLMs) for research. There are three main ways researchers can use LLMs in ARC:

• https://llm.arc.vt.edu offers a no-effort web interface for a LLM. Users can use this tool for casual queries. Researchers should not use this tool for high-risk data. It does not offer API access. ARC will continously update it to run the best performing model publicly available from Hugging Face. The inference runs on GPUs within the ARC infrastructure. No data is sent to any 3rd party outside of the university. No data, prompts, or logs are preserved. Access is restricted through the VT network or VPN. It does not require to have an account with ARC.

• https://ood.arc.vt.edu offers a dedicated LLM via Open OnDemand. Users can use this tool for intensive queries. It offers web and API access secured via tokens using the OpenAI API. Users can select their preferred model to run from a list of models publicly available on Hugging Face. The inference runs on GPUs within the ARC infrastructure. No data is sent to any 3rd party outside of the university. Data, prompts, and logs are preserved within the user’s directory. Access is restricted through the VT network or VPN. It requires to have an account with ARC and a compute allocation.

• Advanced custom development via personalized Slurm scripts. Users can download any software to their user directory or run centrally-installed software (e.g. ollama) combined with custom Slurm scripts. Models downloaded from Hugging Face are available at /common/data/models/. Access is restricted through the VT network or VPN. It requires to have an account with ARC and a compute allocation.