Video Tutorials
ARC provides a number of video tutorials on our channel on video.vt.edu. In particular, the following sequence walks a user through the fundamentals of ARC usage in less than an hour:
Login
These videos will walk the user through accessing our systems for the first time (and streamlining access for subsequent logins):
Accessing Software
The following videos will walk the user through accessing software that ARC has installed or through setting up your own packages:
Scheduler interaction (job submission)
The following will walk the user through the process of submitting interactive jobs for testing/development and batch jobs for production research runs:
Note that these videos require a VT Login to access. Also, each video has a table of contents that can be used to skip between sections; this can be accessed by clicking the “hamburger” (three horizontal bars) button at the top left of the video.
We are actively updating this page.
Compute Nodes on Clusters
Each ARC cluster has multiple partitions. A partition is a set or collection of compute nodes. Often partitions (sets of compute nodes) are all of one type (i.e., architecture). However, sometimes one partition has nodes with different architectures. This video shows how to understand the partitions and compute node types on ARC clusters, which is a requirement for making good choices about how to get your computations done on the clusters.
Accessing ARC Clusters
This video covers ssh, Open OnDemand, and VPN as tools to access ARC clusters.
Different Drives on ARC Clusters
This video describes the various mounts/directories for doing different types of work.
Environments
Environments are collections of software that tailor an otherwise “basic” computing ecosystem into one that supports your particular computing needs.
Motivation
This video explains the software environment that exists when a user logs on and why this is not typically sufficient for most user needs. Environments are described as a way for users to customize their context to do work. Several videos below explain how to construct environments and make use of them.
Modules
Modules are software that enables you to customize your environment to get your work done. Modules are fundamental to clusters. This video explains how to use them.
Virtual environments (VEs)
There is a section below with several videos on how to structure directories where you put your VEs, how to build VEs, and how to use VEs.
How to Create and Use Virtual Environments
How to structure your VEs.
Because you can access all files created on any of the main three clusters (TC, Owl, Falcon), it is useful to structure your virtual environments so that you know, for a given VE, which cluster, partition, and compute node type you can run the VE on. (A VE must be used on the same [cluster, partition, compute node type] triple as it was constructed.)
This video shows how you might structure your directories where your VEs are located.
How to Create and Use a Python Conda Virtual Environment (VE) on Owl
This video shows you how to create a conda virtual environment on Owl. The process for other clusters are very similar.
How to Create and Use a Python Pip-Venv Virtual Environment (VE) on Owl
This video shows you how to create a pip-venv virtual environment on Owl. The process for other clusters are very similar.
For Python.
These videos are how to build VEs for various clusters and VE types when working in Python.
conda VE on Owl.
conda VE on Falcon.
pip VE on Owl.
pip VE on Falcon.
conda VE for Jupyter notebooks.
For Julia.
These videos are how to build VEs for various clusters and VE types when working in Python.
virtual environment
How to Use Virtual Environments (VEs)
Given the VEs built in the previous section, the videos in this section use those VEs to execute computations.
For Python.
Using a conda VE on Owl.
Using a conda VE on Falcon.
Using a pip VE on Owl.
Using a pip VE on Falcon.
Using a conda VE with Jupyter notebooks.
For Julia.
These videos are how to build VEs for various clusters and VE types when working in Python.
Using a VE specified outside Julia source code.
Using a VE specified inside Julia source code.
Running Jobs on ARC Clusters
The two dominant ways to run jobs on ARC clusters are:
interactive jobs
batch jobs.
Interactive Jobs
This video provides an example of an interactive job.
Batch Jobs
This video provides an example of a batch job.