Midway (x86 cluster) -------------------- Midway is a cluster maintained by the Research Computing Center at the University of Chicago. Midway uses Slurm to handle job scheduling. For more details about Midway, and to request an account, visit http://rcc.uchicago.edu. Connecting to a login node ~~~~~~~~~~~~~~~~~~~~~~~~~~ Once you have access to Midway, connect to a Midway login node. ----- $ ssh userid@midway.rcc.uchicago.edu ----- Loading Swift ~~~~~~~~~~~~~ Swift is available on Midway as a module. To load the Swift, run: ----- $ module load swift ----- Example sites.xml ~~~~~~~~~~~~~~~~~ Below is an example that uses two of the queues available on Midway, sandyb and westmere. Be sure to adjust walltime, work directory, and other options as needed. ----- 16 00:05:00 100 100 sandyb 10000 /scratch/midway/{env.USER}/work 12 00:05:00 100 100 westmere 10000 /scratch/midway/{env.USER}/work ----- Example sites.xml for use with MPI ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Below is an example sites.xml that is suitable for running with MPI. Jobtype must be set to single. The value you set for ppn will determine the number of cores/slots your application uses per node. The value of count will set the number of nodes to request. The example below requests 2 nodes with 12 slots per node. ----- 1 12 _WALLTIME_ 100 100 westmere 10000 single 2 /scratch/midway/{env.USER}/work ----- Defining non-standard Slurm options ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ A Slurm submit script has many settings and options. Swift knows about many of the basic Slurm settings, like how to define a project or a queue, but it does not know about every setting. Swift provides a simple way to pass-thru your own settings into the Slurm submit script. The general way to do this is: ----- value ----- Here is one specific example. Slurm has the ability to notify users via email when a job is done. To make this happen, the Slurm submit script that Swift generates needs a line that contains "--mail-type=END". The following line will make it happen. ----- END ----- Any valid Slurm setting can be set in a similar way (see the sbatch man page for a list of all settings). Various tips for running MPI jobs ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ * You'll need to load an MPI module. Run "module load openmpi" to add to your path. * The app that Swift runs should be a wrapper script that invokes your MPI application by running "mpiexec /path/to/yourMPIApp"