Midway (x86 cluster)
--------------------
Midway is a cluster maintained by the Research Computing Center
at the University of Chicago. Midway uses Slurm to handle job
scheduling. For more details about Midway, and to request
an account, visit http://rcc.uchicago.edu.
Connecting to a login node
~~~~~~~~~~~~~~~~~~~~~~~~~~
Once you have access to Midway, connect to
a Midway login node.
-----
$ ssh userid@midway.rcc.uchicago.edu
-----
Loading Swift
~~~~~~~~~~~~~
Swift is available on Midway as a module. To load the
Swift, run:
-----
$ module load swift
-----
Example sites.xml
~~~~~~~~~~~~~~~~~
Below is an example that uses two of the queues available
on Midway, sandyb and westmere. Be sure to adjust walltime,
work directory, and other options as needed.
-----
16
00:05:00
100
100
sandyb
10000
/scratch/midway/{env.USER}/work
12
00:05:00
100
100
westmere
10000
/scratch/midway/{env.USER}/work
-----
Example sites.xml for use with MPI
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Below is an example sites.xml that is suitable for running with MPI.
Jobtype must be set to single. The value you set for ppn
will determine the number of cores/slots your application uses per node.
The value of count will set the number of nodes to request. The example
below requests 2 nodes with 12 slots per node.
-----
1
12
_WALLTIME_
100
100
westmere
10000
single
2
/scratch/midway/{env.USER}/work
-----
Defining non-standard Slurm options
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
A Slurm submit script has many settings and options. Swift knows about many of
the basic Slurm settings, like how to define a project or a queue, but it does
not know about every setting. Swift provides a simple way to pass-thru your own
settings into the Slurm submit script.
The general way to do this is:
-----
value
-----
Here is one specific example. Slurm has the ability to notify users via email
when a job is done. To make this happen, the Slurm submit script that Swift
generates needs a line that contains "--mail-type=END". The following line
will make it happen.
-----
END
-----
Any valid Slurm setting can be set in a similar way (see the sbatch man page
for a list of all settings).
Various tips for running MPI jobs
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
* You'll need to load an MPI module. Run "module load openmpi" to add to your path.
* The app that Swift runs should be a wrapper script that invokes your
MPI application by running "mpiexec /path/to/yourMPIApp"