Swift Cookbook

This section takes you through the installation of the Swift system on your computer. We will start with the prerequisites as explained in the subsequent section.

This is simple. We will be using a pre-compiled version of Swift that can be downloaded from here. Download and untar the latest precompiled version as follows:

Your first Swift script Hello Swift-World!

A good sanity check that Swift is set up and running OK locally is this:

A good first tutorial in using Swift is at: http://www.ci.uchicago.edu/swift/guides/tutorial.php. Follow the steps in that tutorial to learn how to run a few simple scripts on the login host.

Following is a more involved Swift script.

A description of Swift Commandline Options

Also includes a description of Swift inputs and outputs.

Owing to its multi-threaded architecture it is often the case that the underlying java virtual machine gets into deadlock situations or Swift hangs because of other complications in its operations. Under such situations, Swift hang-checker chips in and gives indication of the situation.

I had a .rlog file from a Swift run that ran out of time. I resumed the run using the -resume flag described in section 16.2 of the Swift User Guide and it picked up where it left off. Then I killed it because I wanted to make changes to my sites file.

No new rlog file was emitted but it did recognize the progress that had been made, the 96 tasks that finished sucessfully above and resumed from 2558 tasks finished.

Notes to add (from Mike):

swift -resume ftdock-[id].0.rlog \[rest of the exact command line from initial run\]

Arrays can be passed to Swift in one of the following ways:

We will use the following Swift example, called "catsn" throughout the present section of this cookbook:

Beagle is a University of Chicago’s Computational Institute super computer dedicated to bioinformatics research. It is a cray XE6 machine running PBS job scheduler. More on Beagle could be found here.

Swift is available on Beagle as a software module. This recipe will walk you through a very simple example running Swift on Beagle. The prerequisites for running this example on Beagle is an access to and a project allocation on Beagle. The example is a simplest form of a real application which involves reading from a disc file and writing onto another disc file. It runs the /bin/cat Unix command to read an input file called data.txt and write it to an output file in a folder outdir, called f.nnn.out.

The steps are as follows:

step 1. Load the Swift module on Beagle commandline as follows: $ module load swift

step 2. Create and change to a directory where your Swift related work (source, inputs, outputs) will stay. (say, $ mkdir swift-lab, followed by, $ cd swift-lab)

step 3. copy the folder at /home/ketan/labs/catsn to the above directory. ($cp -r /home/ketan/catsn . followed by $ cd catsn).

step 4. In the sites file: sites.xml make the following two changes:

step 5. Run the example using following commandline:

You can further change the value of -n to any arbitrary number to run that many number of concurrent cat

step 6. Check the output in the generated outdir directory (ls outdir)

Swift on PADS To execute your Swift script on the PADS cluster use this command:

where the contents of a simple pbs.xml sites file could be:

Fusion is an Argonne-LCRC computational facility. More information on Fusion can be found here.

In the current section, we present a recipe to use Swift on Fusion system. Fusion employs a PBS schedular. Consequently, we will use the Coasters PBS provider. The jobmanager string will be "local:pbs" as follows:

The complete sites.xml file is as follows:

This section describes how to get Swift running on the OSG Grid. We will use a manual coaster setup to get Swift running on OSG.

The following figure shows an abstract scheme for the manual coasters setup on OSG.

In the following steps, we will go through the process of manually setting

This section explains a step by step procedure on getting Swift running on the Bionimbus cloud. We will use the manual coasters configuration on the Bionimbus cloud.

step1. Connect to the gateway (ssh gatewayx.lac.uic.edu)

step2. Start a virtual machine (euca-run-instances -n 1 -t m1.small emi-17EB1170)

step3. Start the coaster-service on gateway coaster-service -port 1984 -localport 35753 -nosec

step4. Start the Swift-script from the gateway using normal Swift commandline

swift -config cf -tc.file tc -sites.file sites.xml yourscript.swift -aparam=999

(See below for a sample sites.xml for this run)

step5. Connect back to gateway from virtual machines using reverse ssh tunneling as follows:

ssh -R *:5000:localhost:5000 root@10.101.8.50 sleep 999

WHERE: *=network interface, should remain the same on all cases

localhost=the gateway host, should remain the same

5000(LEFT OF localhost)=the port number on localhost to listen to **THIS WILL vary depending upon which port you want to listen to

5000(RIGHT OF localhost)=the port on target host that you want to forward

root@10.101.8.50=the ip of the Virtual Machine on Bionimbus cloud, this will vary based on what ip you get for your Virtual Machine instance

#On anywhere as long as provide the correct callback uri: here the "http://140.221.9.110:42195" is the callback uri of previous ones

step6. Start the worker from the virtual machine worker.pl http://localhost:42195 tmp /tmp # where 42195 is the port where the coaster service is listening to the workers

(assuming the log is titled swift-run.log)

or

When reporting problems to swift-user@ci.uchicago.edu, please attach the following files and information: