Running Swift
-------------

[[running:running-swift]]
Invoking Swift
~~~~~~~~~~~~~~

Swift can be invoked from the command line using the following syntax:

[listing, shell]
----
swift <options> [<file> [<arguments>]]
----

In general, +<file>+ is a Swift program and is required unless an *immediate*
option is specified, which causes Swift to perform a certain action without 
running any programs.

The +<arguments>+ can be specified and they will be passed to the Swift program
where they can be accessed using either the <<library-v1:arg, arg>> or
<<library-v2:args, args>> functions.

The valid options are as follows:

+-help+ or +-h+:: Tells Swift to displays usage and option information and quit

+-version+:: Displays the Swift version and quits

+-recompile+::
      Forces Swift to re-compile the invoked Swift script. While Swift
      is meant to detect when recompilation is necessary, in some
      special cases it fails to do so. This flag helps with those
      special cases.

+-dryrun+::
      Runs the SwiftScript program without submitting any jobs (can be
      used to get a graph)

+-resume <logfile>+::
      Resumes the execution using a log file

+-config <file>+::
      Indicates the run configuration file to be used for this run. If
      no custom configuration search path is specified, this will
      replace the default run configuration file (+./swift.conf+) if it
      exists, or append to the configuration search path otherwise. If
      a search path is specified, the value of this argument will be
      appended to the search path.

+-configpath <directories>+::
      Specifies a custom configuration search path. If supplied, the
      default configuration search path will be ignored and
      configurations will be loaded from files specified in the value
      of this argument. The files in the value must be separated using
      the operating system's path separator.

+-listconfig (files | full)+::
      Lists details about the loaded configuration. If 'files', lists only
      the configuration files used, while 'full' lists both the files
      and all the property values.

+-sitelist+::
      Prints a list of sites available in the swift configuration

+-verbose+ or +-v+::
      Increases the level of output that Swift produces on the console
      to include more detail about the execution

+-debug+ or +-d+::
      Increases the level of output that Swift produces on the console
      to include lots of detail about the execution

+-logfile <file>+::
      Specifies a file where log messages should go to. By default
      Swift uses the name of the SwiftScript program being run and
      additional information to make the name unique.

+-cdmfile <file>+::
      Specifies a CDM policy file.

+-runid <string>+::
      Specifies the run identifier. This must be unique for every
      invocation of a script and is used in several places to keep
      files from different runs cleanly separated. By default, a
      datestamp and random number are used to generate a run
      identifier.

+-ui <summary | text | TUI[:port] | http[:[password@]port]+::
      Indicates how swift should display run-time information. The
      following are valid values:
      +
        +none+:::: does not print any progress information
       
        +summary+:::: (default) - causes swift to regularly print a count of 
        jobs for each state that a job can be in
        
        +text+:::: regularly prints a more detailed table with Swift run-time
        information
        
        +TUI+:::: displays Swift run-time information using an interactive text 
        user interface. The terminal from which Swift is run must support 
        standard ANSI/VT100 escape sequences. If a port is specified, the 
        interface will also be available via +telnet+ at the specified port.
      
        +http+:::: enables an http server allowing access to swift run-time 
        information using a web browser. If the port is zero or unspecified,
        Swift will print the URL of the UI service.

+-reducedLogging+::
      Makes logging more terse by disabling provenance information and
      low-level task messages

+-minimalLogging+::
      Makes logging much more terse: reports warnings only

+-pauseOnStart+::
      Pauses execution on start. Useful for attaching a debugger or
      profiler to the Swift process

+-e <string>+::
      Runs the swift script code contained in +<string>+

In addition, a number of 
<<configuration:general-swift-options, general Swift options>> can be
specified as command line options. Properties specified on the command line
always override all other properties. The properties that can be specified
on the command line are: <<gprop:sites, sites>>, <<gprop:hostName, hostName>>,
<<gprop:TCPPortRange, TCPPortRange>>, <<gprop:lazyErrors, lazyErrors>>,
<<gprop:keepSiteDir, keepSiteDir>>, 
<<gprop:alwaysTransferWrapperLog, alwaysTransferWrapperLog>>, 
<<gprop:logProvenance, logProvenance>>+, 
<<gprop:fileGCEnabled, fileGCEnabled>>, 
<<gprop:mappingCheckerEnabled, mappingCheckerEnabled>>, 
<<gprop:tracingEnabled, tracingEnabled>>,
<<gprop:maxForeachThreads, maxForeachThreads>>,
<<gprop:CDMFile, CDMFile>>


Run directories
~~~~~~~~~~~~~~~
Every time a Swift program is run, a run directory gets created in the 
directory from which Swift is invoked. The run directory has the name of 
+runNNN+, where +NNN+ is a sequence of three digits starting at +001+ and is 
incremented for each subsequent run.

The run directories are most useful for debugging. They contain the following
files:

+swift.log+:: The main Swift log file
+scriptname-runNNN.d+:: A debug directory containing application invocation logs
+scripts+:: (optional) A directory that may contain scheduler scripts used by 
some of the execution mechanisms (e.g. +PBS+, +LSF+, +SLURM+) to submit jobs to 
resource managers

Tools
~~~~~

Monitoring Tools
^^^^^^^^^^^^^^^^

Monitoring tools can be started using the +-ui+ command line argument when 
invoking Swift. The two most notable versions are the *TUI* and the *HTTP UI*.

TUI
+++

The TUI displays an interactive console interface which allows some basic monitoring.

image:tui-main.png[width="100%"]

HTTP UI
+++++++

The HTTP UI is a more featured monitoring tool for Swift. It supports plotting of various Swift metrics as well as live worker node data (if used with
<<configuration:coasters, Coasters>>). 

In order to use the HTTP UI, Swift should be invoked with the +-ui http[:<port>]+ command line option. If a port is specified, Swift will bind the monitoring service to that port, otherwise a random TCP port will be used and Swift will print the monitor URL on the command line. 


Troubleshooting
~~~~~~~~~~~~~~~

Performance Tips
~~~~~~~~~~~~~~~~

Shared Filesystems::
  * Shared filesystems can be performance bottlenecks. One of the reasons this happens is because they need to ensure various forms of consistency when accessed from multiple locations concurrently. In addition, shared filesystems are in general used concurrently by multiple applications and users. With Swift, it is possible to avoid shared filesystems entirely, which can lead to dramatic performance improvements in some cases. This can be done by having both the Swift installation and the input files on a local disk, then selecting a work directory that lives in local disk (or ramdisk) on the compute node, and by using provider staging.
  * If the an application uses large files that would not fit on compute node local storage, one can use the above strategy to avoid shared filesystems in general, but then map the large files using "direct://" mode. This will ensure that most files avoid shared filesystems entirely, and use shared filesystems only when strictly necessary.
  
The Swift Runtime::
  * Avoid using the built-in monitoring tools. Instead, use the +swift-log-info+ tool with the +-f+ option, which incrementally parses a Swift run log as it is generated and can provide the same information as the built-in monitoring tools.
  * After ensuring that a Swift application runs correctly, when running in production mode, disable costly features used for troubleshooting, such as:
    ** Run with the +minimalLogging+ option. Large Swift runs can generate a lot of logging information. Writing the log to disk, as well as constructing all the log messages can be expensive.
    ** Disable the mapping checker using +mappingCheckerEnabled: false+ in +swift.conf+. The mapping checker uses memory and causes thread contention.
    ** Disable replication unless you know you need it.
    ** Set +statusMode: provider+ in +swift.conf+. This avoids some I/O.
    ** Unless you are running on unreliable resources, set +executionRetries+ to 0. This allows the Swift garbage collector to free memory associated with file names and application command line arguments as soon as a task is sent for execution. By contrast, if +executionRetries+ is not zero, Swift needs to keep file and argument information in memory until it is known that the application has completed successfully (or the maximum number of retries is exceeded).
    ** If you are using Coasters, and the Coaster Service is on the same machine as the Swift client, use +staging: service-local+ instead of +staging: local+. There is no need to read files on the client and proxy them through the service and the service can directly read those files.