⇦ ⇨ - previous / next font in set
, . - decrease / increase font size
DEL - remove font from current set Home - restore font set to initial

Swift/T - High Performance Dataflow Computing

Introduction

Swift/T is a completely new implementation of the Swift language for high-performance computing. In this implementation, the Swift script is translated into an MPI program that uses the Turbine (hence, /T) and ADLB runtime libraries for highly scalable dataflow processing over MPI, without single-node bottlenecks.

User Guide

Gallery

Downloads

Language Overview

Swift is a naturally concurrent language with C-like syntax. It is primarily used to manage calls to leaf tasks- external functions written in C, C++, Fortran, Python, R, Tcl, Julia, Qt Script, or executable programs. The Swift language coordinates the distribution of data to these tasks and schedules them for concurrent execution across the nodes of a large system.

Swift has conventional programming structures- functions, if, for, arrays, etc. Some functions are connected to leaf tasks, which are expected to do the bulk of the computationally intensive work. Tasks run when their input data is available. Data produced by a task may be fed into the next using syntax like:

y = f(x); z = g(y);

If tasks are able to run concurrently, they do: in this, case, two executions of g() run concurrently.

y = f(x); z1 = g(y, 1); z2 = g(y, 2);

Swift loops and other features provide additional concurrency constructs. Data movement is implicitly performed over MPI.

The Swift compiler, STC, breaks a Swift script into leaf tasks and control tasks. These are managed at runtime by the scalable, distributed-memory runtime consisting of Turbine and ADLB. For example, the following code is broken up into the task diagram:

int X = 100, Y = 100; int A[][]; int B[]; foreach x in [0:X-1] { foreach y in [0:Y-1] { if (check(x, y)) { A[x][y] = g(f(x), f(y)); } else { A[x][y] = 0; } } B[x] = sum(A[x]); }

Differences from Swift/K

The previous implementation of the Swift language is Swift/K, which runs on the Karajan grid workflow engine (hence, /K). Karajan and its libraries excel at wide-area computation, exploiting diverse schedulers (PBS, Condor, etc.) and data transfer technologies.

Swift/K is designed to execute a workflow of program executions across wide area resources.
Swift/T is designed to translate a Swift script into an MPI program composing native code libraries for execution on a single large system.

Migration to Swift/T

Swift/T is able to execute Swift/K-style app functions and is a natural migration from Swift/K. Key differences include:

Enhanced performance: 1.5 billion tasks/s
Ability to call native code functions (C, C++, Fortran)
Ability to execute scripts in embedded interpreters (Python, R, Tcl, Julia, etc.)
Enhanced builtin libraries (string, math, system, etc.)
Runs as a single MPI job

Advanced model exploration workflows

Swift/T enables high-performance model exploration workflows, such as parameter search, optimization, and classification, to run at large scale. See the
Extreme-scale Model Exploration Workflows with Swift/T (EMEWS) web site
for more information.