Advanced Photon Source

An Office of Science National User Facility

SDDS and SDDS-Compliant Programs: A Modular System for Accelerator Design, Simulation, Control, and Analysis

System Analysis and Control
Accelerator Simulation
Obtaining Software


At the Advanced Photon Source, we use a unique framework for accelerator simulation, control, and analysis. Our method is based on the use of a common self-describing file protocol and a toolkit of commandline programs that manipulate these files. In addition, we use Tcl/Tk scripts to orchestrate the tools and create graphical user interfaces.

The heart of this method is SDDS, which stands for "Self-Describing Data Sets." It is the name of our self-describing file protocol as well as the name of the toolkit we have created to manipulate these files. SDDS software offers capabilities comparable to commercial analysis packages, but it is free, unlicensed, and open source. We support several platforms and languages, including C/C++, Fortran, Java, Tcl/Tk, and Python. As a result, SDDS is in use at several accelerator laboratories around the world.

This web page provides a brief introduction to our methodology and collects various resources in one place.


If you are new to SDDS, a good place to start is Getting Started with SDDS. This document provides an introduction to the concept behind SDDS and also a series of progressive examples. You can download sample data so you can work along with the examples. Below in this document, you will find links to presentations on SDDS as well as published papers .

The main documentation page for Accelerator Operations & Physics software contains links to manuals for the SDDS toolkit, SDDS-compliant EPICS tools, and Tcl/Tk extensions, as well as documentation for accelerator simulation programs supported by Accelerator Operations & Physics. A self-installing documentation package is available for Windows platforms, and may be obtained from the software download page.

System Analysis and Control

There are two "Toolkits" that use SDDS files. The "SDDS Toolkit" is completely general in application and can be used with data from any source, including a control system or simulation. It provides many features, such as device-independent graphics, fitting, statistics, histograms, digital signal processing, image analysis, and matrix operations. A more complete listing of capabilities is available in the manual.

Although SDDS was developed in order to help us control and analyze the APS accelerators, it is fully generic and can be used for many other control and analysis applications. Since APS uses the EPICS controls system, our data collection and control tools (the "SDDS/EPICS toolkit") use Channel Access. However, this is generally a very small part of the code and these tools could easily be adapted to other control systems. As described in the manual, the SDDS/EPICS toolkit includes programs for time-series data collection; glitch- and trigger-based data collection, configuration save, restore, and ramping; feedback; feedforward; optimization; and experiment execution.

At APS, we build all of our high-level accelerator applications using these tools, orchestrated by Tcl/Tk scripts. Tcl/Tk provides a very fast way to develop graphical user interfaces. SDDS gives functionality and "muscle" to these scripts, by putting the computationally intensive part of any job into compiled code. In addition, with SDDS one doesn't need to program in order to analyze data or control a process. Very frequently, large amounts of data can be reduced without any of the tradiational headaches of coding, such as defining variables, defining procedures, and setting up loops.

Accelerator Simulation

Several accelerator simulations are "SDDS-compliant," meaning that they use SDDS files for input and output. As far as the SDDS toolkit is concerned, a simulation program is just another source of data and requires no special treatment. We use the same tools in the office for simulation as we use in the control room for accelerator analysis.

It is usually very easy to make an existing simulation program SDDS-compliant, particularly for the output. The advantage of this is that SDDS then provides a free, robust, portable postprocessing facility for use with your simulation. That means it is no longer necessary to waste time adding and porting customized graphics and postprocessing features for your physics code. In addition, when you add physics to the code, the graphics and postprocessing are immediately available as long as the new data goes into SDDS files.

If you want to use Accelerator Operations & Physics' simulation codes, you can obtain source code and executables from our Web site, as discussed below. We also provide libraries for accessing SDDS files from various languages, including C/C++ and Fortran.

  • elegant is an accelerator modeling code that performs many functions. Its basic function is 6D tracking, which it performs using matrices (up to second order), symplectic integration, numerical integration, or a user-defined mixture. It computes Twiss parameters, transport matrices, radiation integrals, correction matrices, amplification factors, and floor coordinates. It also performs optimization, including optimization of radiation integrals, floor coordinates, transport matrices, and beam properties from tracking. A number of time-dependent elements are supported, such as rf cavities, kickers, and ramping machines. The code provides simulation of various collective effects, such as wakes and coherent synchrotron radiation.

    A description of elegant and some applications is available here. The applications include complete design of a small storage ring, top-up safety tracking for the APS, and design of a bunch compression system. elegant was also used for the first-ever start-to-end jitter simulations of an x-ray SASE FEL.

  • spiffe is a 2.5-D fully-electromagnetic particle-in-cell code for rf gun simulation. elegant will read the output of spiffe directly, integrated allowing simulations of a gun and linac. Since elegant simulates alpha magnets to third order, this is very convenient for thermionic rf gun simulations. (APS uses thermionic rf guns as injectors.) spiffe was used for design of the APS thermionic rf guns. A sample of some output from spiffe is available here.
  • shower is an EGS4 wrapper that provides easy setup and postprocessing for electron-gamma shower simulations. It is easier to use than EGS4 because one doesn't have to write any code.
  • GENESIS, by Sven Reiche, is a 3-D time-dependent FEL code. APS distributes an SDDS-compliant version of this code.


    A number of presentations are available on the Web that discuss the ideas behind our system.

  • The SDDS Toolkit is a brief introductory talk on SDDS and its use at APS. This talk covers the philosophy behind SDDS but has little detail.
  • Those desiring more detail should consult SDDS: A Modular Toolkit for Accelerator Simulation, Control, and Operation. This has a heavy emphasis on the capabilities of the SDDS/EPICS toolkit, which is used to acquire data from the EPICS control system and perform various control functions (e.g., feedback, feedforward, and optimization). Detailed examples are given, including bits of code and sample output. Accelerator physicists may find this one interesting, as may those involved in accelerator operations.

    Obtaining Software

    As mentioned above, all of our software is open source and available for downloading. For LINUX and Windows, we also distribute compiled code, so that you don't have to build it yourself. For other platforms, you may obtain the code (which is all in C) and build it locally.

    New users are often confused about what they need to download. At minimum, to use any of our software, you'll need to download the "SDDS Toolkit," which is a collection of commandline programs for processing and displaying SDDS files. If you want to run simulations, you'll need to download the simulations of interest. If you want to access data from the EPICS controls system, you'll need to download the "SDDS/EPICS toolkit." You don't need this if you are only interested in simulations, and you don't need the simulations if you are only interested in generic control systems applications.

    Another common pitfall is to download an upgrade of, say, elegant without downloading the upgrade of the SDDS toolkit. You should always upgrade the toolkit when you upgrade anything else. Generally, if you are using several of our products, you should download the most recent version for all of them whenever upgrading any one product.

    If you are a LINUX user, go to the LINUX installation guide. We use Redhat Package Manager to distribute the executables. It is relatively easy to build the software yourself if you use another type of LINUX.

    If you are a Windows users, go to the Windows installation guide. We supply self-installing executables for both codes and manuals. We strongly recommend getting the CYGWIN shell for use on Windows (a link is on the installation guide page) as it is far superior to DOS.

    If you are going to build from source, you'll need to obtain code from the source code page. You will need the program gnumake to build the code. To build the SDDS and SDDS/EPICS toolkits, you'll need the source for those toolkits, along with the EPICS configuration files. If you also want to build the simulation programs, you'll need the source for the simulations, along with the OAG configuration files. Help building locally may be obtained from Robert Soliday


    Papers on SDDS and its use in accelerator control and analysis.

  • M. Borland, ``A Universal Postprocessing Toolkit for Accelerator Simulation and Data Analysis,'' Proceedings of the 1998 ICAP Conference, Sept. 14-18, 1998, Monterey, California. PDF.
  • M. Borland, ``Applications Toolkit for Accelerator Control and Analysis,'' Proceedings of the 1997 Particle Accelerator Conference, May 12-16, 1997, Vancouver, Canada. PDF.
  • M. Borland et al, ``Doing Accelerator Physics Using SDDS, UNIX, and EPICS,'' Proceedings of the 1995 ICALEPS Conference, October, 1995, Chicago, Illinois. PDF.
  • M. Borland, L. Emery, ``The Self-Describing Data Sets File Protocol and Toolkit,'' Proceedings of the 1995 ICALEPS Conference, October, 1995, Chicago, Illinois. PDF.
  • M. Borland, ``A Self-Describing File Protocol for Simulation Integration and Shared Postprocessors,'' Proceedings of the 1995 Particle Accelerator Conference, May 1-5, 1995, Dallas, Texas. PDF.
  • L. Emery, ``Commissioning Software Tools at the Advanced Photon Source,'' Proceedings of the 1995 Particle Accelerator Conference, May 1-5, 1995, Dallas, Texas. PDF.

    Papers on SDDS and its use in accelerator simulations.

  • M. Borland, et al., ``Start-to-End Jitter Simulations of the Linac Coherent Light Source,'' Proceedings of the 2001 Particle Accelerator Conference, Chicago. PDF.
  • M. Borland,``Potential Production of Ultra-Short Electron Bunches with the Advanced Photon Source Linac,'' Proceedings of the 2001 Particle Accelerator Conference, Chicago. PDF.
  • M. Borland, ``elegant: A Flexible SDDS-Compliant Code for Accelerator Simulation,'' APS LS-287, presented at ICAP 2000, Darmstadt, Germany. PDF.
  • M. Borland, ``Design and performance simulations of the bunch compressor for the Advanced Photon Source Low-Energy Undulator Test Line free-electron laser,'' Phys. Rev. ST Accel. Beams 4, 074201 (2001). PDF.
  • M. Borland, L. Emery, ``Tracking Studies of Top-Up Safety for the Advanced Photon Source,'' Proceedings of 1999 Particle Accelerator Conference, March 29-April 2, New York, 2319-2321. PDF.
  • L. Emery, ``Beam Simulation and Radiation Dose Calculation at the Advanced Photon Source with shower, an Interface Program to the EGS4 Code System,'' Proceedings of the 1995 Particle Accelerator Conference, May 1-5, 1995, Dallas, Texas. PDF.

    modified 10/23/2001 by