An Office of Science National User Facility
SDDS and SDDS-Compliant Programs: A Modular System for Accelerator Design, Simulation, Control, and Analysis
At the Advanced Photon Source, we use a unique framework for accelerator simulation, control, and analysis. Our method is based on the use of a common self-describing file protocol and a toolkit of commandline programs that manipulate these files. In addition, we use Tcl/Tk scripts to orchestrate the tools and create graphical user interfaces.
The heart of this method is SDDS, which stands for "Self-Describing Data Sets." It is the name of our self-describing file protocol as well as the name of the toolkit we have created to manipulate these files. SDDS software offers capabilities comparable to commercial analysis packages, but it is free, unlicensed, and open source. We support several platforms and languages, including C/C++, Fortran, Java, Tcl/Tk, and Python. As a result, SDDS is in use at several accelerator laboratories around the world.
This web page provides a brief introduction to our methodology and collects various resources in one place.
If you are new to SDDS, a good place to start is Getting Started with SDDS. This document provides an introduction to the concept behind SDDS and also a series of progressive examples. You can download sample data so you can work along with the examples. Below in this document, you will find links to presentations on SDDS as well as published papers .
The main documentation page for Accelerator Operations & Physics software contains links to manuals for the SDDS toolkit, SDDS-compliant EPICS tools, and Tcl/Tk extensions, as well as documentation for accelerator simulation programs supported by Accelerator Operations & Physics. A self-installing documentation package is available for Windows platforms, and may be obtained from the software download page.
There are two "Toolkits" that use SDDS files. The "SDDS Toolkit" is completely general in application and can be used with data from any source, including a control system or simulation. It provides many features, such as device-independent graphics, fitting, statistics, histograms, digital signal processing, image analysis, and matrix operations. A more complete listing of capabilities is available in the manual.
Although SDDS was developed in order to help us control and analyze the APS accelerators, it is fully generic and can be used for many other control and analysis applications. Since APS uses the EPICS controls system, our data collection and control tools (the "SDDS/EPICS toolkit") use Channel Access. However, this is generally a very small part of the code and these tools could easily be adapted to other control systems. As described in the manual, the SDDS/EPICS toolkit includes programs for time-series data collection; glitch- and trigger-based data collection, configuration save, restore, and ramping; feedback; feedforward; optimization; and experiment execution.
At APS, we build all of our high-level accelerator applications using these tools, orchestrated by Tcl/Tk scripts. Tcl/Tk provides a very fast way to develop graphical user interfaces. SDDS gives functionality and "muscle" to these scripts, by putting the computationally intensive part of any job into compiled code. In addition, with SDDS one doesn't need to program in order to analyze data or control a process. Very frequently, large amounts of data can be reduced without any of the tradiational headaches of coding, such as defining variables, defining procedures, and setting up loops.
Several accelerator simulations are "SDDS-compliant," meaning that they use SDDS files for input and output. As far as the SDDS toolkit is concerned, a simulation program is just another source of data and requires no special treatment. We use the same tools in the office for simulation as we use in the control room for accelerator analysis.
It is usually very easy to make an existing simulation program SDDS-compliant, particularly for the output. The advantage of this is that SDDS then provides a free, robust, portable postprocessing facility for use with your simulation. That means it is no longer necessary to waste time adding and porting customized graphics and postprocessing features for your physics code. In addition, when you add physics to the code, the graphics and postprocessing are immediately available as long as the new data goes into SDDS files.
If you want to use Accelerator Operations & Physics' simulation codes, you can obtain source code and executables from our Web site, as discussed below. We also provide libraries for accessing SDDS files from various languages, including C/C++ and Fortran.
A description of elegant and some applications is available here. The applications include complete design of a small storage ring, top-up safety tracking for the APS, and design of a bunch compression system. elegant was also used for the first-ever start-to-end jitter simulations of an x-ray SASE FEL.
A number of presentations are available on the Web that discuss the ideas behind our system.
As mentioned above, all of our software is open source and available for downloading. For LINUX and Windows, we also distribute compiled code, so that you don't have to build it yourself. For other platforms, you may obtain the code (which is all in C) and build it locally.
New users are often confused about what they need to download. At minimum, to use any of our software, you'll need to download the "SDDS Toolkit," which is a collection of commandline programs for processing and displaying SDDS files. If you want to run simulations, you'll need to download the simulations of interest. If you want to access data from the EPICS controls system, you'll need to download the "SDDS/EPICS toolkit." You don't need this if you are only interested in simulations, and you don't need the simulations if you are only interested in generic control systems applications.
Another common pitfall is to download an upgrade of, say, elegant without downloading the upgrade of the SDDS toolkit. You should always upgrade the toolkit when you upgrade anything else. Generally, if you are using several of our products, you should download the most recent version for all of them whenever upgrading any one product.
If you are a LINUX user, go to the LINUX installation guide. We use Redhat Package Manager to distribute the executables. It is relatively easy to build the software yourself if you use another type of LINUX.
If you are a Windows users, go to the Windows installation guide. We supply self-installing executables for both codes and manuals. We strongly recommend getting the CYGWIN shell for use on Windows (a link is on the installation guide page) as it is far superior to DOS.
If you are going to build from source, you'll need to obtain code from the source code page. You will need the program gnumake to build the code. To build the SDDS and SDDS/EPICS toolkits, you'll need the source for those toolkits, along with the EPICS configuration files. If you also want to build the simulation programs, you'll need the source for the simulations, along with the OAG configuration files. Help building locally may be obtained from Robert Soliday firstname.lastname@example.org.
Papers on SDDS and its use in accelerator control and analysis.
Papers on SDDS and its use in accelerator simulations.
modified 10/23/2001 by email@example.com