A Brief Introduction to the SDDS Toolkit
Michael Borland
Table of Contents
A Brief Introduction to the SDDS ToolkitThe SDDS Toolkit
Michael Borland
Operations Analysis Group
What do the following items from recent operations logbooks have in common?
· SCR files of the injectors were saved.
· RMS beam motion is: x=4.028um, y=1.821um.
· 102mA stored beam, running orbit correction.
· Storage Ring tunes are: x=0.1948, y=0.2753.
· The Storage Ring RF configuration was compared to the reference file.
· Steering complete for 33ID.
· G. Decker is collecting fast beam history data.
· No problems detected with 48 data loggers.
· Louis Emery topped up the ring to 100mA using the top-up software.
· Singlet bunch pattern studies were performed by M. Borland until R. Merl arrived to do the top-up current monitor studies.
Answer:
All of these entries are referring to activities that depend on a software system called the SDDS Toolkit.
The Toolkit was not designed with any of these activities in mind.
Paradoxically, this is why it is so useful.
· SDDS stands for "Self Describing Data Sets."
· SDDS is just a standardized way to store and access data, i.e., a "file protocol."
· SDDS also refers to a group of ~85 programs that use this file protocol.
· These programs are the "tools" in the SDDS Toolkit.
· A hammer, a saw, a drill, etc., can be used together or sequentially to create and modify physical objects.
· The programs in the SDDS toolkit can be used to sequentially transform SDDS data sets.
· Within some limits, it isn't determined ahead of time what physical objects can be modified or what can be created.
· SDDS toolkit programs are generic and operate on any SDDS data set. The meaning of the operations is not predetermined.
· Both physical and SDDS tools can be used in arbitrary sequences of arbitrary length. The capability of the toolkit grows very rapidly with the number of tools.
· Every new tool that is created makes the existing ones more useful, without any advance planning or coordination by developers.
· A new tool need not be useful by itself in order to be very useful as part of a toolkit.
Most SDDS tools produce no directly useful result. This freedom makes new tools much easier to create.
Three types of SDDS-compliant programs
· Data display
· plotting (2 programs)
· printing data as formatted text
· summarizing data set contents
· Data processing
· equation evaluation
· data filtering and outlier removal
· statistics, histograms, and correlations
· fitting and smoothing
· matrix operations (e.g., SVD)
· cross-referencing, sorting, and collation
· FFTs and digital filtering
· Data collection from EPICS
· logging data at fixed time intervals
· event-driven data logging
· alarm logging
· n-dimensional experiments
· save/restore of EPICS data
· Control functions for EPICS
· generalized feedback control
· generalized optimization
· Because the SDDS tools are commandline driven, they can be embedded in scripts..
· Tcl/Tk is used to make graphical user interfaces (GUIs) that depend on SDDS tools for computational "muscle," data collection, and data display.
· Engineers and physicists can use SDDS tools directly to develop new algorithms. Once finished, those results can easily be put into a GUI script.
· Automated data collection
· ~19000 channels of time-series data
· ~1000 channels of glitch data
· ~9000 channels of alarm data
· storage ring beam dump data
· Used to create high-level applications for operators, engineers, and physicists:
· beamline steering
· orbit correction
· configuration control
· routine monitoring
· history review
· problem diagnosis
· Used by engineers and physicists for
· data collection
· automated experiments
· analysis of accelerator and simulation data
· equipment checkout
· Self-describing data files require more information in the file besides the data itself.
· At minimum, a self-describing file protocol
· requires that every data element in the file has a name.
· forbids access to data except via the name.
· SDDS is a specific self-describing data protocol, developed at APS for accelerator commissioning.
· Highly successful application of SDDS to commissioning lead to its use for operations.
· An SDDS file consists of
· A file header describing a structure composed of an arbitrary number of parameters and arrays, and a data table of arbitrary rows and columns.
· Zero or more instances of the structure.
· There are many extremely general self-describing file protocols around today.
· In using these protocols, users find it necessary to create elaborate data standards of their own, which inhibits use of the toolkit approach.
· In contrast, the SDDS file protocol is simple enough to be used in "daily life," but general enough to be widely useful.
· Only the simplicity of the data model makes the SDDS Toolkit feasible.
· Accelerator configuration data:
· Parameters: time stamp, configuration description, username, etc.
· Columns: process variable name, value, access mode, category, subcategory, tolerance, etc.
· Storage ring orbit glitch records
· Parameters: time of glitch, trigger conditions, etc.
· Columns: readouts of all BPMs, time of readout, beam current, etc.
· IPNS---Used for data logging, analysis, and display.
· CEA (France)---Used by a group of particle physicists doing underwater experiments in the Mediteranean.
· SRRC (Taiwan)---Installed by request.
· LEDA (LANL)---Installed by request.
· CEBAF (TJNAF)---Installed by request.
· Look for an upcoming four-part class on using SDDS.
· See the OAG web page at
APS Overview --->
Accelerator Systems Division --->
Operations Analysis --->
OAG Software Documentation
(https://www.aps.anl.gov/Accelerator-Operations-Physics/Documentation)
In particular, see
· SDDS tools, and
· SDDS Compliant EPICS tools.