2.1 Introduction

An SDDS file is referred to as a “data set”. Each data set consists of an ASCII header describing the data that is stored in the file, followed by zero or more “data pages” or “data tables” (the former term is preferred, though the latter is used in many places). The data may be in ASCII or unformatted (i.e., “binary”). Each data page is an instance of the structure defined by the header. That is, while the specific data may vary from page to page, the structure of the data may not.

Three types of entities may be present in each page: parameters, arrays, and columns. Each of these may contain data of a single data type, with the choices being long and short integer, single and double precision floating point, single character, and character string. The names, units, data types, and so forth of these entities are defined in the header.

Parameters are scalar entities. That is, each parameter defined in the header has a single value for each page. Each such value may be a single number or a single character string, for example.

Arrays are multidimensional entities with potentially varying numbers of elements. While there is no restriction on the number of dimensions an array may contain, this quantity is fixed throughout the file for each array. However, the size of the array may vary from page to page. Thus, a given two-dimensional array might be 2x2 in one page, 3x5 in the next, etc.

Columns are vector entities. All columns in a data set are organized into a single table, called the “tabular data section”. Thus, all columns must contain the same number of entries, that number being the number of rows in the table. There is no restriction on how many rows the tabular data may contain, nor on the mixing of data types in the tabular data.

It is possible to design more sophisticated data protocols than SDDS, and this has in fact been done. However, the more flexible a protocol is, the more difficult it becomes to write generic programs that operate on data. Experience with SDDS has shown that there is very little data that cannot be conveniently stored in one or more SDDS files. In fact, most applications need only the parameter and tabular data facilities. Frequently, complex data is separated into several parallel files; the SDDS toolkit provides support for multifile operations that make this convenient.

The following is an example of a very simple SDDS file. Users who would prefer not to read the detailed description of the protocol in the next section may profit from using this example as a guide.

At this point, those who are new to SDDS may wish to skip to the Manual Pages Overview in order to get a feel for the capabilities of the Toolkit. The details of SDDS protocol, the subject of the next section, are less important than what can be done with data once it is in SDDS protocol.