SDDS_internal.h File Reference

Detailed Description

Internal definitions and function declarations for SDDS with LZMA support.

This header file contains internal macros, structures, and function prototypes used by the SDDS (Self Describing Data Sets) library to handle LZMA-compressed files. It includes definitions for the lzmafile structure, buffer sizes, and various binary and ASCII I/O routines tailored for SDDS datasets.

The file ensures that internal components are only included once using include guards and conditionally includes the LZMA library when necessary.

Key Components:

• Structures:
  • lzmafile: Represents an LZMA-compressed file with associated stream and buffer.
• Function Prototypes:
  • lzma_open, lzma_close, lzma_read, lzma_write: Basic file operations for LZMA files.
  • SDDS_WriteBinaryArrays, SDDS_ReadBinaryArrays, etc.: SDDS-specific binary I/O functions.
  • SDDS_WriteAsciiArrays, SDDS_ReadAsciiArrays, etc.: SDDS-specific ASCII I/O functions.
  • Additional utility and processing functions for handling SDDS datasets.
• Macros:
  • LZMA_BUF_SIZE: Defines the buffer size for LZMA operations.
  • TABLE_LENGTH_INCREMENT: Specifies the increment size for table allocations.
  • Command definitions for various SDDS operations.

Dependencies:

• Requires the LZMA library (lzma.h) for compression support.
• Conditionally includes zLib support if defined.

Usage:

This file is intended for internal use within the SDDS library and should not be included directly by external applications. It provides the necessary infrastructure to manage compressed data streams and perform efficient I/O operations on SDDS datasets.

Note

Ensure that the LZMA and zLib libraries are properly linked when compiling components that include this header.

Copyright

• (c) 2002 The University of Chicago, as Operator of Argonne National Laboratory.
• (c) 2002 The Regents of the University of California, as Operator of Los Alamos National Laboratory.

License This file is distributed under the terms of the Software License Agreement

found in the file LICENSE included with this distribution.

Authors

M. Borland, C. Saunders, R. Soliday, H. Shang

Definition in file SDDS_internal.h.

#include "SDDS.h"
#include <lzma.h>

Go to the source code of this file.

Functions
void *	lzma_open (const char *path, const char *mode)
int	lzma_close (struct lzmafile *file)
long	lzma_read (struct lzmafile *file, void *buf, size_t count)
long	lzma_write (struct lzmafile *file, const void *buf, size_t count)
int	lzma_printf (struct lzmafile *file, const char *format,...)
int	lzma_puts (const char *s, struct lzmafile *file)
int	lzma_putc (int c, struct lzmafile *file)
char *	lzma_gets (char *s, int size, struct lzmafile *file)
int	lzma_eof (struct lzmafile *file)
long	lzma_tell (struct lzmafile *file)
int	lzma_seek (struct lzmafile *file, long offset, int whence)
void *	UnpackLZMAOpen (char *filename)
char *	fgetsLZMASkipComments (SDDS_DATASET *SDDS_dataset, char *s, int32_t slen, struct lzmafile *lzmafp, char skip_char)
	Reads a line from a LZMA-compressed file while skipping comment lines.
char *	fgetsLZMASkipCommentsResize (SDDS_DATASET *SDDS_dataset, char **s, int32_t *slen, struct lzmafile *lzmafp, char skip_char)
	Reads a line from a LZMA-compressed file with dynamic buffer resizing while skipping comment lines.
int32_t	SDDS_LZMAWriteBinaryString (char *string, struct lzmafile *lzmafp, SDDS_FILEBUFFER *fBuffer)
	Writes a binary string to a file with LZMA compression.
char *	SDDS_ReadNonNativeLZMABinaryString (struct lzmafile *lzmafp, SDDS_FILEBUFFER *fBuffer, int32_t skip)
	Reads a non-native endian binary string from an LZMA-compressed file.
int32_t	SDDS_LZMAWriteNonNativeBinaryString (char *string, struct lzmafile *lzmafp, SDDS_FILEBUFFER *fBuffer)
	Writes a non-native endian binary string to an LZMA-compressed file.
int32_t	SDDS_WriteBinaryArrays (SDDS_DATASET *SDDS_dataset)
	Writes the binary arrays of the SDDS dataset to a file.
int32_t	SDDS_WriteBinaryColumns (SDDS_DATASET *SDDS_dataset)
	Writes the binary columns of an SDDS dataset to the associated file.
int32_t	SDDS_WriteNonNativeBinaryColumns (SDDS_DATASET *SDDS_dataset)
	Writes non-native endian binary columns of an SDDS dataset to the associated file.
int32_t	SDDS_WriteBinaryParameters (SDDS_DATASET *SDDS_dataset)
	Writes the binary parameters of the SDDS dataset.
int32_t	SDDS_WriteBinaryPage (SDDS_DATASET *SDDS_dataset)
int32_t	SDDS_WriteBinaryRow (SDDS_DATASET *SDDS_dataset, int64_t row)
	Writes a single binary row of an SDDS dataset to the associated file.
int32_t	SDDS_UpdateBinaryPage (SDDS_DATASET *SDDS_dataset, uint32_t mode)
	Updates the binary page of an SDDS dataset.
int32_t	SDDS_UpdateNonNativeBinaryPage (SDDS_DATASET *SDDS_dataset, uint32_t mode)
	Updates a non-native endian binary page in an SDDS dataset.
int32_t	SDDS_fseek (FILE *fp, int64_t offset, int32_t dir)
	Sets the file position indicator for a given file stream with retry logic.
char *	SDDS_ReadBinaryString (FILE *fp, SDDS_FILEBUFFER *fBuffer, int32_t skip)
	Reads a binary string from a file with buffering.
int32_t	SDDS_ReadBinaryArrays (SDDS_DATASET *SDDS_dataset)
	Reads binary arrays from an SDDS dataset.
int32_t	SDDS_ReadBinaryColumns (SDDS_DATASET *SDDS_dataset, int64_t sparse_interval, int64_t sparse_offset)
	Reads the binary columns from an SDDS dataset.
int32_t	SDDS_ReadNonNativeBinaryColumns (SDDS_DATASET *SDDS_dataset)
	Reads the non-native endian binary columns from an SDDS dataset.
int32_t	SDDS_ReadBinaryParameters (SDDS_DATASET *SDDS_dataset)
	Reads binary parameters from the specified SDDS dataset.
int32_t	SDDS_ReadBinaryPage (SDDS_DATASET *SDDS_dataset, int64_t sparse_interval, int64_t sparse_offset, int32_t sparse_statistics)
	Reads a binary page from an SDDS dataset.
int32_t	SDDS_ReadBinaryPageLastRows (SDDS_DATASET *SDDS_dataset, int64_t last_rows)
	Reads the last specified number of rows from a binary page of an SDDS dataset.
int32_t	SDDS_ReadBinaryPageDetailed (SDDS_DATASET *SDDS_dataset, int64_t sparse_interval, int64_t sparse_offset, int64_t last_rows, int32_t sparse_statistics)
	Reads a binary page from an SDDS dataset with detailed options.
int32_t	SDDS_ReadBinaryRow (SDDS_DATASET *SDDS_dataset, int64_t row, int32_t skip)
	Reads a binary row from the specified SDDS dataset.
int32_t	SDDS_ReadNonNativePageLastRows (SDDS_DATASET *SDDS_dataset, int64_t last_rows)
	Reads the last few rows from a non-native endian page in an SDDS dataset.
int32_t	SDDS_ReadNonNativeBinaryPageDetailed (SDDS_DATASET *SDDS_dataset, int64_t sparse_interval, int64_t sparse_offset, int64_t last_rows)
	Reads a detailed non-native endian binary page from an SDDS dataset.
int32_t	SDDS_ReadNonNativePageDetailed (SDDS_DATASET *SDDS_dataset, uint32_t mode, int64_t sparse_interval, int64_t sparse_offset, int64_t last_rows)
	Reads a detailed non-native endian page from an SDDS dataset.
int32_t	SDDS_ReadNonNativeBinaryPageLastRows (SDDS_DATASET *SDDS_dataset, int64_t last_rows)
	Reads the last few rows from a non-native endian binary page in an SDDS dataset.
int32_t	SDDS_GetTerminateMode (void)
int32_t	SDDS_AllocateColumnFlags (SDDS_DATASET *SDDS_target)
int32_t	SDDS_WriteAsciiArrays (SDDS_DATASET *SDDS_dataset, FILE *fp)
	Writes the arrays of an SDDS dataset in ASCII format to a file.
int32_t	SDDS_WriteAsciiParameters (SDDS_DATASET *SDDS_dataset, FILE *fp)
	Writes the parameter data of an SDDS dataset in ASCII format to a file.
void	SDDS_AppendParameterComment (PARAMETER_DEFINITION *definition, char *text, char *string)
int32_t	SDDS_WriteAsciiRow (SDDS_DATASET *SDDS_dataset, int64_t row, FILE *fp)
	Writes a single row of data in ASCII format to a file.
int32_t	SDDS_WriteAsciiPage (SDDS_DATASET *SDDS_dataset)
	Writes a page of data in ASCII format to the SDDS dataset.
int32_t	SDDS_ReadAsciiArrays (SDDS_DATASET *SDDS_dataset)
	Reads the arrays from an ASCII file into the SDDS dataset.
int32_t	SDDS_ReadAsciiParameters (SDDS_DATASET *SDDS_dataset)
	Reads the parameters from an ASCII file into the SDDS dataset.
int32_t	SDDS_AsciiDataExpected (SDDS_DATASET *SDDS_dataset)
	Checks whether the SDDS dataset expects ASCII data input.
int32_t	SDDS_ReadAsciiPageLastRows (SDDS_DATASET *SDDS_dataset, int64_t last_rows)
	Reads the last specified number of rows from an ASCII page of an SDDS dataset.
int32_t	SDDS_ReadAsciiPageDetailed (SDDS_DATASET *SDDS_dataset, int64_t sparse_interval, int64_t sparse_offset, int64_t last_rows, int32_t sparse_statistics)
	Reads a detailed page of data from an ASCII file into an SDDS dataset with optional sparsity and statistics.
int32_t	SDDS_LZMAWriteAsciiParameters (SDDS_DATASET *SDDS_dataset, struct lzmafile *lzmafp)
	Writes the parameter data of an SDDS dataset in ASCII format to an LZMA compressed file.
int32_t	SDDS_LZMAWriteAsciiArrays (SDDS_DATASET *SDDS_dataset, struct lzmafile *lzmafp)
	Writes the arrays of an SDDS dataset in ASCII format to an LZMA compressed file.
int32_t	SDDS_LZMAWriteAsciiRow (SDDS_DATASET *SDDS_dataset, int64_t row, struct lzmafile *lzmafp)
	Writes a single row of data in ASCII format to an LZMA compressed file.
int32_t	SDDS_CopyColumn (SDDS_DATASET *SDDS_dataset, int32_t target, int32_t source)
	Copies data from a source column to a target column within an SDDS dataset.
int32_t	SDDS_CopyParameter (SDDS_DATASET *SDDS_dataset, int32_t target, int32_t source)
	Copies a parameter from a source index to a target index within an SDDS dataset.
int32_t	SDDS_TransferRow (SDDS_DATASET *SDDS_dataset, int64_t target, int64_t source)
	Transfers data from a source row to a target row within an SDDS dataset.
int64_t	SDDS_GetSelectedRowIndex (SDDS_DATASET *SDDS_dataset, int64_t srow_index)
	Retrieves the actual row index corresponding to a selected row position within the current data table.
int32_t	SDDS_CheckTable (SDDS_DATASET *SDDS_dataset, const char *caller)
int32_t	SDDS_AdvanceCounter (int32_t *counter, int32_t *max_count, int32_t n_indices)
	Advances a multi-dimensional counter based on maximum counts for each dimension.
void	SDDS_FreePointerArray (void **data, int32_t dimensions, int32_t *dimension)
	Frees a multi-dimensional pointer array created by SDDS_MakePointerArray.
int32_t	SDDS_WriteVersion (int32_t version_number, FILE *fp)
	Writes the SDDS protocol version to a standard file.
int32_t	SDDS_WriteDescription (char *description, char *contents, FILE *fp)
	Writes the SDDS description section to a standard file.
int32_t	SDDS_WriteColumnDefinition (COLUMN_DEFINITION *column, FILE *fp)
	Writes a column definition to a standard file.
int32_t	SDDS_WriteParameterDefinition (PARAMETER_DEFINITION *parameter, FILE *fp)
	Writes a parameter definition to a standard file.
int32_t	SDDS_WriteAssociateDefinition (ASSOCIATE_DEFINITION *associate, FILE *fp)
	Writes an associate definition to a standard file.
int32_t	SDDS_WriteArrayDefinition (ARRAY_DEFINITION *array_definition, FILE *fp)
	Writes an array definition to a standard file.
int32_t	SDDS_WriteDataMode (SDDS_LAYOUT *layout, FILE *fp)
	Writes the data mode section to a standard file.
int32_t	SDDS_LZMAWriteVersion (int32_t version_number, struct lzmafile *lzmafp)
	Writes the SDDS protocol version to an LZMA-compressed file.
int32_t	SDDS_LZMAWriteDescription (char *description, char *contents, struct lzmafile *lzmafp)
	Writes the SDDS description section to an LZMA-compressed file.
int32_t	SDDS_LZMAWriteColumnDefinition (COLUMN_DEFINITION *column, struct lzmafile *lzmafp)
	Writes a column definition to an LZMA-compressed file.
int32_t	SDDS_LZMAWriteParameterDefinition (PARAMETER_DEFINITION *parameter, struct lzmafile *lzmafp)
	Writes a parameter definition to an LZMA-compressed file.
int32_t	SDDS_LZMAWriteAssociateDefinition (ASSOCIATE_DEFINITION *associate, struct lzmafile *lzmafp)
	Writes an associate definition to an LZMA-compressed file.
int32_t	SDDS_LZMAWriteArrayDefinition (ARRAY_DEFINITION *array_definition, struct lzmafile *lzmafp)
	Writes an array definition to an LZMA-compressed file.
int32_t	SDDS_LZMAWriteDataMode (SDDS_LAYOUT *layout, struct lzmafile *lzmafp)
	Writes the data mode section to an LZMA-compressed file.
int32_t	SDDS_ProcessDescription (SDDS_DATASET *SDDS_dataset, char *s)
	Process the description section of the SDDS dataset.
int32_t	SDDS_ProcessColumnDefinition (SDDS_DATASET *SDDS_dataset, char *s)
	Process the column definition section of the SDDS dataset.
int32_t	SDDS_ProcessParameterDefinition (SDDS_DATASET *SDDS_dataset, char *s)
	Process the parameter definition section of the SDDS dataset.
int32_t	SDDS_ProcessArrayDefinition (SDDS_DATASET *SDDS_dataset, char *s)
	Process the array definition section of the SDDS dataset.
FILE *	SDDS_ProcessIncludeCommand (SDDS_DATASET *SDDS_dataset, char *s)
	Process the include command within the SDDS dataset.
int32_t	SDDS_ProcessAssociateDefinition (SDDS_DATASET *SDDS_dataset, char *s)
	Process the associate definition section of the SDDS dataset.
int32_t	SDDS_ProcessDataMode (SDDS_DATASET *SDDS_dataset, char *s)
	Process the data mode section of the SDDS dataset.
int32_t	SDDS1_ProcessDescription (SDDS_DATASET *SDDS_dataset, char *s)
	Process the description section for SDDS protocol version 1.
int32_t	SDDS1_ProcessColumnDefinition (SDDS_DATASET *SDDS_dataset, char *s)
	Processes and defines a column within an SDDS dataset.
int32_t	SDDS1_ProcessParameterDefinition (SDDS_DATASET *SDDS_dataset, char *s)
	Processes and defines a parameter within an SDDS dataset.
int32_t	SDDS1_ProcessArrayDefinition (SDDS_DATASET *SDDS_dataset, char *s)
	Processes and defines an array within an SDDS dataset.
FILE *	SDDS1_ProcessIncludeCommand (SDDS_DATASET *SDDS_dataset, char *s)
	Processes an include command by opening the specified file.
int32_t	SDDS1_ProcessAssociateDefinition (SDDS_DATASET *SDDS_dataset, char *s)
	Processes and defines an associate within an SDDS dataset.
int32_t	SDDS1_ProcessDataMode (SDDS_DATASET *SDDS_dataset, char *s)
	Processes and sets the data mode for an SDDS dataset.
int32_t	SDDS_ReadLayout (SDDS_DATASET *SDDS_dataset, FILE *fp)
int32_t	SDDS_LZMAReadLayout (SDDS_DATASET *SDDS_dataset, struct lzmafile *lzmafp)
int32_t	SDDS_UpdateAsciiPage (SDDS_DATASET *SDDS_dataset, uint32_t mode)
	Updates the current ASCII page of an SDDS dataset with new data.
uint32_t	SDDS_GetSpecialCommentsModes (SDDS_DATASET *SDDS_dataset)
	Retrieves the current special comments modes set in the SDDS dataset.
void	SDDS_ResetSpecialCommentsModes (SDDS_DATASET *SDDS_dataset)
	Resets the special comments modes in the SDDS dataset.
void	SDDS_ParseSpecialComments (SDDS_DATASET *SDDS_dataset, char *s)
	Parses and processes special comment commands within the SDDS dataset.
void	SDDS_FreeTableStrings (SDDS_DATASET *SDDS_dataset)

Function Documentation

fgetsLZMASkipComments()

char * fgetsLZMASkipComments	(	SDDS_DATASET *	SDDS_dataset,
		char *	s,
		int32_t	slen,
		struct lzmafile *	lzmafp,
		char	skip_char )

Reads a line from a LZMA-compressed file while skipping comment lines.

This function reads lines from the specified LZMA-compressed file stream, ignoring lines that begin with the specified skip_char. It also processes special comment lines that start with !# by parsing them using SDDS_ParseSpecialComments.

Parameters

[in]	SDDS_dataset	Pointer to the SDDS_DATASET structure, used for processing comments.
[out]	s	Pointer to a character array where the read line will be stored.
[in]	slen	The maximum number of characters to read into s.
[in]	lzmafp	Pointer to the lzmafile structure representing the LZMA-compressed file stream.
[in]	skip_char	Character indicating the start of a comment line. Lines beginning with this character will be skipped.

Returns

On success, returns the pointer s containing the read line. If the end of the file is reached or an error occurs, returns NULL.

Note

The function modifies the buffer s by removing comments as determined by SDDS_CutOutComments.

See also

SDDS_CutOutComments

SDDS_ParseSpecialComments

lzma_gets

Definition at line 1594 of file SDDS_utils.c.

                                                                                                                          {
  while (lzma_gets(s, slen, lzmafp)) {
    if (s[0] != skip_char) {
      SDDS_CutOutComments(SDDS_dataset, s, skip_char);
      return (s);
    } else if (s[1] == '#') {
      SDDS_ParseSpecialComments(SDDS_dataset, s + 2);
    }
  }
  return (NULL);
}

fgetsLZMASkipCommentsResize()

char * fgetsLZMASkipCommentsResize	(	SDDS_DATASET *	SDDS_dataset,
		char **	s,
		int32_t *	slen,
		struct lzmafile *	lzmafp,
		char	skip_char )

Reads a line from a LZMA-compressed file with dynamic buffer resizing while skipping comment lines.

This function reads lines from the specified LZMA-compressed file stream, ignoring lines that begin with the specified skip_char. If a line exceeds the current buffer size, the buffer is dynamically resized to accommodate the entire line. It also processes special comment lines that start with !# by parsing them using SDDS_ParseSpecialComments.

Parameters

[in]	SDDS_dataset	Pointer to the SDDS_DATASET structure, used for processing comments.
[in,out]	s	Pointer to a pointer to a character array where the read line will be stored. This buffer may be resized if necessary.
[in,out]	slen	Pointer to an int32_t variable specifying the current size of the buffer *s. This value may be updated if the buffer is resized.
[in]	lzmafp	Pointer to the lzmafile structure representing the LZMA-compressed file stream.
[in]	skip_char	Character indicating the start of a comment line. Lines beginning with this character will be skipped.

Returns

On success, returns the pointer *s containing the read line. If the end of the file is reached or an error occurs, returns NULL.

Note

The caller is responsible for managing the memory of the buffer *s, including freeing it when no longer needed.

See also

SDDS_CutOutComments

SDDS_ParseSpecialComments

SDDS_Realloc

lzma_gets

Definition at line 1626 of file SDDS_utils.c.

                                                                                                                                  {
  int32_t spaceLeft, length, newLine;
  char *sInsert, *fgetsReturn;
 
  sInsert = *s;
  spaceLeft = *slen;
  newLine = 1;
  while ((fgetsReturn = lzma_gets(sInsert, spaceLeft, lzmafp))) {
    if (newLine && sInsert[0] == '!')
      continue;
    SDDS_CutOutComments(SDDS_dataset, sInsert, skip_char);
    length = strlen(sInsert);
    if (sInsert[length - 1] != '\n' && !lzma_eof(lzmafp)) {
      /* buffer wasn't long enough to get the whole line.  Resize and add more data. */
      spaceLeft = *slen;
      *slen = *slen * 2;
      *s = SDDS_Realloc(*s, sizeof(**s) * *slen);
      sInsert = *s + strlen(*s);
      newLine = 0;
    } else
      break;
  }
  if (!fgetsReturn)
    return NULL;
  return (*s);
}

lzma_close()

int lzma_close

(

struct lzmafile *

file

)

Definition at line 136 of file SDDS_lzma.c.

                                      {
  int ret, outsize;
  unsigned char buf[BUF_SIZE]; /* buffer used when flushing remaining
                                  output data in write mode */
  if (!file)
    return -1;
  if (file->mode == 'w') {
    /* flush LZMA output buffer */
    for (;;) {
      file->str.next_out = buf;
      file->str.avail_out = BUF_SIZE;
      ret = lzma_code(&file->str, LZMA_FINISH);
      if (ret != LZMA_STREAM_END && ret != LZMA_OK) {
        fprintf(stderr, "lzma_close error: encoding failed: %d\n", ret);
        lzma_end(&file->str);
        fclose(file->fp);
        free(file);
        return EOF;
      }
      outsize = BUF_SIZE - file->str.avail_out;
      if (fwrite(buf, 1, outsize, file->fp) != outsize) {
        lzma_end(&file->str);
        fclose(file->fp);
        free(file);
        return EOF;
      }
      if (ret == LZMA_STREAM_END)
        break;
    }
  }
  lzma_end(&file->str);
  ret = fclose(file->fp);
  free(file);
  return ret;
}

lzma_eof()

int lzma_eof

(

struct lzmafile *

file

)

Definition at line 415 of file SDDS_lzma.c.

                                    {
  lzma_stream *lstr;
  lstr = &file->str;
  if (lstr->avail_in == 0) {
    return feof(file->fp);
  } else {
    return 0;
  }
}

lzma_gets()

char * lzma_gets	(	char *	s,
		int	size,
		struct lzmafile *	file )

Definition at line 217 of file SDDS_lzma.c.

                                                          {
  int ret;
  int i = 0;
  lzma_stream *lstr;
  if (file->mode != 'r')
    return NULL;
  if (s == NULL || size < 1)
    return NULL;
  s[0] = '\0';
  lstr = &file->str;
  lstr->next_out = (void *)s;
 
  /* decompress until newline or EOF or output buffer is full */
  while (1) {
    if (lstr->avail_in == 0) {
      /* refill input buffer */
      ret = fread(file->rdbuf, 1, BUF_SIZE, file->fp);
      if (ret == 0) {
        break; /* EOF */
      }
      lstr->next_in = file->rdbuf; /* buffer containing lzma data just read */
      lstr->avail_in = ret;        /* number of bytes read */
    }
    if (i + 1 == size) {
      s[i] = '\0';
      break;
    }
    lstr->avail_out = 1;
    ret = lzma_code(lstr, LZMA_RUN);
    /* this fills up lstr->next_out and decreases lstr->avail_out */
    /* it also emptys lstr->next_in and decreases lstr->avail_in */
    if (ret != LZMA_OK && ret != LZMA_STREAM_END) {
      fprintf(stderr, "lzma_gets error: decoding failed: %d\n", ret);
      return NULL;
    }
    if (ret == LZMA_STREAM_END) { /* EOF */
      s[i + 1] = '\0';
      break;
    }
    if (s[i] == 10) { /* 10 is the value for \n */
      if (i > 0) {    /* we sometimes get \10\10 */
        if ((i == 1) && (s[0] == 32)) {
          /* when uncompressing the lzma stream we some times end 
             up with \10\32\10 instead of a simple \10 */
        } else {
          s[i + 1] = '\0';
          break;
        }
      }
    }
    i++;
  }
  return s;
}

lzma_open()

void * lzma_open	(	const char *	path,
		const char *	mode )

Definition at line 99 of file SDDS_lzma.c.

                                                    {
  int ret;
 
  /* initialize LZMA stream */
  struct lzmafile *lf = malloc(sizeof(struct lzmafile));
  if (!lf)
    return NULL;
  lf->fp = fopen(path, mode);
  if (!lf->fp) {
    free(lf);
    return NULL;
  }
  lf->str = lzma_stream_init;
  lf->mode = mode[0];
  if (mode[0] == 'r') {
#if LZMA_VERSION <= UINT32_C(49990030)
    ret = lzma_auto_decoder(&lf->str, NULL, NULL);
#else
    ret = lzma_auto_decoder(&lf->str, -1, 0);
#endif
    lf->str.avail_in = 0;
  } else {
    ret = LZMA_EASY_ENCODER(&lf->str, SDDS_GetLockedLZMACompressionLevel());
  }
  if (ret != LZMA_OK) {
    fprintf(stderr, "lzma_open error: %d\n", ret);
    lzma_end(&lf->str);
    fclose(lf->fp);
    free(lf);
    return NULL;
  }
  return (void *)lf;
}

lzma_printf()

int lzma_printf	(	struct lzmafile *	file,
		const char *	format,
			... )

Definition at line 392 of file SDDS_lzma.c.

                                                                {
  size_t size = 32768;
  int len;
  unsigned char in[32768];
  va_list va;
  va_start(va, format);
 
  in[size - 1] = 0;
  (void)vsnprintf((char *)in, size, format, va);
  va_end(va);
  len = strlen((char *)in);
 
  /* check that printf() results fit in buffer */
  if (len <= 0 || len >= (int)size || in[size - 1] != 0) {
    fprintf(stderr, "lzma_printf error: the printf results do not fit in the buffer\n");
    return -1;
  }
  in[len] = '\0';
  len = lzma_write(file, in, len);
 
  return len;
}

lzma_putc()

int lzma_putc	(	int	c,
		struct lzmafile *	file )

Definition at line 356 of file SDDS_lzma.c.

                                            {
  int ret;
  lzma_stream *lstr = &file->str;
 
  unsigned char bufout[1]; /* compressed output buffer */
  char buf[1];
 
  if (file->mode != 'w') {
    fprintf(stderr, "lzma_putc error: file was not opened for writting\n");
    return EOF;
  }
  buf[0] = c;
 
  lstr->next_in = (void *)buf;
  lstr->avail_in = 1;
  while (lstr->avail_in) {
    lstr->next_out = bufout;
    lstr->avail_out = 1;
    ret = lzma_code(lstr, LZMA_RUN);
    if (ret != LZMA_OK) {
      fprintf(stderr, "lzma_putc error: encoding failed: %d\n", ret);
      return EOF;
    }
    ret = fwrite(bufout, 1, 1 - lstr->avail_out, file->fp);
    if (ret != 1 - lstr->avail_out) {
      fprintf(stderr, "lzma_putc error\n");
      return EOF;
    }
  }
  return (unsigned char)c;
}

lzma_puts()

int lzma_puts	(	const char *	s,
		struct lzmafile *	file )

Definition at line 311 of file SDDS_lzma.c.

                                                    {
  int ret;
  lzma_stream *lstr = &file->str;
  int count;
  unsigned char *bufout; /* compressed output buffer */
  char *buf;
 
  if (file->mode != 'w') {
    fprintf(stderr, "lzma_puts error: file was not opened for writting\n");
    return EOF;
  }
  count = strlen(s);
  bufout = malloc(sizeof(unsigned char) * count);
  buf = malloc(sizeof(char) * count);
  //strncpy(buf, s, count);
  memcpy(buf, s, count);
 
  lstr->next_in = (void *)buf;
  lstr->avail_in = count;
  while (lstr->avail_in) {
    lstr->next_out = bufout;
    lstr->avail_out = count;
    ret = lzma_code(lstr, LZMA_RUN);
    if (ret != LZMA_OK) {
      fprintf(stderr, "lzma_puts error: encoding failed: %d\n", ret);
      free(bufout);
      free(buf);
      return EOF;
    }
    ret = fwrite(bufout, 1, count - lstr->avail_out, file->fp);
    if (ret != count - lstr->avail_out) {
      fprintf(stderr, "lzma_puts error\n");
      free(bufout);
      free(buf);
      return EOF;
    }
  }
  free(bufout);
  free(buf);
  return count;
}

lzma_read()

long lzma_read	(	struct lzmafile *	file,
		void *	buf,
		size_t	count )

Definition at line 176 of file SDDS_lzma.c.

                                                               {
  int ret;
  lzma_stream *lstr;
  if (file->mode != 'r')
    return -1;
 
  lstr = &file->str;
  lstr->next_out = buf;
  lstr->avail_out = count;
 
  /* decompress until EOF or output buffer is full */
  while (lstr->avail_out) {
    if (lstr->avail_in == 0) {
      /* refill input buffer */
      ret = fread(file->rdbuf, 1, BUF_SIZE, file->fp);
      if (ret == 0) {
        break; /* EOF */
      }
      lstr->next_in = file->rdbuf; /* buffer containing lzma data just read */
      lstr->avail_in = ret;        /* number of bytes read */
    }
    ret = lzma_code(lstr, LZMA_RUN);
    /* this fills up lstr->next_out and decreases lstr->avail_out */
    /* it also emptys lstr->next_in and decreases lstr->avail_in */
    if (ret != LZMA_OK && ret != LZMA_STREAM_END) {
      fprintf(stderr, "lzma_read error: decoding failed: %d\n", ret);
      return -1;
    }
    if (ret == LZMA_STREAM_END) {
      break; /* EOF */
    }
  }
  return count - lstr->avail_out; /* length of buf that has valid data */
}

lzma_seek()

int lzma_seek	(	struct lzmafile *	file,
		long	offset,
		int	whence )

Definition at line 434 of file SDDS_lzma.c.

                                                              {
  return fseek(file->fp, offset, whence);
}

lzma_tell()

long lzma_tell

(

struct lzmafile *

file

)

Definition at line 430 of file SDDS_lzma.c.

                                      {
  return ftell(file->fp);
}

lzma_write()

long lzma_write	(	struct lzmafile *	file,
		const void *	buf,
		size_t	count )

Definition at line 275 of file SDDS_lzma.c.

                                                                      {
  int ret;
  lzma_stream *lstr = &file->str;
  unsigned char *bufout; /* compressed output buffer */
  bufout = malloc(sizeof(char) * count);
 
  if (file->mode != 'w') {
    fprintf(stderr, "lzma_write error: file was not opened for writting\n");
    free(bufout);
    return -1;
  }
  lstr->next_in = buf;
  lstr->avail_in = count;
  while (lstr->avail_in) {
    lstr->next_out = bufout;
    lstr->avail_out = count;
    ret = lzma_code(lstr, LZMA_RUN);
    if (ret != LZMA_OK) {
      fprintf(stderr, "lzma_write error: encoding failed: %d\n", ret);
      free(bufout);
      return -1;
    }
    ret = fwrite(bufout, 1, count - lstr->avail_out, file->fp);
    if (ret != count - lstr->avail_out) {
      fprintf(stderr, "lzma_write error\n");
      free(bufout);
      return -1;
    }
  }
  free(bufout);
  return count;
}

SDDS1_ProcessArrayDefinition()

int32_t SDDS1_ProcessArrayDefinition ( SDDS_DATASET * SDDS_dataset, char * s )

extern

Processes and defines an array within an SDDS dataset.

This function parses an array definition string, extracts the necessary array information, and defines the array in the given SDDS dataset.

Parameters

SDDS_dataset	Pointer to the SDDS dataset where the array will be defined.
s	String containing the array definition in namelist format.

Returns

• Returns 1 on successful processing and definition of the array.
• Returns 0 if an error occurs during parsing or definition.

Definition at line 916 of file SDDS_process.c.

                                                                          {
  ARRAY_DEFINITION arrayDef;
  int32_t code;
 
  arrayDef.name = arrayDef.symbol = arrayDef.units = arrayDef.description = arrayDef.format_string = arrayDef.group_name = NULL;
  arrayDef.type = -1;
  arrayDef.field_length = 0;
  arrayDef.dimensions = 1;
 
  if (!SDDS_ParseNamelist((void *)&arrayDef, SDDS_ArrayFieldInformation, SDDS_ARRAY_FIELDS, s)) {
    SDDS_SetError("Problem parsing array namelist");
    return 0;
  }
  code = SDDS_DefineArray(SDDS_dataset, arrayDef.name, arrayDef.symbol, arrayDef.units, arrayDef.description, arrayDef.format_string, arrayDef.type, arrayDef.field_length, arrayDef.dimensions, arrayDef.group_name);
  if (arrayDef.name)
    free(arrayDef.name);
  if (arrayDef.symbol)
    free(arrayDef.symbol);
  if (arrayDef.units)
    free(arrayDef.units);
  if (arrayDef.description)
    free(arrayDef.description);
  if (arrayDef.format_string)
    free(arrayDef.format_string);
  if (arrayDef.group_name)
    free(arrayDef.group_name);
 
  if (code < 0) {
    SDDS_SetError("Unable to process array definition--call to define array failed (SDDS1_ProcessArrayDefinition)");
    return (0);
  }
  return (1);
}

SDDS1_ProcessAssociateDefinition()

int32_t SDDS1_ProcessAssociateDefinition ( SDDS_DATASET * SDDS_dataset, char * s )

extern

Processes and defines an associate within an SDDS dataset.

This function parses an associate definition string, extracts the necessary associate information, and defines the associate in the given SDDS dataset.

Parameters

SDDS_dataset	Pointer to the SDDS dataset where the associate will be defined.
s	String containing the associate definition in namelist format.

Returns

• Returns 1 on successful processing and definition of the associate.
• Returns 0 if an error occurs during parsing or definition.

Definition at line 847 of file SDDS_process.c.

                                                                              {
  ASSOCIATE_DEFINITION assocDef;
  int32_t code;
 
  assocDef.name = assocDef.filename = assocDef.path = assocDef.description = assocDef.contents = NULL;
  assocDef.sdds = 0;
  if (!SDDS_ParseNamelist((void *)&assocDef, SDDS_AssociateFieldInformation, SDDS_ASSOCIATE_FIELDS, s)) {
    SDDS_SetError("Problem parsing associate namelist");
    return 0;
  }
 
  code = SDDS_DefineAssociate(SDDS_dataset, assocDef.name, assocDef.filename, assocDef.path, assocDef.description, assocDef.contents, assocDef.sdds);
  if (code < 0) {
    SDDS_SetError("Unable to process associate definition--call to define associate failed (SDDS1_ProcessAssociateDefinition)");
    return (0);
  }
  return (1);
}

SDDS1_ProcessColumnDefinition()

int32_t SDDS1_ProcessColumnDefinition ( SDDS_DATASET * SDDS_dataset, char * s )

extern

Processes and defines a column within an SDDS dataset.

This function parses a column definition string, extracts the necessary column information, and defines the column in the given SDDS dataset.

Parameters

SDDS_dataset	Pointer to the SDDS dataset where the column will be defined.
s	String containing the column definition in namelist format.

Returns

• Returns 1 on successful processing and definition of the column.
• Returns 0 if an error occurs during parsing or definition.

Definition at line 730 of file SDDS_process.c.

                                                                           {
  COLUMN_DEFINITION colDef;
  int32_t code;
 
  colDef.name = colDef.symbol = colDef.units = colDef.description = colDef.format_string = NULL;
  colDef.type = -1;
  colDef.field_length = 0;
 
  if (!SDDS_ParseNamelist((void *)&colDef, SDDS_ColumnFieldInformation, SDDS_COLUMN_FIELDS, s)) {
    SDDS_SetError("Problem parsing column namelist");
    return 0;
  }
  code = SDDS_DefineColumn(SDDS_dataset, colDef.name, colDef.symbol, colDef.units, colDef.description, colDef.format_string, colDef.type, colDef.field_length);
  if (colDef.name)
    free(colDef.name);
  if (colDef.symbol)
    free(colDef.symbol);
  if (colDef.units)
    free(colDef.units);
  if (colDef.description)
    free(colDef.description);
  if (colDef.format_string)
    free(colDef.format_string);
 
  if (code < 0) {
    SDDS_SetError("Unable to process column definition--call to define column failed (SDDS1_ProcessColumnDefinition)");
    return (0);
  }
  return (1);
}

SDDS1_ProcessDataMode()

int32_t SDDS1_ProcessDataMode ( SDDS_DATASET * SDDS_dataset, char * s )

extern

Processes and sets the data mode for an SDDS dataset.

This function parses a data mode definition string, extracts the necessary data mode information, and sets the data mode in the given SDDS dataset.

Parameters

SDDS_dataset	Pointer to the SDDS dataset where the data mode will be set.
s	String containing the data mode definition in namelist format.

Returns

• Returns 1 on successful processing and setting of the data mode.
• Returns 0 if an error occurs during parsing or if mandatory fields are missing.

Definition at line 879 of file SDDS_process.c.

                                                                   {
  DATA_MODE *dataMode;
  dataMode = &(SDDS_dataset->layout.data_mode);
  dataMode->mode = 0;
  dataMode->endian = 0;
  dataMode->lines_per_row = 1;
  dataMode->no_row_counts = dataMode->additional_header_lines = 0;
  dataMode->fixed_row_count = 0;
  dataMode->column_major = 0;
  if (!SDDS_ParseNamelist((void *)dataMode, SDDS_DataFieldInformation, SDDS_DATA_FIELDS, s)) {
    SDDS_SetError("Problem parsing data namelist");
    return 0;
  }
  if (dataMode->mode == 0) {
    SDDS_SetError("Problem with data namelist: mode not given.");
    return 0;
  }
  if (dataMode->mode == SDDS_ASCII && dataMode->lines_per_row < 0) {
    SDDS_SetError("Unable to process data mode--lines_per_row is invalid (SDDS1_ProcessDataMode)");
    return (0);
  }
  return 1;
}

SDDS1_ProcessDescription()

int32_t SDDS1_ProcessDescription ( SDDS_DATASET * SDDS_dataset, char * s )

extern

Process the description section for SDDS protocol version 1.

Parameters

SDDS_dataset	Pointer to the SDDS_DATASET structure.
s	Pointer to the description string.

Returns

Returns 1 on success, 0 on failure.

Definition at line 703 of file SDDS_process.c.

                                                                      {
  SDDS_dataset->layout.description = NULL;
  SDDS_dataset->layout.contents = NULL;
 
  if (!SDDS_ParseNamelist((void *)&SDDS_dataset->layout, SDDS_DescriptionFieldInformation, SDDS_DESCRIPTION_FIELDS, s)) {
    SDDS_SetError("Problem parsing description namelist");
    return 0;
  }
#if DEBUG
  fprintf(stderr, "Description scanned: description=>%s<, contents = >%s<\n", SDDS_dataset->layout.description, SDDS_dataset->layout.contents);
#endif
  return 1;
}

SDDS1_ProcessIncludeCommand()

FILE * SDDS1_ProcessIncludeCommand ( SDDS_DATASET * SDDS_dataset, char * s )

extern

Processes an include command by opening the specified file.

This function parses an include command string to extract the filename, then attempts to open the file for reading. It returns a file pointer if successful.

Parameters

SDDS_dataset	Pointer to the SDDS dataset (unused in current implementation).
s	String containing the include command in namelist format.

Returns

• Returns a pointer to the opened FILE on success.
• Returns NULL if parsing fails or the file cannot be opened.

Definition at line 818 of file SDDS_process.c.

                                                                       {
  FILE *fp;
  char *filename;
 
  filename = NULL;
  if (!SDDS_ParseNamelist((void *)&filename, SDDS_IncludeFieldInformation, SDDS_INCLUDE_FIELDS, s)) {
    SDDS_SetError("Problem parsing include namelist");
    return 0;
  }
  if (!filename || !(fp = fopen(filename, "r"))) {
    SDDS_SetError("Unable to process include command--invalid/nonexistent file (SDDS1_ProcessIncludeCommand)");
    return (NULL);
  }
  return (fp);
}

SDDS1_ProcessParameterDefinition()

int32_t SDDS1_ProcessParameterDefinition ( SDDS_DATASET * SDDS_dataset, char * s )

extern

Processes and defines a parameter within an SDDS dataset.

This function parses a parameter definition string, extracts the necessary parameter information, and defines the parameter in the given SDDS dataset.

Parameters

SDDS_dataset	Pointer to the SDDS dataset where the parameter will be defined.
s	String containing the parameter definition in namelist format.

Returns

• Returns 1 on successful processing and definition of the parameter.
• Returns 0 if an error occurs during parsing or definition.

Definition at line 774 of file SDDS_process.c.

                                                                              {
  PARAMETER_DEFINITION paramDef;
  int32_t code;
 
  paramDef.name = paramDef.symbol = paramDef.units = paramDef.description = paramDef.format_string = paramDef.fixed_value = NULL;
  paramDef.type = -1;
 
  if (!SDDS_ParseNamelist((void *)&paramDef, SDDS_ParameterFieldInformation, SDDS_PARAMETER_FIELDS, s)) {
    SDDS_SetError("Problem parsing parameter namelist");
    return 0;
  }
  code = SDDS_DefineParameter(SDDS_dataset, paramDef.name, paramDef.symbol, paramDef.units, paramDef.description, paramDef.format_string, paramDef.type, paramDef.fixed_value);
  if (paramDef.name)
    free(paramDef.name);
  if (paramDef.symbol)
    free(paramDef.symbol);
  if (paramDef.units)
    free(paramDef.units);
  if (paramDef.description)
    free(paramDef.description);
  if (paramDef.format_string)
    free(paramDef.format_string);
 
  if (code < 0) {
    SDDS_SetError("Unable to process parameter definition--call to define parameter failed (SDDS1_ProcessParameterDefinition)");
    return (0);
  }
  return (1);
}

SDDS_AdvanceCounter()

int32_t SDDS_AdvanceCounter ( int32_t * counter, int32_t * max_count, int32_t n_indices )

extern

Advances a multi-dimensional counter based on maximum counts for each dimension.

This helper function increments a multi-dimensional counter array, handling carry-over for each dimension. It is typically used for iterating over multi-dimensional arrays in a nested loop fashion.

Parameters

counter	Pointer to an array of integers representing the current count in each dimension.
max_count	Pointer to an array of integers representing the maximum count for each dimension.
n_indices	The number of dimensions (indices) in the counter and max_count arrays.

Returns

Returns the index of the dimension that was incremented. If all dimensions have been fully iterated over, returns -1 to indicate completion.

See also

SDDS_SetArrayVararg, SDDS_SetArray, SDDS_AdvanceCounter

Definition at line 1497 of file SDDS_dataprep.c.

                                                                                     {
  int32_t i;
 
  for (i = n_indices - 1; i >= 0; i--)
    if (counter[i] != (max_count[i] - 1))
      break;
  if (i == -1)
    return (-1);
 
  for (i = n_indices - 1; i >= 0; i--) {
    if (counter[i] < (max_count[i] - 1)) {
      counter[i]++;
      break;
    } else {
      counter[i] = 0;
    }
  }
  return (i);
}

SDDS_AllocateColumnFlags()

int32_t SDDS_AllocateColumnFlags ( SDDS_DATASET * SDDS_target )

extern

Allocates memory for column flags and column order arrays in the specified SDDS dataset.

This function allocates memory for the column_flag and column_order arrays based on the number of columns defined in the dataset's layout. It initializes column_flag to 1 and column_order to 0 and 1 respectively.

Parameters

SDDS_target

Pointer to the SDDS_DATASET structure where column flags will be allocated.

Returns

Returns 1 on successful allocation and initialization. On failure, returns 0 and records an error message.

See also

SDDS_Malloc, SDDS_SetMemory, SDDS_SetError

Definition at line 42 of file SDDS_dataprep.c.

                                                            {
  if (SDDS_target->layout.n_columns &&
      ((!(SDDS_target->column_flag = (int32_t *)SDDS_Malloc(sizeof(int32_t) * SDDS_target->layout.n_columns)) ||
        !(SDDS_target->column_order = (int32_t *)SDDS_Malloc(sizeof(int32_t) * SDDS_target->layout.n_columns))) ||
       (!SDDS_SetMemory(SDDS_target->column_flag, SDDS_target->layout.n_columns, SDDS_LONG, (int32_t)1, (int32_t)0) || 
        !SDDS_SetMemory(SDDS_target->column_order, SDDS_target->layout.n_columns, SDDS_LONG, (int32_t)0, (int32_t)1)))) {
    SDDS_SetError("Unable to allocate column flags--memory allocation failure (SDDS_AllocateColumnFlags)");
    return 0;
  }
  return 1;
}

SDDS_AsciiDataExpected()

int32_t SDDS_AsciiDataExpected ( SDDS_DATASET * SDDS_dataset )

extern

Checks whether the SDDS dataset expects ASCII data input.

This function determines if the provided SDDS dataset is expecting ASCII data. The dataset expects ASCII data if it has columns, arrays, or parameters without fixed values. If the dataset only contains parameters with fixed values and no columns or arrays, it does not expect ASCII data.

Parameters

SDDS_dataset

Pointer to the SDDS dataset to check.

Returns

Returns 1 if ASCII data is expected, or 0 if ASCII data is not expected.

Definition at line 2140 of file SDDS_ascii.c.

                                                           {
  int32_t i;
  if (SDDS_dataset->layout.n_columns || SDDS_dataset->layout.n_arrays)
    return (1);
  for (i = 0; i < SDDS_dataset->layout.n_parameters; i++)
    if (!SDDS_dataset->layout.parameter_definition[i].fixed_value)
      return (1);
  return (0);
}

SDDS_CopyColumn()

int32_t SDDS_CopyColumn ( SDDS_DATASET * SDDS_dataset, int32_t target, int32_t source )

extern

Copies data from a source column to a target column within an SDDS dataset.

This function duplicates the data from the specified source column to the target column in the provided SDDS dataset. It handles both string and non-string data types appropriately, ensuring that memory is managed correctly for string entries.

Parameters

SDDS_dataset	Pointer to the SDDS_DATASET structure representing the dataset.
target	The index of the target column where data will be copied to.
source	The index of the source column from which data will be copied.

Returns

On success, returns 1. On failure, returns 0 and sets an appropriate error message.

Return values

1	Indicates that the column copy was successful.
0	Indicates that an error occurred (e.g., invalid dataset, out-of-range indices, memory allocation failure, string copy failure).

Note

• Both target and source must be valid column indices within the dataset.
• The function does not handle the allocation of new columns; it assumes that the target column already exists.

See also

SDDS_DeleteColumn, SDDS_DeleteUnsetColumns, SDDS_TransferRow

Definition at line 3929 of file SDDS_extract.c.

                                                                                    {
  COLUMN_DEFINITION *cd_target, *cd_source;
  if (!SDDS_CheckDataset(SDDS_dataset, "SDDS_CopyColumn"))
    return (0);
  if (target < 0 || source < 0 || target >= SDDS_dataset->layout.n_columns || source >= SDDS_dataset->layout.n_columns) {
    SDDS_SetError("Unable to copy column--target or source index out of range (SDDS_CopyColumn");
    return (0);
  }
  cd_target = SDDS_dataset->layout.column_definition + target;
  cd_source = SDDS_dataset->layout.column_definition + source;
  SDDS_dataset->column_flag[target] = SDDS_dataset->column_flag[source];
  if (SDDS_dataset->n_rows_allocated) {
    if (cd_target->type != cd_source->type) {
      if (!(SDDS_dataset->data[target] = SDDS_Realloc(SDDS_dataset->data[target], SDDS_type_size[cd_source->type - 1] * SDDS_dataset->n_rows_allocated))) {
        SDDS_SetError("Unable to copy column--memory allocation failure (SDDS_CopyColumn)");
        return (0);
      }
    }
    if (cd_source->type != SDDS_STRING)
      memcpy(SDDS_dataset->data[target], SDDS_dataset->data[source], SDDS_type_size[cd_source->type - 1] * SDDS_dataset->n_rows);
    else if (!SDDS_CopyStringArray(SDDS_dataset->data[target], SDDS_dataset->data[source], SDDS_dataset->n_rows)) {
      SDDS_SetError("Unable to copy column--string copy failure (SDDS_CopyColumn)");
      return (0);
    }
  }
  memcpy((char *)cd_target, (char *)cd_source, sizeof(*cd_target));
  return (1);
}

SDDS_CopyParameter()

int32_t SDDS_CopyParameter ( SDDS_DATASET * SDDS_dataset, int32_t target, int32_t source )

extern

Copies a parameter from a source index to a target index within an SDDS dataset.

This function duplicates the parameter data from the source index to the target index in the provided SDDS dataset. It handles both string and non-string data types appropriately, ensuring that memory is managed correctly for string entries.

Parameters

SDDS_dataset	Pointer to the SDDS_DATASET structure representing the dataset.
target	The index of the target parameter where data will be copied to.
source	The index of the source parameter from which data will be copied.

Returns

On success, returns 1. On failure, returns 0 and sets an appropriate error message.

Return values

1	Indicates that the parameter copy was successful.
0	Indicates that an error occurred (e.g., invalid dataset, out-of-range indices, memory allocation failure, string copy failure).

Note

• Both target and source must be valid parameter indices within the dataset.
• The function assumes that the target parameter already exists and is intended to be overwritten.

See also

SDDS_DeleteParameter, SDDS_CopyArray

Definition at line 4018 of file SDDS_extract.c.

                                                                                       {
  PARAMETER_DEFINITION *cd_target, *cd_source;
  if (!SDDS_CheckDataset(SDDS_dataset, "SDDS_CopyParameter"))
    return (0);
  if (target < 0 || source < 0 || target >= SDDS_dataset->layout.n_parameters || source >= SDDS_dataset->layout.n_parameters) {
    SDDS_SetError("Unable to copy parameter--target or source index out of range (SDDS_CopyParameter");
    return (0);
  }
  cd_target = SDDS_dataset->layout.parameter_definition + target;
  cd_source = SDDS_dataset->layout.parameter_definition + source;
  if (SDDS_dataset->parameter) {
    if (cd_target->type != cd_source->type) {
      if (!(SDDS_dataset->parameter[target] = SDDS_Realloc(SDDS_dataset->data[target], SDDS_type_size[cd_source->type - 1]))) {
        SDDS_SetError("Unable to copy parameter--memory allocation failure (SDDS_CopyParameter)");
        return (0);
      }
    }
    if (cd_source->type != SDDS_STRING)
      memcpy(SDDS_dataset->parameter[target], SDDS_dataset->parameter[source], SDDS_type_size[cd_source->type - 1]);
    else if (!SDDS_CopyStringArray(SDDS_dataset->parameter[target], SDDS_dataset->parameter[source], 1)) {
      SDDS_SetError("Unable to copy parameter--string copy failure (SDDS_CopyParameter)");
      return (0);
    }
  }
  memcpy((char *)cd_target, (char *)cd_source, sizeof(*cd_target));
  return (1);
}

SDDS_FreePointerArray()

void SDDS_FreePointerArray ( void ** data, int32_t dimensions, int32_t * dimension )

extern

Frees a multi-dimensional pointer array created by SDDS_MakePointerArray.

This function recursively deallocates a multi-dimensional pointer array that was previously created using SDDS_MakePointerArray or SDDS_MakePointerArrayRecursively. It ensures that all pointer layers are properly freed to prevent memory leaks.

Parameters

[in]	data	Pointer to the multi-dimensional pointer array to be freed.
[in]	dimensions	The number of dimensions in the pointer array.
[in]	dimension	An array specifying the size of each dimension.

Note

• The function assumes that the pointer array was created using SDDS library functions.
• It does not free the actual data block pointed to by the pointer array.

See also

SDDS_MakePointerArrayRecursively

free

Definition at line 3087 of file SDDS_utils.c.

{
  if (!data || !dimension || !dimensions)
    return;
  if (dimensions > 1) {
    SDDS_FreePointerArray((void **)(data[0]), dimensions - 1, dimension + 1);
    free(data);
  }
}

SDDS_FreeTableStrings()

void SDDS_FreeTableStrings

(

SDDS_DATASET *

SDDS_dataset

)

Frees the strings in the current table of the SDDS dataset.

This function frees any strings stored in the data columns of the current table. It does not free strings from parameters or arrays.

Parameters

SDDS_dataset

The SDDS dataset to free table strings from.

Definition at line 1448 of file SDDS_input.c.

                                                       {
  int64_t i, j;
  char **ptr;
  /* free stored strings */
  if (!SDDS_dataset)
    return;
  for (i = 0; i < SDDS_dataset->layout.n_columns; i++)
    if (SDDS_dataset->layout.column_definition[i].type == SDDS_STRING) {
      ptr = (char **)SDDS_dataset->data[i];
      for (j = 0; j < SDDS_dataset->n_rows; j++, ptr++)
        if (*ptr) {
          free(*ptr);
          *ptr = NULL;
        }
    }
}

SDDS_fseek()

int32_t SDDS_fseek ( FILE * fp, int64_t offset, int32_t dir )

extern

Sets the file position indicator for a given file stream with retry logic.

Attempts to set the file position indicator for the specified file stream (fp) to a new position defined by offset and dir. The function retries the fseek operation up to FSEEK_TRIES times in case of transient failures, implementing a delay between attempts.

Parameters

fp	Pointer to the FILE stream whose position indicator is to be set.
offset	Number of bytes to offset from the position specified by dir.
dir	Positioning directive, which can be one of: SEEK_SET to set the position relative to the beginning of the file, SEEK_CUR to set the position relative to the current position, SEEK_END to set the position relative to the end of the file.

Returns

• Returns 0 if the operation is successful.
• Returns -1 if all retry attempts fail to set the file position.

The function attempts to set the file position using fseek. If fseek fails, it sleeps for 1 second (or 1 second using nanosleep on vxWorks systems) before retrying. After FSEEK_TRIES unsuccessful attempts, it reports a warning and returns -1.

Note

• The function is designed to handle temporary file access issues by retrying the fseek operation.
• It is not suitable for non-recoverable fseek errors, which will cause it to fail after retries.

Definition at line 1279 of file SDDS_binary.c.

                                                          {
  int32_t try;
#if defined(vxWorks)
  struct timespec rqtp;
  rqtp.tv_sec = 1;
  rqtp.tv_nsec = 0;
#endif
  for (try = 0; try < FSEEK_TRIES; try++) {
    if (fseek(fp, offset, dir) == -1) {
#if defined(vxWorks)
      nanosleep(&rqtp, NULL);
#else
      sleep(1);
#endif
    } else
      break;
  }
  if (try == 0)
    return 0;
  if (try == FSEEK_TRIES) {
    fputs("warning: fseek problems--unable to recover\n", stderr);
    return -1;
  }
  fputs("warning: fseek problems--recovered\n", stderr);
  return 0;
}

SDDS_GetSelectedRowIndex()

int64_t SDDS_GetSelectedRowIndex ( SDDS_DATASET * SDDS_dataset, int64_t srow_index )

extern

Retrieves the actual row index corresponding to a selected row position within the current data table.

This function maps a selected row index (i.e., the position among rows marked as "of interest") to its corresponding actual row index within the dataset's data table.

Parameters

SDDS_dataset	Pointer to the SDDS_DATASET structure representing the data set.
srow_index	Zero-based index representing the position of the selected row among all rows marked as "of interest".

Returns

• Non-negative integer representing the actual row index within the dataset's data table.
• -1 if an error occurs (e.g., invalid dataset, tabular data not present, srow_index out of range).

Warning

• Ensure that srow_index is within the valid range [0, SDDS_CountRowsOfInterest(SDDS_dataset) - 1] to avoid out-of-range errors.

Note

• This function is useful when iterating over selected rows and needing to access their actual positions within the dataset.

See also

• SDDS_CountRowsOfInterest
• SDDS_GetValue
• SDDS_GetValueAsDouble

Definition at line 1702 of file SDDS_extract.c.

                                                                                 {
  int64_t i, j;
  if (!SDDS_CheckDataset(SDDS_dataset, "SDDS_GetSelectedRowIndex"))
    return (-1);
  if (!SDDS_CheckTabularData(SDDS_dataset, "SDDS_GetSelectedRowIndex"))
    return (-1);
  for (i = j = 0; i < SDDS_dataset->n_rows; i++) {
    if (SDDS_dataset->row_flag[i] && j++ == srow_index)
      break;
  }
  if (i == SDDS_dataset->n_rows)
    return (-1);
  return (i);
}

SDDS_GetSpecialCommentsModes()

uint32_t SDDS_GetSpecialCommentsModes

(

SDDS_DATASET *

SDDS_dataset

)

Retrieves the current special comments modes set in the SDDS dataset.

This function returns the current set of special comment flags that have been parsed and set within the dataset. These flags indicate the presence of specific configurations such as endianness and fixed row counts.

Parameters

[in]

SDDS_dataset

Pointer to the SDDS_DATASET structure representing the dataset.

Returns

• A uint32_t value representing the combined set of special comment flags.
• 0 if no special comments have been set.

Note

• Special comment flags are typically set by parsing comment strings using functions like SDDS_ParseSpecialComments.

Warning

• Ensure that the dataset is properly initialized before calling this function.

See also

SDDS_ParseSpecialComments, SDDS_ResetSpecialCommentsModes

Definition at line 5292 of file SDDS_utils.c.

                                                                  {
  return SDDS_dataset->layout.commentFlags;
}

SDDS_GetTerminateMode()

int32_t SDDS_GetTerminateMode ( void )

extern

Definition at line 1346 of file SDDS_input.c.

                                    {
  int32_t mode;
  mdb_thread_lock(&terminateModeLock);
  mode = terminateMode;
  mdb_thread_unlock(&terminateModeLock);
  return mode;
}

SDDS_LZMAReadLayout()

int32_t SDDS_LZMAReadLayout ( SDDS_DATASET * SDDS_dataset, struct lzmafile * lzmafp )

extern

Reads the header layout of an SDDS dataset from a file with LZMA compression.

Parameters

SDDS_dataset	The SDDS dataset structure to store the layout information.
lzmafp	The LZMA file pointer to the SDDS file.

Returns

Returns 1 on success, 0 on failure.

Definition at line 681 of file SDDS_input.c.

                                                                                 {
  char buffer[SDDS_MAXLINE];
  char *groupName, *ptr;
  FILE *fp1;
  int32_t retval, bigEndianMachine;
  uint32_t commentFlags;
 
  if (!lzmafp) {
    SDDS_SetError("Unable to read layout--NULL file pointer (SDDS_LZMAReadLayout)");
    return (0);
  }
  if (SDDS_dataset->layout.depth == 0) {
    if (SDDS_dataset->layout.disconnected) {
      SDDS_SetError("Can't read layout--file is disconnected (SDDS_LZMAReadLayout)");
      return 0;
    }
    if (!SDDS_CheckDataset(SDDS_dataset, "SDDS_LZMAReadLayout")) {
      lzma_close(lzmafp);
      return (0);
    }
    SDDS_dataset->layout.layout_written = 1; /* it is already in the file */
    if (!lzma_gets(SDDS_dataset->layout.s, SDDS_MAXLINE, lzmafp)) {
      lzma_close(lzmafp);
      SDDS_SetError("Unable to read layout--no header lines found (SDDS_LZMAReadLayout)");
      return (0);
    }
    if (strncmp(SDDS_dataset->layout.s, "SDDS", 4) != 0) {
      lzma_close(lzmafp);
      SDDS_SetError("Unable to read layout--no header lines found (SDDS_LZMAReadLayout)");
      return (0);
    }
    if (sscanf(SDDS_dataset->layout.s + 4, "%" SCNd32, &SDDS_dataset->layout.version) != 1) {
      lzma_close(lzmafp);
      SDDS_SetError("Unable to read layout--no version number on first line (SDDS_LZMAReadLayout)");
      return (0);
    }
    SDDS_ResetSpecialCommentsModes(SDDS_dataset);
    SDDS_dataset->layout.data_command_seen = 0;
  }
  while (SDDS_GetLZMANamelist(SDDS_dataset, SDDS_dataset->layout.s, SDDS_MAXLINE, lzmafp)) {
#if DEBUG
    strcpy(buffer, SDDS_dataset->layout.s);
#endif
    groupName = SDDS_dataset->layout.s + 1;
    if (!(ptr = strpbrk(SDDS_dataset->layout.s, " \t"))) {
      SDDS_SetError("Unable to read layout---no groupname in namelist (SDDS_LZMAReadLayout)");
      return 0;
    }
    *ptr = 0;
    switch (match_string(groupName, SDDS_command, SDDS_NUM_COMMANDS, EXACT_MATCH)) {
    case SDDS_DESCRIPTION_COMMAND:
      if (!SDDS_ProcessDescription(SDDS_dataset, ptr + 1)) {
        lzma_close(lzmafp);
        SDDS_SetError("Unable to process description (SDDS_LZMAReadLayout)");
        return (0);
      }
      break;
    case SDDS_COLUMN_COMMAND:
      if (!SDDS_ProcessColumnDefinition(SDDS_dataset, ptr + 1)) {
        lzma_close(lzmafp);
        SDDS_SetError("Unable to process column definition (SDDS_LZMAReadLayout)");
        return (0);
      }
      break;
    case SDDS_PARAMETER_COMMAND:
      if (!SDDS_ProcessParameterDefinition(SDDS_dataset, ptr + 1)) {
        lzma_close(lzmafp);
        SDDS_SetError("Unable to process parameter definition (SDDS_LZMAReadLayout)");
        return (0);
      }
      break;
    case SDDS_ASSOCIATE_COMMAND:
#if RW_ASSOCIATES != 0
      if (!SDDS_ProcessAssociateDefinition(SDDS_dataset, ptr + 1)) {
        lzma_close(lzmafp);
        SDDS_SetError("Unable to process associate definition (SDDS_LZMAReadLayout)");
        return (0);
      }
#endif
      break;
    case SDDS_DATA_COMMAND:
      if (!SDDS_ProcessDataMode(SDDS_dataset, ptr + 1)) {
        lzma_close(lzmafp);
        SDDS_SetError("Unable to process data mode (SDDS_LZMAReadLayout)");
        return (0);
      }
      if (SDDS_dataset->layout.data_command_seen) {
        /* should never happen */
        lzma_close(lzmafp);
        SDDS_SetError("Unable to read layout--multiple data commands (SDDS_LZMAReadLayout)");
        return (0);
      }
      if (!SDDS_SaveLayout(SDDS_dataset)) {
        SDDS_SetError("Unable to read layout--couldn't save layout (SDDS_LZMAReadLayout)");
        return (0);
      }
      SDDS_dataset->layout.data_command_seen = 1;
      commentFlags = SDDS_GetSpecialCommentsModes(SDDS_dataset);
      if ((commentFlags & SDDS_BIGENDIAN_SEEN) && (commentFlags & SDDS_LITTLEENDIAN_SEEN)) {
        SDDS_SetError("Unable to read data as it says it is both big and little endian (SDDS_LZMAReadLayout)");
        return (0);
      }
      bigEndianMachine = SDDS_IsBigEndianMachine();
      SDDS_dataset->swapByteOrder = SDDS_dataset->layout.byteOrderDeclared = 0;
      SDDS_dataset->autoRecover = 0;
      if ((commentFlags & SDDS_BIGENDIAN_SEEN) || (SDDS_dataset->layout.data_mode.endian == SDDS_BIGENDIAN)) {
        SDDS_dataset->layout.byteOrderDeclared = SDDS_BIGENDIAN_SEEN;
        if (!bigEndianMachine)
          SDDS_dataset->swapByteOrder = 1;
      }
      if ((commentFlags & SDDS_LITTLEENDIAN_SEEN) || (SDDS_dataset->layout.data_mode.endian == SDDS_LITTLEENDIAN)) {
        SDDS_dataset->layout.byteOrderDeclared = SDDS_LITTLEENDIAN_SEEN;
        if (bigEndianMachine)
          SDDS_dataset->swapByteOrder = 1;
      }
      if ((commentFlags & SDDS_FIXED_ROWCOUNT_SEEN) || (SDDS_dataset->layout.data_mode.fixed_row_count))
        if (!SDDS_SetAutoReadRecovery(SDDS_dataset, SDDS_AUTOREADRECOVER))
          return (0);
      return (1);
    case SDDS_INCLUDE_COMMAND:
      if (!(fp1 = SDDS_ProcessIncludeCommand(SDDS_dataset, ptr + 1))) {
        lzma_close(lzmafp);
        SDDS_SetError("Unable to process include command (SDDS_LZMAReadLayout)");
        return (0);
      }
      SDDS_dataset->layout.depth += 1;
      retval = SDDS_ReadLayout(SDDS_dataset, fp1);
      SDDS_dataset->layout.depth -= 1;
      fclose(fp1);
      if (retval == 0) {
        return (0);
      }
      if (SDDS_dataset->layout.data_command_seen) {
        return (1);
      }
      break;
    case SDDS_ARRAY_COMMAND:
      if (!SDDS_ProcessArrayDefinition(SDDS_dataset, ptr + 1)) {
        lzma_close(lzmafp);
        SDDS_SetError("Unable to process array definition (SDDS_LZMAReadLayout)");
        return (0);
      }
      break;
    default:
      lzma_close(lzmafp);
      sprintf(buffer, "Unknown layout entry %s given (SDDS_LZMAReadLayout)", groupName);
      SDDS_SetError(buffer);
      return (0);
    }
  }
  /* on recursive calls, it's okay to hit EOF */
  if ((lzma_eof(lzmafp) && SDDS_dataset->layout.depth != 0) || SDDS_dataset->layout.data_command_seen)
    return (1);
  return (0);
}

SDDS_LZMAWriteArrayDefinition()

int32_t SDDS_LZMAWriteArrayDefinition ( ARRAY_DEFINITION * array_definition, struct lzmafile * lzmafp )

extern

Writes an array definition to an LZMA-compressed file.

This function outputs the definition of a single array in the SDDS layout to an LZMA-compressed file. It includes fields such as name, symbol, units, description, format string, group name, data type, and dimensions. The definition is enclosed within &array and &end tags.

Parameters

array_definition	Pointer to the array definition structure.
lzmafp	The LZMA-compressed file pointer where the array definition will be written.

Returns

Returns 1 on success, 0 if the file pointer is NULL or the array type is invalid.

Definition at line 722 of file SDDS_write.c.

                                                                                                   {
  if (!lzmafp || array_definition->type <= 0 || array_definition->type > SDDS_NUM_TYPES)
    return (0);
 
  lzma_puts("&array ", lzmafp);
  SDDS_LZMAPrintNamelistField(lzmafp, "name", array_definition->name);
  SDDS_LZMAPrintNamelistField(lzmafp, "symbol", SDDS_BlankToNull(array_definition->symbol));
  SDDS_LZMAPrintNamelistField(lzmafp, "units", SDDS_BlankToNull(array_definition->units));
  SDDS_LZMAPrintNamelistField(lzmafp, "description", SDDS_BlankToNull(array_definition->description));
  SDDS_LZMAPrintNamelistField(lzmafp, "format_string", SDDS_BlankToNull(array_definition->format_string));
  SDDS_LZMAPrintNamelistField(lzmafp, "group_name", SDDS_BlankToNull(array_definition->group_name));
  SDDS_LZMAPrintNamelistField(lzmafp, "type", SDDS_type_name[array_definition->type - 1]);
  if (array_definition->dimensions != 1) /* 1 is default */
    lzma_printf(lzmafp, "dimensions=%" PRId32 ", ", array_definition->dimensions);
  lzma_puts(" &end\n", lzmafp);
  return (1);
}

SDDS_LZMAWriteAsciiArrays()

int32_t SDDS_LZMAWriteAsciiArrays ( SDDS_DATASET * SDDS_dataset, struct lzmafile * lzmafp )

extern

Writes the arrays of an SDDS dataset in ASCII format to an LZMA compressed file.

This function writes all arrays contained in the provided SDDS dataset to the specified LZMA compressed file in ASCII format. For each array, it writes the dimensions, a description line, and the array elements formatted according to their data type.

Parameters

SDDS_dataset	Pointer to the SDDS dataset containing the arrays to be written.
lzmafp	Pointer to the LZMA file stream where the array data will be written.

Returns

Returns 1 on success, or 0 on error. If an error occurs, an error message is set via SDDS_SetError().

Note

The function checks the dataset for consistency before writing. It uses SDDS_LZMAWriteTypedValue() to write each array element according to its data type. Each array element is written, with up to 6 elements per line.

Definition at line 680 of file SDDS_ascii.c.

                                                                                       {
  int32_t i, j;
  SDDS_LAYOUT *layout;
  /*  char *predefined_format; */
  ARRAY_DEFINITION *array_definition;
  SDDS_ARRAY *array;
 
  if (!SDDS_CheckDataset(SDDS_dataset, "SDDS_LZMAWriteAsciiArrays"))
    return (0);
  layout = &SDDS_dataset->layout;
  for (j = 0; j < layout->n_arrays; j++) {
    array_definition = layout->array_definition + j;
    array = &SDDS_dataset->array[j];
    for (i = 0; i < array_definition->dimensions; i++)
      lzma_printf(lzmafp, "%" PRId32 " ", array->dimension[i]);
    lzma_printf(lzmafp, "          ! %" PRId32 "-dimensional array %s:\n", array_definition->dimensions, array_definition->name);
    for (i = 0; i < array->elements;) {
      if (!SDDS_LZMAWriteTypedValue(array->data, i, array_definition->type, NULL, lzmafp)) {
        SDDS_SetError("Unable to write array--couldn't write ASCII data (SDDS_LZMAWriteAsciiArrays)");
        return (0);
      }
      i++;
      if (i % 6 == 0 || i == array->elements)
        lzma_putc('\n', lzmafp);
      else
        lzma_putc(' ', lzmafp);
    }
  }
  return (1);
}

SDDS_LZMAWriteAsciiParameters()

int32_t SDDS_LZMAWriteAsciiParameters ( SDDS_DATASET * SDDS_dataset, struct lzmafile * lzmafp )

extern

Writes the parameter data of an SDDS dataset in ASCII format to an LZMA compressed file.

This function writes all parameters of the provided SDDS dataset to the specified LZMA compressed file in ASCII format. Only parameters that do not have fixed values are written. Each parameter value is written on a new line.

Parameters

SDDS_dataset	Pointer to the SDDS dataset containing the parameter data.
lzmafp	Pointer to the LZMA file stream where the ASCII data will be written.

Returns

Returns 1 on success, or 0 on error. If an error occurs, an error message is set via SDDS_SetError().

Note

The function checks the dataset for consistency before writing. It uses SDDS_LZMAWriteTypedValue() to write each parameter value according to its data type.

Definition at line 565 of file SDDS_ascii.c.

                                                                                           {
  int32_t i;
  SDDS_LAYOUT *layout;
  /*  char *predefined_format; */
 
  if (!SDDS_CheckDataset(SDDS_dataset, "SDDS_LZMAWriteAsciiParameters"))
    return (0);
  layout = &SDDS_dataset->layout;
  for (i = 0; i < layout->n_parameters; i++) {
    if (layout->parameter_definition[i].fixed_value)
      continue;
    if (!SDDS_LZMAWriteTypedValue(SDDS_dataset->parameter[i], 0, layout->parameter_definition[i].type, NULL, lzmafp)) {
      SDDS_SetError("Unable to write ascii parameters (SDDS_LZMAWriteAsciiParameters)");
      return 0;
    }
    lzma_putc('\n', lzmafp);
  }
  return (1);
}

SDDS_LZMAWriteAsciiRow()

int32_t SDDS_LZMAWriteAsciiRow ( SDDS_DATASET * SDDS_dataset, int64_t row, struct lzmafile * lzmafp )

extern

Writes a single row of data in ASCII format to an LZMA compressed file.

This function writes the specified row from the SDDS dataset to the provided LZMA compressed file in ASCII format. The data is formatted according to the data types of the columns. Supports multi-line rows as specified by the data mode settings in the dataset layout.

Parameters

SDDS_dataset	Pointer to the SDDS dataset containing the data.
row	Zero-based index of the row to write.
lzmafp	Pointer to the LZMA file stream where the row data will be written.

Returns

Returns 1 on success, or 0 on error. If an error occurs, an error message is set via SDDS_SetError().

Note

The function checks the dataset for consistency before writing. It uses SDDS_LZMAWriteTypedValue() to write each column value according to its data type. The number of values per line is determined by the lines_per_row setting in the dataset layout.

Definition at line 826 of file SDDS_ascii.c.

                                                                                                 {
  int32_t newline_needed;
  int64_t i, n_per_line, line;
  /*  int32_t  embedded_quotes_present; */
  SDDS_LAYOUT *layout;
  /*  char *predefined_format, *s; */
 
  if (!SDDS_CheckDataset(SDDS_dataset, "SDDS_LZMAWriteAsciiRow"))
    return (0);
  layout = &SDDS_dataset->layout;
  if (SDDS_dataset->layout.data_mode.lines_per_row <= 0)
    SDDS_dataset->layout.data_mode.lines_per_row = 1;
  n_per_line = SDDS_dataset->layout.n_columns / SDDS_dataset->layout.data_mode.lines_per_row;
  line = 1;
  newline_needed = 0;
  for (i = 0; i < layout->n_columns; i++) {
    if (!SDDS_LZMAWriteTypedValue(SDDS_dataset->data[i], row, layout->column_definition[i].type, NULL, lzmafp)) {
      SDDS_SetError("Unable to write ascii row (SDDS_LZMAWriteAsciiRow)");
      return 0;
    }
    if ((i + 1) % n_per_line == 0 && line != SDDS_dataset->layout.data_mode.lines_per_row) {
      newline_needed = 0;
      lzma_putc('\n', lzmafp);
      line++;
    } else {
      lzma_putc(' ', lzmafp);
      newline_needed = 1;
    }
  }
  if (newline_needed)
    lzma_putc('\n', lzmafp);
  return (1);
}

SDDS_LZMAWriteAssociateDefinition()

int32_t SDDS_LZMAWriteAssociateDefinition ( ASSOCIATE_DEFINITION * associate_definition, struct lzmafile * lzmafp )

extern

Writes an associate definition to an LZMA-compressed file.

This function outputs the definition of an associate in the SDDS layout to an LZMA-compressed file. It includes fields such as name, filename, contents, path, description, and an SDDS flag. The definition is enclosed within &associate and &end tags.

Parameters

associate_definition	Pointer to the associate definition structure.
lzmafp	The LZMA-compressed file pointer where the associate definition will be written.

Returns

Returns 1 on success, 0 if the file pointer is NULL.

Definition at line 520 of file SDDS_write.c.

                                                                                                               {
  if (!lzmafp)
    return (0);
 
  lzma_puts("&associate ", lzmafp);
  SDDS_LZMAPrintNamelistField(lzmafp, "name", associate_definition->name);
  SDDS_LZMAPrintNamelistField(lzmafp, "filename", SDDS_BlankToNull(associate_definition->filename));
  SDDS_LZMAPrintNamelistField(lzmafp, "contents", SDDS_BlankToNull(associate_definition->contents));
  SDDS_LZMAPrintNamelistField(lzmafp, "path", SDDS_BlankToNull(associate_definition->path));
  SDDS_LZMAPrintNamelistField(lzmafp, "description", SDDS_BlankToNull(associate_definition->description));
  lzma_printf(lzmafp, "sdds=%" PRId32, associate_definition->sdds);
  lzma_puts(" &end\n", lzmafp);
  return (1);
}

SDDS_LZMAWriteBinaryString()

int32_t SDDS_LZMAWriteBinaryString	(	char *	string,
		struct lzmafile *	lzmafp,
		SDDS_FILEBUFFER *	fBuffer )

Writes a binary string to a file with LZMA compression.

This function writes a binary string to the specified LZMA-compressed file by first writing the length of the string followed by the string's content. If the input string is NULL, an empty string is written instead. The writing operation utilizes LZMA buffered write functions to ensure data is compressed appropriately.

Parameters

[in]	string	The null-terminated string to be written. If NULL, an empty string is written.
[in]	lzmafp	The LZMA file pointer to write to. Must be a valid, open LZMA-compressed file in write mode.
[in,out]	fBuffer	Pointer to the file buffer used for buffered writing operations.

Returns

int32_t Returns 1 on success, 0 on failure.

Return values

1	Operation was successful.
0	An error occurred during writing.

Definition at line 2674 of file SDDS_binary.c.

                                                                                                    {
  int32_t length;
  static const char dummy_string[] = "";
  if (!string)
    string = (char *)dummy_string;
  length = strlen(string);
  if (!SDDS_LZMABufferedWrite(&length, sizeof(length), lzmafp, fBuffer)) {
    SDDS_SetError("Unable to write string--error writing length");
    return (0);
  }
  if (length && !SDDS_LZMABufferedWrite(string, sizeof(*string) * length, lzmafp, fBuffer)) {
    SDDS_SetError("Unable to write string--error writing contents");
    return (0);
  }
  return (1);
}

SDDS_LZMAWriteColumnDefinition()

int32_t SDDS_LZMAWriteColumnDefinition ( COLUMN_DEFINITION * column_definition, struct lzmafile * lzmafp )

extern

Writes a column definition to an LZMA-compressed file.

This function outputs the definition of a single column in the SDDS layout to an LZMA-compressed file. It includes fields such as name, symbol, units, description, format string, and data type. The definition is enclosed within &column and &end tags.

Parameters

column_definition	Pointer to the column definition structure.
lzmafp	The LZMA-compressed file pointer where the column definition will be written.

Returns

Returns 1 on success, 0 if the file pointer is NULL or the column type is invalid.

Definition at line 356 of file SDDS_write.c.

                                                                                                      {
  if (!lzmafp || column_definition->type <= 0 || column_definition->type > SDDS_NUM_TYPES)
    return (0);
 
  lzma_puts("&column ", lzmafp);
  SDDS_LZMAPrintNamelistField(lzmafp, "name", column_definition->name);
  SDDS_LZMAPrintNamelistField(lzmafp, "symbol", SDDS_BlankToNull(column_definition->symbol));
  SDDS_LZMAPrintNamelistField(lzmafp, "units", SDDS_BlankToNull(column_definition->units));
  SDDS_LZMAPrintNamelistField(lzmafp, "description", SDDS_BlankToNull(column_definition->description));
  SDDS_LZMAPrintNamelistField(lzmafp, "format_string", SDDS_BlankToNull(column_definition->format_string));
  SDDS_LZMAPrintNamelistField(lzmafp, "type", SDDS_type_name[column_definition->type - 1]);
  lzma_puts(" &end\n", lzmafp);
  return (1);
}

SDDS_LZMAWriteDataMode()

int32_t SDDS_LZMAWriteDataMode ( SDDS_LAYOUT * layout, struct lzmafile * lzmafp )

extern

Writes the data mode section to an LZMA-compressed file.

This function outputs the data mode settings of the SDDS layout to the specified LZMA-compressed file pointer. It includes settings such as mode, lines per row, row counts, endianess, column-major order, and fixed row counts. The section is enclosed within &data and &end tags.

Parameters

layout	Pointer to the SDDS layout structure containing data mode settings.
lzmafp	The LZMA-compressed file pointer where the data mode section will be written.

Returns

Returns 1 on success, 0 if the file pointer is NULL or the data mode is invalid.

Definition at line 614 of file SDDS_write.c.

                                                                             {
  if (!lzmafp || layout->data_mode.mode < 0 || layout->data_mode.mode > SDDS_NUM_DATA_MODES)
    return (0);
 
  lzma_puts("&data ", lzmafp);
  SDDS_LZMAPrintNamelistField(lzmafp, "mode", SDDS_data_mode[layout->data_mode.mode - 1]);
  if (layout->data_mode.lines_per_row > 1)
    lzma_printf(lzmafp, "lines_per_row=%" PRId32 ", ", layout->data_mode.lines_per_row);
  if (layout->data_mode.no_row_counts)
    lzma_printf(lzmafp, "no_row_counts=1, ");
  if (layout->version >= 3) {
    if (layout->data_mode.mode == SDDS_BINARY) {
      if (layout->byteOrderDeclared == SDDS_BIGENDIAN)
        lzma_printf(lzmafp, "endian=big, ");
      else
        lzma_printf(lzmafp, "endian=little, ");
      if (layout->data_mode.column_major)
        lzma_printf(lzmafp, "column_major_order=1, ");
    }
    if (layout->data_mode.fixed_row_count)
      lzma_printf(lzmafp, "fixed_row_count=1, ");
  }
  lzma_puts("&end\n", lzmafp);
  return (1);
}

SDDS_LZMAWriteDescription()

int32_t SDDS_LZMAWriteDescription ( char * text, char * contents, struct lzmafile * lzmafp )

extern

Writes the SDDS description section to an LZMA-compressed file.

This function writes the description section of the SDDS layout, including optional text and contents fields. It encapsulates the data within &description and &end tags in an LZMA-compressed file.

Parameters

text	The descriptive text for the SDDS layout.
contents	The contents description for the SDDS layout.
lzmafp	The LZMA-compressed file pointer where the description will be written.

Returns

Returns 1 on success, 0 if the file pointer is NULL.

Definition at line 280 of file SDDS_write.c.

                                                                                       {
  if (!lzmafp)
    return 0;
  if (!text && !contents)
    return 1;
  lzma_puts("&description ", lzmafp);
  SDDS_LZMAPrintNamelistField(lzmafp, "text", text);
  SDDS_LZMAPrintNamelistField(lzmafp, "contents", contents);
  lzma_puts("&end\n", lzmafp);
  return 1;
}

SDDS_LZMAWriteNonNativeBinaryString()

int32_t SDDS_LZMAWriteNonNativeBinaryString	(	char *	string,
		struct lzmafile *	lzmafp,
		SDDS_FILEBUFFER *	fBuffer )

Writes a non-native endian binary string to an LZMA-compressed file.

This function writes a binary string to the specified LZMA-compressed file pointer, handling non-native endianness. It first writes the length of the string as a 32-bit integer with byte order swapped. If the string is not to be skipped, it then writes the string data itself followed by a null terminator. If the input string is NULL, an empty string is written instead.

Parameters

[in]	string	The string to write. If NULL, an empty string is written.
[in]	lzmafp	Pointer to the LZMAFILE where the string will be written.
[in]	fBuffer	Pointer to the SDDS_FILEBUFFER structure used for buffered writing.

Returns

int32_t Returns 1 on successful writing of the string, or 0 if an error occurred.

Return values

1	The string was successfully written and byte-swapped.
0	An error occurred during the write operation, such as I/O failures or memory allocation issues.

Note

The caller is responsible for ensuring that the LZMAFILE pointer lzmafp is valid and open for writing. This function does not perform memory allocation for the string; it assumes that the string is already allocated and managed appropriately.

Definition at line 5763 of file SDDS_binary.c.

                                                                                                             {
  int32_t length;
  static const char dummy_string[] = "";
  if (!string)
    string = (char *)dummy_string;
  length = strlen(string);
  SDDS_SwapLong(&length);
  if (!SDDS_LZMABufferedWrite(&length, sizeof(length), lzmafp, fBuffer)) {
    SDDS_SetError("Unable to write string--error writing length");
    return (0);
  }
  SDDS_SwapLong(&length);
  if (length && !SDDS_LZMABufferedWrite(string, sizeof(*string) * length, lzmafp, fBuffer)) {
    SDDS_SetError("Unable to write string--error writing contents");
    return (0);
  }
  return (1);
}

SDDS_LZMAWriteParameterDefinition()

int32_t SDDS_LZMAWriteParameterDefinition ( PARAMETER_DEFINITION * parameter_definition, struct lzmafile * lzmafp )

extern

Writes a parameter definition to an LZMA-compressed file.

This function outputs the definition of a single parameter in the SDDS layout to an LZMA-compressed file. It includes fields such as name, symbol, units, description, format string, data type, and fixed value. The definition is enclosed within &parameter and &end tags.

Parameters

parameter_definition	Pointer to the parameter definition structure.
lzmafp	The LZMA-compressed file pointer where the parameter definition will be written.

Returns

Returns 1 on success, 0 if the file pointer is NULL or the parameter type is invalid.

Definition at line 438 of file SDDS_write.c.

                                                                                                               {
  if (!lzmafp || parameter_definition->type <= 0 || parameter_definition->type > SDDS_NUM_TYPES)
    return (0);
  lzma_puts("&parameter ", lzmafp);
  SDDS_LZMAPrintNamelistField(lzmafp, "name", parameter_definition->name);
  SDDS_LZMAPrintNamelistField(lzmafp, "symbol", SDDS_BlankToNull(parameter_definition->symbol));
  SDDS_LZMAPrintNamelistField(lzmafp, "units", SDDS_BlankToNull(parameter_definition->units));
  SDDS_LZMAPrintNamelistField(lzmafp, "description", SDDS_BlankToNull(parameter_definition->description));
  SDDS_LZMAPrintNamelistField(lzmafp, "format_string", SDDS_BlankToNull(parameter_definition->format_string));
  SDDS_LZMAPrintNamelistField(lzmafp, "type", SDDS_type_name[parameter_definition->type - 1]);
  SDDS_LZMAPrintNamelistField(lzmafp, "fixed_value", parameter_definition->fixed_value);
  lzma_puts("&end\n", lzmafp);
  return (1);
}

SDDS_LZMAWriteVersion()

int32_t SDDS_LZMAWriteVersion ( int32_t version_number, struct lzmafile * lzmafp )

extern

Writes the SDDS protocol version to an LZMA-compressed file.

This function outputs the SDDS protocol version string to the provided LZMA-compressed file pointer. Maintaining the protocol version is essential for ensuring the integrity of the SDDS file format.

Parameters

version_number	The SDDS protocol version number to write.
lzmafp	The LZMA-compressed file pointer where the version string will be written.

Returns

Returns 1 on successful write, 0 if the file pointer is NULL.

Definition at line 77 of file SDDS_write.c.

                                                                               {
  if (!lzmafp)
    return (0);
  lzma_printf(lzmafp, "SDDS%" PRId32 "\n", version_number);
  return (1);
}

SDDS_ParseSpecialComments()

void SDDS_ParseSpecialComments	(	SDDS_DATASET *	SDDS_dataset,
		char *	s )

Parses and processes special comment commands within the SDDS dataset.

This function interprets special commands embedded within comment strings of the SDDS dataset. Supported special commands include:

• big-endian
• little-endian
• fixed-rowcount

Each recognized command updates the commentFlags field within the dataset's layout to reflect the presence of these special configurations.

Parameters

[in,out]	SDDS_dataset	Pointer to the SDDS_DATASET structure representing the dataset. The commentFlags field will be updated based on the parsed commands.
[in]	s	Pointer to the null-terminated string containing special comment commands to be parsed.

Note

• This function is intended to be used internally by the SDDS library to handle special configurations specified in comments.
• Only the predefined special commands are recognized and processed.

Warning

• Unrecognized commands within the comment string are ignored.
• Ensure that the input string s is properly formatted and null-terminated to prevent undefined behavior.

See also

SDDS_GetSpecialCommentsModes, SDDS_ResetSpecialCommentsModes

Definition at line 5342 of file SDDS_utils.c.

                                                                    {
  char buffer[SDDS_MAXLINE];
  int32_t i;
  if (SDDS_dataset == NULL)
    return;
  while (SDDS_GetToken(s, buffer, SDDS_MAXLINE) > 0) {
    for (i = 0; i < COMMENT_COMMANDS; i++) {
      if (strcmp(buffer, commentCommandName[i]) == 0) {
        SDDS_dataset->layout.commentFlags |= commentCommandFlag[i];
        break;
      }
    }
  }
}

SDDS_ProcessArrayDefinition()

int32_t SDDS_ProcessArrayDefinition ( SDDS_DATASET * SDDS_dataset, char * s )

extern

Process the array definition section of the SDDS dataset.

Parameters

SDDS_dataset	Pointer to the SDDS_DATASET structure.
s	Pointer to the array definition string.

Returns

Returns 1 on success, 0 on failure.

Definition at line 243 of file SDDS_process.c.

                                                                         {
  if (!SDDS_CheckDataset(SDDS_dataset, "SDDS_ProcessArrayDefinition"))
    return (0);
  if (!s) {
    SDDS_SetError("Unable to process array definition--namelist text pointer NULL (SDDS_ProcessArrayDefinition)");
    return (0);
  }
  switch (SDDS_dataset->layout.version) {
  case 1:
    return (SDDS1_ProcessArrayDefinition(SDDS_dataset, s));
  case 2:
    return (SDDS1_ProcessArrayDefinition(SDDS_dataset, s));
  case 3:
    return (SDDS1_ProcessArrayDefinition(SDDS_dataset, s));
  case 4:
    return (SDDS1_ProcessArrayDefinition(SDDS_dataset, s));
  case 5:
    return (SDDS1_ProcessArrayDefinition(SDDS_dataset, s));
  default:
    SDDS_SetError("Unable to process array definition--protocol version number is invalid (SDDS_ProcessArrayDefinition)");
    return (0);
  }
}

SDDS_ProcessAssociateDefinition()

int32_t SDDS_ProcessAssociateDefinition ( SDDS_DATASET * SDDS_dataset, char * s )

extern

Process the associate definition section of the SDDS dataset.

Parameters

SDDS_dataset	Pointer to the SDDS_DATASET structure.
s	Pointer to the associate definition string.

Returns

Returns 1 on success, 0 on failure.

Definition at line 181 of file SDDS_process.c.

                                                                             {
  if (!SDDS_CheckDataset(SDDS_dataset, "SDDS_ProcessAssociateDefinition"))
    return (0);
  if (!s) {
    SDDS_SetError("Unable to process associate definition--namelist text pointer NULL (SDDS_ProcessAssociateDefinition)");
    return (0);
  }
  switch (SDDS_dataset->layout.version) {
  case 1:
    return (SDDS1_ProcessAssociateDefinition(SDDS_dataset, s));
  case 2:
    return (SDDS1_ProcessAssociateDefinition(SDDS_dataset, s));
  case 3:
    return (SDDS1_ProcessAssociateDefinition(SDDS_dataset, s));
  case 4:
    return (SDDS1_ProcessAssociateDefinition(SDDS_dataset, s));
  case 5:
    return (SDDS1_ProcessAssociateDefinition(SDDS_dataset, s));
  default:
    SDDS_SetError("Unable to process associate definition--protocol version number is invalid (SDDS_ProcessAssociateDefinition)");
    return (0);
  }
}

SDDS_ProcessColumnDefinition()

int32_t SDDS_ProcessColumnDefinition ( SDDS_DATASET * SDDS_dataset, char * s )

extern

Process the column definition section of the SDDS dataset.

Parameters

SDDS_dataset	Pointer to the SDDS_DATASET structure.
s	Pointer to the column definition string.

Returns

Returns 1 on success, 0 on failure.

Definition at line 88 of file SDDS_process.c.

                                                                          {
  if (!SDDS_CheckDataset(SDDS_dataset, "SDDS1_ProcessColumnDefinition"))
    return (0);
  if (!s) {
    SDDS_SetError("Unable to process column definition--namelist text pointer NULL (SDDS1_ProcessColumnDefinition)");
    return (0);
  }
  switch (SDDS_dataset->layout.version) {
  case 1:
    return (SDDS1_ProcessColumnDefinition(SDDS_dataset, s));
  case 2:
    return (SDDS1_ProcessColumnDefinition(SDDS_dataset, s));
  case 3:
    return (SDDS1_ProcessColumnDefinition(SDDS_dataset, s));
  case 4:
    return (SDDS1_ProcessColumnDefinition(SDDS_dataset, s));
  case 5:
    return (SDDS1_ProcessColumnDefinition(SDDS_dataset, s));
  default:
    SDDS_SetError("Unable to process column definition--protocol version number is invalid (SDDS_ProcessColumnDefinition)");
    return (0);
  }
}

SDDS_ProcessDataMode()

int32_t SDDS_ProcessDataMode ( SDDS_DATASET * SDDS_dataset, char * s )

extern

Process the data mode section of the SDDS dataset.

Parameters

SDDS_dataset	Pointer to the SDDS_DATASET structure.
s	Pointer to the data mode string.

Returns

Returns 1 on success, 0 on failure.

Definition at line 212 of file SDDS_process.c.

                                                                  {
  if (!SDDS_CheckDataset(SDDS_dataset, "SDDS_ProcessDataMode"))
    return (0);
  if (!s) {
    SDDS_SetError("Unable to process data mode--namelist text pointer is NULL (SDDS_ProcessDataMode)");
    return (0);
  }
  switch (SDDS_dataset->layout.version) {
  case 1:
    return (SDDS1_ProcessDataMode(SDDS_dataset, s));
  case 2:
    return (SDDS1_ProcessDataMode(SDDS_dataset, s));
  case 3:
    return (SDDS1_ProcessDataMode(SDDS_dataset, s));
  case 4:
    return (SDDS1_ProcessDataMode(SDDS_dataset, s));
  case 5:
    return (SDDS1_ProcessDataMode(SDDS_dataset, s));
  default:
    SDDS_SetError("Unable to process data mode--protocol version number is invalid (SDDS_ProcessDataMode)");
    return (0);
  }
}

SDDS_ProcessDescription()

int32_t SDDS_ProcessDescription ( SDDS_DATASET * SDDS_dataset, char * s )

extern

Process the description section of the SDDS dataset.

Parameters

SDDS_dataset	Pointer to the SDDS_DATASET structure.
s	Pointer to the description string.

Returns

Returns 1 on success, 0 on failure.

Definition at line 57 of file SDDS_process.c.

                                                                     {
  if (!SDDS_CheckDataset(SDDS_dataset, "SDDS_ProcessDescription"))
    return (0);
  if (!s) {
    SDDS_SetError("Unable to process description--namelist text pointer is NULL (SDDS_ProcessDescription)");
    return (0);
  }
  switch (SDDS_dataset->layout.version) {
  case 1:
    return (SDDS1_ProcessDescription(SDDS_dataset, s));
  case 2:
    return (SDDS1_ProcessDescription(SDDS_dataset, s));
  case 3:
    return (SDDS1_ProcessDescription(SDDS_dataset, s));
  case 4:
    return (SDDS1_ProcessDescription(SDDS_dataset, s));
  case 5:
    return (SDDS1_ProcessDescription(SDDS_dataset, s));
  default:
    SDDS_SetError("Unable to process description--protocol version number is invalid (SDDS_ProcessDescription)");
    return (0);
  }
}

SDDS_ProcessIncludeCommand()

FILE * SDDS_ProcessIncludeCommand ( SDDS_DATASET * SDDS_dataset, char * s )

extern

Process the include command within the SDDS dataset.

Parameters

SDDS_dataset	Pointer to the SDDS_DATASET structure.
s	Pointer to the include command string.

Returns

Returns a FILE pointer on success, NULL on failure.

Definition at line 150 of file SDDS_process.c.

                                                                      {
  if (!SDDS_CheckDataset(SDDS_dataset, "SDDS_ProcessIncludeCommand"))
    return (NULL);
  if (!s) {
    SDDS_SetError("Unable to process parameter definition--namelist text pointer NULL (SDDS_ProcessIncludeCommand)");
    return (NULL);
  }
  switch (SDDS_dataset->layout.version) {
  case 1:
    return (SDDS1_ProcessIncludeCommand(SDDS_dataset, s));
  case 2:
    return (SDDS1_ProcessIncludeCommand(SDDS_dataset, s));
  case 3:
    return (SDDS1_ProcessIncludeCommand(SDDS_dataset, s));
  case 4:
    return (SDDS1_ProcessIncludeCommand(SDDS_dataset, s));
  case 5:
    return (SDDS1_ProcessIncludeCommand(SDDS_dataset, s));
  default:
    SDDS_SetError("Unable to process parameter definition--protocol version number is invalid (SDDS_ProcessIncludeCommand)");
    return (0);
  }
}

SDDS_ProcessParameterDefinition()

int32_t SDDS_ProcessParameterDefinition ( SDDS_DATASET * SDDS_dataset, char * s )

extern

Process the parameter definition section of the SDDS dataset.

Parameters

SDDS_dataset	Pointer to the SDDS_DATASET structure.
s	Pointer to the parameter definition string.

Returns

Returns 1 on success, 0 on failure.

Definition at line 119 of file SDDS_process.c.

                                                                             {
  if (!SDDS_CheckDataset(SDDS_dataset, "SDDS_ProcessParameterDefinition"))
    return (0);
  if (!s) {
    SDDS_SetError("Unable to process parameter definition--namelist text pointer NULL (SDDS_ProcessParameterDefinition)");
    return (0);
  }
  switch (SDDS_dataset->layout.version) {
  case 1:
    return (SDDS1_ProcessParameterDefinition(SDDS_dataset, s));
  case 2:
    return (SDDS1_ProcessParameterDefinition(SDDS_dataset, s));
  case 3:
    return (SDDS1_ProcessParameterDefinition(SDDS_dataset, s));
  case 4:
    return (SDDS1_ProcessParameterDefinition(SDDS_dataset, s));
  case 5:
    return (SDDS1_ProcessParameterDefinition(SDDS_dataset, s));
  default:
    SDDS_SetError("Unable to process parameter definition--protocol version number is invalid (SDDS_ProcessParameterDefinition)");
    return (0);
  }
}

SDDS_ReadAsciiArrays()

int32_t SDDS_ReadAsciiArrays ( SDDS_DATASET * SDDS_dataset )

extern

Reads the arrays from an ASCII file into the SDDS dataset.

This function reads array data from an ASCII file and stores it in the provided SDDS dataset. It supports reading from regular files, as well as GZIP and LZMA compressed files if enabled.

Parameters

SDDS_dataset

Pointer to the SDDS dataset where the arrays will be stored.

Returns

Returns 1 on success, 0 on error, or -1 if end-of-file is reached before any arrays are read.

Note

The function checks the dataset for consistency before reading. It reads array dimensions and elements, handling multi-dimensional arrays appropriately. The array elements are scanned and stored according to their data types.

Definition at line 1047 of file SDDS_ascii.c.

                                                         {
  int32_t i, j, length;
  SDDS_LAYOUT *layout;
  char *buffer = NULL;
  int32_t bufferSize = 0;
  char *bigBuffer = NULL;
  int32_t bigBufferSize = 0;
  char *bigBufferCopy;
  int32_t bigBufferCopySize;
  /*  ARRAY_DEFINITION *array_definition; */
  SDDS_ARRAY *array;
#if defined(zLib)
  gzFile gzfp = NULL;
#endif
  FILE *fp = NULL;
  struct lzmafile *lzmafp = NULL;
 
  layout = &SDDS_dataset->layout;
#if defined(zLib)
  if (SDDS_dataset->layout.gzipFile) {
    gzfp = layout->gzfp;
  } else {
#endif
    if (SDDS_dataset->layout.lzmaFile) {
      lzmafp = layout->lzmafp;
    } else {
      fp = layout->fp;
    }
#if defined(zLib)
  }
#endif
  if (layout->n_arrays > 0) {
    if (!(bigBuffer = SDDS_Malloc(sizeof(*bigBuffer) * (bigBufferSize = INITIAL_BIG_BUFFER_SIZE)))) {
      SDDS_SetError("Unable to read parameters--buffer allocation failure (SDDS_ReadAsciiArrays)");
      return (0);
    }
  }
  for (i = 0; i < layout->n_arrays; i++) {
#if defined(zLib)
    if (SDDS_dataset->layout.gzipFile) {
      if (!fgetsGZipSkipCommentsResize(SDDS_dataset, &bigBuffer, &bigBufferSize, gzfp, '!') || SDDS_StringIsBlank(bigBuffer)) {
        if (i == 0)
          return (-1);
        SDDS_SetError("Unable to read array--dimensions missing (SDDS_ReadAsciiArrays)");
        return (0);
      }
    } else {
#endif
      if (SDDS_dataset->layout.lzmaFile) {
        if (!fgetsLZMASkipCommentsResize(SDDS_dataset, &bigBuffer, &bigBufferSize, lzmafp, '!') || SDDS_StringIsBlank(bigBuffer)) {
          if (i == 0)
            return (-1);
          SDDS_SetError("Unable to read array--dimensions missing (SDDS_ReadAsciiArrays)");
          return (0);
        }
      } else {
        if (!fgetsSkipCommentsResize(SDDS_dataset, &bigBuffer, &bigBufferSize, fp, '!') || SDDS_StringIsBlank(bigBuffer)) {
          if (i == 0)
            return (-1);
          SDDS_SetError("Unable to read array--dimensions missing (SDDS_ReadAsciiArrays)");
          return (0);
        }
      }
#if defined(zLib)
    }
#endif
    if (!(array = SDDS_dataset->array + i)) {
      SDDS_SetError("Unable to read array--pointer to structure storage area is NULL (SDDS_ReadAsciiArrays)");
      return (0);
    }
    if (array->definition && !SDDS_FreeArrayDefinition(array->definition)) {
      SDDS_SetError("Unable to get array--array definition corrupted (SDDS_ReadAsciiArrays)");
      return (0);
    }
    if (!SDDS_CopyArrayDefinition(&array->definition, layout->array_definition + i)) {
      SDDS_SetError("Unable to read array--definition copy failed (SDDS_ReadAsciiArrays)");
      return (0);
    }
    /*if (array->dimension) free(array->dimension); */
    if (!(array->dimension = SDDS_Realloc(array->dimension, sizeof(*array->dimension) * array->definition->dimensions))) {
      SDDS_SetError("Unable to read array--allocation failure (SDDS_ReadAsciiArrays)");
      return (0);
    }
    array->elements = 1;
    if ((length = strlen(bigBuffer)) >= bufferSize) {
      if (!(buffer = SDDS_Realloc(buffer, sizeof(*buffer) * (bufferSize = 2 * length)))) {
        SDDS_SetError("Unable to scan data--allocation failure (SDDS_ReadAsciiArrays");
        return (0);
      }
    }
    for (j = 0; j < array->definition->dimensions; j++) {
      if (SDDS_GetToken(bigBuffer, buffer, SDDS_MAXLINE) <= 0 || sscanf(buffer, "%" SCNd32, array->dimension + j) != 1 || array->dimension[j] < 0) {
        SDDS_SetError("Unable to read array--dimensions missing or negative (SDDS_ReadAsciiArrays)");
        return (0);
      }
      array->elements *= array->dimension[j];
    }
    if (!array->elements)
      continue;
    if (array->data)
      free(array->data);
    array->data = array->pointer = NULL;
    if (!(array->data = SDDS_Realloc(array->data, array->elements * SDDS_type_size[array->definition->type - 1]))) {
      SDDS_SetError("Unable to read array--allocation failure (SDDS_ReadAsciiArrays)");
      return (0);
    }
    SDDS_ZeroMemory(array->data, array->elements * SDDS_type_size[array->definition->type - 1]);
    j = 0;
    bigBuffer[0] = 0;
    do {
      if (SDDS_StringIsBlank(bigBuffer)) {
        bigBuffer[0] = 0;
#if defined(zLib)
        if (SDDS_dataset->layout.gzipFile) {
          if (!fgetsGZipSkipCommentsResize(SDDS_dataset, &bigBuffer, &bigBufferSize, gzfp, '!') || SDDS_StringIsBlank(bigBuffer)) {
            SDDS_SetError("Unable to read array--dimensions missing (SDDS_ReadAsciiArrays)");
            return (0);
          }
        } else {
#endif
          if (SDDS_dataset->layout.lzmaFile) {
            if (!fgetsLZMASkipCommentsResize(SDDS_dataset, &bigBuffer, &bigBufferSize, lzmafp, '!') || SDDS_StringIsBlank(bigBuffer)) {
              SDDS_SetError("Unable to read array--dimensions missing (SDDS_ReadAsciiArrays)");
              return (0);
            }
          } else {
            if (!fgetsSkipCommentsResize(SDDS_dataset, &bigBuffer, &bigBufferSize, fp, '!') || SDDS_StringIsBlank(bigBuffer)) {
              SDDS_SetError("Unable to read array--dimensions missing (SDDS_ReadAsciiArrays)");
              return (0);
            }
          }
#if defined(zLib)
        }
#endif
      }
      /* copy the bigBuffer because SDDS_ScanData2 will change the bigBufferCopy pointer that SDDS_ScanData
         cannot do becuase bigBuffer is a static variable. This change was implemented to greatly improve
         the speed of reading a very large line of array elements. The previous version just did a lot of
         strcpy commands to these huge lines which was really slow. */
      bigBufferCopy = bigBuffer;
      bigBufferCopySize = strlen(bigBufferCopy);
      do {
        if (!SDDS_ScanData2(bigBufferCopy, &bigBufferCopy, &bigBufferCopySize, array->definition->type, array->definition->field_length, array->data, j, 0)) {
          SDDS_SetError("Unable to read array--error scanning data element (SDDS_ReadAsciiArrays)");
          return (0);
        }
      } while (++j < array->elements && !SDDS_StringIsBlank(bigBufferCopy));
      bigBuffer[0] = 0;
    } while (j < array->elements);
  }
  if (buffer)
    free(buffer);
  if (bigBuffer)
    free(bigBuffer);
  return (1);
}

SDDS_ReadAsciiPageDetailed()

int32_t SDDS_ReadAsciiPageDetailed ( SDDS_DATASET * SDDS_dataset, int64_t sparse_interval, int64_t sparse_offset, int64_t last_rows, int32_t sparse_statistics )

extern

Reads a detailed page of data from an ASCII file into an SDDS dataset with optional sparsity and statistics.

This function reads a page of data from an ASCII SDDS file into the provided dataset, supporting various options for data sparsity and statistical processing. It allows specifying a sparse interval and offset to read every nth row starting from a specific offset, as well as reading only the last specified number of rows. Additionally, it can compute statistical measures over the sparse data, such as average, median, minimum, or maximum.

Parameters

SDDS_dataset	Pointer to the SDDS dataset where the data will be stored.
sparse_interval	Interval for sparsity; reads every nth row if greater than 1. Use 1 to read every row.
sparse_offset	Offset from the first row to start reading. Rows before this offset are skipped.
last_rows	Number of last rows to read. If greater than 0, only the last last_rows rows are read, overriding sparse_interval and sparse_offset. Use 0 to read all rows according to sparse_interval and sparse_offset.
sparse_statistics	Statistical operation to perform over the sparse data: 0: None (no statistical operation) 1: Average 2: Median 3: Minimum 4: Maximum

Returns

Returns the page number on success, -1 if end-of-file is reached before any data is read, 0 on error.

Note

This function handles reading from regular files, as well as GZIP and LZMA compressed files if enabled. It updates the internal state of the dataset, including the page number and number of rows read. If last_rows is specified and greater than 0, it overrides sparse_interval and sparse_offset. In case of errors during reading, the function attempts to recover if autoRecover is enabled in the dataset.

Definition at line 1273 of file SDDS_ascii.c.

                                                                                                                                                             {
  SDDS_LAYOUT *layout;
  int32_t no_row_counts, end_of_data, retval, lineCount;
  int64_t n_rows, i, j, k, rows_to_store;
  /*  int32_t page_number; */
  char s[SDDS_MAXLINE];
  char *dataRead, *bigBufferCopy;
  int32_t bigBufferCopySize;
  char *bigBuffer = NULL;
  int32_t bigBufferSize = 0;
  /*  char *ptr; */
#if defined(zLib)
  gzFile gzfp = NULL;
#endif
  FILE *fp = NULL;
  void **statData=NULL;
  double statResult;
  struct lzmafile *lzmafp = NULL;
  /*  double value; */
 
  if (SDDS_dataset->autoRecovered)
    return -1;
 
  SDDS_SetReadRecoveryMode(SDDS_dataset, 0);
 
#if defined(zLib)
  if (SDDS_dataset->layout.gzipFile) {
    gzfp = SDDS_dataset->layout.gzfp;
  } else {
#endif
    if (SDDS_dataset->layout.lzmaFile) {
      lzmafp = SDDS_dataset->layout.lzmafp;
    } else {
      fp = SDDS_dataset->layout.fp;
    }
#if defined(zLib)
  }
#endif
  if (SDDS_dataset->page_number == -1)
    /* end of file already hit */
    return -1;
#if defined(zLib)
  if (SDDS_dataset->layout.gzipFile) {
    if (gzeof(gzfp) && SDDS_dataset->page_number > 0)
      /* end of file and not first page about to be read */
      return SDDS_dataset->page_number = -1;
  } else {
#endif
    if (SDDS_dataset->layout.lzmaFile) {
      if (lzma_eof(lzmafp) && SDDS_dataset->page_number > 0)
        /* end of file and not first page about to be read */
        return SDDS_dataset->page_number = -1;
    } else {
      if (feof(fp) && SDDS_dataset->page_number > 0)
        /* end of file and not first page about to be read */
        return SDDS_dataset->page_number = -1;
    }
#if defined(zLib)
  }
#endif
  if (!SDDS_AsciiDataExpected(SDDS_dataset) && SDDS_dataset->page_number != 0)
    /* if no columns or arrays and only fixed value parameters, then only one "page" allowed */
    return (SDDS_dataset->page_number = -1);
 
#if defined(zLib)
  if (SDDS_dataset->layout.gzipFile) {
    if (SDDS_dataset->page_number == 0)
      for (i = 0; i < SDDS_dataset->layout.data_mode.additional_header_lines; i++) {
        if (!fgetsGZipSkipComments(SDDS_dataset, s, SDDS_MAXLINE, gzfp, '!'))
          return (SDDS_dataset->page_number = -1); /* indicates end of data */
      }
  } else {
#endif
    if (SDDS_dataset->layout.lzmaFile) {
      if (SDDS_dataset->page_number == 0)
        for (i = 0; i < SDDS_dataset->layout.data_mode.additional_header_lines; i++) {
          if (!fgetsLZMASkipComments(SDDS_dataset, s, SDDS_MAXLINE, lzmafp, '!'))
            return (SDDS_dataset->page_number = -1); /* indicates end of data */
        }
    } else {
      if (SDDS_dataset->page_number == 0)
        for (i = 0; i < SDDS_dataset->layout.data_mode.additional_header_lines; i++) {
          if (!fgetsSkipComments(SDDS_dataset, s, SDDS_MAXLINE, fp, '!'))
            return (SDDS_dataset->page_number = -1); /* indicates end of data */
        }
    }
#if defined(zLib)
  }
#endif
 
  /* the next call increments the page number */
  if (!SDDS_StartPage(SDDS_dataset, 0)) {
    SDDS_SetError("Unable to read page--couldn't start page (SDDS_ReadAsciiPage)");
    return (0);
  }
 
  layout = &SDDS_dataset->layout;
 
  if ((retval = SDDS_ReadAsciiParameters(SDDS_dataset)) < 1) {
    if (retval)
      return (SDDS_dataset->page_number = retval);
    SDDS_SetError("Unable to read page--couldn't read parameters (SDDS_ReadAsciiPage)");
    return (0);
  }
  if ((retval = SDDS_ReadAsciiArrays(SDDS_dataset)) < 1) {
    if (retval)
      return (SDDS_dataset->page_number = retval);
    SDDS_SetError("Unable to read page--couldn't read arrays (SDDS_ReadAsciiPage)");
    return (0);
  }
 
  if (last_rows < 0)
    last_rows = 0;
  if (sparse_interval <= 0)
    sparse_interval = 1;
  if (sparse_offset < 0)
    sparse_offset = 0;
 
  SDDS_dataset->rowcount_offset = -1;
  if (layout->n_columns) {
    if (!(bigBuffer = SDDS_Malloc(sizeof(*bigBuffer) * (bigBufferSize = INITIAL_BIG_BUFFER_SIZE)))) {
      SDDS_SetError("Unable to read parameters--buffer allocation failure (SDDS_ReadAsciiPage)");
      return (0);
    }
 
    n_rows = 0;
    no_row_counts = 0;
    if (!SDDS_dataset->layout.data_mode.no_row_counts) {
      /* read the number of rows */
#if defined(zLib)
      if (SDDS_dataset->layout.gzipFile) {
        if (!fgetsGZipSkipComments(SDDS_dataset, s, SDDS_MAXLINE, gzfp, '!')) {
          if (bigBuffer)
            free(bigBuffer);
          return (SDDS_dataset->page_number = -1); /* indicates end of data */
        }
      } else {
#endif
        if (SDDS_dataset->layout.lzmaFile) {
          if (!fgetsLZMASkipComments(SDDS_dataset, s, SDDS_MAXLINE, lzmafp, '!')) {
            if (bigBuffer)
              free(bigBuffer);
            return (SDDS_dataset->page_number = -1); /* indicates end of data */
          }
        } else {
          do {
            SDDS_dataset->rowcount_offset = ftell(fp);
            if (!fgets(s, SDDS_MAXLINE, fp)) {
              if (bigBuffer)
                free(bigBuffer);
              return (SDDS_dataset->page_number = -1); /* indicates end of data */
            }
          } while (s[0] == '!');
        }
#if defined(zLib)
      }
#endif
      if (sscanf(s, "%" SCNd64, &n_rows) != 1 || n_rows < 0) {
        SDDS_SetError("Unable to read page--file has no (valid) number-of-rows entry (SDDS_ReadAsciiPage)");
        return (0);
      }
      if (n_rows > SDDS_GetRowLimit()) {
        /* number of rows is "unreasonably" large---treat like end-of-file */
        if (bigBuffer)
          free(bigBuffer);
        return (SDDS_dataset->page_number = -1);
      }
      if (last_rows) {
        sparse_interval = 1;
        sparse_offset = n_rows - last_rows;
        if (sparse_offset < 0)
          sparse_offset = 0;
      }
      rows_to_store = (n_rows - sparse_offset) / sparse_interval + 2;
    } else {
      // fix last_rows when no row count is available
      no_row_counts = 1;
      n_rows = TABLE_LENGTH_INCREMENT;
      rows_to_store = n_rows;
    }
 
    if (rows_to_store >= SDDS_dataset->n_rows_allocated) {
      /* lengthen the page */
      if (!SDDS_LengthenTable(SDDS_dataset, rows_to_store - SDDS_dataset->n_rows_allocated)) {
        SDDS_SetError("Unable to read page--couldn't lengthen data page (SDDS_ReadAsciiPage)");
        return (0);
      }
    }
 
    /* read the page values */
    j = end_of_data = k = 0;
    s[0] = 0;
    if (!no_row_counts && n_rows == 0) {
      SDDS_dataset->n_rows = 0;
      if (bigBuffer)
        free(bigBuffer);
      return (SDDS_dataset->page_number);
    }
    bigBuffer[0] = 0;
    bigBufferCopy = bigBuffer;
    do {
      if (j >= SDDS_dataset->n_rows_allocated) {
        /* lengthen the page */
        if (!SDDS_LengthenTable(SDDS_dataset, TABLE_LENGTH_INCREMENT)) {
          SDDS_SetError("Unable to read page--couldn't lengthen data page (SDDS_ReadAsciiPage)");
          return (0);
        }
      }
      lineCount = 0;
      dataRead = NULL;
      for (i = 0; i < SDDS_dataset->layout.n_columns; i++) {
        if (k == 0) {
          if (sparse_statistics != 0) {
            // Allocate buffer space for statistical sparsing
            if (i == 0) {
              statData = (void**)malloc(SDDS_dataset->layout.n_columns * sizeof(void*));
            }
            if (SDDS_FLOATING_TYPE(layout->column_definition[i].type)) {
              // Not ideal for SDDS_LONGDOUBLE but we may never run across this error
              statData[i] = (double*)calloc(sparse_interval, sizeof(double));
            }
          }
        }
        
        if (layout->column_definition[i].definition_mode & SDDS_WRITEONLY_DEFINITION)
          continue;
        if (SDDS_StringIsBlank(bigBufferCopy)) {
          bigBuffer[0] = 0;
          bigBufferCopy = bigBuffer;
          dataRead = NULL;
#if defined(zLib)
          if (SDDS_dataset->layout.gzipFile) {
            if (!(dataRead = fgetsGZipSkipCommentsResize(SDDS_dataset, &bigBuffer, &bigBufferSize, gzfp, '!')) || SDDS_StringIsBlank(bigBuffer)) {
              SDDS_dataset->n_rows = j;
              if (no_row_counts) {
                /* legitmate end of data set */
                end_of_data = 1;
                break;
              }
              /* error, but may be recoverable */
              gzseek(gzfp, 0L, SEEK_END);
              if (SDDS_dataset->autoRecover) {
                SDDS_dataset->autoRecovered = 1;
                SDDS_ClearErrors();
                if (bigBuffer)
                  free(bigBuffer);
                return (SDDS_dataset->page_number);
              }
              SDDS_SetError("Unable to read page (SDDS_ReadAsciiPage)");
              SDDS_SetReadRecoveryMode(SDDS_dataset, 1);
              return (0);
            }
          } else {
#endif
            if (SDDS_dataset->layout.lzmaFile) {
              if (!(dataRead = fgetsLZMASkipCommentsResize(SDDS_dataset, &bigBuffer, &bigBufferSize, lzmafp, '!')) || SDDS_StringIsBlank(bigBuffer)) {
                SDDS_dataset->n_rows = j;
                if (no_row_counts) {
                  /* legitmate end of data set */
                  end_of_data = 1;
                  break;
                }
                /* error, but may be recoverable */
                lzma_seek(lzmafp, 0L, SEEK_END);
                if (SDDS_dataset->autoRecover) {
                  SDDS_dataset->autoRecovered = 1;
                  SDDS_ClearErrors();
                  if (bigBuffer)
                    free(bigBuffer);
                  return (SDDS_dataset->page_number);
                }
                SDDS_SetError("Unable to read page (SDDS_ReadAsciiPage)");
                SDDS_SetReadRecoveryMode(SDDS_dataset, 1);
                return (0);
              }
            } else {
              if (!(dataRead = fgetsSkipCommentsResize(SDDS_dataset, &bigBuffer, &bigBufferSize, fp, '!')) || SDDS_StringIsBlank(bigBuffer)) {
                SDDS_dataset->n_rows = j;
                if (no_row_counts) {
                  /* legitmate end of data set */
                  end_of_data = 1;
                  break;
                }
                /* error, but may be recoverable */
                fseek(fp, 0L, SEEK_END);
                if (SDDS_dataset->autoRecover) {
                  SDDS_dataset->autoRecovered = 1;
                  SDDS_ClearErrors();
                  if (bigBuffer)
                    free(bigBuffer);
                  return (SDDS_dataset->page_number);
                }
                SDDS_SetError("Unable to read page (SDDS_ReadAsciiPage)");
                SDDS_SetReadRecoveryMode(SDDS_dataset, 1);
                return (0);
              }
            }
#if defined(zLib)
          }
#endif
          lineCount++;
          bigBufferCopy = bigBuffer;
          bigBufferCopySize = strlen(bigBufferCopy);
        }
        if (!SDDS_ScanData2(bigBufferCopy, &bigBufferCopy, &bigBufferCopySize, layout->column_definition[i].type, layout->column_definition[i].field_length, SDDS_dataset->data[i], j, 0)) {
          /* error, but may be recoverable */
          SDDS_dataset->n_rows = j;
#if defined(zLib)
          if (SDDS_dataset->layout.gzipFile) {
            gzseek(gzfp, 0L, SEEK_END);
          } else {
#endif
            if (SDDS_dataset->layout.lzmaFile) {
              lzma_seek(lzmafp, 0L, SEEK_END);
            } else {
              fseek(fp, 0L, SEEK_END);
            }
#if defined(zLib)
          }
#endif
          if (SDDS_dataset->autoRecover) {
            SDDS_dataset->autoRecovered = 1;
            SDDS_ClearErrors();
            if (bigBuffer)
              free(bigBuffer);
            return (SDDS_dataset->page_number);
          }
          SDDS_SetReadRecoveryMode(SDDS_dataset, 1);
          SDDS_SetError("Unable to read page--scanning error (SDDS_ReadAsciiPage)");
          return (0);
        }
        if (sparse_statistics != 0) {
          switch (layout->column_definition[i].type) {
          case SDDS_FLOAT:
            ((double*)statData[i])[k % sparse_interval] = (double)(((float*)SDDS_dataset->data[i])[j]);
            break;
          case SDDS_DOUBLE:
            ((double*)statData[i])[k % sparse_interval] = ((double*)SDDS_dataset->data[i])[j];
            break;
          case SDDS_LONGDOUBLE:
            ((double*)statData[i])[k % sparse_interval] = (double)(((long double*)SDDS_dataset->data[i])[j]);
            break;
          }
          if (SDDS_FLOATING_TYPE(layout->column_definition[i].type)) {
            if (sparse_statistics == 1) {
              // Sparse and get average statistics
              compute_average(&statResult, (double*)statData[i], (k % sparse_interval) + 1);
            } else if (sparse_statistics == 2) {
              // Sparse and get median statistics
              compute_median(&statResult, (double*)statData[i], (k % sparse_interval) + 1);
            } else if (sparse_statistics == 3) {
              // Sparse and get minimum statistics
              statResult = min_in_array((double*)statData[i], (k % sparse_interval) + 1);
            } else if (sparse_statistics == 4) {
              // Sparse and get maximum statistics
              statResult = max_in_array((double*)statData[i], (k % sparse_interval) + 1);
            }
          }
          switch (layout->column_definition[i].type) {
          case SDDS_FLOAT:
            ((float*)SDDS_dataset->data[i])[j] = statResult;
            break;
          case SDDS_DOUBLE:
            ((double*)SDDS_dataset->data[i])[j] = statResult;
            break;
          case SDDS_LONGDOUBLE:
            ((long double*)SDDS_dataset->data[i])[j] = statResult;
            break;
          }
        }
 
#if defined(DEBUG)
        fprintf(stderr, "line remaining = %s\n", bigBuffer);
#endif
      }
      if (end_of_data)
        /* ran out of data for no_row_counts=1 */
        break;
      if (layout->data_mode.lines_per_row != 0 && lineCount != layout->data_mode.lines_per_row) {
        sprintf(s, "Unable to read page--line layout error at line %" PRId64 " of page %" PRId32 " (SDDS_ReadAsciiPage)", j + 1, SDDS_dataset->page_number);
        SDDS_SetError(s);
        /* data ends prematurely, which is an error that may be recoverable */
#if defined(zLib)
        if (SDDS_dataset->layout.gzipFile) {
          gzseek(gzfp, 0L, SEEK_END);
        } else {
#endif
          if (SDDS_dataset->layout.lzmaFile) {
            lzma_seek(lzmafp, 0L, SEEK_END);
          } else {
            fseek(fp, 0L, SEEK_END);
          }
#if defined(zLib)
        }
#endif
        if (SDDS_dataset->autoRecover) {
          SDDS_dataset->autoRecovered = 1;
          SDDS_ClearErrors();
          if (bigBuffer)
            free(bigBuffer);
          return (SDDS_dataset->page_number);
        }
        SDDS_SetReadRecoveryMode(SDDS_dataset, 1);
        SDDS_dataset->n_rows = j;
        return (0);
      }
      if (layout->data_mode.lines_per_row != 0) {
        bigBuffer[0] = 0;
        bigBufferCopy = bigBuffer;
      }
      if (--sparse_offset < 0 && 
          (((sparse_statistics == 0) && (k % sparse_interval == 0)) || ((sparse_statistics != 0) && (k % sparse_interval == sparse_interval - 1))))
        j++;
      k++;
    } while (k < n_rows || no_row_counts);
 
    if (sparse_statistics != 0) {
      for (i = 0; i < SDDS_dataset->layout.n_columns; i++) {
        if (SDDS_FLOATING_TYPE(layout->column_definition[i].type)) {
          free(statData[i]);
        }
      }
      free(statData);
    }
 
    if (end_of_data && !(SDDS_dataset->page_number == 1) && j == 0 && !dataRead) {
      /* For end of data in no_row_counts=1 mode for any page other than the first,
       * an end-of-file is not a valid way to end an empty page (only an incomplete line is)
       */
      if (bigBuffer)
        free(bigBuffer);
      return SDDS_dataset->page_number = -1;
    }
    SDDS_dataset->n_rows = j;
  }
  if (bigBuffer)
    free(bigBuffer);
 
  return (SDDS_dataset->page_number);
}

SDDS_ReadAsciiPageLastRows()

int32_t SDDS_ReadAsciiPageLastRows ( SDDS_DATASET * SDDS_dataset, int64_t last_rows )

extern

Reads the last specified number of rows from an ASCII page of an SDDS dataset.

This function reads only the last specified number of rows from the next ASCII page in the SDDS dataset.

Parameters

SDDS_dataset	Pointer to the SDDS dataset where the data will be stored.
last_rows	Number of rows to read from the end of the page.

Returns

Returns the page number on success, -1 if end-of-file is reached, or 0 on error.

Note

The function utilizes SDDS_ReadAsciiPageDetailed() to perform the actual reading.

Definition at line 1241 of file SDDS_ascii.c.

                                                                                  {
  return SDDS_ReadAsciiPageDetailed(SDDS_dataset, 1, 0, last_rows, 0);
}

SDDS_ReadAsciiParameters()

int32_t SDDS_ReadAsciiParameters ( SDDS_DATASET * SDDS_dataset )

extern

Reads the parameters from an ASCII file into the SDDS dataset.

This function reads parameter data from an ASCII file and stores it in the provided SDDS dataset. It supports reading from regular files, as well as GZIP and LZMA compressed files if enabled.

Parameters

SDDS_dataset

Pointer to the SDDS dataset where the parameters will be stored.

Returns

Returns 1 on success, 0 on error, or -1 if end-of-file is reached before any parameters are read.

Note

The function checks the dataset for consistency before reading. It handles fixed-value parameters appropriately. The function reads each parameter line by line, skipping comments, and stores the values after scanning them according to their data type.

Definition at line 927 of file SDDS_ascii.c.

                                                             {
  int32_t i, first_read;
  SDDS_LAYOUT *layout;
#if defined(zLib)
  gzFile gzfp;
#endif
  FILE *fp;
  struct lzmafile *lzmafp;
  char *bigBuffer = NULL;
  int32_t bigBufferSize = 0;
 
  layout = &SDDS_dataset->layout;
  first_read = 1;
  if (layout->n_parameters > 0) {
    if (!(bigBuffer = SDDS_Malloc(sizeof(*bigBuffer) * (bigBufferSize = INITIAL_BIG_BUFFER_SIZE)))) {
      SDDS_SetError("Unable to read parameters--buffer allocation failure (SDDS_ReadAsciiParameters)");
      return (0);
    }
  }
#if defined(zLib)
  if (SDDS_dataset->layout.gzipFile) {
    gzfp = layout->gzfp;
    for (i = 0; i < layout->n_parameters; i++) {
      if (layout->parameter_definition[i].definition_mode & SDDS_WRITEONLY_DEFINITION)
        continue;
      bigBuffer[0] = 0;
      if (!layout->parameter_definition[i].fixed_value) {
        if (!fgetsGZipSkipCommentsResize(SDDS_dataset, &bigBuffer, &bigBufferSize, gzfp, '!')) {
          if (first_read) {
            if (bigBuffer)
              free(bigBuffer);
            return (-1);
          }
          SDDS_SetError("Unable to read parameters--data ends prematurely (SDDS_ReadAsciiParameters)");
          return (0);
        }
        first_read = 0;
        bigBuffer[strlen(bigBuffer) - 1] = 0;
      } else
        strcpy(bigBuffer, layout->parameter_definition[i].fixed_value);
      if (!SDDS_ScanData(bigBuffer, layout->parameter_definition[i].type, 0, SDDS_dataset->parameter[i], 0, 1)) {
        SDDS_SetError("Unable to read page--parameter scanning error (SDDS_ReadAsciiParameters)");
        return (0);
      }
    }
  } else {
#endif
    if (SDDS_dataset->layout.lzmaFile) {
      lzmafp = layout->lzmafp;
      for (i = 0; i < layout->n_parameters; i++) {
        if (layout->parameter_definition[i].definition_mode & SDDS_WRITEONLY_DEFINITION)
          continue;
        bigBuffer[0] = 0;
        if (!layout->parameter_definition[i].fixed_value) {
          if (!fgetsLZMASkipCommentsResize(SDDS_dataset, &bigBuffer, &bigBufferSize, lzmafp, '!')) {
            if (first_read) {
              if (bigBuffer)
                free(bigBuffer);
              return (-1);
            }
            SDDS_SetError("Unable to read parameters--data ends prematurely (SDDS_ReadAsciiParameters)");
            return (0);
          }
          first_read = 0;
          bigBuffer[strlen(bigBuffer) - 1] = 0;
        } else
          strcpy(bigBuffer, layout->parameter_definition[i].fixed_value);
        if (!SDDS_ScanData(bigBuffer, layout->parameter_definition[i].type, 0, SDDS_dataset->parameter[i], 0, 1)) {
          SDDS_SetError("Unable to read page--parameter scanning error (SDDS_ReadAsciiParameters)");
          return (0);
        }
      }
    } else {
      fp = layout->fp;
      for (i = 0; i < layout->n_parameters; i++) {
        if (layout->parameter_definition[i].definition_mode & SDDS_WRITEONLY_DEFINITION)
          continue;
        bigBuffer[0] = 0;
        if (!layout->parameter_definition[i].fixed_value) {
          if (!fgetsSkipCommentsResize(SDDS_dataset, &bigBuffer, &bigBufferSize, fp, '!')) {
            if (first_read) {
              if (bigBuffer)
                free(bigBuffer);
              return (-1);
            }
            SDDS_SetError("Unable to read parameters--data ends prematurely (SDDS_ReadAsciiParameters)");
            return (0);
          }
          first_read = 0;
          bigBuffer[strlen(bigBuffer) - 1] = 0;
        } else
          strcpy(bigBuffer, layout->parameter_definition[i].fixed_value);
        if (!SDDS_ScanData(bigBuffer, layout->parameter_definition[i].type, 0, SDDS_dataset->parameter[i], 0, 1)) {
          SDDS_SetError("Unable to read page--parameter scanning error (SDDS_ReadAsciiParameters)");
          return (0);
        }
      }
    }
#if defined(zLib)
  }
#endif
  if (bigBuffer)
    free(bigBuffer);
  return (1);
}

SDDS_ReadBinaryArrays()

int32_t SDDS_ReadBinaryArrays ( SDDS_DATASET * SDDS_dataset )

extern

Reads binary arrays from an SDDS dataset.

This function iterates through all array definitions within the specified SDDS dataset and reads their binary data from the underlying file. It handles various compression formats, including uncompressed, LZMA-compressed, and GZIP-compressed files. For each array, the function reads its definition, dimensions, and data elements, allocating and managing memory as necessary. String arrays are handled by reading each string individually, while other data types are read in bulk.

Parameters

[in,out]

SDDS_dataset

Pointer to the SDDS_DATASET structure representing the dataset to read from.

Returns

int32_t Returns 1 on successful reading of all arrays, or 0 if an error occurred.

Return values

1	All arrays were successfully read and stored.
0	An error occurred during the read operation, such as I/O failures, memory allocation issues, or corrupted array definitions.

Note

The caller is responsible for ensuring that the SDDS_dataset structure is properly initialized and that memory allocations for arrays are managed appropriately to prevent memory leaks.

Definition at line 3184 of file SDDS_binary.c.

                                                          {
  int32_t i, j;
  SDDS_LAYOUT *layout;
  /*  char *predefined_format; */
  /*  static char buffer[SDDS_MAXLINE]; */
#if defined(zLib)
  gzFile gzfp = NULL;
#endif
  FILE *fp = NULL;
  struct lzmafile *lzmafp = NULL;
  SDDS_ARRAY *array;
  SDDS_FILEBUFFER *fBuffer;
 
  if (!SDDS_CheckDataset(SDDS_dataset, "SDDS_ReadBinaryArrays"))
    return (0);
  layout = &SDDS_dataset->layout;
  if (!layout->n_arrays)
    return (1);
#if defined(zLib)
  if (SDDS_dataset->layout.gzipFile) {
    gzfp = layout->gzfp;
  } else {
#endif
    if (SDDS_dataset->layout.lzmaFile) {
      lzmafp = layout->lzmafp;
    } else {
      fp = layout->fp;
    }
#if defined(zLib)
  }
#endif
  fBuffer = &SDDS_dataset->fBuffer;
  if (!SDDS_dataset->array) {
    SDDS_SetError("Unable to read array--pointer to structure storage area is NULL (SDDS_ReadBinaryArrays)");
    return (0);
  }
  for (i = 0; i < layout->n_arrays; i++) {
    array = SDDS_dataset->array + i;
    if (array->definition && !SDDS_FreeArrayDefinition(array->definition)) {
      SDDS_SetError("Unable to get array--array definition corrupted (SDDS_ReadBinaryArrays)");
      return (0);
    }
    if (!SDDS_CopyArrayDefinition(&array->definition, layout->array_definition + i)) {
      SDDS_SetError("Unable to read array--definition copy failed (SDDS_ReadBinaryArrays)");
      return (0);
    }
    /*if (array->dimension) free(array->dimension); */
    if (!(array->dimension = SDDS_Realloc(array->dimension, sizeof(*array->dimension) * array->definition->dimensions))) {
      SDDS_SetError("Unable to read array--allocation failure (SDDS_ReadBinaryArrays)");
      return (0);
    }
#if defined(zLib)
    if (SDDS_dataset->layout.gzipFile) {
      if (!SDDS_GZipBufferedRead(array->dimension, sizeof(*array->dimension) * array->definition->dimensions, gzfp, fBuffer, SDDS_LONG, SDDS_dataset->layout.byteOrderDeclared)) {
        SDDS_SetError("Unable to read arrays--failure reading dimensions (SDDS_ReadBinaryArrays)");
        return (0);
      }
    } else {
#endif
      if (SDDS_dataset->layout.lzmaFile) {
        if (!SDDS_LZMABufferedRead(array->dimension, sizeof(*array->dimension) * array->definition->dimensions, lzmafp, fBuffer, SDDS_LONG, SDDS_dataset->layout.byteOrderDeclared)) {
          SDDS_SetError("Unable to read arrays--failure reading dimensions (SDDS_ReadBinaryArrays)");
          return (0);
        }
      } else {
        if (!SDDS_BufferedRead(array->dimension, sizeof(*array->dimension) * array->definition->dimensions, fp, fBuffer, SDDS_LONG, SDDS_dataset->layout.byteOrderDeclared)) {
          SDDS_SetError("Unable to read arrays--failure reading dimensions (SDDS_ReadBinaryArrays)");
          return (0);
        }
      }
#if defined(zLib)
    }
#endif
    array->elements = 1;
    for (j = 0; j < array->definition->dimensions; j++)
      array->elements *= array->dimension[j];
    if (array->data)
      free(array->data);
    array->data = array->pointer = NULL;
    if (array->elements == 0)
      continue;
    if (array->elements < 0) {
      SDDS_SetError("Unable to read array--number of elements is negative (SDDS_ReadBinaryArrays)");
      return (0);
    }
    if (!(array->data = SDDS_Realloc(array->data, array->elements * SDDS_type_size[array->definition->type - 1]))) {
      SDDS_SetError("Unable to read array--allocation failure (SDDS_ReadBinaryArrays)");
      return (0);
    }
    if (array->definition->type == SDDS_STRING) {
#if defined(zLib)
      if (SDDS_dataset->layout.gzipFile) {
        for (j = 0; j < array->elements; j++) {
          if (!(((char **)(array->data))[j] = SDDS_ReadGZipBinaryString(gzfp, fBuffer, 0))) {
            SDDS_SetError("Unable to read arrays--failure reading string (SDDS_ReadBinaryArrays)");
            return (0);
          }
        }
      } else {
#endif
        if (SDDS_dataset->layout.lzmaFile) {
          for (j = 0; j < array->elements; j++) {
            if (!(((char **)(array->data))[j] = SDDS_ReadLZMABinaryString(lzmafp, fBuffer, 0))) {
              SDDS_SetError("Unable to read arrays--failure reading string (SDDS_ReadBinaryArrays)");
              return (0);
            }
          }
        } else {
          for (j = 0; j < array->elements; j++) {
            if (!(((char **)(array->data))[j] = SDDS_ReadBinaryString(fp, fBuffer, 0))) {
              SDDS_SetError("Unable to read arrays--failure reading string (SDDS_ReadBinaryArrays)");
              return (0);
            }
          }
        }
#if defined(zLib)
      }
#endif
    } else {
#if defined(zLib)
      if (SDDS_dataset->layout.gzipFile) {
        if (!SDDS_GZipBufferedRead(array->data, SDDS_type_size[array->definition->type - 1] * array->elements, gzfp, fBuffer, array->definition->type, SDDS_dataset->layout.byteOrderDeclared)) {
          SDDS_SetError("Unable to read arrays--failure reading values (SDDS_ReadBinaryArrays)");
          return (0);
        }
      } else {
#endif
        if (SDDS_dataset->layout.lzmaFile) {
          if (!SDDS_LZMABufferedRead(array->data, SDDS_type_size[array->definition->type - 1] * array->elements, lzmafp, fBuffer, array->definition->type, SDDS_dataset->layout.byteOrderDeclared)) {
            SDDS_SetError("Unable to read arrays--failure reading values (SDDS_ReadBinaryArrays)");
            return (0);
          }
        } else {
          if (!SDDS_BufferedRead(array->data, SDDS_type_size[array->definition->type - 1] * array->elements, fp, fBuffer, array->definition->type, SDDS_dataset->layout.byteOrderDeclared)) {
            SDDS_SetError("Unable to read arrays--failure reading values (SDDS_ReadBinaryArrays)");
            return (0);
          }
        }
#if defined(zLib)
      }
#endif
    }
  }
  return (1);
}

SDDS_ReadBinaryColumns()

int32_t SDDS_ReadBinaryColumns ( SDDS_DATASET * SDDS_dataset, int64_t sparse_interval, int64_t sparse_offset )

extern

Reads the binary columns from an SDDS dataset.

This function iterates through all column definitions within the specified SDDS dataset and reads their binary data from the underlying file. It handles various compression formats, including uncompressed, LZMA-compressed, and GZIP-compressed files. For each column, the function reads data for each row, managing memory allocation for string columns as necessary. Non-string data types are read in bulk for each column.

Parameters

[in,out]

SDDS_dataset

Pointer to the SDDS_DATASET structure representing the dataset to read from.

Returns

int32_t Returns 1 on successful reading of all columns, or 0 if an error occurred.

Return values

1	All columns were successfully read and stored.
0	An error occurred during the read operation, such as I/O failures, memory allocation issues, or corrupted column definitions.

Note

The caller is responsible for ensuring that the SDDS_dataset structure is properly initialized and that memory allocations for columns are managed appropriately to prevent memory leaks.

Definition at line 3349 of file SDDS_binary.c.

                                                                                                           {
  int64_t i, j, k, row;
  SDDS_LAYOUT *layout;
  /*  char *predefined_format; */
  /*  static char buffer[SDDS_MAXLINE]; */
#if defined(zLib)
  gzFile gzfp = NULL;
#endif
  FILE *fp = NULL;
  struct lzmafile *lzmafp = NULL;
  SDDS_FILEBUFFER *fBuffer;
 
  if (!SDDS_CheckDataset(SDDS_dataset, "SDDS_ReadBinaryColumns"))
    return (0);
  layout = &SDDS_dataset->layout;
  if (!layout->n_columns || !SDDS_dataset->n_rows)
    return (1);
#if defined(zLib)
  if (SDDS_dataset->layout.gzipFile) {
    gzfp = layout->gzfp;
  } else {
#endif
    if (SDDS_dataset->layout.lzmaFile) {
      lzmafp = layout->lzmafp;
    } else {
      fp = layout->fp;
    }
#if defined(zLib)
  }
#endif
  fBuffer = &SDDS_dataset->fBuffer;
 
  for (i = 0; i < layout->n_columns; i++) {
    if (layout->column_definition[i].definition_mode & SDDS_WRITEONLY_DEFINITION)
      continue;
    if (layout->column_definition[i].type == SDDS_STRING) {
#if defined(zLib)
      if (SDDS_dataset->layout.gzipFile) {
        for (row = 0; row < SDDS_dataset->n_rows; row++) {
          if (((char ***)SDDS_dataset->data)[i][row])
            free((((char ***)SDDS_dataset->data)[i][row]));
          if (!(((char ***)SDDS_dataset->data)[i][row] = SDDS_ReadGZipBinaryString(gzfp, fBuffer, 0))) {
            SDDS_SetError("Unable to read columns--failure reading string (SDDS_ReadBinaryColumns)");
            return (0);
          }
        }
      } else {
#endif
        if (SDDS_dataset->layout.lzmaFile) {
          for (row = 0; row < SDDS_dataset->n_rows; row++) {
            if (((char ***)SDDS_dataset->data)[i][row])
              free((((char ***)SDDS_dataset->data)[i][row]));
            if (!(((char ***)SDDS_dataset->data)[i][row] = SDDS_ReadLZMABinaryString(lzmafp, fBuffer, 0))) {
              SDDS_SetError("Unable to read columns--failure reading string (SDDS_ReadBinaryColumms)");
              return (0);
            }
          }
        } else {
          for (row = 0; row < SDDS_dataset->n_rows; row++) {
            if (((char ***)SDDS_dataset->data)[i][row])
              free((((char ***)SDDS_dataset->data)[i][row]));
            if (!(((char ***)SDDS_dataset->data)[i][row] = SDDS_ReadBinaryString(fp, fBuffer, 0))) {
              SDDS_SetError("Unable to read columns--failure reading string (SDDS_ReadBinaryColumms)");
              return (0);
            }
          }
        }
#if defined(zLib)
      }
#endif
    } else {
#if defined(zLib)
      if (SDDS_dataset->layout.gzipFile) {
        if (!SDDS_GZipBufferedRead(SDDS_dataset->data[i], SDDS_type_size[layout->column_definition[i].type - 1] * SDDS_dataset->n_rows, gzfp, fBuffer, layout->column_definition[i].type, SDDS_dataset->layout.byteOrderDeclared)) {
          SDDS_SetError("Unable to read columns--failure reading values (SDDS_ReadBinaryColumns)");
          return (0);
        }
      } else {
#endif
        if (SDDS_dataset->layout.lzmaFile) {
          if (!SDDS_LZMABufferedRead(SDDS_dataset->data[i], SDDS_type_size[layout->column_definition[i].type - 1] * SDDS_dataset->n_rows, lzmafp, fBuffer, layout->column_definition[i].type, SDDS_dataset->layout.byteOrderDeclared)) {
            SDDS_SetError("Unable to read columns--failure reading values (SDDS_ReadBinaryColumns)");
            return (0);
          }
        } else {
          if (!SDDS_BufferedRead(SDDS_dataset->data[i], SDDS_type_size[layout->column_definition[i].type - 1] * SDDS_dataset->n_rows, fp, fBuffer, layout->column_definition[i].type, SDDS_dataset->layout.byteOrderDeclared)) {
            SDDS_SetError("Unable to read columns--failure reading values (SDDS_ReadBinaryColumns)");
            return (0);
          }
        }
#if defined(zLib)
      }
#endif
    }
  }
 
  if (sparse_interval == 1 && sparse_offset == 0) {
    return(1);
  }
 
  j = SDDS_dataset->n_rows;
  for (i = 0; i < layout->n_columns; i++) {
    j = k = 0;
    switch (layout->column_definition[i].type) {
    case SDDS_SHORT:
      for (row = sparse_offset; row < SDDS_dataset->n_rows; row++) {
        if (k % sparse_interval == 0) {
          ((short*)SDDS_dataset->data[i])[j] = ((short*)SDDS_dataset->data[i])[row];
          j++;
        }
        k++;
      }
      break; 
    case SDDS_USHORT:
      for (row = sparse_offset; row < SDDS_dataset->n_rows; row++) {
        if (k % sparse_interval == 0) {
          ((unsigned short*)SDDS_dataset->data[i])[j] = ((unsigned short*)SDDS_dataset->data[i])[row];
          j++;
        }
        k++;
      }
      break; 
    case SDDS_LONG:
      for (row = sparse_offset; row < SDDS_dataset->n_rows; row++) {
        if (k % sparse_interval == 0) {
          ((int32_t*)SDDS_dataset->data[i])[j] = ((int32_t*)SDDS_dataset->data[i])[row];
          j++;
        }
        k++;
      }
      break; 
    case SDDS_ULONG:
      for (row = sparse_offset; row < SDDS_dataset->n_rows; row++) {
        if (k % sparse_interval == 0) {
          ((uint32_t*)SDDS_dataset->data[i])[j] = ((uint32_t*)SDDS_dataset->data[i])[row];
          j++;
        }
        k++;
      }
      break; 
    case SDDS_LONG64:
      for (row = sparse_offset; row < SDDS_dataset->n_rows; row++) {
        if (k % sparse_interval == 0) {
          ((int64_t*)SDDS_dataset->data[i])[j] = ((int64_t*)SDDS_dataset->data[i])[row];
          j++;
        }
        k++;
      }
      break; 
    case SDDS_ULONG64:
      for (row = sparse_offset; row < SDDS_dataset->n_rows; row++) {
        if (k % sparse_interval == 0) {
          ((uint64_t*)SDDS_dataset->data[i])[j] = ((uint64_t*)SDDS_dataset->data[i])[row];
          j++;
        }
        k++;
      }
      break; 
    case SDDS_FLOAT:
      for (row = sparse_offset; row < SDDS_dataset->n_rows; row++) {
        if (k % sparse_interval == 0) {
          ((float*)SDDS_dataset->data[i])[j] = ((float*)SDDS_dataset->data[i])[row];
          j++;
        }
        k++;
      }
      break; 
    case SDDS_DOUBLE:
      for (row = sparse_offset; row < SDDS_dataset->n_rows; row++) {
        if (k % sparse_interval == 0) {
          ((double*)SDDS_dataset->data[i])[j] = ((double*)SDDS_dataset->data[i])[row];
          j++;
        }
        k++;
      }
      break; 
    case SDDS_LONGDOUBLE:
      for (row = sparse_offset; row < SDDS_dataset->n_rows; row++) {
        if (k % sparse_interval == 0) {
          ((long double*)SDDS_dataset->data[i])[j] = ((long double*)SDDS_dataset->data[i])[row];
          j++;
        }
        k++;
      }
      break; 
    case SDDS_STRING:
      for (row = sparse_offset; row < SDDS_dataset->n_rows; row++) {
        if (k % sparse_interval == 0) {
          ((char**)SDDS_dataset->data[i])[j] = ((char**)SDDS_dataset->data[i])[row];
          j++;
        }
        k++;
      }
      for (k=j; k<SDDS_dataset->n_rows; k++) {
        if (((char ***)SDDS_dataset->data)[i][k]) {
          free((((char ***)SDDS_dataset->data)[i][k]));
          ((char ***)SDDS_dataset->data)[i][k] = NULL;
        }
      }
 
      break; 
    case SDDS_CHARACTER:
      for (row = sparse_offset; row < SDDS_dataset->n_rows; row++) {
        if (k % sparse_interval == 0) {
          ((char*)SDDS_dataset->data[i])[j] = ((char*)SDDS_dataset->data[i])[row];
          j++;
        }
        k++;
      }
      break; 
    default:
      break;
    }
  }
    
  SDDS_dataset->n_rows = j;
 
  return (1);
}

SDDS_ReadBinaryPage()

int32_t SDDS_ReadBinaryPage ( SDDS_DATASET * SDDS_dataset, int64_t sparse_interval, int64_t sparse_offset, int32_t sparse_statistics )

extern

Reads a binary page from an SDDS dataset.

This function reads a binary page from the specified SDDS dataset. It allows for sparse reading by specifying the sparse_interval and sparse_offset parameters, enabling the reading of data at specified intervals or starting from a specific offset.

Parameters

SDDS_dataset	Pointer to the SDDS_DATASET structure representing the dataset to read from.
sparse_interval	Interval at which to read rows. A value greater than 1 enables sparse reading.
sparse_offset	Number of initial rows to skip before starting to read data.
sparse_statistics	Flag indicating whether to compute statistics during sparse reading: 0: No statistics. 1: Compute average. 2: Compute median. 3: Compute minimum. 4: Compute maximum.

Returns

• Returns the page number on successful read.
• Returns -1 if the end-of-file is reached.
• Returns 0 on error.

The function internally calls SDDS_ReadBinaryPageDetailed with the provided parameters to perform the actual reading. It handles various scenarios, including non-native byte orders and different data layouts (row-major or column-major).

Note

• This function is typically called to read data pages in bulk, allowing for efficient data access by skipping unnecessary rows.
• Sparse statistics can be used to reduce the amount of data by computing aggregated values.
• The function assumes that the dataset has been properly initialized and that the file pointers are correctly set up.

Definition at line 2141 of file SDDS_binary.c.

                                                                                                                                   {
  return SDDS_ReadBinaryPageDetailed(SDDS_dataset, sparse_interval, sparse_offset, 0, sparse_statistics);
}

SDDS_ReadBinaryPageDetailed()

int32_t SDDS_ReadBinaryPageDetailed ( SDDS_DATASET * SDDS_dataset, int64_t sparse_interval, int64_t sparse_offset, int64_t last_rows, int32_t sparse_statistics )

extern

Reads a binary page from an SDDS dataset with detailed options.

This function reads a binary page from the specified SDDS dataset, providing detailed control over the reading process. It supports sparse reading, reading a specific number of rows from the end, and computing statistics on the data.

Typically, this function is not called directly. Instead, it is invoked through higher-level functions such as SDDS_ReadBinaryPage or SDDS_ReadBinaryPageLastRows, which provide simplified interfaces for common reading scenarios.

Parameters

SDDS_dataset	Pointer to the SDDS_DATASET structure representing the dataset to read from.
sparse_interval	Interval at which to read rows. A value greater than 1 enables sparse reading.
sparse_offset	Number of initial rows to skip before starting to read data.
last_rows	The number of rows to read from the end of the dataset. If 0, all rows are read.
sparse_statistics	Flag indicating whether to compute statistics during sparse reading: 0: No statistics. 1: Compute average. 2: Compute median. 3: Compute minimum. 4: Compute maximum.

Returns

• Returns the page number on successful read.
• Returns -1 if the end-of-file is reached.
• Returns 0 on error.

The function performs the following steps:

• Checks if the dataset has been auto-recovered; if so, it returns -1.
• Determines if the dataset uses native or non-native byte order and delegates to SDDS_ReadNonNativePageDetailed if necessary.
• Initializes file pointers based on the compression format (standard, gzip, LZMA).
• Allocates and initializes the buffer for reading if not already allocated.
• Reads the number of rows from the binary file, handling both 32-bit and 64-bit row counts.
• Validates the row count and ensures it does not exceed predefined limits.
• Adjusts for column-major layouts by calling SDDS_ReadBinaryColumns if necessary.
• Handles sparse reading by skipping rows based on sparse_interval and sparse_offset.
• If sparse_statistics is enabled, computes the specified statistics (average, median, min, max) on floating-point data.
• Handles errors by setting appropriate error messages and managing recovery modes.

Note

• This function provides extensive control over the reading process, allowing for optimized data access.
• Ensure that all parameters are set correctly to avoid unintended data skips or miscomputations.
• The function assumes that the dataset has been properly initialized and that the file pointers are correctly set up.
• Compression support (zLib for gzip, LZMA libraries) must be enabled during compilation for handling compressed files.

Definition at line 2223 of file SDDS_binary.c.

                                                                                                                                                              {
  int32_t n_rows32;
  int64_t n_rows, i, j, k, alloc_rows, rows_to_store, mod;
 
  /*  int32_t page_number, i; */
#if defined(zLib)
  gzFile gzfp = NULL;
#endif
  FILE *fp = NULL;
  struct lzmafile *lzmafp = NULL;
  SDDS_FILEBUFFER *fBuffer;
  void **statData=NULL;
  double statResult;
 
  if (SDDS_dataset->autoRecovered)
    return -1;
  if (SDDS_dataset->swapByteOrder) {
    return SDDS_ReadNonNativePageDetailed(SDDS_dataset, 0, sparse_interval, sparse_offset, last_rows);
  }
 
  /*  static char s[SDDS_MAXLINE]; */
  n_rows = 0;
  SDDS_SetReadRecoveryMode(SDDS_dataset, 0);
#if defined(zLib)
  if (SDDS_dataset->layout.gzipFile) {
    gzfp = SDDS_dataset->layout.gzfp;
  } else {
#endif
    if (SDDS_dataset->layout.lzmaFile) {
      lzmafp = SDDS_dataset->layout.lzmafp;
    } else {
      fp = SDDS_dataset->layout.fp;
    }
#if defined(zLib)
  }
#endif
  fBuffer = &SDDS_dataset->fBuffer;
  if (!fBuffer->buffer) {
    int32_t bufferSize = SDDS_GetLockedDefaultIOBufferSize();
    if (bufferSize == 0 && (SDDS_dataset->layout.popenUsed || !SDDS_dataset->layout.filename) && (sparse_interval > 1 || sparse_offset > 0 || last_rows > 0)) {
      SDDS_SetError("The IO buffer size is 0 for data being read from a pipe with sparsing.  This is not supported.");
      return 0;
    }
    if (!(fBuffer->buffer = fBuffer->data = SDDS_Malloc(sizeof(char) * (bufferSize + 1)))) {
      SDDS_SetError("Unable to do buffered read--allocation failure");
      return 0;
    }
    fBuffer->bufferSize = bufferSize;
    fBuffer->bytesLeft = 0;
  }
  SDDS_dataset->rowcount_offset = -1;
#if defined(zLib)
  if (SDDS_dataset->layout.gzipFile) {
    if (!SDDS_GZipBufferedRead(&n_rows32, sizeof(n_rows32), gzfp, &SDDS_dataset->fBuffer, SDDS_LONG, SDDS_dataset->layout.byteOrderDeclared)) {
      if (gzeof(gzfp))
        return (SDDS_dataset->page_number = -1);
      SDDS_SetError("Unable to read page--failure reading number of rows (SDDS_ReadBinaryPageDetailed)");
      return (0);
    }
    if (n_rows32 == INT32_MIN) {
      if (!SDDS_GZipBufferedRead(&n_rows, sizeof(n_rows), gzfp, &SDDS_dataset->fBuffer, SDDS_LONG64, SDDS_dataset->layout.byteOrderDeclared)) {
        if (gzeof(gzfp))
          return (SDDS_dataset->page_number = -1);
        SDDS_SetError("Unable to read page--failure reading number of rows (SDDS_ReadBinaryPageDetailed)");
        return (0);
      }
    } else {
      n_rows = n_rows32;
    }
  } else {
#endif
    /* This value will only be valid if read buffering is turned off, which is done for
     * certain append operations!   Should really modify SDDS_BufferedRead and SDDS_BufferedWrite
     * to provide ftell capability.
     */
    if (SDDS_dataset->layout.lzmaFile) {
      if (!SDDS_LZMABufferedRead(&n_rows32, sizeof(n_rows32), lzmafp, &SDDS_dataset->fBuffer, SDDS_LONG, SDDS_dataset->layout.byteOrderDeclared)) {
        if (lzma_eof(lzmafp))
          return (SDDS_dataset->page_number = -1);
        SDDS_SetError("Unable to read page--failure reading number of rows (SDDS_ReadBinaryPageDetailed)");
        return (0);
      }
      if (n_rows32 == INT32_MIN) {
        if (!SDDS_LZMABufferedRead(&n_rows, sizeof(n_rows), lzmafp, &SDDS_dataset->fBuffer, SDDS_LONG64, SDDS_dataset->layout.byteOrderDeclared)) {
          if (lzma_eof(lzmafp))
            return (SDDS_dataset->page_number = -1);
          SDDS_SetError("Unable to read page--failure reading number of rows (SDDS_ReadBinaryPageDetailed)");
          return (0);
        }
      } else {
        n_rows = n_rows32;
      }
    } else {
      SDDS_dataset->rowcount_offset = ftell(fp);
      if (!SDDS_BufferedRead(&n_rows32, sizeof(n_rows32), fp, &SDDS_dataset->fBuffer, SDDS_LONG, SDDS_dataset->layout.byteOrderDeclared)) {
        if (feof(fp))
          return (SDDS_dataset->page_number = -1);
        SDDS_SetError("Unable to read page--failure reading number of rows (SDDS_ReadBinaryPageDetailed)");
        return (0);
      }
      if (n_rows32 == INT32_MIN) {
        if (!SDDS_BufferedRead(&n_rows, sizeof(n_rows), fp, &SDDS_dataset->fBuffer, SDDS_LONG64, SDDS_dataset->layout.byteOrderDeclared)) {
          if (feof(fp))
            return (SDDS_dataset->page_number = -1);
          SDDS_SetError("Unable to read page--failure reading number of rows (SDDS_ReadBinaryPageDetailed)");
          return (0);
        }
      } else {
        n_rows = n_rows32;
      }
    }
#if defined(zLib)
  }
#endif
 
#if defined(DEBUG)
  fprintf(stderr, "Expect %" PRId64 " rows of data\n", n_rows);
#endif
  if (n_rows < 0) {
    SDDS_SetError("Unable to read page--negative number of rows (SDDS_ReadBinaryPageDetailed)");
    return (0);
  }
  if (SDDS_dataset->layout.byteOrderDeclared == 0) {
    if (n_rows > 10000000) {
      SDDS_SetError("Unable to read page--endian byte order not declared and suspected to be non-native. (SDDS_ReadBinaryPageDetailed)");
      return (0);
    }
  }
  if (n_rows > SDDS_GetRowLimit()) {
    /* the number of rows is "unreasonably" large---treat like end-of-file */
    return (SDDS_dataset->page_number = -1);
  }
  if (last_rows < 0)
    last_rows = 0;
  /* Fix this limitation later */
 
  if (last_rows) {
    sparse_interval = 1;
    sparse_offset = n_rows - last_rows;
  }
  if (sparse_interval <= 0)
    sparse_interval = 1;
  if (sparse_offset < 0)
    sparse_offset = 0;
 
  rows_to_store = (n_rows - sparse_offset) / sparse_interval + 2;
  alloc_rows = rows_to_store - SDDS_dataset->n_rows_allocated;
 
  if (!SDDS_StartPage(SDDS_dataset, 0) || !SDDS_LengthenTable(SDDS_dataset, alloc_rows)) {
    SDDS_SetError("Unable to read page--couldn't start page (SDDS_ReadBinaryPageDetailed)");
    return (0);
  }
 
  /* read the parameter values */
  if (!SDDS_ReadBinaryParameters(SDDS_dataset)) {
    SDDS_SetError("Unable to read page--parameter reading error (SDDS_ReadBinaryPageDetailed)");
    return (0);
  }
 
  /* read the array values */
  if (!SDDS_ReadBinaryArrays(SDDS_dataset)) {
    SDDS_SetError("Unable to read page--array reading error (SDDS_ReadBinaryPageDetailed)");
    return (0);
  }
  if (SDDS_dataset->layout.data_mode.column_major) {
    SDDS_dataset->n_rows = n_rows;
    if (sparse_statistics == 0) {
      if (!SDDS_ReadBinaryColumns(SDDS_dataset, sparse_interval, sparse_offset)) {
        SDDS_SetError("Unable to read page--column reading error (SDDS_ReadBinaryPageDetailed)");
        return (0);
      }
      return (SDDS_dataset->page_number);
    } else {
      /* allocate space for full columns if needed */
      if (SDDS_dataset->n_rows_allocated < n_rows) {
        if (!SDDS_LengthenTable(SDDS_dataset, n_rows - SDDS_dataset->n_rows_allocated)) {
          SDDS_SetError("Unable to read page--couldn't start page (SDDS_ReadBinaryPageDetailed)");
          return (0);
        }
      }
      if (!SDDS_ReadBinaryColumns(SDDS_dataset, 1, 0)) {
        SDDS_SetError("Unable to read page--column reading error (SDDS_ReadBinaryPageDetailed)");
        return (0);
      }
      statData = (void**)malloc(SDDS_dataset->layout.n_columns * sizeof(void*));
      for (i = 0; i < SDDS_dataset->layout.n_columns; i++) {
        statData[i] = NULL;
        if (SDDS_FLOATING_TYPE(SDDS_dataset->layout.column_definition[i].type))
          statData[i] = (double*)calloc(sparse_interval, sizeof(double));
      }
      n_rows -= sparse_offset;
      for (j = k = 0; j < n_rows; j++) {
        for (i = 0; i < SDDS_dataset->layout.n_columns; i++) {
          switch (SDDS_dataset->layout.column_definition[i].type) {
          case SDDS_FLOAT:
            if (statData[i])
              ((double*)statData[i])[j % sparse_interval] = (double)(((float*)SDDS_dataset->data[i])[j + sparse_offset]);
            break;
          case SDDS_DOUBLE:
            if (statData[i])
              ((double*)statData[i])[j % sparse_interval] = ((double*)SDDS_dataset->data[i])[j + sparse_offset];
            break;
          case SDDS_LONGDOUBLE:
            if (statData[i])
              ((double*)statData[i])[j % sparse_interval] = (double)(((long double*)SDDS_dataset->data[i])[j + sparse_offset]);
            break;
          case SDDS_STRING:
            if (((char ***)SDDS_dataset->data)[i][k])
              free(((char ***)SDDS_dataset->data)[i][k]);
            ((char ***)SDDS_dataset->data)[i][k] = ((char ***)SDDS_dataset->data)[i][j + sparse_offset];
            ((char ***)SDDS_dataset->data)[i][j + sparse_offset] = NULL;
            break;
          case SDDS_SHORT:
            ((short*)SDDS_dataset->data[i])[k] = ((short*)SDDS_dataset->data[i])[j + sparse_offset];
            break;
          case SDDS_USHORT:
            ((unsigned short*)SDDS_dataset->data[i])[k] = ((unsigned short*)SDDS_dataset->data[i])[j + sparse_offset];
            break;
          case SDDS_LONG:
            ((int32_t*)SDDS_dataset->data[i])[k] = ((int32_t*)SDDS_dataset->data[i])[j + sparse_offset];
            break;
          case SDDS_ULONG:
            ((uint32_t*)SDDS_dataset->data[i])[k] = ((uint32_t*)SDDS_dataset->data[i])[j + sparse_offset];
            break;
          case SDDS_LONG64:
            ((int64_t*)SDDS_dataset->data[i])[k] = ((int64_t*)SDDS_dataset->data[i])[j + sparse_offset];
            break;
          case SDDS_ULONG64:
            ((uint64_t*)SDDS_dataset->data[i])[k] = ((uint64_t*)SDDS_dataset->data[i])[j + sparse_offset];
            break;
          case SDDS_CHARACTER:
            ((char*)SDDS_dataset->data[i])[k] = ((char*)SDDS_dataset->data[i])[j + sparse_offset];
            break;
          default:
            break;
          }
          if (statData[i]) {
            if (sparse_statistics == 1)
              compute_average(&statResult, (double*)statData[i], (j % sparse_interval) + 1);
            else if (sparse_statistics == 2)
              compute_median(&statResult, (double*)statData[i], (j % sparse_interval) + 1);
            else if (sparse_statistics == 3)
              statResult = min_in_array((double*)statData[i], (j % sparse_interval) + 1);
            else if (sparse_statistics == 4)
              statResult = max_in_array((double*)statData[i], (j % sparse_interval) + 1);
            switch (SDDS_dataset->layout.column_definition[i].type) {
            case SDDS_FLOAT:
              ((float*)SDDS_dataset->data[i])[k] = statResult;
              break;
            case SDDS_DOUBLE:
              ((double*)SDDS_dataset->data[i])[k] = statResult;
              break;
            case SDDS_LONGDOUBLE:
              ((long double*)SDDS_dataset->data[i])[k] = statResult;
              break;
            }
          }
        }
        if (j % sparse_interval == sparse_interval - 1)
          k++;
      }
      for (i = 0; i < SDDS_dataset->layout.n_columns; i++) {
        if (SDDS_dataset->layout.column_definition[i].type == SDDS_STRING) {
          int64_t r;
          for (r = k; r < SDDS_dataset->n_rows; r++) {
            if (((char ***)SDDS_dataset->data)[i][r]) {
              free(((char ***)SDDS_dataset->data)[i][r]);
              ((char ***)SDDS_dataset->data)[i][r] = NULL;
            }
          }
        }
        if (statData[i])
          free(statData[i]);
      }
      free(statData);
      SDDS_dataset->n_rows = k;
      return (SDDS_dataset->page_number);
    }
  }
  if ((sparse_interval <= 1) && (sparse_offset == 0)) {
    for (j = 0; j < n_rows; j++) {
      if (!SDDS_ReadBinaryRow(SDDS_dataset, j, 0)) {
        SDDS_dataset->n_rows = j;
        if (SDDS_dataset->autoRecover) {
#if defined(DEBUG)
          fprintf(stderr, "Doing auto-read recovery\n");
#endif
          SDDS_dataset->autoRecovered = 1;
          SDDS_ClearErrors();
          return (SDDS_dataset->page_number);
        }
        SDDS_SetError("Unable to read page--error reading data row (SDDS_ReadBinaryPageDetailed)");
        SDDS_SetReadRecoveryMode(SDDS_dataset, 1);
        return (0);
      }
    }
    SDDS_dataset->n_rows = j;
    return (SDDS_dataset->page_number);
  } else {
    for (j = 0; j < sparse_offset; j++) {
      if (!SDDS_ReadBinaryRow(SDDS_dataset, 0, 1)) {
        SDDS_dataset->n_rows = 0;
        if (SDDS_dataset->autoRecover) {
          SDDS_dataset->autoRecovered = 1;
          SDDS_ClearErrors();
          return (SDDS_dataset->page_number);
        }
        SDDS_SetError("Unable to read page--error reading data row (SDDS_ReadBinaryPageDetailed)");
        SDDS_SetReadRecoveryMode(SDDS_dataset, 1);
        return (0);
      }
    }
    n_rows -= sparse_offset;
    if (sparse_statistics != 0) {
      // Allocate buffer space for statistical sparsing
      statData = (void**)malloc(SDDS_dataset->layout.n_columns * sizeof(void*));
      for (i = 0; i < SDDS_dataset->layout.n_columns; i++) {
        if (SDDS_FLOATING_TYPE(SDDS_dataset->layout.column_definition[i].type)) {
          // Not ideal for SDDS_LONGDOUBLE but we may never run across this error
          statData[i] = (double*)calloc(sparse_interval, sizeof(double));
        }
      }
      for (j = k = 0; j < n_rows; j++) {
        if (!SDDS_ReadBinaryRow(SDDS_dataset, k, 0)) {
          SDDS_dataset->n_rows = k;
          if (SDDS_dataset->autoRecover) {
            SDDS_dataset->autoRecovered = 1;
            SDDS_ClearErrors();
            return (SDDS_dataset->page_number);
          }
          SDDS_SetError("Unable to read page--error reading data row (SDDS_ReadBinaryPageDetailed)");
          SDDS_SetReadRecoveryMode(SDDS_dataset, 1);
          return (0);
        }
        for (i = 0; i < SDDS_dataset->layout.n_columns; i++) {
          switch (SDDS_dataset->layout.column_definition[i].type) {
          case SDDS_FLOAT:
            ((double*)statData[i])[j % sparse_interval] = (double)(((float*)SDDS_dataset->data[i])[k]);
            break;
          case SDDS_DOUBLE:
            ((double*)statData[i])[j % sparse_interval] = ((double*)SDDS_dataset->data[i])[k];
            break;
          case SDDS_LONGDOUBLE:
            ((double*)statData[i])[j % sparse_interval] = (double)(((long double*)SDDS_dataset->data[i])[k]);
            break;
          }
          if (SDDS_FLOATING_TYPE(SDDS_dataset->layout.column_definition[i].type)) {
            if (sparse_statistics == 1) {
              // Sparse and get average statistics
              compute_average(&statResult, (double*)statData[i], (j % sparse_interval) + 1);
            } else if (sparse_statistics == 2) {
              // Sparse and get median statistics
              compute_median(&statResult, (double*)statData[i], (j % sparse_interval) + 1);
            } else if (sparse_statistics == 3) {
              // Sparse and get minimum statistics
              statResult = min_in_array((double*)statData[i], (j % sparse_interval) + 1);
            } else if (sparse_statistics == 4) {
              // Sparse and get maximum statistics
              statResult = max_in_array((double*)statData[i], (j % sparse_interval) + 1);
            }
          }
          switch (SDDS_dataset->layout.column_definition[i].type) {
          case SDDS_FLOAT:
            ((float*)SDDS_dataset->data[i])[k] = statResult;
            break;
          case SDDS_DOUBLE:
            ((double*)SDDS_dataset->data[i])[k] = statResult;
            break;
          case SDDS_LONGDOUBLE:
            ((long double*)SDDS_dataset->data[i])[k] = statResult;
            break;
          }
        }
        if (j % sparse_interval == sparse_interval - 1) {
          k++;
        }
      }
      for (i = 0; i < SDDS_dataset->layout.n_columns; i++) {
        if (SDDS_FLOATING_TYPE(SDDS_dataset->layout.column_definition[i].type)) {
          free(statData[i]);
        }
      }
      free(statData);
    } else {
      for (j = k = 0; j < n_rows; j++) {
        if (!SDDS_ReadBinaryRow(SDDS_dataset, k, mod = j % sparse_interval)) {
          SDDS_dataset->n_rows = k;
          if (SDDS_dataset->autoRecover) {
            SDDS_dataset->autoRecovered = 1;
            SDDS_ClearErrors();
            return (SDDS_dataset->page_number);
          }
          SDDS_SetError("Unable to read page--error reading data row (SDDS_ReadBinaryPageDetailed)");
          SDDS_SetReadRecoveryMode(SDDS_dataset, 1);
          return (0);
        }
        k += mod ? 0 : 1;
      }
    }
    SDDS_dataset->n_rows = k;
    return (SDDS_dataset->page_number);
  }
}

SDDS_ReadBinaryPageLastRows()

int32_t SDDS_ReadBinaryPageLastRows ( SDDS_DATASET * SDDS_dataset, int64_t last_rows )

extern

Reads the last specified number of rows from a binary page of an SDDS dataset.

This function reads the last last_rows rows from the binary page of the specified SDDS dataset. It is useful for retrieving recent data entries without processing the entire dataset.

Parameters

SDDS_dataset	Pointer to the SDDS_DATASET structure representing the dataset to read from.
last_rows	The number of rows to read from the end of the dataset.

Returns

• Returns the page number on successful read.
• Returns -1 if the end-of-file is reached.
• Returns 0 on error.

The function internally calls SDDS_ReadBinaryPageDetailed with sparse_interval set to 1, sparse_offset set to 0, and last_rows as specified. This configuration ensures that only the last last_rows rows are read from the dataset.

Note

• This function is particularly useful for applications that need to display or process the most recent data entries.
• Ensure that last_rows does not exceed the total number of rows in the dataset to avoid errors.
• The function assumes that the dataset has been properly initialized and that the file pointers are correctly set up.

Definition at line 2171 of file SDDS_binary.c.

                                                                                   {
  return SDDS_ReadBinaryPageDetailed(SDDS_dataset, 1, 0, last_rows, 0);
}

SDDS_ReadBinaryParameters()

int32_t SDDS_ReadBinaryParameters ( SDDS_DATASET * SDDS_dataset )

extern

Reads binary parameters from the specified SDDS dataset.

This function iterates through all the parameters defined in the SDDS dataset layout and reads their values from the underlying file. It handles different data types, including strings, and manages memory allocation for string parameters. Depending on the dataset's compression settings (uncompressed, LZMA, or GZIP), it uses the appropriate reading functions to retrieve the parameter values.

Parameters

[in,out]

SDDS_dataset

Pointer to the SDDS_DATASET structure representing the dataset to read from.

Returns

int32_t Returns 1 on successfully reading all binary parameters, or 0 if an error occurred.

Return values

1	All parameters were successfully read and stored.
0	An error occurred during the read operation, such as I/O errors, data type mismatches, or memory allocation failures.

Note

Parameters with the 'fixed_value' attribute are handled by scanning the fixed value string instead of reading from the file. String parameters are dynamically allocated and should be freed by the caller when no longer needed.

Definition at line 3072 of file SDDS_binary.c.

                                                              {
  int32_t i;
  SDDS_LAYOUT *layout;
  /*  char *predefined_format; */
  char buffer[SDDS_MAXLINE];
#if defined(zLib)
  gzFile gzfp = NULL;
#endif
  FILE *fp = NULL;
  struct lzmafile *lzmafp = NULL;
  SDDS_FILEBUFFER *fBuffer;
 
  if (!SDDS_CheckDataset(SDDS_dataset, "SDDS_ReadBinaryParameters"))
    return (0);
  layout = &SDDS_dataset->layout;
  if (!layout->n_parameters)
    return (1);
#if defined(zLib)
  if (SDDS_dataset->layout.gzipFile) {
    gzfp = layout->gzfp;
  } else {
#endif
    if (SDDS_dataset->layout.lzmaFile) {
      lzmafp = layout->lzmafp;
    } else {
      fp = layout->fp;
    }
#if defined(zLib)
  }
#endif
  fBuffer = &SDDS_dataset->fBuffer;
  for (i = 0; i < layout->n_parameters; i++) {
    if (layout->parameter_definition[i].definition_mode & SDDS_WRITEONLY_DEFINITION)
      continue;
    if (layout->parameter_definition[i].fixed_value) {
      strcpy(buffer, layout->parameter_definition[i].fixed_value);
      if (!SDDS_ScanData(buffer, layout->parameter_definition[i].type, 0, SDDS_dataset->parameter[i], 0, 1)) {
        SDDS_SetError("Unable to read page--parameter scanning error (SDDS_ReadBinaryParameters)");
        return (0);
      }
    } else if (layout->parameter_definition[i].type == SDDS_STRING) {
      if (*(char **)SDDS_dataset->parameter[i])
        free(*(char **)SDDS_dataset->parameter[i]);
#if defined(zLib)
      if (SDDS_dataset->layout.gzipFile) {
        if (!(*((char **)SDDS_dataset->parameter[i]) = SDDS_ReadGZipBinaryString(gzfp, fBuffer, 0))) {
          SDDS_SetError("Unable to read parameters--failure reading string (SDDS_ReadBinaryParameters)");
          return (0);
        }
      } else {
#endif
        if (SDDS_dataset->layout.lzmaFile) {
          if (!(*((char **)SDDS_dataset->parameter[i]) = SDDS_ReadLZMABinaryString(lzmafp, fBuffer, 0))) {
            SDDS_SetError("Unable to read parameters--failure reading string (SDDS_ReadBinaryParameters)");
            return (0);
          }
        } else {
          if (!(*((char **)SDDS_dataset->parameter[i]) = SDDS_ReadBinaryString(fp, fBuffer, 0))) {
            SDDS_SetError("Unable to read parameters--failure reading string (SDDS_ReadBinaryParameters)");
            return (0);
          }
        }
#if defined(zLib)
      }
#endif
    } else {
#if defined(zLib)
      if (SDDS_dataset->layout.gzipFile) {
        if (!SDDS_GZipBufferedRead(SDDS_dataset->parameter[i], SDDS_type_size[layout->parameter_definition[i].type - 1], gzfp, fBuffer, layout->parameter_definition[i].type, SDDS_dataset->layout.byteOrderDeclared)) {
          SDDS_SetError("Unable to read parameters--failure reading value (SDDS_ReadBinaryParameters)");
          return (0);
        }
      } else {
#endif
        if (SDDS_dataset->layout.lzmaFile) {
          if (!SDDS_LZMABufferedRead(SDDS_dataset->parameter[i], SDDS_type_size[layout->parameter_definition[i].type - 1], lzmafp, fBuffer, layout->parameter_definition[i].type, SDDS_dataset->layout.byteOrderDeclared)) {
            SDDS_SetError("Unable to read parameters--failure reading value (SDDS_ReadBinaryParameters)");
            return (0);
          }
        } else {
          if (!SDDS_BufferedRead(SDDS_dataset->parameter[i], SDDS_type_size[layout->parameter_definition[i].type - 1], fp, fBuffer, layout->parameter_definition[i].type, SDDS_dataset->layout.byteOrderDeclared)) {
            SDDS_SetError("Unable to read parameters--failure reading value (SDDS_ReadBinaryParameters)");
            return (0);
          }
        }
#if defined(zLib)
      }
#endif
    }
  }
  return (1);
}

SDDS_ReadBinaryRow()

int32_t SDDS_ReadBinaryRow ( SDDS_DATASET * SDDS_dataset, int64_t row, int32_t skip )

extern

Reads a binary row from the specified SDDS dataset.

This function reads a single row of data from the given SDDS dataset. Depending on the dataset's configuration, it handles uncompressed, LZMA-compressed, or GZIP-compressed files. For each column in the dataset, the function reads the appropriate data type. If a column is of type string, it reads the string using the corresponding string reading function. If the 'skip' parameter is set, the function skips reading the data without storing it.

Parameters

[in,out]	SDDS_dataset	Pointer to the SDDS_DATASET structure representing the dataset to read from.
[in]	row	The row number to read. Must be within the allocated range of rows in the dataset.
[in]	skip	If non-zero, the function skips reading the data for each column without storing it.

Returns

int32_t Returns 1 on successful reading of the row, or 0 if an error occurred.

Return values

1	The row was successfully read and stored (or skipped).
0	An error occurred during reading, such as I/O errors or memory allocation failures.

Note

This function may modify the dataset's data structures by allocating memory for string columns. Ensure that the dataset is properly initialized and that memory is managed appropriately.

Definition at line 2836 of file SDDS_binary.c.

                                                                                  {
  int64_t i, type, size;
  SDDS_LAYOUT *layout;
#if defined(zLib)
  gzFile gzfp;
#endif
  FILE *fp;
  struct lzmafile *lzmafp;
  SDDS_FILEBUFFER *fBuffer;
 
  if (!SDDS_CheckDataset(SDDS_dataset, "SDDS_ReadBinaryRow"))
    return (0);
  layout = &SDDS_dataset->layout;
  fBuffer = &SDDS_dataset->fBuffer;
 
#if defined(zLib)
  if (SDDS_dataset->layout.gzipFile) {
    gzfp = layout->gzfp;
    for (i = 0; i < layout->n_columns; i++) {
      if (layout->column_definition[i].definition_mode & SDDS_WRITEONLY_DEFINITION)
        continue;
      if ((type = layout->column_definition[i].type) == SDDS_STRING) {
        if (!skip) {
          if (((char ***)SDDS_dataset->data)[i][row])
            free((((char ***)SDDS_dataset->data)[i][row]));
          if (!(((char ***)SDDS_dataset->data)[i][row] = SDDS_ReadGZipBinaryString(gzfp, fBuffer, 0))) {
            SDDS_SetError("Unable to read rows--failure reading string (SDDS_ReadBinaryRows)");
            return (0);
          }
        } else {
          if (!SDDS_ReadGZipBinaryString(gzfp, fBuffer, 1)) {
            SDDS_SetError("Unable to read rows--failure reading string (SDDS_ReadBinaryRows)");
            return 0;
          }
        }
      } else {
        size = SDDS_type_size[type - 1];
        if (!SDDS_GZipBufferedRead(skip ? NULL : (char *)SDDS_dataset->data[i] + row * size, size, gzfp, fBuffer, type, SDDS_dataset->layout.byteOrderDeclared)) {
          SDDS_SetError("Unable to read row--failure reading value (SDDS_ReadBinaryRow)");
          return (0);
        }
      }
    }
  } else {
#endif
    if (SDDS_dataset->layout.lzmaFile) {
      lzmafp = layout->lzmafp;
      for (i = 0; i < layout->n_columns; i++) {
        if (layout->column_definition[i].definition_mode & SDDS_WRITEONLY_DEFINITION)
          continue;
        if ((type = layout->column_definition[i].type) == SDDS_STRING) {
          if (!skip) {
            if (((char ***)SDDS_dataset->data)[i][row])
              free((((char ***)SDDS_dataset->data)[i][row]));
            if (!(((char ***)SDDS_dataset->data)[i][row] = SDDS_ReadLZMABinaryString(lzmafp, fBuffer, 0))) {
              SDDS_SetError("Unable to read rows--failure reading string (SDDS_ReadBinaryRows)");
              return (0);
            }
          } else {
            if (!SDDS_ReadLZMABinaryString(lzmafp, fBuffer, 1)) {
              SDDS_SetError("Unable to read rows--failure reading string (SDDS_ReadBinaryRows)");
              return 0;
            }
          }
        } else {
          size = SDDS_type_size[type - 1];
          if (!SDDS_LZMABufferedRead(skip ? NULL : (char *)SDDS_dataset->data[i] + row * size, size, lzmafp, fBuffer, type, SDDS_dataset->layout.byteOrderDeclared)) {
            SDDS_SetError("Unable to read row--failure reading value (SDDS_ReadBinaryRow)");
            return (0);
          }
        }
      }
    } else {
      fp = layout->fp;
      for (i = 0; i < layout->n_columns; i++) {
        if (layout->column_definition[i].definition_mode & SDDS_WRITEONLY_DEFINITION)
          continue;
        if ((type = layout->column_definition[i].type) == SDDS_STRING) {
          if (!skip) {
            if (((char ***)SDDS_dataset->data)[i][row])
              free((((char ***)SDDS_dataset->data)[i][row]));
            if (!(((char ***)SDDS_dataset->data)[i][row] = SDDS_ReadBinaryString(fp, fBuffer, 0))) {
              SDDS_SetError("Unable to read rows--failure reading string (SDDS_ReadBinaryRows)");
              return (0);
            }
          } else {
            if (!SDDS_ReadBinaryString(fp, fBuffer, 1)) {
              SDDS_SetError("Unable to read rows--failure reading string (SDDS_ReadBinaryRows)");
              return 0;
            }
          }
        } else {
          size = SDDS_type_size[type - 1];
          if (!SDDS_BufferedRead(skip ? NULL : (char *)SDDS_dataset->data[i] + row * size, size, fp, fBuffer, type, SDDS_dataset->layout.byteOrderDeclared)) {
            SDDS_SetError("Unable to read row--failure reading value (SDDS_ReadBinaryRow)");
            return (0);
          }
        }
      }
    }
#if defined(zLib)
  }
#endif
  return (1);
}

SDDS_ReadBinaryString()

char * SDDS_ReadBinaryString ( FILE * fp, SDDS_FILEBUFFER * fBuffer, int32_t skip )

extern

Reads a binary string from a file with buffering.

This function reads a binary string from the specified file by first reading the length of the string and then reading the string content based on the length. If the 'skip' parameter is set, the string data is skipped over instead of being stored. The function allocates memory for the string, which should be freed by the caller when no longer needed.

Parameters

[in]	fp	The file pointer to read from. Must be an open file in binary read mode.
[in,out]	fBuffer	Pointer to the file buffer used for buffered reading operations.
[in]	skip	If non-zero, the string data is skipped without being stored.

Returns

char* Returns a pointer to the read null-terminated string on success, or NULL if an error occurred.

Return values

NULL	An error occurred during reading or memory allocation.
Non-NULL	Pointer to the read string.

Definition at line 2741 of file SDDS_binary.c.

                                                                              {
  int32_t length;
  char *string;
 
  if (!SDDS_BufferedRead(&length, sizeof(length), fp, fBuffer, SDDS_LONG, 0) || length < 0)
    return (0);
  if (!(string = SDDS_Malloc(sizeof(*string) * (length + 1))))
    return (NULL);
  if (length && !SDDS_BufferedRead(skip ? NULL : string, sizeof(*string) * length, fp, fBuffer, SDDS_STRING, 0))
    return (NULL);
  string[length] = 0;
  return (string);
}

SDDS_ReadLayout()

int32_t SDDS_ReadLayout ( SDDS_DATASET * SDDS_dataset, FILE * fp )

extern

Reads the header layout of an SDDS dataset from a file.

Parameters

SDDS_dataset	The SDDS dataset structure to store the layout information.
fp	The file pointer to the SDDS file.

Returns

Returns 1 on success, 0 on failure.

Definition at line 518 of file SDDS_input.c.

                                                              {
  char buffer[SDDS_MAXLINE];
  char *groupName, *ptr;
  FILE *fp1;
  int32_t retval, bigEndianMachine;
  uint32_t commentFlags;
 
  if (!fp) {
    SDDS_SetError("Unable to read layout--NULL file pointer (SDDS_ReadLayout)");
    return (0);
  }
  if (SDDS_dataset->layout.depth == 0) {
    if (SDDS_dataset->layout.disconnected) {
      SDDS_SetError("Can't read layout--file is disconnected (SDDS_ReadLayout)");
      return 0;
    }
    if (!SDDS_CheckDataset(SDDS_dataset, "SDDS_ReadLayout")) {
      fclose(fp);
      return (0);
    }
    SDDS_dataset->layout.layout_written = 1; /* it is already in the file */
    if (!fgets(SDDS_dataset->layout.s, SDDS_MAXLINE, fp)) {
      fclose(fp);
      SDDS_SetError("Unable to read layout--no header lines found (SDDS_ReadLayout)");
      return (0);
    }
    if (strncmp(SDDS_dataset->layout.s, "SDDS", 4) != 0) {
      fclose(fp);
      SDDS_SetError("Unable to read layout--no header lines found (SDDS_ReadLayout)");
      return (0);
    }
    if (sscanf(SDDS_dataset->layout.s + 4, "%" SCNd32, &SDDS_dataset->layout.version) != 1) {
      fclose(fp);
      SDDS_SetError("Unable to read layout--no version number on first line (SDDS_ReadLayout)");
      return (0);
    }
    SDDS_ResetSpecialCommentsModes(SDDS_dataset);
    SDDS_dataset->layout.data_command_seen = 0;
  }
  while (SDDS_GetNamelist(SDDS_dataset, SDDS_dataset->layout.s, SDDS_MAXLINE, fp)) {
#if DEBUG
    strcpy(buffer, SDDS_dataset->layout.s);
#endif
    groupName = SDDS_dataset->layout.s + 1;
    if (!(ptr = strpbrk(SDDS_dataset->layout.s, " \t"))) {
      SDDS_SetError("Unable to read layout---no groupname in namelist (SDDS_ReadLayout)");
      return 0;
    }
    *ptr = 0;
    switch (match_string(groupName, SDDS_command, SDDS_NUM_COMMANDS, EXACT_MATCH)) {
    case SDDS_DESCRIPTION_COMMAND:
      if (!SDDS_ProcessDescription(SDDS_dataset, ptr + 1)) {
        fclose(fp);
        SDDS_SetError("Unable to process description (SDDS_ReadLayout)");
        return (0);
      }
      break;
    case SDDS_COLUMN_COMMAND:
      if (!SDDS_ProcessColumnDefinition(SDDS_dataset, ptr + 1)) {
        fclose(fp);
        SDDS_SetError("Unable to process column definition (SDDS_ReadLayout)");
        return (0);
      }
      break;
    case SDDS_PARAMETER_COMMAND:
      if (!SDDS_ProcessParameterDefinition(SDDS_dataset, ptr + 1)) {
        fclose(fp);
        SDDS_SetError("Unable to process parameter definition (SDDS_ReadLayout)");
        return (0);
      }
      break;
    case SDDS_ASSOCIATE_COMMAND:
#if RW_ASSOCIATES != 0
      if (!SDDS_ProcessAssociateDefinition(SDDS_dataset, ptr + 1)) {
        fclose(fp);
        SDDS_SetError("Unable to process associate definition (SDDS_ReadLayout)");
        return (0);
      }
#endif
      break;
    case SDDS_DATA_COMMAND:
      if (!SDDS_ProcessDataMode(SDDS_dataset, ptr + 1)) {
        fclose(fp);
        SDDS_SetError("Unable to process data mode (SDDS_ReadLayout)");
        return (0);
      }
      if (SDDS_dataset->layout.data_command_seen) {
        /* should never happen */
        fclose(fp);
        SDDS_SetError("Unable to read layout--multiple data commands (SDDS_ReadLayout)");
        return (0);
      }
      if (!SDDS_SaveLayout(SDDS_dataset)) {
        SDDS_SetError("Unable to read layout--couldn't save layout (SDDS_ReadLayout)");
        return (0);
      }
      SDDS_dataset->layout.data_command_seen = 1;
      commentFlags = SDDS_GetSpecialCommentsModes(SDDS_dataset);
      if ((commentFlags & SDDS_BIGENDIAN_SEEN) && (commentFlags & SDDS_LITTLEENDIAN_SEEN)) {
        SDDS_SetError("Unable to read data as it says it is both big and little endian (SDDS_ReadLayout)");
        return (0);
      }
      bigEndianMachine = SDDS_IsBigEndianMachine();
      SDDS_dataset->swapByteOrder = SDDS_dataset->layout.byteOrderDeclared = 0;
      SDDS_dataset->autoRecover = 0;
      if ((commentFlags & SDDS_BIGENDIAN_SEEN) || (SDDS_dataset->layout.data_mode.endian == SDDS_BIGENDIAN)) {
        SDDS_dataset->layout.byteOrderDeclared = SDDS_BIGENDIAN_SEEN;
        if (!bigEndianMachine)
          SDDS_dataset->swapByteOrder = 1;
      }
      if ((commentFlags & SDDS_LITTLEENDIAN_SEEN) || (SDDS_dataset->layout.data_mode.endian == SDDS_LITTLEENDIAN)) {
        SDDS_dataset->layout.byteOrderDeclared = SDDS_LITTLEENDIAN_SEEN;
        if (bigEndianMachine)
          SDDS_dataset->swapByteOrder = 1;
      }
      if ((commentFlags & SDDS_FIXED_ROWCOUNT_SEEN) || (SDDS_dataset->layout.data_mode.fixed_row_count))
        if (!SDDS_SetAutoReadRecovery(SDDS_dataset, SDDS_AUTOREADRECOVER))
          return (0);
      return (1);
    case SDDS_INCLUDE_COMMAND:
      if (!(fp1 = SDDS_ProcessIncludeCommand(SDDS_dataset, ptr + 1))) {
        fclose(fp);
        SDDS_SetError("Unable to process include command (SDDS_ReadLayout)");
        return (0);
      }
      SDDS_dataset->layout.depth += 1;
      retval = SDDS_ReadLayout(SDDS_dataset, fp1);
      SDDS_dataset->layout.depth -= 1;
      fclose(fp1);
      if (retval == 0) {
        return (0);
      }
      if (SDDS_dataset->layout.data_command_seen) {
        return (1);
      }
      break;
    case SDDS_ARRAY_COMMAND:
      if (!SDDS_ProcessArrayDefinition(SDDS_dataset, ptr + 1)) {
        fclose(fp);
        SDDS_SetError("Unable to process array definition (SDDS_ReadLayout)");
        return (0);
      }
      break;
    default:
      fclose(fp);
      sprintf(buffer, "Unknown layout entry %s given (SDDS_ReadLayout)", groupName);
      SDDS_SetError(buffer);
      return (0);
    }
  }
  /* on recursive calls, it's okay to hit EOF */
  if ((feof(fp) && SDDS_dataset->layout.depth != 0) || SDDS_dataset->layout.data_command_seen)
    return (1);
  return (0);
}

SDDS_ReadNonNativeBinaryColumns()

int32_t SDDS_ReadNonNativeBinaryColumns ( SDDS_DATASET * SDDS_dataset )

extern

Reads the non-native endian binary columns from an SDDS dataset.

This function is similar to SDDS_ReadBinaryColumns but specifically handles columns with non-native endianness. It iterates through all column definitions within the specified SDDS dataset and reads their binary data from the underlying file, ensuring that the byte order is correctly swapped to match the system's native endianness. The function supports various compression formats, including uncompressed, LZMA-compressed, and GZIP-compressed files.

Parameters

[in,out]

SDDS_dataset

Pointer to the SDDS_DATASET structure representing the dataset to read from.

Returns

int32_t Returns 1 on successful reading and byte-swapping of all columns, or 0 if an error occurred.

Return values

1	All non-native endian columns were successfully read and byte-swapped.
0	An error occurred during the read or byte-swapping operation, such as I/O failures, memory allocation issues, or corrupted column definitions.

Note

This function assumes that the dataset's byte order has been declared and that the underlying file's byte order differs from the system's native byte order. Proper initialization and configuration of the SDDS_dataset structure are required before calling this function.

Definition at line 3590 of file SDDS_binary.c.

                                                                    {
  int64_t i, row;
  SDDS_LAYOUT *layout;
  /*  char *predefined_format; */
  /*  static char buffer[SDDS_MAXLINE]; */
#if defined(zLib)
  gzFile gzfp = NULL;
#endif
  FILE *fp = NULL;
  struct lzmafile *lzmafp = NULL;
  SDDS_FILEBUFFER *fBuffer;
 
  if (!SDDS_CheckDataset(SDDS_dataset, "SDDS_ReadNonNativeBinaryColumns"))
    return (0);
  layout = &SDDS_dataset->layout;
  if (!layout->n_columns || !SDDS_dataset->n_rows)
    return (1);
#if defined(zLib)
  if (SDDS_dataset->layout.gzipFile) {
    gzfp = layout->gzfp;
  } else {
#endif
    if (SDDS_dataset->layout.lzmaFile) {
      lzmafp = layout->lzmafp;
    } else {
      fp = layout->fp;
    }
#if defined(zLib)
  }
#endif
  fBuffer = &SDDS_dataset->fBuffer;
 
  for (i = 0; i < layout->n_columns; i++) {
    if (layout->column_definition[i].definition_mode & SDDS_WRITEONLY_DEFINITION)
      continue;
    if (layout->column_definition[i].type == SDDS_STRING) {
#if defined(zLib)
      if (SDDS_dataset->layout.gzipFile) {
        for (row = 0; row < SDDS_dataset->n_rows; row++) {
          if (((char ***)SDDS_dataset->data)[i][row])
            free((((char ***)SDDS_dataset->data)[i][row]));
          if (!(((char ***)SDDS_dataset->data)[i][row] = SDDS_ReadNonNativeGZipBinaryString(gzfp, fBuffer, 0))) {
            SDDS_SetError("Unable to read columns--failure reading string (SDDS_ReadNonNativeBinaryColumns)");
            return (0);
          }
        }
      } else {
#endif
        if (SDDS_dataset->layout.lzmaFile) {
          for (row = 0; row < SDDS_dataset->n_rows; row++) {
            if (((char ***)SDDS_dataset->data)[i][row])
              free((((char ***)SDDS_dataset->data)[i][row]));
            if (!(((char ***)SDDS_dataset->data)[i][row] = SDDS_ReadNonNativeLZMABinaryString(lzmafp, fBuffer, 0))) {
              SDDS_SetError("Unable to read columns--failure reading string (SDDS_ReadNonNativeBinaryColumms)");
              return (0);
            }
          }
        } else {
          for (row = 0; row < SDDS_dataset->n_rows; row++) {
            if (((char ***)SDDS_dataset->data)[i][row])
              free((((char ***)SDDS_dataset->data)[i][row]));
            if (!(((char ***)SDDS_dataset->data)[i][row] = SDDS_ReadNonNativeBinaryString(fp, fBuffer, 0))) {
              SDDS_SetError("Unable to read columns--failure reading string (SDDS_ReadNonNativeBinaryColumms)");
              return (0);
            }
          }
        }
#if defined(zLib)
      }
#endif
    } else {
#if defined(zLib)
      if (SDDS_dataset->layout.gzipFile) {
        if (!SDDS_GZipBufferedRead(SDDS_dataset->data[i], SDDS_type_size[layout->column_definition[i].type - 1] * SDDS_dataset->n_rows, gzfp, fBuffer, layout->column_definition[i].type, SDDS_dataset->layout.byteOrderDeclared)) {
          SDDS_SetError("Unable to read columns--failure reading values (SDDS_ReadNonNativeBinaryColumns)");
          return (0);
        }
      } else {
#endif
        if (SDDS_dataset->layout.lzmaFile) {
          if (!SDDS_LZMABufferedRead(SDDS_dataset->data[i], SDDS_type_size[layout->column_definition[i].type - 1] * SDDS_dataset->n_rows, lzmafp, fBuffer, layout->column_definition[i].type, SDDS_dataset->layout.byteOrderDeclared)) {
            SDDS_SetError("Unable to read columns--failure reading values (SDDS_ReadNonNativeBinaryColumns)");
            return (0);
          }
        } else {
          if (!SDDS_BufferedRead(SDDS_dataset->data[i], SDDS_type_size[layout->column_definition[i].type - 1] * SDDS_dataset->n_rows, fp, fBuffer, layout->column_definition[i].type, SDDS_dataset->layout.byteOrderDeclared)) {
            SDDS_SetError("Unable to read columns--failure reading values (SDDS_ReadNonNativeBinaryColumns)");
            return (0);
          }
        }
#if defined(zLib)
      }
#endif
    }
  }
  return (1);
}

SDDS_ReadNonNativeBinaryPageDetailed()

int32_t SDDS_ReadNonNativeBinaryPageDetailed ( SDDS_DATASET * SDDS_dataset, int64_t sparse_interval, int64_t sparse_offset, int64_t last_rows )

extern

Reads a detailed non-native endian binary page from an SDDS dataset.

This function reads a binary page from the specified SDDS dataset, handling data with non-native endianness. It supports both sparse reading and reading of the last few rows based on the provided parameters. The function performs necessary byte order conversions to ensure correct data interpretation on the host system.

Parameters

[in,out]	SDDS_dataset	Pointer to the SDDS_DATASET structure representing the dataset to read from.
[in]	sparse_interval	Interval between rows to be read for sparsity.
[in]	sparse_offset	Offset to start reading rows for sparsity.
[in]	last_rows	Number of last rows to read from the dataset.

Returns

int32_t Returns the page number on success, or 0 on failure.

Return values

>0	Page number successfully read.
0	An error occurred during the read operation, such as I/O failures, data corruption, or unsupported data modes.

Note

This function handles various compression formats, including uncompressed, LZMA-compressed, and GZIP-compressed files. It manages memory allocation for parameters, arrays, and columns, ensuring that data is correctly stored and byte-swapped as necessary. The function also updates the dataset's row count and handles auto-recovery in case of errors.

Definition at line 4361 of file SDDS_binary.c.

                                                                                                                                            {
  int32_t n_rows32 = 0;
  int64_t n_rows, j, k, alloc_rows, rows_to_store, mod;
  /*  int32_t page_number, i; */
#if defined(zLib)
  gzFile gzfp = NULL;
#endif
  FILE *fp = NULL;
  struct lzmafile *lzmafp = NULL;
  SDDS_FILEBUFFER *fBuffer;
 
  /*  static char s[SDDS_MAXLINE]; */
  n_rows = 0;
  SDDS_SetReadRecoveryMode(SDDS_dataset, 0);
#if defined(zLib)
  if (SDDS_dataset->layout.gzipFile) {
    gzfp = SDDS_dataset->layout.gzfp;
  } else {
#endif
    if (SDDS_dataset->layout.lzmaFile) {
      lzmafp = SDDS_dataset->layout.lzmafp;
    } else {
      fp = SDDS_dataset->layout.fp;
    }
#if defined(zLib)
  }
#endif
  fBuffer = &SDDS_dataset->fBuffer;
  if (!fBuffer->buffer) {
    int32_t bufferSize = SDDS_GetLockedDefaultIOBufferSize();
    if (!(fBuffer->buffer = fBuffer->data = SDDS_Malloc(sizeof(char) * bufferSize))) {
      SDDS_SetError("Unable to do buffered read--allocation failure");
      return 0;
    }
    fBuffer->bufferSize = bufferSize;
    fBuffer->bytesLeft = 0;
  }
  SDDS_dataset->rowcount_offset = -1;
#if defined(zLib)
  if (SDDS_dataset->layout.gzipFile) {
    if (!SDDS_GZipBufferedRead(&n_rows32, sizeof(n_rows32), gzfp, &SDDS_dataset->fBuffer, SDDS_LONG, SDDS_dataset->layout.byteOrderDeclared)) {
      if (gzeof(gzfp))
        return (SDDS_dataset->page_number = -1);
      SDDS_SetError("Unable to read page--failure reading number of rows (SDDS_ReadNonNativeBinaryPage)");
      return (0);
    }
    SDDS_SwapLong(&n_rows32);
    if (n_rows32 == INT32_MIN) {
      if (!SDDS_GZipBufferedRead(&n_rows, sizeof(n_rows), gzfp, &SDDS_dataset->fBuffer, SDDS_LONG64, SDDS_dataset->layout.byteOrderDeclared)) {
        if (gzeof(gzfp))
          return (SDDS_dataset->page_number = -1);
        SDDS_SetError("Unable to read page--failure reading number of rows (SDDS_ReadNonNativeBinaryPage)");
        return (0);
      }
      SDDS_SwapLong64(&n_rows);
    } else {
      n_rows = n_rows32;
    }
  } else {
#endif
    /* This value will only be valid if read buffering is turned off, which is done for
     * certain append operations!   Should really modify SDDS_BufferedRead and SDDS_BufferedWrite
     * to provide ftell capability.
     */
    if (SDDS_dataset->layout.lzmaFile) {
      if (!SDDS_LZMABufferedRead(&n_rows32, sizeof(n_rows32), lzmafp, &SDDS_dataset->fBuffer, SDDS_LONG, SDDS_dataset->layout.byteOrderDeclared)) {
        if (lzma_eof(lzmafp))
          return (SDDS_dataset->page_number = -1);
        SDDS_SetError("Unable to read page--failure reading number of rows (SDDS_ReadNonNativeBinaryPage)");
        return (0);
      }
      SDDS_SwapLong(&n_rows32);
      if (n_rows32 == INT32_MIN) {
        if (!SDDS_LZMABufferedRead(&n_rows, sizeof(n_rows), lzmafp, &SDDS_dataset->fBuffer, SDDS_LONG64, SDDS_dataset->layout.byteOrderDeclared)) {
          if (lzma_eof(lzmafp))
            return (SDDS_dataset->page_number = -1);
          SDDS_SetError("Unable to read page--failure reading number of rows (SDDS_ReadNonNativeBinaryPage)");
          return (0);
        }
        SDDS_SwapLong64(&n_rows);
      } else {
        n_rows = n_rows32;
      }
    } else {
      SDDS_dataset->rowcount_offset = ftell(fp);
      if (!SDDS_BufferedRead(&n_rows32, sizeof(n_rows32), fp, &SDDS_dataset->fBuffer, SDDS_LONG, SDDS_dataset->layout.byteOrderDeclared)) {
        if (feof(fp))
          return (SDDS_dataset->page_number = -1);
        SDDS_SetError("Unable to read page--failure reading number of rows (SDDS_ReadNonNativeBinaryPage)");
        return (0);
      }
      SDDS_SwapLong(&n_rows32);
      if (n_rows32 == INT32_MIN) {
        if (!SDDS_BufferedRead(&n_rows, sizeof(n_rows), fp, &SDDS_dataset->fBuffer, SDDS_LONG64, SDDS_dataset->layout.byteOrderDeclared)) {
          if (feof(fp))
            return (SDDS_dataset->page_number = -1);
          SDDS_SetError("Unable to read page--failure reading number of rows (SDDS_ReadNonNativeBinaryPage)");
          return (0);
        }
        SDDS_SwapLong64(&n_rows);
      } else {
        n_rows = n_rows32;
      }
    }
#if defined(zLib)
  }
#endif
  if (n_rows < 0) {
    SDDS_SetError("Unable to read page--negative number of rows (SDDS_ReadNonNativeBinaryPage)");
    return (0);
  }
  if (last_rows < 0)
    last_rows = 0;
  /* Fix this limitation later */
  if (SDDS_dataset->layout.data_mode.column_major) {
    sparse_interval = 1;
    sparse_offset = 0;
    last_rows = 0;
  }
  if (last_rows) {
    sparse_interval = 1;
    sparse_offset = n_rows - last_rows;
    rows_to_store = last_rows + 2;
    alloc_rows = rows_to_store - SDDS_dataset->n_rows_allocated;
  }
  if (sparse_interval <= 0)
    sparse_interval = 1;
  if (sparse_offset < 0)
    sparse_offset = 0;
 
  rows_to_store = (n_rows - sparse_offset) / sparse_interval + 2;
  alloc_rows = rows_to_store - SDDS_dataset->n_rows_allocated;
  if (!SDDS_StartPage(SDDS_dataset, 0) || !SDDS_LengthenTable(SDDS_dataset, alloc_rows)) {
    SDDS_SetError("Unable to read page--couldn't start page (SDDS_ReadNonNativeBinaryPage)");
    return (0);
  }
 
  /* read the parameter values */
  if (!SDDS_ReadNonNativeBinaryParameters(SDDS_dataset)) {
    SDDS_SetError("Unable to read page--parameter reading error (SDDS_ReadNonNativeBinaryPage)");
    return (0);
  }
 
  /* read the array values */
  if (!SDDS_ReadNonNativeBinaryArrays(SDDS_dataset)) {
    SDDS_SetError("Unable to read page--array reading error (SDDS_ReadNonNativeBinaryPage)");
    return (0);
  }
  if (SDDS_dataset->layout.data_mode.column_major) {
    SDDS_dataset->n_rows = n_rows;
    if (!SDDS_ReadNonNativeBinaryColumns(SDDS_dataset)) {
      SDDS_SetError("Unable to read page--column reading error (SDDS_ReadNonNativeBinaryPage)");
      return (0);
    }
    SDDS_SwapEndsColumnData(SDDS_dataset);
    return (SDDS_dataset->page_number);
  }
  if ((sparse_interval <= 1) && (sparse_offset == 0)) {
    for (j = 0; j < n_rows; j++) {
      if (!SDDS_ReadNonNativeBinaryRow(SDDS_dataset, j, 0)) {
        SDDS_dataset->n_rows = j - 1;
        if (SDDS_dataset->autoRecover) {
          SDDS_ClearErrors();
          SDDS_SwapEndsColumnData(SDDS_dataset);
          return (SDDS_dataset->page_number);
        }
        SDDS_SetError("Unable to read page--error reading data row (SDDS_ReadNonNativeBinaryPage)");
        SDDS_SetReadRecoveryMode(SDDS_dataset, 1);
        return (0);
      }
    }
    SDDS_dataset->n_rows = j;
    SDDS_SwapEndsColumnData(SDDS_dataset);
    return (SDDS_dataset->page_number);
  } else {
    for (j = 0; j < sparse_offset; j++) {
      if (!SDDS_ReadNonNativeBinaryRow(SDDS_dataset, 0, 1)) {
        SDDS_dataset->n_rows = 0;
        if (SDDS_dataset->autoRecover) {
          SDDS_ClearErrors();
          SDDS_SwapEndsColumnData(SDDS_dataset);
          return (SDDS_dataset->page_number);
        }
        SDDS_SetError("Unable to read page--error reading data row (SDDS_ReadNonNativeBinaryPage)");
        SDDS_SetReadRecoveryMode(SDDS_dataset, 1);
        return (0);
      }
    }
    n_rows -= sparse_offset;
    for (j = k = 0; j < n_rows; j++) {
      if (!SDDS_ReadNonNativeBinaryRow(SDDS_dataset, k, mod = j % sparse_interval)) {
        SDDS_dataset->n_rows = k - 1;
        if (SDDS_dataset->autoRecover) {
          SDDS_ClearErrors();
          SDDS_SwapEndsColumnData(SDDS_dataset);
          return (SDDS_dataset->page_number);
        }
        SDDS_SetError("Unable to read page--error reading data row (SDDS_ReadNonNativeBinaryPage)");
        SDDS_SetReadRecoveryMode(SDDS_dataset, 1);
        return (0);
      }
      k += mod ? 0 : 1;
    }
    SDDS_dataset->n_rows = k;
    SDDS_SwapEndsColumnData(SDDS_dataset);
    return (SDDS_dataset->page_number);
  }
}

SDDS_ReadNonNativeBinaryPageLastRows()

int32_t SDDS_ReadNonNativeBinaryPageLastRows ( SDDS_DATASET * SDDS_dataset, int64_t last_rows )

extern

Reads the last few rows from a non-native endian binary page in an SDDS dataset.

This function reads the specified number of last rows from a binary page in the given SDDS dataset, handling data with non-native endianness. It performs necessary byte order conversions to ensure correct data interpretation on the host system.

Parameters

[in,out]	SDDS_dataset	Pointer to the SDDS_DATASET structure representing the dataset to read from.
[in]	last_rows	Number of last rows to read from the dataset.

Returns

int32_t Returns the number of rows read on success, or 0 on failure.

Return values

>0	Number of rows successfully read.
0	An error occurred during the read operation, such as I/O failures, data corruption, or unsupported data modes.

Note

This function is a wrapper for SDDS_ReadNonNativeBinaryPageDetailed with specific parameters to read the last few rows. It should be used when only the most recent rows are needed.

Definition at line 4334 of file SDDS_binary.c.

                                                                                            {
  return SDDS_ReadNonNativeBinaryPageDetailed(SDDS_dataset, 1, 0, last_rows);
}

SDDS_ReadNonNativeLZMABinaryString()

char * SDDS_ReadNonNativeLZMABinaryString	(	struct lzmafile *	lzmafp,
		SDDS_FILEBUFFER *	fBuffer,
		int32_t	skip )

Reads a non-native endian binary string from an LZMA-compressed file.

This function reads a binary string from the specified LZMA-compressed file pointer, handling non-native endianness. It first reads the length of the string, swaps its byte order if necessary, allocates memory for the string, reads the string data, and null-terminates it.

Parameters

[in]	lzmafp	Pointer to the LZMAFILE from which to read the string.
[in,out]	fBuffer	Pointer to the SDDS_FILEBUFFER structure used for buffered reading.
[in]	skip	If non-zero, the function will skip reading the string data, useful for sparse reading.

Returns

char* Returns a pointer to the read string on success, or NULL if an error occurred.

Return values

Non-NULL	Pointer to the newly allocated string.
NULL	An error occurred during reading or memory allocation.

Note

The caller is responsible for freeing the returned string to prevent memory leaks.

Definition at line 5035 of file SDDS_binary.c.

                                                                                                          {
  int32_t length;
  char *string;
 
  if (!SDDS_LZMABufferedRead(&length, sizeof(length), lzmafp, fBuffer, SDDS_LONG, 0))
    return (0);
  SDDS_SwapLong(&length);
  if (length < 0)
    return (0);
  if (!(string = SDDS_Malloc(sizeof(*string) * (length + 1))))
    return (NULL);
  if (length && !SDDS_LZMABufferedRead(skip ? NULL : string, sizeof(*string) * length, lzmafp, fBuffer, SDDS_STRING, 0))
    return (NULL);
  string[length] = 0;
  return (string);
}

SDDS_ReadNonNativePageDetailed()

int32_t SDDS_ReadNonNativePageDetailed ( SDDS_DATASET * SDDS_dataset, uint32_t mode, int64_t sparse_interval, int64_t sparse_offset, int64_t last_rows )

extern

Reads a detailed non-native endian page from an SDDS dataset.

This function reads a page of data from the specified SDDS dataset, handling data with non-native endianness. It supports both ASCII and binary data modes, performing necessary byte order conversions to ensure correct data interpretation on the host system. Additionally, it allows for sparse reading and reading of the last few rows based on the provided parameters.

Parameters

[in,out]	SDDS_dataset	Pointer to the SDDS_DATASET structure representing the dataset to read from.
[in]	mode	Mode flag to support future expansion.
[in]	sparse_interval	Interval between rows to be read for sparsity.
[in]	sparse_offset	Offset to start reading rows for sparsity.
[in]	last_rows	Number of last rows to read from the dataset.

Returns

int32_t Returns the number of rows read on success, or 0 on failure.

Return values

>0	Number of rows successfully read.
0	An error occurred during the read operation, such as I/O failures, data corruption, or unsupported data modes.

Note

This function handles various compression formats, including uncompressed, LZMA-compressed, and GZIP-compressed files. It manages memory allocation for parameters, arrays, and columns, ensuring that data is correctly stored and byte-swapped as necessary.

Definition at line 4182 of file SDDS_binary.c.

{
  int32_t retval;
  /*  SDDS_LAYOUT layout_copy; */
 
  if (!SDDS_CheckDataset(SDDS_dataset, "SDDS_ReadNonNativePageDetailed"))
    return (0);
  if (SDDS_dataset->layout.disconnected) {
    SDDS_SetError("Can't read page--file is disconnected (SDDS_ReadNonNativePageDetailed)");
    return 0;
  }
#if defined(zLib)
  if (SDDS_dataset->layout.gzipFile) {
    if (!SDDS_dataset->layout.gzfp) {
      SDDS_SetError("Unable to read page--NULL file pointer (SDDS_ReadNonNativePageDetailed)");
      return (0);
    }
  } else {
#endif
    if (SDDS_dataset->layout.lzmaFile) {
      if (!SDDS_dataset->layout.lzmafp) {
        SDDS_SetError("Unable to read page--NULL file pointer (SDDS_ReadNonNativePageDetailed)");
        return (0);
      }
    } else {
      if (!SDDS_dataset->layout.fp) {
        SDDS_SetError("Unable to read page--NULL file pointer (SDDS_ReadNonNativePageDetailed)");
        return (0);
      }
    }
#if defined(zLib)
  }
#endif
  if (SDDS_dataset->original_layout.data_mode.mode == SDDS_ASCII) {
    if ((retval = SDDS_ReadAsciiPage(SDDS_dataset, sparse_interval, sparse_offset, 0)) < 1) {
      return (retval);
    }
  } else if (SDDS_dataset->original_layout.data_mode.mode == SDDS_BINARY) {
    if ((retval = SDDS_ReadNonNativeBinaryPage(SDDS_dataset, sparse_interval, sparse_offset)) < 1) {
      return (retval);
    }
  } else {
    SDDS_SetError("Unable to read page--unrecognized data mode (SDDS_ReadNonNativePageDetailed)");
    return (0);
  }
  return (retval);
}

SDDS_ReadNonNativePageLastRows()

int32_t SDDS_ReadNonNativePageLastRows ( SDDS_DATASET * SDDS_dataset, int64_t last_rows )

extern

Reads the last few rows from a non-native endian page in an SDDS dataset.

This function reads the specified number of last rows from the non-native endian page of the given SDDS dataset. It handles data with non-native endianness, performing necessary byte order conversions to ensure correct data interpretation on the host system.

Parameters

[in,out]	SDDS_dataset	Pointer to the SDDS_DATASET structure representing the dataset to read from.
[in]	last_rows	Number of last rows to read from the dataset.

Returns

int32_t Returns the number of rows read on success, or 0 on failure.

Return values

>0	Number of rows successfully read.
0	An error occurred during the read operation, such as I/O failures, data corruption, or unsupported data modes.

Note

This function is a wrapper for SDDS_ReadNonNativePageDetailed with specific parameters to read the last few rows. It should be used when only the most recent rows are needed.

Definition at line 4249 of file SDDS_binary.c.

                                                                                      {
  int32_t retval;
  if (!SDDS_CheckDataset(SDDS_dataset, "SDDS_ReadNonNativePageLastRows"))
    return (0);
  if (SDDS_dataset->layout.disconnected) {
    SDDS_SetError("Can't read page--file is disconnected (SDDS_ReadNonNativePageLastRows)");
    return 0;
  }
#if defined(zLib)
  if (SDDS_dataset->layout.gzipFile) {
    if (!SDDS_dataset->layout.gzfp) {
      SDDS_SetError("Unable to read page--NULL file pointer (SDDS_ReadNonNativePageLastRows)");
      return (0);
    }
  } else {
#endif
    if (SDDS_dataset->layout.lzmaFile) {
      if (!SDDS_dataset->layout.lzmafp) {
        SDDS_SetError("Unable to read page--NULL file pointer (SDDS_ReadNonNativePageLastRows)");
        return (0);
      }
    } else {
      if (!SDDS_dataset->layout.fp) {
        SDDS_SetError("Unable to read page--NULL file pointer (SDDS_ReadNonNativePageLastRows)");
        return (0);
      }
    }
#if defined(zLib)
  }
#endif
  if (SDDS_dataset->original_layout.data_mode.mode == SDDS_ASCII) {
    if ((retval = SDDS_ReadAsciiPageLastRows(SDDS_dataset, last_rows)) < 1) {
      return (retval);
    }
  } else if (SDDS_dataset->original_layout.data_mode.mode == SDDS_BINARY) {
    if ((retval = SDDS_ReadNonNativeBinaryPageLastRows(SDDS_dataset, last_rows)) < 1) {
      return (retval);
    }
  } else {
    SDDS_SetError("Unable to read page--unrecognized data mode (SDDS_ReadNonNativePageLastRows)");
    return (0);
  }
  return (retval);
}

SDDS_ResetSpecialCommentsModes()

void SDDS_ResetSpecialCommentsModes

(

SDDS_DATASET *

SDDS_dataset

)

Resets the special comments modes in the SDDS dataset.

This function clears all special comment flags that have been set within the dataset. After calling this function, the dataset will no longer have any special configurations related to comments.

Parameters

[in,out]

SDDS_dataset

Pointer to the SDDS_DATASET structure representing the dataset. The commentFlags field will be reset to 0.

Note

• Use this function to clear all special configurations before setting new ones or when resetting the dataset's state.

Warning

• This operation cannot be undone. Ensure that you no longer require the existing special comment configurations before resetting.

See also

SDDS_GetSpecialCommentsModes, SDDS_ParseSpecialComments

Definition at line 5312 of file SDDS_utils.c.

                                                                {
  SDDS_dataset->layout.commentFlags = 0;
}

SDDS_TransferRow()

int32_t SDDS_TransferRow ( SDDS_DATASET * SDDS_dataset, int64_t target, int64_t source )

extern

Transfers data from a source row to a target row within an SDDS dataset.

This function copies all column data from the specified source row to the target row in the provided SDDS dataset. It handles both string and non-string data types appropriately, ensuring that memory is managed correctly for string entries.

Parameters

SDDS_dataset	Pointer to the SDDS_DATASET structure representing the dataset.
target	The index of the target row where data will be copied to.
source	The index of the source row from which data will be copied.

Returns

On success, returns 1. On failure, returns 0 and sets an appropriate error message.

Return values

1	Indicates that the row transfer was successful.
0	Indicates that an error occurred (e.g., invalid dataset, out-of-range indices, memory allocation failure, string copy failure).

Note

• Both target and source must be valid row indices within the dataset.
• The function does not allocate or deallocate rows; it only copies data between existing rows.

See also

SDDS_DeleteUnsetRows, SDDS_CopyColumn

Definition at line 3803 of file SDDS_extract.c.

                                                                                     {
  int32_t size;
  int64_t i;
  if (!SDDS_CheckDataset(SDDS_dataset, "SDDS_TransferRow"))
    return (0);
  for (i = 0; i < SDDS_dataset->layout.n_columns; i++) {
    if (SDDS_dataset->layout.column_definition[i].type != SDDS_STRING) {
      size = SDDS_type_size[SDDS_dataset->layout.column_definition[i].type - 1];
      memcpy((char *)SDDS_dataset->data[i] + target * size, (char *)SDDS_dataset->data[i] + source * size, size);
    } else {
      if (((char ***)SDDS_dataset->data)[i][target])
        free(((char ***)SDDS_dataset->data)[i][target]);
      ((char ***)SDDS_dataset->data)[i][target] = NULL;
      if (!SDDS_CopyString(((char ***)SDDS_dataset->data)[i] + target, ((char ***)SDDS_dataset->data)[i][source]))
        return ((int32_t)0);
    }
  }
  return (1);
}

SDDS_UpdateAsciiPage()

int32_t SDDS_UpdateAsciiPage ( SDDS_DATASET * SDDS_dataset, uint32_t mode )

extern

Updates the current ASCII page of an SDDS dataset with new data.

This function updates the ASCII page of the provided SDDS dataset, appending any new rows that have been added since the last write. It handles updating the row count in the file and ensures data consistency. If the dataset is not currently writing a page, it initiates a new page write.

Parameters

SDDS_dataset	Pointer to the SDDS dataset to update.
mode	Mode flags that control the update behavior. Common modes include: FLUSH_TABLE: Flushes the data table after writing.

Returns

Returns 1 on success, or 0 on error. If an error occurs, an error message is set via SDDS_SetError().

Note

This function cannot be used with compressed files (GZIP or LZMA). It requires a regular ASCII file. It also handles updating internal state variables such as the number of rows written and the last row written.

Definition at line 2167 of file SDDS_ascii.c.

                                                                        {
  FILE *fp;
  int32_t code;
  int64_t i, rows, offset;
  SDDS_FILEBUFFER *fBuffer;
 
#ifdef DEBUG
  fprintf(stderr, "%" PRId64 " virtual rows present, first=%" PRId32 "\n", SDDS_CountRowsOfInterest(SDDS_dataset), SDDS_dataset->first_row_in_mem);
#endif
  if (!SDDS_CheckDataset(SDDS_dataset, "SDDS_UpdateAsciiPage"))
    return (0);
#if defined(zLib)
  if (SDDS_dataset->layout.gzipFile) {
    SDDS_SetError("Unable to perform page updates on a gzip file (SDDS_UpdateAsciiPage)");
    return 0;
  }
#endif
  if (SDDS_dataset->layout.lzmaFile) {
    SDDS_SetError("Unable to perform page updates on an .lzma or .xz file (SDDS_UpdateAsciiPage)");
    return 0;
  }
  if (!SDDS_dataset->writing_page) {
    if (!(code = SDDS_WriteAsciiPage(SDDS_dataset)))
      return 0;
    if (mode & FLUSH_TABLE) {
      SDDS_FreeTableStrings(SDDS_dataset);
      SDDS_dataset->first_row_in_mem = SDDS_CountRowsOfInterest(SDDS_dataset);
      SDDS_dataset->last_row_written = -1;
      SDDS_dataset->n_rows = 0;
    }
    return code;
  }
  if (!(fp = SDDS_dataset->layout.fp)) {
    SDDS_SetError("Unable to update page--file pointer is NULL (SDDS_UpdateAsciiPage)");
    return (0);
  }
  fBuffer = &SDDS_dataset->fBuffer;
  if (!SDDS_FlushBuffer(fp, fBuffer)) {
    SDDS_SetError("Unable to write page--buffer flushing problem (SDDS_UpdateAsciiPage)");
    return 0;
  }
  offset = ftell(fp);
 
  rows = SDDS_CountRowsOfInterest(SDDS_dataset) + SDDS_dataset->first_row_in_mem;
  if (rows == SDDS_dataset->n_rows_written)
    return (1);
  if (rows < SDDS_dataset->n_rows_written) {
    SDDS_SetError("Unable to update page--new number of rows less than previous number (SDDS_UpdateAsciiPage)");
    return (0);
  }
  if ((!SDDS_dataset->layout.data_mode.fixed_row_count) || (((rows + rows - SDDS_dataset->n_rows_written) / SDDS_dataset->layout.data_mode.fixed_row_increment) != (rows / SDDS_dataset->layout.data_mode.fixed_row_increment))) {
    if (!SDDS_dataset->layout.data_mode.no_row_counts) {
      if (SDDS_fseek(fp, SDDS_dataset->rowcount_offset, SEEK_SET) == -1) {
        SDDS_SetError("Unable to update page--failure doing fseek (SDDS_UpdateAsciiPage)");
        return (0);
      }
      /* overwrite the existing row count */
      if (SDDS_dataset->layout.data_mode.fixed_row_count) {
        if ((rows - SDDS_dataset->n_rows_written) + 1 > SDDS_dataset->layout.data_mode.fixed_row_increment) {
          SDDS_dataset->layout.data_mode.fixed_row_increment = (rows - SDDS_dataset->n_rows_written) + 1;
        }
        fprintf(fp, "%20" PRId64 "\n", ((rows / SDDS_dataset->layout.data_mode.fixed_row_increment) + 2) * SDDS_dataset->layout.data_mode.fixed_row_increment);
      } else
        fprintf(fp, "%20" PRId64 "\n", rows);
      if (SDDS_fseek(fp, offset, SEEK_SET) == -1) {
        SDDS_SetError("Unable to update page--failure doing fseek to end of page (SDDS_UpdateAsciiPage)");
        return (0);
      }
    }
  }
  for (i = SDDS_dataset->last_row_written + 1; i < SDDS_dataset->n_rows; i++)
    if (SDDS_dataset->row_flag[i])
      SDDS_WriteAsciiRow(SDDS_dataset, i, fp);
  if (!SDDS_FlushBuffer(fp, fBuffer)) {
    SDDS_SetError("Unable to write page--buffer flushing problem (SDDS_UpdateAsciiPage)");
    return 0;
  }
  SDDS_dataset->last_row_written = SDDS_dataset->n_rows - 1;
  SDDS_dataset->n_rows_written = rows;
  if (mode & FLUSH_TABLE) {
    SDDS_FreeTableStrings(SDDS_dataset);
    SDDS_dataset->first_row_in_mem = rows;
    SDDS_dataset->last_row_written = -1;
    SDDS_dataset->n_rows = 0;
  }
  return (1);
}

SDDS_UpdateBinaryPage()

int32_t SDDS_UpdateBinaryPage ( SDDS_DATASET * SDDS_dataset, uint32_t mode )

extern

Updates the binary page of an SDDS dataset.

This function updates the binary page of the specified SDDS dataset based on the provided mode. It handles writing the dataset's binary data to the associated file, managing buffering, and handling different file formats such as gzip and LZMA if applicable.

Parameters

SDDS_dataset	Pointer to the SDDS_DATASET structure representing the dataset to update.
mode	Bitmask indicating the update mode. It can be: 0 for a standard update. FLUSH_TABLE to flush the table after updating.

Returns

• Returns 1 on successful update.
• Returns 0 if an error occurs during the update process.

The function performs several checks before updating:

• Checks the environment variable SDDS_OUTPUT_ENDIANESS to determine if a non-native binary update is required.
• Validates the dataset structure.
• Ensures that the dataset is not using gzip or LZMA compression, or is not in column-major data mode.
• Handles writing the binary page, updating row counts, and managing buffer flushing.

Note

• The function is not thread-safe and should be called in a synchronized context.
• Requires that the dataset has been properly initialized and populated with data.

Definition at line 1098 of file SDDS_binary.c.

                                                                         {
  FILE *fp;
  int64_t i, rows, offset, code, fixed_rows;
  int32_t min32 = INT32_MIN, rows32;
  SDDS_FILEBUFFER *fBuffer;
  char *outputEndianess = NULL;
 
  if ((outputEndianess = getenv("SDDS_OUTPUT_ENDIANESS"))) {
    if (((strncmp(outputEndianess, "big", 3) == 0) && (SDDS_IsBigEndianMachine() == 0)) || ((strncmp(outputEndianess, "little", 6) == 0) && (SDDS_IsBigEndianMachine() == 1)))
      return SDDS_UpdateNonNativeBinaryPage(SDDS_dataset, mode);
  }
 
#ifdef DEBUG
  fprintf(stderr, "%" PRId64 " virtual rows present, first=%" PRId64 "\n", SDDS_CountRowsOfInterest(SDDS_dataset), SDDS_dataset->first_row_in_mem);
#endif
  if (!SDDS_CheckDataset(SDDS_dataset, "SDDS_UpdateBinaryPage"))
    return (0);
#if defined(zLib)
  if (SDDS_dataset->layout.gzipFile) {
    SDDS_SetError("Unable to perform page updates on a gzip file (SDDS_UpdateBinaryPage)");
    return 0;
  }
#endif
  if (SDDS_dataset->layout.lzmaFile) {
    SDDS_SetError("Unable to perform page updates on an .lzma or .xz file (SDDS_UpdateBinaryPage)");
    return 0;
  }
  if (SDDS_dataset->layout.data_mode.column_major) {
    SDDS_SetError("Unable to perform page updates on column major order file. (SDDS_UpdateBinaryPage)");
    return 0;
  }
  if (!SDDS_dataset->writing_page) {
#ifdef DEBUG
    fprintf(stderr, "Page not being written---calling SDDS_UpdateBinaryPage\n");
#endif
    if (!(code = SDDS_WriteBinaryPage(SDDS_dataset)))
      return 0;
    if (mode & FLUSH_TABLE) {
      SDDS_FreeTableStrings(SDDS_dataset);
      SDDS_dataset->first_row_in_mem = SDDS_CountRowsOfInterest(SDDS_dataset);
      SDDS_dataset->last_row_written = -1;
      SDDS_dataset->n_rows = 0;
    }
    return code;
  }
 
  if (!(fp = SDDS_dataset->layout.fp)) {
    SDDS_SetError("Unable to update page--file pointer is NULL (SDDS_UpdateBinaryPage)");
    return (0);
  }
  fBuffer = &SDDS_dataset->fBuffer;
  if (!SDDS_FlushBuffer(fp, fBuffer)) {
    SDDS_SetError("Unable to write page--buffer flushing problem (SDDS_UpdateBinaryPage)");
    return 0;
  }
  offset = ftell(fp);
 
  rows = SDDS_CountRowsOfInterest(SDDS_dataset) + SDDS_dataset->first_row_in_mem;
#ifdef DEBUG
  fprintf(stderr, "%" PRId64 " rows stored in table, %" PRId64 " already written\n", rows, SDDS_dataset->n_rows_written);
#endif
  if (rows == SDDS_dataset->n_rows_written)
    return (1);
  if (rows < SDDS_dataset->n_rows_written) {
    SDDS_SetError("Unable to update page--new number of rows less than previous number (SDDS_UpdateBinaryPage)");
    return (0);
  }
  if ((!SDDS_dataset->layout.data_mode.fixed_row_count) || (((rows + rows - SDDS_dataset->n_rows_written) / SDDS_dataset->layout.data_mode.fixed_row_increment) != (rows / SDDS_dataset->layout.data_mode.fixed_row_increment))) {
    if (SDDS_fseek(fp, SDDS_dataset->rowcount_offset, 0) == -1) {
      SDDS_SetError("Unable to update page--failure doing fseek (SDDS_UpdateBinaryPage)");
      return (0);
    }
    if (SDDS_dataset->layout.data_mode.fixed_row_count) {
      if ((rows - SDDS_dataset->n_rows_written) + 1 > SDDS_dataset->layout.data_mode.fixed_row_increment) {
        SDDS_dataset->layout.data_mode.fixed_row_increment = (rows - SDDS_dataset->n_rows_written) + 1;
      }
      fixed_rows = ((rows / SDDS_dataset->layout.data_mode.fixed_row_increment) + 2) * SDDS_dataset->layout.data_mode.fixed_row_increment;
#if defined(DEBUG)
      fprintf(stderr, "Setting %" PRId64 " fixed rows\n", fixed_rows);
#endif
      if ((fixed_rows > INT32_MAX) && (SDDS_dataset->n_rows_written <= INT32_MAX)) {
        SDDS_SetError("Unable to update page--crossed the INT32_MAX row boundary (SDDS_UpdateBinaryPage)");
        return (0);
      }
      if (fixed_rows > INT32_MAX) {
        if (fwrite(&min32, sizeof(min32), 1, fp) != 1) {
          SDDS_SetError("Unable to update page--failure writing number of rows (SDDS_UpdateBinaryPage)");
          return (0);
        }
        if (fwrite(&fixed_rows, sizeof(fixed_rows), 1, fp) != 1) {
          SDDS_SetError("Unable to update page--failure writing number of rows (SDDS_UpdateBinaryPage)");
          return (0);
        }
      } else {
        rows32 = (int32_t)fixed_rows;
        if (fwrite(&rows32, sizeof(rows32), 1, fp) != 1) {
          SDDS_SetError("Unable to update page--failure writing number of rows (SDDS_UpdateBinaryPage)");
          return (0);
        }
      }
    } else {
#if defined(DEBUG)
      fprintf(stderr, "Setting %" PRId64 " rows\n", rows);
#endif
      if ((rows > INT32_MAX) && (SDDS_dataset->n_rows_written <= INT32_MAX)) {
        SDDS_SetError("Unable to update page--crossed the INT32_MAX row boundary (SDDS_UpdateBinaryPage)");
        return (0);
      }
      if (rows > INT32_MAX) {
        if (fwrite(&min32, sizeof(min32), 1, fp) != 1) {
          SDDS_SetError("Unable to update page--failure writing number of rows (SDDS_UpdateBinaryPage)");
          return (0);
        }
        if (fwrite(&rows, sizeof(rows), 1, fp) != 1) {
          SDDS_SetError("Unable to update page--failure writing number of rows (SDDS_UpdateBinaryPage)");
          return (0);
        }
      } else {
        rows32 = (int32_t)rows;
        if (fwrite(&rows32, sizeof(rows32), 1, fp) != 1) {
          SDDS_SetError("Unable to update page--failure writing number of rows (SDDS_UpdateBinaryPage)");
          return (0);
        }
      }
    }
    if (SDDS_fseek(fp, offset, 0) == -1) {
      SDDS_SetError("Unable to update page--failure doing fseek to end of page (SDDS_UpdateBinaryPage)");
      return (0);
    }
  }
  for (i = SDDS_dataset->last_row_written + 1; i < SDDS_dataset->n_rows; i++)
    if (SDDS_dataset->row_flag[i] && !SDDS_WriteBinaryRow(SDDS_dataset, i)) {
      SDDS_SetError("Unable to update page--failure writing row (SDDS_UpdateBinaryPage)");
      return (0);
    }
#ifdef DEBUG
  fprintf(stderr, "Flushing buffer\n");
#endif
  if (!SDDS_FlushBuffer(fp, fBuffer)) {
    SDDS_SetError("Unable to write page--buffer flushing problem (SDDS_UpdateBinaryPage)");
    return 0;
  }
  SDDS_dataset->last_row_written = SDDS_dataset->n_rows - 1;
  SDDS_dataset->n_rows_written = rows;
  if (mode & FLUSH_TABLE) {
    SDDS_FreeTableStrings(SDDS_dataset);
    SDDS_dataset->first_row_in_mem = rows;
    SDDS_dataset->last_row_written = -1;
    SDDS_dataset->n_rows = 0;
  }
  return (1);
}

SDDS_UpdateNonNativeBinaryPage()

int32_t SDDS_UpdateNonNativeBinaryPage ( SDDS_DATASET * SDDS_dataset, uint32_t mode )

extern

Updates a non-native endian binary page in an SDDS dataset.

This function updates an existing binary page in the specified SDDS dataset, handling byte order reversal to convert between little-endian and big-endian formats. It supports updating rows based on the provided mode flags, such as flushing the table. The function ensures that the buffer is flushed before performing the update and writes any new rows that have been flagged for writing. It also handles fixed row counts and manages byte order conversions as necessary.

Parameters

[in,out]	SDDS_dataset	Pointer to the SDDS_DATASET structure representing the dataset to update.
[in]	mode	Bitmask indicating the update mode (e.g., FLUSH_TABLE).

Returns

int32_t Returns 1 on successful update of the binary page, or 0 if an error occurred.

Return values

1	The binary page was successfully updated and byte-swapped.
0	An error occurred during the update operation, such as I/O failures, invalid row counts, or corrupted dataset definitions.

Note

This function modifies the dataset's internal structures during the update process. Ensure that the dataset is properly initialized and opened for writing before invoking this function. After updating, the dataset's state is updated to reflect the changes made to the page.

Definition at line 5844 of file SDDS_binary.c.

                                                                                  {
  FILE *fp;
  int32_t code, min32 = INT32_MIN, rows32;
  int64_t i, rows, offset, fixed_rows;
  SDDS_FILEBUFFER *fBuffer;
 
  if (!SDDS_CheckDataset(SDDS_dataset, "SDDS_UpdateNonNativeBinaryPage"))
    return (0);
#if defined(zLib)
  if (SDDS_dataset->layout.gzipFile) {
    SDDS_SetError("Unable to perform page updates on a gzip file (SDDS_UpdateNonNativeBinaryPage)");
    return 0;
  }
#endif
  if (SDDS_dataset->layout.lzmaFile) {
    SDDS_SetError("Unable to perform page updates on .lzma or .xz files (SDDS_UpdateNonNativeBinaryPage)");
    return 0;
  }
  if (SDDS_dataset->layout.data_mode.column_major) {
    SDDS_SetError("Unable to perform page updates on a column major order file (SDDS_UpdateNonNativeBinaryPage)");
    return 0;
  }
  if (!SDDS_dataset->writing_page) {
#ifdef DEBUG
    fprintf(stderr, "Page not being written---calling SDDS_UpdateNonNativeBinaryPage\n");
#endif
    if (!(code = SDDS_WriteNonNativeBinaryPage(SDDS_dataset))) {
      return 0;
    }
    if (mode & FLUSH_TABLE) {
      SDDS_FreeTableStrings(SDDS_dataset);
      SDDS_dataset->first_row_in_mem = SDDS_CountRowsOfInterest(SDDS_dataset);
      SDDS_dataset->last_row_written = -1;
      SDDS_dataset->n_rows = 0;
    }
    return code;
  }
 
  if (!(fp = SDDS_dataset->layout.fp)) {
    SDDS_SetError("Unable to update page--file pointer is NULL (SDDS_UpdateNonNativeBinaryPage)");
    return (0);
  }
  fBuffer = &SDDS_dataset->fBuffer;
  if (!SDDS_FlushBuffer(fp, fBuffer)) {
    SDDS_SetError("Unable to write page--buffer flushing problem (SDDS_UpdateNonNativeBinaryPage)");
    return 0;
  }
  offset = ftell(fp);
 
  rows = SDDS_CountRowsOfInterest(SDDS_dataset) + SDDS_dataset->first_row_in_mem;
#ifdef DEBUG
  fprintf(stderr, "%" PRId64 " rows stored in table, %" PRId32 " already written\n", rows, SDDS_dataset->n_rows_written);
#endif
  if (rows == SDDS_dataset->n_rows_written) {
    return (1);
  }
  if (rows < SDDS_dataset->n_rows_written) {
    SDDS_SetError("Unable to update page--new number of rows less than previous number (SDDS_UpdateNonNativeBinaryPage)");
    return (0);
  }
  SDDS_SwapLong(&min32);
  if ((!SDDS_dataset->layout.data_mode.fixed_row_count) || (((rows + rows - SDDS_dataset->n_rows_written) / SDDS_dataset->layout.data_mode.fixed_row_increment) != (rows / SDDS_dataset->layout.data_mode.fixed_row_increment))) {
    if (SDDS_fseek(fp, SDDS_dataset->rowcount_offset, 0) == -1) {
      SDDS_SetError("Unable to update page--failure doing fseek (SDDS_UpdateNonNativeBinaryPage)");
      return (0);
    }
    if (SDDS_dataset->layout.data_mode.fixed_row_count) {
      if ((rows - SDDS_dataset->n_rows_written) + 1 > SDDS_dataset->layout.data_mode.fixed_row_increment) {
        SDDS_dataset->layout.data_mode.fixed_row_increment = (rows - SDDS_dataset->n_rows_written) + 1;
      }
      fixed_rows = ((rows / SDDS_dataset->layout.data_mode.fixed_row_increment) + 2) * SDDS_dataset->layout.data_mode.fixed_row_increment;
#if defined(DEBUG)
      fprintf(stderr, "Setting %" PRId64 " fixed rows\n", fixed_rows);
#endif
      if ((fixed_rows > INT32_MAX) && (SDDS_dataset->n_rows_written <= INT32_MAX)) {
        SDDS_SetError("Unable to update page--crossed the INT32_MAX row boundary (SDDS_UpdateNonNativeBinaryPage)");
        return (0);
      }
      if (fixed_rows > INT32_MAX) {
        if (fwrite(&min32, sizeof(min32), 1, fp) != 1) {
          SDDS_SetError("Unable to update page--failure writing number of rows (SDDS_UpdateBinaryPage)");
          return (0);
        }
        SDDS_SwapLong64(&fixed_rows);
        if (fwrite(&fixed_rows, sizeof(fixed_rows), 1, fp) != 1) {
          SDDS_SetError("Unable to update page--failure writing number of rows (SDDS_UpdateNonNativeBinaryPage)");
          return (0);
        }
        SDDS_SwapLong64(&fixed_rows);
      } else {
        rows32 = (int32_t)fixed_rows;
        SDDS_SwapLong(&rows32);
        if (fwrite(&rows32, sizeof(rows32), 1, fp) != 1) {
          SDDS_SetError("Unable to update page--failure writing number of rows (SDDS_UpdateNonNativeBinaryPage)");
          return (0);
        }
      }
    } else {
#if defined(DEBUG)
      fprintf(stderr, "Setting %" PRId64 " rows\n", rows);
#endif
      if ((rows > INT32_MAX) && (SDDS_dataset->n_rows_written <= INT32_MAX)) {
        SDDS_SetError("Unable to update page--crossed the INT32_MAX row boundary (SDDS_UpdateNonNativeBinaryPage)");
        return (0);
      }
      if (rows > INT32_MAX) {
        if (fwrite(&min32, sizeof(min32), 1, fp) != 1) {
          SDDS_SetError("Unable to update page--failure writing number of rows (SDDS_UpdateBinaryPage)");
          return (0);
        }
        SDDS_SwapLong64(&rows);
        if (fwrite(&rows, sizeof(rows), 1, fp) != 1) {
          SDDS_SetError("Unable to update page--failure writing number of rows (SDDS_UpdateNonNativeBinaryPage)");
          return (0);
        }
        SDDS_SwapLong64(&rows);
      } else {
        rows32 = (int32_t)rows;
        SDDS_SwapLong(&rows32);
        if (fwrite(&rows32, sizeof(rows32), 1, fp) != 1) {
          SDDS_SetError("Unable to update page--failure writing number of rows (SDDS_UpdateNonNativeBinaryPage)");
          return (0);
        }
      }
    }
    if (SDDS_fseek(fp, offset, 0) == -1) {
      SDDS_SetError("Unable to update page--failure doing fseek to end of page (SDDS_UpdateNonNativeBinaryPage)");
      return (0);
    }
  }
  SDDS_SwapEndsColumnData(SDDS_dataset);
  for (i = SDDS_dataset->last_row_written + 1; i < SDDS_dataset->n_rows; i++) {
    if (SDDS_dataset->row_flag[i] && !SDDS_WriteNonNativeBinaryRow(SDDS_dataset, i)) {
      SDDS_SetError("Unable to update page--failure writing row (SDDS_UpdateNonNativeBinaryPage)");
      return (0);
    }
  }
  SDDS_SwapEndsColumnData(SDDS_dataset);
#ifdef DEBUG
  fprintf(stderr, "Flushing buffer\n");
#endif
  if (!SDDS_FlushBuffer(fp, fBuffer)) {
    SDDS_SetError("Unable to write page--buffer flushing problem (SDDS_UpdateNonNativeBinaryPage)");
    return 0;
  }
  SDDS_dataset->last_row_written = SDDS_dataset->n_rows - 1;
  SDDS_dataset->n_rows_written = rows;
  if (mode & FLUSH_TABLE) {
    SDDS_FreeTableStrings(SDDS_dataset);
    SDDS_dataset->first_row_in_mem = rows;
    SDDS_dataset->last_row_written = -1;
    SDDS_dataset->n_rows = 0;
  }
  return (1);
}

SDDS_WriteArrayDefinition()

int32_t SDDS_WriteArrayDefinition ( ARRAY_DEFINITION * array_definition, FILE * fp )

extern

Writes an array definition to a standard file.

This function outputs the definition of a single array in the SDDS layout. It includes fields such as name, symbol, units, description, format string, group name, data type, and dimensions. The definition is enclosed within &array and &end tags.

Parameters

array_definition	Pointer to the array definition structure.
fp	The file pointer where the array definition will be written.

Returns

Returns 1 on success, 0 if the file pointer is NULL or the array type is invalid.

Definition at line 692 of file SDDS_write.c.

                                                                                {
  if (!fp || array_definition->type <= 0 || array_definition->type > SDDS_NUM_TYPES)
    return (0);
 
  fputs("&array ", fp);
  SDDS_PrintNamelistField(fp, "name", array_definition->name);
  SDDS_PrintNamelistField(fp, "symbol", SDDS_BlankToNull(array_definition->symbol));
  SDDS_PrintNamelistField(fp, "units", SDDS_BlankToNull(array_definition->units));
  SDDS_PrintNamelistField(fp, "description", SDDS_BlankToNull(array_definition->description));
  SDDS_PrintNamelistField(fp, "format_string", SDDS_BlankToNull(array_definition->format_string));
  SDDS_PrintNamelistField(fp, "group_name", SDDS_BlankToNull(array_definition->group_name));
  SDDS_PrintNamelistField(fp, "type", SDDS_type_name[array_definition->type - 1]);
  if (array_definition->dimensions != 1) /* 1 is default */
    fprintf(fp, "dimensions=%" PRId32 ", ", array_definition->dimensions);
  fputs(" &end\n", fp);
  return (1);
}

SDDS_WriteAsciiArrays()

int32_t SDDS_WriteAsciiArrays ( SDDS_DATASET * SDDS_dataset, FILE * fp )

extern

Writes the arrays of an SDDS dataset in ASCII format to a file.

This function writes all arrays contained in the provided SDDS dataset to the specified file pointer in ASCII format. For each array, it writes the dimensions, a description line, and the array elements formatted according to their data type.

Parameters

SDDS_dataset	Pointer to the SDDS dataset containing the arrays to be written.
fp	File pointer to the ASCII file where the array data will be written.

Returns

Returns 1 on success, or 0 on error. If an error occurs, an error message is set via SDDS_SetError().

Note

The function checks the dataset for consistency before writing. It uses SDDS_WriteTypedValue() to write each array element according to its data type. Each array element is written, with up to 6 elements per line.

Definition at line 635 of file SDDS_ascii.c.

                                                                    {
  int32_t i, j;
  SDDS_LAYOUT *layout;
  /*  char *predefined_format; */
  ARRAY_DEFINITION *array_definition;
  SDDS_ARRAY *array;
 
  if (!SDDS_CheckDataset(SDDS_dataset, "SDDS_WriteAsciiArrays"))
    return (0);
  layout = &SDDS_dataset->layout;
  for (j = 0; j < layout->n_arrays; j++) {
    array_definition = layout->array_definition + j;
    array = &SDDS_dataset->array[j];
    for (i = 0; i < array_definition->dimensions; i++)
      fprintf(fp, "%" PRId32 " ", array->dimension[i]);
    fprintf(fp, "          ! %" PRId32 "-dimensional array %s:\n", array_definition->dimensions, array_definition->name);
    for (i = 0; i < array->elements;) {
      if (!SDDS_WriteTypedValue(array->data, i, array_definition->type, NULL, fp)) {
        SDDS_SetError("Unable to write array--couldn't write ASCII data (SDDS_WriteAsciiArrays)");
        return (0);
      }
      i++;
      if (i % 6 == 0 || i == array->elements)
        fputc('\n', fp);
      else
        fputc(' ', fp);
    }
  }
  return (1);
}

SDDS_WriteAsciiPage()

int32_t SDDS_WriteAsciiPage ( SDDS_DATASET * SDDS_dataset )

extern

Writes a page of data in ASCII format to the SDDS dataset.

This function writes the current page of data from the SDDS dataset to an ASCII file. It handles writing to regular files, as well as LZMA and GZIP compressed files if enabled. The function writes the page number, parameters, arrays, and rows of data according to the SDDS format.

Parameters

SDDS_dataset

Pointer to the SDDS dataset containing the data to be written.

Returns

Returns 1 on success, or 0 on error. If an error occurs, an error message is set via SDDS_SetError().

Note

This function checks the dataset for consistency before writing. It also updates internal state variables such as the last row written and the number of rows written.

Definition at line 410 of file SDDS_ascii.c.

                                                        {
#if defined(zLib)
  gzFile gzfp;
#endif
  FILE *fp;
  struct lzmafile *lzmafp;
  int64_t i, rows;
 
  if (!SDDS_CheckDataset(SDDS_dataset, "SDDS_WriteAsciiPage"))
    return (0);
#if defined(zLib)
  if (SDDS_dataset->layout.gzipFile) {
    if (!(gzfp = SDDS_dataset->layout.gzfp)) {
      SDDS_SetError("Unable to write page--file pointer is NULL (SDDS_WriteAsciiPage)");
      return (0);
    }
    if (SDDS_dataset->layout.data_mode.no_row_counts && (SDDS_dataset->page_number > 1 || SDDS_dataset->file_had_data))
      gzputc(gzfp, '\n');
    gzprintf(gzfp, "! page number %" PRId32 "\n", SDDS_dataset->page_number);
 
    if (!SDDS_GZipWriteAsciiParameters(SDDS_dataset, gzfp) || !SDDS_GZipWriteAsciiArrays(SDDS_dataset, gzfp))
      return 0;
    rows = 0;
    if (SDDS_dataset->layout.n_columns) {
      rows = SDDS_CountRowsOfInterest(SDDS_dataset);
      if (!SDDS_dataset->layout.data_mode.no_row_counts) {
        SDDS_dataset->rowcount_offset = gztell(gzfp);
        if (SDDS_dataset->layout.data_mode.fixed_row_count) {
          gzprintf(gzfp, "%20" PRId64 "\n", ((rows / SDDS_dataset->layout.data_mode.fixed_row_increment) + 2) * SDDS_dataset->layout.data_mode.fixed_row_increment);
        } else
          gzprintf(gzfp, "%20" PRId64 "\n", rows);
      }
      for (i = 0; i < SDDS_dataset->n_rows; i++)
        if (SDDS_dataset->row_flag[i] && !SDDS_GZipWriteAsciiRow(SDDS_dataset, i, gzfp))
          return 0;
    }
    SDDS_dataset->last_row_written = SDDS_dataset->n_rows - 1;
    SDDS_dataset->n_rows_written = rows;
    SDDS_dataset->writing_page = 1;
    /*gzflush(gzfp, Z_FULL_FLUSH); */
  } else {
#endif
    if (SDDS_dataset->layout.lzmaFile) {
      if (!(lzmafp = SDDS_dataset->layout.lzmafp)) {
        SDDS_SetError("Unable to write page--file pointer is NULL (SDDS_WriteAsciiPage)");
        return (0);
      }
      if (SDDS_dataset->layout.data_mode.no_row_counts && (SDDS_dataset->page_number > 1 || SDDS_dataset->file_had_data))
        lzma_putc('\n', lzmafp);
      lzma_printf(lzmafp, "! page number %" PRId32 "\n", SDDS_dataset->page_number);
 
      if (!SDDS_LZMAWriteAsciiParameters(SDDS_dataset, lzmafp) || !SDDS_LZMAWriteAsciiArrays(SDDS_dataset, lzmafp))
        return 0;
      rows = 0;
      if (SDDS_dataset->layout.n_columns) {
        rows = SDDS_CountRowsOfInterest(SDDS_dataset);
        if (!SDDS_dataset->layout.data_mode.no_row_counts) {
          SDDS_dataset->rowcount_offset = lzma_tell(lzmafp);
          if (SDDS_dataset->layout.data_mode.fixed_row_count)
            lzma_printf(lzmafp, "%20" PRId64 "\n", ((rows / SDDS_dataset->layout.data_mode.fixed_row_increment) + 2) * SDDS_dataset->layout.data_mode.fixed_row_increment);
          else
            lzma_printf(lzmafp, "%20" PRId64 "\n", rows);
        }
        for (i = 0; i < SDDS_dataset->n_rows; i++)
          if (SDDS_dataset->row_flag[i] && !SDDS_LZMAWriteAsciiRow(SDDS_dataset, i, lzmafp))
            return 0;
      }
      SDDS_dataset->last_row_written = SDDS_dataset->n_rows - 1;
      SDDS_dataset->n_rows_written = rows;
      SDDS_dataset->writing_page = 1;
    } else {
      if (!(fp = SDDS_dataset->layout.fp)) {
        SDDS_SetError("Unable to write page--file pointer is NULL (SDDS_WriteAsciiPage)");
        return (0);
      }
      if (SDDS_dataset->layout.data_mode.no_row_counts && (SDDS_dataset->page_number > 1 || SDDS_dataset->file_had_data))
        fputc('\n', fp);
      fprintf(fp, "! page number %" PRId32 "\n", SDDS_dataset->page_number);
 
      if (!SDDS_WriteAsciiParameters(SDDS_dataset, fp) || !SDDS_WriteAsciiArrays(SDDS_dataset, fp))
        return 0;
      rows = 0;
      if (SDDS_dataset->layout.n_columns) {
        rows = SDDS_CountRowsOfInterest(SDDS_dataset);
        if (!SDDS_dataset->layout.data_mode.no_row_counts) {
          SDDS_dataset->rowcount_offset = ftell(fp);
          if (SDDS_dataset->layout.data_mode.fixed_row_count)
            fprintf(fp, "%20" PRId64 "\n", ((rows / SDDS_dataset->layout.data_mode.fixed_row_increment) + 2) * SDDS_dataset->layout.data_mode.fixed_row_increment);
          else
            fprintf(fp, "%20" PRId64 "\n", rows);
        }
        for (i = 0; i < SDDS_dataset->n_rows; i++)
          if (SDDS_dataset->row_flag[i] && !SDDS_WriteAsciiRow(SDDS_dataset, i, fp))
            return 0;
      }
      SDDS_dataset->last_row_written = SDDS_dataset->n_rows - 1;
      SDDS_dataset->n_rows_written = rows;
      SDDS_dataset->writing_page = 1;
      fflush(fp);
    }
#if defined(zLib)
  }
#endif
 
  return (1);
}

SDDS_WriteAsciiParameters()

int32_t SDDS_WriteAsciiParameters ( SDDS_DATASET * SDDS_dataset, FILE * fp )

extern

Writes the parameter data of an SDDS dataset in ASCII format to a file.

This function writes all parameters of the SDDS dataset to the provided file pointer in ASCII format. Only parameters that do not have fixed values are written. Each parameter value is written on a new line.

Parameters

SDDS_dataset	Pointer to the SDDS dataset containing the parameter data.
fp	File pointer to the ASCII file where the parameter data will be written.

Returns

Returns 1 on success, or 0 on error. If an error occurs, an error message is set via SDDS_SetError().

Note

The function checks the dataset for consistency before writing. It uses SDDS_WriteTypedValue() to write each parameter value according to its data type.

Definition at line 531 of file SDDS_ascii.c.

                                                                        {
  int32_t i;
  SDDS_LAYOUT *layout;
  /*  char *predefined_format; */
 
  if (!SDDS_CheckDataset(SDDS_dataset, "SDDS_WriteAsciiParameters"))
    return (0);
  layout = &SDDS_dataset->layout;
  for (i = 0; i < layout->n_parameters; i++) {
    if (layout->parameter_definition[i].fixed_value)
      continue;
    if (!SDDS_WriteTypedValue(SDDS_dataset->parameter[i], 0, layout->parameter_definition[i].type, NULL, fp)) {
      SDDS_SetError("Unable to write ascii parameters (SDDS_WriteAsciiParameters)");
      return 0;
    }
    fputc('\n', fp);
  }
  return (1);
}

SDDS_WriteAsciiRow()

int32_t SDDS_WriteAsciiRow ( SDDS_DATASET * SDDS_dataset, int64_t row, FILE * fp )

extern

Writes a single row of data in ASCII format to a file.

This function writes the specified row from the SDDS dataset to the provided file pointer in ASCII format. The data is formatted according to the data types of the columns. Supports multi-line rows as specified by the data mode settings in the dataset layout.

Parameters

SDDS_dataset	Pointer to the SDDS dataset containing the data.
row	Zero-based index of the row to write.
fp	File pointer to the ASCII file where the row data will be written.

Returns

Returns 1 on success, or 0 on error. If an error occurs, an error message is set via SDDS_SetError().

Note

The function checks the dataset for consistency before writing. It uses SDDS_WriteTypedValue() to write each column value according to its data type. The number of values per line is determined by the lines_per_row setting in the dataset layout.

Definition at line 775 of file SDDS_ascii.c.

                                                                              {
  int32_t newline_needed;
  int64_t i, n_per_line, line;
  /*  int32_t  embedded_quotes_present; */
  SDDS_LAYOUT *layout;
  /*  char *predefined_format, *s; */
 
  if (!SDDS_CheckDataset(SDDS_dataset, "SDDS_WriteAsciiRow"))
    return (0);
  layout = &SDDS_dataset->layout;
  if (SDDS_dataset->layout.data_mode.lines_per_row <= 0)
    SDDS_dataset->layout.data_mode.lines_per_row = 1;
  n_per_line = SDDS_dataset->layout.n_columns / SDDS_dataset->layout.data_mode.lines_per_row;
  line = 1;
  newline_needed = 0;
  for (i = 0; i < layout->n_columns; i++) {
    if (!SDDS_WriteTypedValue(SDDS_dataset->data[i], row, layout->column_definition[i].type, NULL, fp)) {
      SDDS_SetError("Unable to write ascii row (SDDS_WriteAsciiRow)");
      return 0;
    }
    if ((i + 1) % n_per_line == 0 && line != SDDS_dataset->layout.data_mode.lines_per_row) {
      newline_needed = 0;
      fputc('\n', fp);
      line++;
    } else {
      fputc(' ', fp);
      newline_needed = 1;
    }
  }
  if (newline_needed)
    fputc('\n', fp);
  return (1);
}

SDDS_WriteAssociateDefinition()

int32_t SDDS_WriteAssociateDefinition ( ASSOCIATE_DEFINITION * associate_definition, FILE * fp )

extern

Writes an associate definition to a standard file.

This function outputs the definition of an associate in the SDDS layout. It includes fields such as name, filename, contents, path, description, and an SDDS flag. The definition is enclosed within &associate and &end tags.

Parameters

associate_definition	Pointer to the associate definition structure.
fp	The file pointer where the associate definition will be written.

Returns

Returns 1 on success, 0 if the file pointer is NULL.

Definition at line 493 of file SDDS_write.c.

                                                                                            {
  if (!fp)
    return (0);
 
  fputs("&associate ", fp);
  SDDS_PrintNamelistField(fp, "name", associate_definition->name);
  SDDS_PrintNamelistField(fp, "filename", SDDS_BlankToNull(associate_definition->filename));
  SDDS_PrintNamelistField(fp, "contents", SDDS_BlankToNull(associate_definition->contents));
  SDDS_PrintNamelistField(fp, "path", SDDS_BlankToNull(associate_definition->path));
  SDDS_PrintNamelistField(fp, "description", SDDS_BlankToNull(associate_definition->description));
  fprintf(fp, "sdds=%" PRId32, associate_definition->sdds);
  fputs(" &end\n", fp);
  return (1);
}

SDDS_WriteBinaryArrays()

int32_t SDDS_WriteBinaryArrays ( SDDS_DATASET * SDDS_dataset )

extern

Writes the binary arrays of the SDDS dataset to a file.

This function writes all arrays defined in the SDDS dataset to the associated binary file. It handles arrays with and without dimensions, and manages different compression formats such as gzip and LZMA if enabled.

Parameters

SDDS_dataset

Pointer to the SDDS_DATASET structure containing the dataset to write.

Returns

• Returns 1 on successful write of all arrays.
• Returns 0 if an error occurs during the writing process.

The function performs the following steps:

• Validates the dataset structure.
• Iterates over all arrays defined in the dataset layout.
• For each array:
  • If the array has no dimensions, it writes zeroes for each defined dimension.
  • If the array has dimensions, it writes the dimension sizes using the appropriate compression method.
  • If the array type is SDDS_STRING, it writes each string element using the appropriate compression method.
  • Otherwise, it writes the array's data using buffered write functions.

The function handles different file formats:

• For gzip files, it uses SDDS_GZipWriteBinaryString and SDDS_GZipBufferedWrite.
• For LZMA files, it uses SDDS_LZMAWriteBinaryString and SDDS_LZMABufferedWrite.
• For standard binary files, it uses SDDS_WriteBinaryString and SDDS_BufferedWrite.

Note

• The function assumes that the dataset has been properly initialized and that all arrays are correctly allocated.
• Compression support (zLib or LZMA) must be enabled during compilation for handling compressed files.

Definition at line 1554 of file SDDS_binary.c.

                                                           {
  int32_t i, j, zero = 0;
  SDDS_LAYOUT *layout;
#if defined(zLib)
  gzFile gzfp;
#endif
  FILE *fp;
  struct lzmafile *lzmafp;
  SDDS_FILEBUFFER *fBuffer;
 
  if (!SDDS_CheckDataset(SDDS_dataset, "SDDS_WriteBinaryArrays"))
    return (0);
  layout = &SDDS_dataset->layout;
#if defined(zLib)
  if (SDDS_dataset->layout.gzipFile) {
    gzfp = layout->gzfp;
    fBuffer = &SDDS_dataset->fBuffer;
    for (i = 0; i < layout->n_arrays; i++) {
      if (!SDDS_dataset->array[i].dimension) {
        for (j = 0; j < layout->array_definition[i].dimensions; j++)
          if (!SDDS_GZipBufferedWrite(&zero, sizeof(zero), gzfp, fBuffer)) {
            SDDS_SetError("Unable to write null array--failure writing dimensions (SDDS_WriteBinaryArrays)");
            return 0;
          }
        continue;
      }
      if (!SDDS_GZipBufferedWrite(SDDS_dataset->array[i].dimension, sizeof(*(SDDS_dataset->array)[i].dimension) * layout->array_definition[i].dimensions, gzfp, fBuffer)) {
        SDDS_SetError("Unable to write arrays--failure writing dimensions (SDDS_WriteBinaryArrays)");
        return (0);
      }
      if (layout->array_definition[i].type == SDDS_STRING) {
        for (j = 0; j < SDDS_dataset->array[i].elements; j++) {
          if (!SDDS_GZipWriteBinaryString(((char **)SDDS_dataset->array[i].data)[j], gzfp, fBuffer)) {
            SDDS_SetError("Unable to write arrays--failure writing string (SDDS_WriteBinaryArrays)");
            return (0);
          }
        }
      } else if (!SDDS_GZipBufferedWrite(SDDS_dataset->array[i].data, SDDS_type_size[layout->array_definition[i].type - 1] * SDDS_dataset->array[i].elements, gzfp, fBuffer)) {
        SDDS_SetError("Unable to write arrays--failure writing values (SDDS_WriteBinaryArrays)");
        return (0);
      }
    }
  } else {
#endif
    if (SDDS_dataset->layout.gzipFile) {
      lzmafp = layout->lzmafp;
      fBuffer = &SDDS_dataset->fBuffer;
      for (i = 0; i < layout->n_arrays; i++) {
        if (!SDDS_dataset->array[i].dimension) {
          for (j = 0; j < layout->array_definition[i].dimensions; j++)
            if (!SDDS_LZMABufferedWrite(&zero, sizeof(zero), lzmafp, fBuffer)) {
              SDDS_SetError("Unable to write null array--failure writing dimensions (SDDS_WriteBinaryArrays)");
              return 0;
            }
          continue;
        }
        if (!SDDS_LZMABufferedWrite(SDDS_dataset->array[i].dimension, sizeof(*(SDDS_dataset->array)[i].dimension) * layout->array_definition[i].dimensions, lzmafp, fBuffer)) {
          SDDS_SetError("Unable to write arrays--failure writing dimensions (SDDS_WriteBinaryArrays)");
          return (0);
        }
        if (layout->array_definition[i].type == SDDS_STRING) {
          for (j = 0; j < SDDS_dataset->array[i].elements; j++) {
            if (!SDDS_LZMAWriteBinaryString(((char **)SDDS_dataset->array[i].data)[j], lzmafp, fBuffer)) {
              SDDS_SetError("Unable to write arrays--failure writing string (SDDS_WriteBinaryArrays)");
              return (0);
            }
          }
        } else if (!SDDS_LZMABufferedWrite(SDDS_dataset->array[i].data, SDDS_type_size[layout->array_definition[i].type - 1] * SDDS_dataset->array[i].elements, lzmafp, fBuffer)) {
          SDDS_SetError("Unable to write arrays--failure writing values (SDDS_WriteBinaryArrays)");
          return (0);
        }
      }
    } else {
      fp = layout->fp;
      fBuffer = &SDDS_dataset->fBuffer;
      for (i = 0; i < layout->n_arrays; i++) {
        if (!SDDS_dataset->array[i].dimension) {
          for (j = 0; j < layout->array_definition[i].dimensions; j++)
            if (!SDDS_BufferedWrite(&zero, sizeof(zero), fp, fBuffer)) {
              SDDS_SetError("Unable to write null array--failure writing dimensions (SDDS_WriteBinaryArrays)");
              return 0;
            }
          continue;
        }
        if (!SDDS_BufferedWrite(SDDS_dataset->array[i].dimension, sizeof(*(SDDS_dataset->array)[i].dimension) * layout->array_definition[i].dimensions, fp, fBuffer)) {
          SDDS_SetError("Unable to write arrays--failure writing dimensions (SDDS_WriteBinaryArrays)");
          return (0);
        }
        if (layout->array_definition[i].type == SDDS_STRING) {
          for (j = 0; j < SDDS_dataset->array[i].elements; j++) {
            if (!SDDS_WriteBinaryString(((char **)SDDS_dataset->array[i].data)[j], fp, fBuffer)) {
              SDDS_SetError("Unable to write arrays--failure writing string (SDDS_WriteBinaryArrays)");
              return (0);
            }
          }
        } else if (!SDDS_BufferedWrite(SDDS_dataset->array[i].data, SDDS_type_size[layout->array_definition[i].type - 1] * SDDS_dataset->array[i].elements, fp, fBuffer)) {
          SDDS_SetError("Unable to write arrays--failure writing values (SDDS_WriteBinaryArrays)");
          return (0);
        }
      }
    }
#if defined(zLib)
  }
#endif
  return (1);
}

SDDS_WriteBinaryColumns()

int32_t SDDS_WriteBinaryColumns ( SDDS_DATASET * SDDS_dataset )

extern

Writes the binary columns of an SDDS dataset to the associated file.

This function iterates over each column defined in the SDDS dataset layout and writes its data to the binary file. It handles different data types, including strings and numeric types, and supports various compression formats such as gzip and LZMA if enabled.

Depending on the dataset's configuration, the function writes directly to a standard binary file, a gzip-compressed file, or an LZMA-compressed file. It also handles sparse data by only writing rows flagged for inclusion.

Parameters

SDDS_dataset

Pointer to the SDDS_DATASET structure representing the dataset to write.

Returns

• Returns 1 on successful writing of all columns.
• Returns 0 if an error occurs during the writing process.

The function performs the following steps:

• Validates the dataset structure using SDDS_CheckDataset.
• Determines the file format (standard, gzip, LZMA) and initializes the corresponding file pointer.
• Iterates through each column in the dataset:
  • For string columns, writes each string entry individually using the appropriate write function.
  • For numeric columns, writes the entire column data in a buffered manner if all rows are flagged; otherwise, writes individual row entries.
• Handles errors by setting appropriate error messages and aborting the write operation.

Note

• The function assumes that the dataset has been properly initialized and populated with data.
• Compression support (zLib for gzip, LZMA libraries) must be enabled during compilation for handling compressed files.
• The function is not thread-safe and should be called in a synchronized context.

Definition at line 1694 of file SDDS_binary.c.

                                                            {
  int64_t i, row, rows, type, size;
  SDDS_LAYOUT *layout;
#if defined(zLib)
  gzFile gzfp;
#endif
  FILE *fp;
  struct lzmafile *lzmafp;
  SDDS_FILEBUFFER *fBuffer;
 
  if (!SDDS_CheckDataset(SDDS_dataset, "SDDS_WriteBinaryColumns"))
    return (0);
  layout = &SDDS_dataset->layout;
  fBuffer = &SDDS_dataset->fBuffer;
  rows = SDDS_CountRowsOfInterest(SDDS_dataset);
#if defined(zLib)
  if (SDDS_dataset->layout.gzipFile) {
    gzfp = layout->gzfp;
    for (i = 0; i < layout->n_columns; i++) {
      type = layout->column_definition[i].type;
      size = SDDS_type_size[type - 1];
      if (type == SDDS_STRING) {
        for (row = 0; row < SDDS_dataset->n_rows; row++) {
          if (SDDS_dataset->row_flag[row] && !SDDS_GZipWriteBinaryString(*((char **)SDDS_dataset->data[i] + row), gzfp, fBuffer)) {
            SDDS_SetError("Unable to write arrays--failure writing string (SDDS_WriteBinaryColumns)");
            return (0);
          }
        }
      } else {
        if (rows == SDDS_dataset->n_rows) {
          if (!SDDS_GZipBufferedWrite(SDDS_dataset->data[i], size * rows, gzfp, fBuffer)) {
            SDDS_SetError("Unable to write columns--failure writing values (SDDS_WriteBinaryColumns)");
            return (0);
          }
        } else {
          for (row = 0; row < SDDS_dataset->n_rows; row++) {
            if (SDDS_dataset->row_flag[row] && !SDDS_GZipBufferedWrite((char *)SDDS_dataset->data[i] + row * size, size, gzfp, fBuffer)) {
              SDDS_SetError("Unable to write columns--failure writing values (SDDS_WriteBinaryColumns)");
              return (0);
            }
          }
        }
      }
    }
  } else {
#endif
    if (SDDS_dataset->layout.lzmaFile) {
      lzmafp = layout->lzmafp;
      for (i = 0; i < layout->n_columns; i++) {
        type = layout->column_definition[i].type;
        size = SDDS_type_size[type - 1];
        if (layout->column_definition[i].type == SDDS_STRING) {
          for (row = 0; row < SDDS_dataset->n_rows; row++) {
            if (SDDS_dataset->row_flag[row] && !SDDS_LZMAWriteBinaryString(*((char **)SDDS_dataset->data[i] + row), lzmafp, fBuffer)) {
              SDDS_SetError("Unable to write arrays--failure writing string (SDDS_WriteBinaryColumns)");
              return (0);
            }
          }
        } else {
          if (rows == SDDS_dataset->n_rows) {
            if (!SDDS_LZMABufferedWrite(SDDS_dataset->data[i], size * rows, lzmafp, fBuffer)) {
              SDDS_SetError("Unable to write columns--failure writing values (SDDS_WriteBinaryColumns)");
              return (0);
            }
          } else {
            for (row = 0; row < SDDS_dataset->n_rows; row++) {
              if (SDDS_dataset->row_flag[row] && !SDDS_LZMABufferedWrite((char *)SDDS_dataset->data[i] + row * size, size, lzmafp, fBuffer)) {
                SDDS_SetError("Unable to write columns--failure writing values (SDDS_WriteBinaryColumns)");
                return (0);
              }
            }
          }
        }
      }
    } else {
      fp = layout->fp;
      for (i = 0; i < layout->n_columns; i++) {
        type = layout->column_definition[i].type;
        size = SDDS_type_size[type - 1];
        if (layout->column_definition[i].type == SDDS_STRING) {
          for (row = 0; row < SDDS_dataset->n_rows; row++) {
            if (SDDS_dataset->row_flag[row] && !SDDS_WriteBinaryString(*((char **)SDDS_dataset->data[i] + row), fp, fBuffer)) {
              SDDS_SetError("Unable to write arrays--failure writing string (SDDS_WriteBinaryColumns)");
              return (0);
            }
          }
        } else {
          if (rows == SDDS_dataset->n_rows) {
            if (!SDDS_BufferedWrite(SDDS_dataset->data[i], size * rows, fp, fBuffer)) {
              SDDS_SetError("Unable to write columns--failure writing values (SDDS_WriteBinaryColumns)");
              return (0);
            }
          } else {
            for (row = 0; row < SDDS_dataset->n_rows; row++) {
              if (SDDS_dataset->row_flag[row] && !SDDS_BufferedWrite((char *)SDDS_dataset->data[i] + row * size, size, fp, fBuffer)) {
                SDDS_SetError("Unable to write columns--failure writing values (SDDS_WriteBinaryColumns)");
                return (0);
              }
            }
          }
        }
      }
    }
#if defined(zLib)
  }
#endif
  return (1);
}

SDDS_WriteBinaryPage()

int32_t SDDS_WriteBinaryPage ( SDDS_DATASET * SDDS_dataset )

extern

Writes a binary page of data to an SDDS dataset, handling compression and buffering.

This function writes a binary page (including parameters, arrays, and row data) to the SDDS dataset pointed to by SDDS_dataset. It handles different file types, including regular files, LZMA-compressed files, and GZIP-compressed files, and uses buffering to improve write performance.

The function performs the following steps:

• Checks for output endianess and writes a non-native binary page if needed.
• Determines the number of rows to write and calculates any fixed row counts.
• Writes the number of rows to the file.
• Writes parameters, arrays, and column data using the appropriate write functions.
• Flushes the buffer to ensure all data is written.

It uses the appropriate buffered write functions (SDDS_BufferedWrite, SDDS_LZMABufferedWrite, or SDDS_GZipBufferedWrite) depending on the file type.

Parameters

SDDS_dataset

Pointer to the SDDS_DATASET structure representing the SDDS dataset to write to.

Returns

Returns 1 on success; returns 0 on error.

Note

The function sets error messages using SDDS_SetError if any step fails.

Definition at line 767 of file SDDS_binary.c.

                                                         {
#if defined(zLib)
  gzFile gzfp;
#endif
  FILE *fp;
  struct lzmafile *lzmafp;
  int64_t i, rows, fixed_rows;
  int32_t min32 = INT32_MIN, rows32;
  /*  static char buffer[SDDS_MAXLINE]; */
  SDDS_FILEBUFFER *fBuffer;
  char *outputEndianess = NULL;
 
  if ((outputEndianess = getenv("SDDS_OUTPUT_ENDIANESS"))) {
    if (((strncmp(outputEndianess, "big", 3) == 0) && (SDDS_IsBigEndianMachine() == 0)) || ((strncmp(outputEndianess, "little", 6) == 0) && (SDDS_IsBigEndianMachine() == 1)))
      return SDDS_WriteNonNativeBinaryPage(SDDS_dataset);
  }
 
  if (!SDDS_CheckDataset(SDDS_dataset, "SDDS_WriteBinaryPage"))
    return (0);
 
#if defined(zLib)
  if (SDDS_dataset->layout.gzipFile) {
    if (!(gzfp = SDDS_dataset->layout.gzfp)) {
      SDDS_SetError("Unable to write page--file pointer is NULL (SDDS_WriteBinaryPage)");
      return (0);
    }
    fBuffer = &SDDS_dataset->fBuffer;
 
    if (!fBuffer->buffer) {
      int32_t bufferSize = SDDS_GetLockedDefaultIOBufferSize();
      if (!(fBuffer->buffer = fBuffer->data = SDDS_Malloc(sizeof(char) * (bufferSize + 1)))) {
        SDDS_SetError("Unable to do buffered read--allocation failure (SDDS_WriteBinaryPage)");
        return 0;
      }
      fBuffer->bufferSize = bufferSize;
      fBuffer->bytesLeft = bufferSize;
    }
 
    rows = SDDS_CountRowsOfInterest(SDDS_dataset);
    SDDS_dataset->rowcount_offset = gztell(gzfp);
    if (SDDS_dataset->layout.data_mode.fixed_row_count) {
      fixed_rows = ((rows / SDDS_dataset->layout.data_mode.fixed_row_increment) + 2) * SDDS_dataset->layout.data_mode.fixed_row_increment;
      if (fixed_rows > INT32_MAX) {
        if (!SDDS_GZipBufferedWrite(&min32, sizeof(min32), gzfp, fBuffer)) {
          SDDS_SetError("Unable to write page--failure writing number of rows (SDDS_WriteBinaryPage)");
          return (0);
        }
        if (!SDDS_GZipBufferedWrite(&fixed_rows, sizeof(fixed_rows), gzfp, fBuffer)) {
          SDDS_SetError("Unable to write page--failure writing number of rows (SDDS_WriteBinaryPage)");
          return (0);
        }
      } else {
        rows32 = (int32_t)fixed_rows;
        if (!SDDS_GZipBufferedWrite(&rows32, sizeof(rows32), gzfp, fBuffer)) {
          SDDS_SetError("Unable to write page--failure writing number of rows (SDDS_WriteBinaryPage)");
          return (0);
        }
      }
    } else {
      if (rows > INT32_MAX) {
        if (!SDDS_GZipBufferedWrite(&min32, sizeof(min32), gzfp, fBuffer)) {
          SDDS_SetError("Unable to write page--failure writing number of rows (SDDS_WriteBinaryPage)");
          return (0);
        }
        if (!SDDS_GZipBufferedWrite(&rows, sizeof(rows), gzfp, fBuffer)) {
          SDDS_SetError("Unable to write page--failure writing number of rows (SDDS_WriteBinaryPage)");
          return (0);
        }
      } else {
        rows32 = (int32_t)rows;
        if (!SDDS_GZipBufferedWrite(&rows32, sizeof(rows32), gzfp, fBuffer)) {
          SDDS_SetError("Unable to write page--failure writing number of rows (SDDS_WriteBinaryPage)");
          return (0);
        }
      }
    }
    if (!SDDS_WriteBinaryParameters(SDDS_dataset)) {
      SDDS_SetError("Unable to write page--parameter writing problem (SDDS_WriteBinaryPage)");
      return 0;
    }
    if (!SDDS_WriteBinaryArrays(SDDS_dataset)) {
      SDDS_SetError("Unable to write page--array writing problem (SDDS_WriteBinaryPage)");
      return 0;
    }
    if (SDDS_dataset->layout.n_columns) {
      if (SDDS_dataset->layout.data_mode.column_major) {
        if (!SDDS_WriteBinaryColumns(SDDS_dataset)) {
          SDDS_SetError("Unable to write page--column writing problem (SDDS_WriteBinaryPage)");
          return 0;
        }
      } else {
        for (i = 0; i < SDDS_dataset->n_rows; i++) {
          if (SDDS_dataset->row_flag[i] && !SDDS_WriteBinaryRow(SDDS_dataset, i)) {
            SDDS_SetError("Unable to write page--row writing problem (SDDS_WriteBinaryPage)");
            return 0;
          }
        }
      }
    }
    if (!SDDS_GZipFlushBuffer(gzfp, fBuffer)) {
      SDDS_SetError("Unable to write page--buffer flushing problem (SDDS_WriteBinaryPage)");
      return 0;
    }
    SDDS_dataset->last_row_written = SDDS_dataset->n_rows - 1;
    SDDS_dataset->n_rows_written = rows;
    SDDS_dataset->writing_page = 1;
  } else {
#endif
    if (SDDS_dataset->layout.lzmaFile) {
      if (!(lzmafp = SDDS_dataset->layout.lzmafp)) {
        SDDS_SetError("Unable to write page--file pointer is NULL (SDDS_WriteBinaryPage)");
        return (0);
      }
      fBuffer = &SDDS_dataset->fBuffer;
 
      if (!fBuffer->buffer) {
        int32_t bufferSize = SDDS_GetLockedDefaultIOBufferSize();
        if (!(fBuffer->buffer = fBuffer->data = SDDS_Malloc(sizeof(char) * (bufferSize + 1)))) {
          SDDS_SetError("Unable to do buffered read--allocation failure (SDDS_WriteBinaryPage)");
          return 0;
        }
        fBuffer->bufferSize = bufferSize;
        fBuffer->bytesLeft = bufferSize;
      }
      rows = SDDS_CountRowsOfInterest(SDDS_dataset);
      SDDS_dataset->rowcount_offset = lzma_tell(lzmafp);
      if (SDDS_dataset->layout.data_mode.fixed_row_count) {
        fixed_rows = ((rows / SDDS_dataset->layout.data_mode.fixed_row_increment) + 2) * SDDS_dataset->layout.data_mode.fixed_row_increment;
        if (fixed_rows > INT32_MAX) {
          if (!SDDS_LZMABufferedWrite(&min32, sizeof(min32), lzmafp, fBuffer)) {
            SDDS_SetError("Unable to write page--failure writing number of rows (SDDS_WriteBinaryPage)");
            return (0);
          }
          if (!SDDS_LZMABufferedWrite(&fixed_rows, sizeof(fixed_rows), lzmafp, fBuffer)) {
            SDDS_SetError("Unable to write page--failure writing number of rows (SDDS_WriteBinaryPage)");
            return (0);
          }
        } else {
          rows32 = (int32_t)fixed_rows;
          if (!SDDS_LZMABufferedWrite(&rows32, sizeof(rows32), lzmafp, fBuffer)) {
            SDDS_SetError("Unable to write page--failure writing number of rows (SDDS_WriteBinaryPage)");
            return (0);
          }
        }
      } else {
        if (rows > INT32_MAX) {
          if (!SDDS_LZMABufferedWrite(&min32, sizeof(min32), lzmafp, fBuffer)) {
            SDDS_SetError("Unable to write page--failure writing number of rows (SDDS_WriteBinaryPage)");
            return (0);
          }
          if (!SDDS_LZMABufferedWrite(&rows, sizeof(rows), lzmafp, fBuffer)) {
            SDDS_SetError("Unable to write page--failure writing number of rows (SDDS_WriteBinaryPage)");
            return (0);
          }
        } else {
          rows32 = (int32_t)rows;
          if (!SDDS_LZMABufferedWrite(&rows32, sizeof(rows32), lzmafp, fBuffer)) {
            SDDS_SetError("Unable to write page--failure writing number of rows (SDDS_WriteBinaryPage)");
            return (0);
          }
        }
      }
      if (!SDDS_WriteBinaryParameters(SDDS_dataset)) {
        SDDS_SetError("Unable to write page--parameter writing problem (SDDS_WriteBinaryPage)");
        return 0;
      }
      if (!SDDS_WriteBinaryArrays(SDDS_dataset)) {
        SDDS_SetError("Unable to write page--array writing problem (SDDS_WriteBinaryPage)");
        return 0;
      }
      if (SDDS_dataset->layout.n_columns) {
        if (SDDS_dataset->layout.data_mode.column_major) {
          if (!SDDS_WriteBinaryColumns(SDDS_dataset)) {
            SDDS_SetError("Unable to write page--column writing problem (SDDS_WriteBinaryPage)");
            return 0;
          }
        } else {
          for (i = 0; i < SDDS_dataset->n_rows; i++) {
            if (SDDS_dataset->row_flag[i] && !SDDS_WriteBinaryRow(SDDS_dataset, i)) {
              SDDS_SetError("Unable to write page--row writing problem (SDDS_WriteBinaryPage)");
              return 0;
            }
          }
        }
      }
      if (!SDDS_LZMAFlushBuffer(lzmafp, fBuffer)) {
        SDDS_SetError("Unable to write page--buffer flushing problem (SDDS_WriteBinaryPage)");
        return 0;
      }
      SDDS_dataset->last_row_written = SDDS_dataset->n_rows - 1;
      SDDS_dataset->n_rows_written = rows;
      SDDS_dataset->writing_page = 1;
    } else {
      if (!(fp = SDDS_dataset->layout.fp)) {
        SDDS_SetError("Unable to write page--file pointer is NULL (SDDS_WriteBinaryPage)");
        return (0);
      }
      fBuffer = &SDDS_dataset->fBuffer;
 
      if (!fBuffer->buffer) {
        int32_t bufferSize = SDDS_GetLockedDefaultIOBufferSize();
        if (!(fBuffer->buffer = fBuffer->data = SDDS_Malloc(sizeof(char) * (bufferSize + 1)))) {
          SDDS_SetError("Unable to do buffered read--allocation failure (SDDS_WriteBinaryPage)");
          return 0;
        }
        fBuffer->bufferSize = bufferSize;
        fBuffer->bytesLeft = bufferSize;
      }
 
      /* Flush any existing data in the output buffer so we can determine the
       * row count offset for the file.  This is probably unnecessary.
       */
      if (!SDDS_FlushBuffer(fp, fBuffer)) {
        SDDS_SetError("Unable to write page--buffer flushing problem (SDDS_WriteBinaryPage)");
        return 0;
      }
 
      /* output the row count and determine its byte offset in the file */
      rows = SDDS_CountRowsOfInterest(SDDS_dataset);
      SDDS_dataset->rowcount_offset = ftell(fp);
      if (SDDS_dataset->layout.data_mode.fixed_row_count) {
        fixed_rows = ((rows / SDDS_dataset->layout.data_mode.fixed_row_increment) + 2) * SDDS_dataset->layout.data_mode.fixed_row_increment;
#if defined(DEBUG)
        fprintf(stderr, "setting %" PRId64 " fixed rows\n", fixed_rows);
#endif
        if (fixed_rows > INT32_MAX) {
          if (!SDDS_BufferedWrite(&min32, sizeof(min32), fp, fBuffer)) {
            SDDS_SetError("Unable to write page--failure writing number of rows (SDDS_WriteBinaryPage)");
            return (0);
          }
          if (!SDDS_BufferedWrite(&fixed_rows, sizeof(fixed_rows), fp, fBuffer)) {
            SDDS_SetError("Unable to write page--failure writing number of rows (SDDS_WriteBinaryPage)");
            return (0);
          }
        } else {
          rows32 = (int32_t)fixed_rows;
          if (!SDDS_BufferedWrite(&rows32, sizeof(rows32), fp, fBuffer)) {
            SDDS_SetError("Unable to write page--failure writing number of rows (SDDS_WriteBinaryPage)");
            return (0);
          }
        }
      } else {
#if defined(DEBUG)
        fprintf(stderr, "setting %" PRId64 " rows\n", rows);
#endif
        if (rows > INT32_MAX) {
          if (!SDDS_BufferedWrite(&min32, sizeof(min32), fp, fBuffer)) {
            SDDS_SetError("Unable to write page--failure writing number of rows (SDDS_WriteBinaryPage)");
            return (0);
          }
          if (!SDDS_BufferedWrite(&rows, sizeof(rows), fp, fBuffer)) {
            SDDS_SetError("Unable to write page--failure writing number of rows (SDDS_WriteBinaryPage)");
            return (0);
          }
        } else {
          rows32 = (int32_t)rows;
          if (!SDDS_BufferedWrite(&rows32, sizeof(rows32), fp, fBuffer)) {
            SDDS_SetError("Unable to write page--failure writing number of rows (SDDS_WriteBinaryPage)");
            return (0);
          }
        }
      }
 
      /* write the data, using buffered I/O */
      if (!SDDS_WriteBinaryParameters(SDDS_dataset)) {
        SDDS_SetError("Unable to write page--parameter writing problem (SDDS_WriteBinaryPage)");
        return 0;
      }
      if (!SDDS_WriteBinaryArrays(SDDS_dataset)) {
        SDDS_SetError("Unable to write page--array writing problem (SDDS_WriteBinaryPage)");
        return 0;
      }
      if (SDDS_dataset->layout.n_columns) {
        if (SDDS_dataset->layout.data_mode.column_major) {
          if (!SDDS_WriteBinaryColumns(SDDS_dataset)) {
            SDDS_SetError("Unable to write page--column writing problem (SDDS_WriteBinaryPage)");
            return 0;
          }
        } else {
          for (i = 0; i < SDDS_dataset->n_rows; i++) {
            if (SDDS_dataset->row_flag[i] && !SDDS_WriteBinaryRow(SDDS_dataset, i)) {
              SDDS_SetError("Unable to write page--row writing problem (SDDS_WriteBinaryPage)");
              return 0;
            }
          }
        }
      }
      /* flush the page */
      if (!SDDS_FlushBuffer(fp, fBuffer)) {
        SDDS_SetError("Unable to write page--buffer flushing problem (SDDS_WriteBinaryPage)");
        return 0;
      }
      SDDS_dataset->last_row_written = SDDS_dataset->n_rows - 1;
      SDDS_dataset->n_rows_written = rows;
      SDDS_dataset->writing_page = 1;
    }
#if defined(zLib)
  }
#endif
  return (1);
}

SDDS_WriteBinaryParameters()

int32_t SDDS_WriteBinaryParameters ( SDDS_DATASET * SDDS_dataset )

extern

Writes the binary parameters of the SDDS dataset.

This function writes all non-fixed parameters of the SDDS dataset to the associated binary file. It handles different compression formats such as gzip and LZMA if enabled.

Parameters

SDDS_dataset

Pointer to the SDDS_DATASET structure containing the dataset to write.

Returns

• Returns 1 on successful write of all parameters.
• Returns 0 if an error occurs during the writing process.

The function performs the following steps:

• Validates the dataset structure.
• Iterates over all parameters defined in the dataset layout.
• For each parameter:
  • If it is a fixed value, it is skipped.
  • If the parameter is of type SDDS_STRING, it writes the string using the appropriate compression method.
  • Otherwise, it writes the parameter's value using buffered write functions.

The function handles different file formats:

• For gzip files, it uses SDDS_GZipWriteBinaryString and SDDS_GZipBufferedWrite.
• For LZMA files, it uses SDDS_LZMAWriteBinaryString and SDDS_LZMABufferedWrite.
• For standard binary files, it uses SDDS_WriteBinaryString and SDDS_BufferedWrite.

Note

• The function assumes that the dataset has been properly initialized and that all parameters are correctly allocated.
• Compression support (zLib or LZMA) must be enabled during compilation for handling compressed files.

Definition at line 1447 of file SDDS_binary.c.

                                                               {
  int32_t i;
  SDDS_LAYOUT *layout;
  /*  char *predefined_format; */
  /*  static char buffer[SDDS_MAXLINE]; */
#if defined(zLib)
  gzFile gzfp;
#endif
  FILE *fp;
  struct lzmafile *lzmafp;
  SDDS_FILEBUFFER *fBuffer;
 
  if (!SDDS_CheckDataset(SDDS_dataset, "SDDS_WriteBinaryParameters"))
    return (0);
  layout = &SDDS_dataset->layout;
#if defined(zLib)
  if (SDDS_dataset->layout.gzipFile) {
    gzfp = layout->gzfp;
    fBuffer = &SDDS_dataset->fBuffer;
    for (i = 0; i < layout->n_parameters; i++) {
      if (layout->parameter_definition[i].fixed_value)
        continue;
      if (layout->parameter_definition[i].type == SDDS_STRING) {
        if (!SDDS_GZipWriteBinaryString(*((char **)SDDS_dataset->parameter[i]), gzfp, fBuffer)) {
          SDDS_SetError("Unable to write parameters--failure writing string (SDDS_WriteBinaryParameters)");
          return (0);
        }
      } else if (!SDDS_GZipBufferedWrite(SDDS_dataset->parameter[i], SDDS_type_size[layout->parameter_definition[i].type - 1], gzfp, fBuffer)) {
        SDDS_SetError("Unable to write parameters--failure writing value (SDDS_WriteBinaryParameters)");
        return (0);
      }
    }
  } else {
#endif
    if (SDDS_dataset->layout.lzmaFile) {
      lzmafp = layout->lzmafp;
      fBuffer = &SDDS_dataset->fBuffer;
      for (i = 0; i < layout->n_parameters; i++) {
        if (layout->parameter_definition[i].fixed_value)
          continue;
        if (layout->parameter_definition[i].type == SDDS_STRING) {
          if (!SDDS_LZMAWriteBinaryString(*((char **)SDDS_dataset->parameter[i]), lzmafp, fBuffer)) {
            SDDS_SetError("Unable to write parameters--failure writing string (SDDS_WriteBinaryParameters)");
            return (0);
          }
        } else if (!SDDS_LZMABufferedWrite(SDDS_dataset->parameter[i], SDDS_type_size[layout->parameter_definition[i].type - 1], lzmafp, fBuffer)) {
          SDDS_SetError("Unable to write parameters--failure writing value (SDDS_WriteBinaryParameters)");
          return (0);
        }
      }
    } else {
      fp = layout->fp;
      fBuffer = &SDDS_dataset->fBuffer;
      for (i = 0; i < layout->n_parameters; i++) {
        if (layout->parameter_definition[i].fixed_value)
          continue;
        if (layout->parameter_definition[i].type == SDDS_STRING) {
          if (!SDDS_WriteBinaryString(*((char **)SDDS_dataset->parameter[i]), fp, fBuffer)) {
            SDDS_SetError("Unable to write parameters--failure writing string (SDDS_WriteBinaryParameters)");
            return (0);
          }
        } else if (!SDDS_BufferedWrite(SDDS_dataset->parameter[i], SDDS_type_size[layout->parameter_definition[i].type - 1], fp, fBuffer)) {
          SDDS_SetError("Unable to write parameters--failure writing value (SDDS_WriteBinaryParameters)");
          return (0);
        }
      }
    }
#if defined(zLib)
  }
#endif
  return (1);
}

SDDS_WriteBinaryRow()

int32_t SDDS_WriteBinaryRow ( SDDS_DATASET * SDDS_dataset, int64_t row )

extern

Writes a single binary row of an SDDS dataset to the associated file.

This function writes the data of a specified row within the SDDS dataset to the binary file. It handles different data types, including strings and numeric types, and supports various compression formats such as gzip and LZMA if enabled.

Depending on the dataset's configuration, the function writes directly to a standard binary file, a gzip-compressed file, or an LZMA-compressed file.

Parameters

SDDS_dataset	Pointer to the SDDS_DATASET structure representing the dataset.
row	The zero-based index of the row to write.

Returns

• Returns 1 on successful writing of the row.
• Returns 0 if an error occurs during the writing process.

The function performs the following steps:

• Validates the dataset structure using SDDS_CheckDataset.
• Determines the file format (standard, gzip, LZMA) and initializes the corresponding file pointer.
• Iterates through each column in the dataset:
  • For string columns, writes the string entry of the specified row using the appropriate write function.
  • For numeric columns, writes the data of the specified row using buffered write functions.
• Handles errors by setting appropriate error messages and aborting the write operation.

Note

• The function assumes that the dataset has been properly initialized and that the specified row exists.
• Compression support (zLib for gzip, LZMA libraries) must be enabled during compilation for handling compressed files.
• The function is not thread-safe and should be called in a synchronized context.

Definition at line 1978 of file SDDS_binary.c.

                                                                     {
  int64_t i, type, size;
  SDDS_LAYOUT *layout;
#if defined(zLib)
  gzFile gzfp;
#endif
  FILE *fp;
  struct lzmafile *lzmafp;
  SDDS_FILEBUFFER *fBuffer;
 
  if (!SDDS_CheckDataset(SDDS_dataset, "SDDS_WriteBinaryRow"))
    return (0);
  layout = &SDDS_dataset->layout;
#if defined(zLib)
  if (SDDS_dataset->layout.gzipFile) {
    gzfp = layout->gzfp;
    fBuffer = &SDDS_dataset->fBuffer;
    for (i = 0; i < layout->n_columns; i++) {
      if ((type = layout->column_definition[i].type) == SDDS_STRING) {
        if (!SDDS_GZipWriteBinaryString(*((char **)SDDS_dataset->data[i] + row), gzfp, fBuffer)) {
          SDDS_SetError("Unable to write rows--failure writing string (SDDS_WriteBinaryRows)");
          return (0);
        }
      } else {
        size = SDDS_type_size[type - 1];
        if (!SDDS_GZipBufferedWrite((char *)SDDS_dataset->data[i] + row * size, size, gzfp, fBuffer)) {
          SDDS_SetError("Unable to write row--failure writing value (SDDS_WriteBinaryRow)");
          return (0);
        }
      }
    }
  } else {
#endif
    if (SDDS_dataset->layout.lzmaFile) {
      lzmafp = layout->lzmafp;
      fBuffer = &SDDS_dataset->fBuffer;
      for (i = 0; i < layout->n_columns; i++) {
        if ((type = layout->column_definition[i].type) == SDDS_STRING) {
          if (!SDDS_LZMAWriteBinaryString(*((char **)SDDS_dataset->data[i] + row), lzmafp, fBuffer)) {
            SDDS_SetError("Unable to write rows--failure writing string (SDDS_WriteBinaryRows)");
            return (0);
          }
        } else {
          size = SDDS_type_size[type - 1];
          if (!SDDS_LZMABufferedWrite((char *)SDDS_dataset->data[i] + row * size, size, lzmafp, fBuffer)) {
            SDDS_SetError("Unable to write row--failure writing value (SDDS_WriteBinaryRow)");
            return (0);
          }
        }
      }
    } else {
      fp = layout->fp;
      fBuffer = &SDDS_dataset->fBuffer;
      for (i = 0; i < layout->n_columns; i++) {
        if ((type = layout->column_definition[i].type) == SDDS_STRING) {
          if (!SDDS_WriteBinaryString(*((char **)SDDS_dataset->data[i] + row), fp, fBuffer)) {
            SDDS_SetError("Unable to write rows--failure writing string (SDDS_WriteBinaryRows)");
            return (0);
          }
        } else {
          size = SDDS_type_size[type - 1];
          if (!SDDS_BufferedWrite((char *)SDDS_dataset->data[i] + row * size, size, fp, fBuffer)) {
            SDDS_SetError("Unable to write row--failure writing value (SDDS_WriteBinaryRow)");
            return (0);
          }
        }
      }
    }
#if defined(zLib)
  }
#endif
  return (1);
}

SDDS_WriteColumnDefinition()

int32_t SDDS_WriteColumnDefinition ( COLUMN_DEFINITION * column_definition, FILE * fp )

extern

Writes a column definition to a standard file.

This function outputs the definition of a single column in the SDDS layout. It includes fields such as name, symbol, units, description, format string, and data type. The definition is enclosed within &column and &end tags.

Parameters

column_definition	Pointer to the column definition structure.
fp	The file pointer where the column definition will be written.

Returns

Returns 1 on success, 0 if the file pointer is NULL or the column type is invalid.

Definition at line 329 of file SDDS_write.c.

                                                                                   {
  if (!fp || column_definition->type <= 0 || column_definition->type > SDDS_NUM_TYPES)
    return (0);
 
  fputs("&column ", fp);
  SDDS_PrintNamelistField(fp, "name", column_definition->name);
  SDDS_PrintNamelistField(fp, "symbol", SDDS_BlankToNull(column_definition->symbol));
  SDDS_PrintNamelistField(fp, "units", SDDS_BlankToNull(column_definition->units));
  SDDS_PrintNamelistField(fp, "description", SDDS_BlankToNull(column_definition->description));
  SDDS_PrintNamelistField(fp, "format_string", SDDS_BlankToNull(column_definition->format_string));
  SDDS_PrintNamelistField(fp, "type", SDDS_type_name[column_definition->type - 1]);
  fputs(" &end\n", fp);
  return (1);
}

SDDS_WriteDataMode()

int32_t SDDS_WriteDataMode ( SDDS_LAYOUT * layout, FILE * fp )

extern

Writes the data mode section to a standard file.

This function outputs the data mode settings of the SDDS layout to the specified file pointer. It includes settings such as mode, lines per row, row counts, endianess, column-major order, and fixed row counts. The section is enclosed within &data and &end tags.

Parameters

layout	Pointer to the SDDS layout structure containing data mode settings.
fp	The file pointer where the data mode section will be written.

Returns

Returns 1 on success, 0 if the file pointer is NULL or the data mode is invalid.

Definition at line 576 of file SDDS_write.c.

                                                          {
  if (!fp || layout->data_mode.mode < 0 || layout->data_mode.mode > SDDS_NUM_DATA_MODES)
    return (0);
 
  fputs("&data ", fp);
  SDDS_PrintNamelistField(fp, "mode", SDDS_data_mode[layout->data_mode.mode - 1]);
  if (layout->data_mode.lines_per_row > 1)
    fprintf(fp, "lines_per_row=%" PRId32 ", ", layout->data_mode.lines_per_row);
  if (layout->data_mode.no_row_counts)
    fprintf(fp, "no_row_counts=1, ");
  if (layout->version >= 3) {
    if (layout->data_mode.mode == SDDS_BINARY) {
      if (layout->byteOrderDeclared == SDDS_BIGENDIAN)
        fprintf(fp, "endian=big, ");
      else
        fprintf(fp, "endian=little, ");
      if (layout->data_mode.column_major)
        fprintf(fp, "column_major_order=1, ");
    }
    if (layout->data_mode.fixed_row_count)
      fprintf(fp, "fixed_row_count=1, ");
  }
  fputs("&end\n", fp);
  return (1);
}

SDDS_WriteDescription()

int32_t SDDS_WriteDescription ( char * text, char * contents, FILE * fp )

extern

Writes the SDDS description section to a standard file.

This function writes the description section of the SDDS layout, including optional text and contents fields. It encapsulates the data within &description and &end tags.

Parameters

text	The descriptive text for the SDDS layout.
contents	The contents description for the SDDS layout.
fp	The file pointer where the description will be written.

Returns

Returns 1 on success, 0 if the file pointer is NULL.

Definition at line 256 of file SDDS_write.c.

                                                                    {
  if (!fp)
    return 0;
  if (!text && !contents)
    return 1;
  fputs("&description ", fp);
  SDDS_PrintNamelistField(fp, "text", text);
  SDDS_PrintNamelistField(fp, "contents", contents);
  fputs("&end\n", fp);
  return 1;
}

SDDS_WriteNonNativeBinaryColumns()

int32_t SDDS_WriteNonNativeBinaryColumns ( SDDS_DATASET * SDDS_dataset )

extern

Writes non-native endian binary columns of an SDDS dataset to the associated file.

This function iterates over each column defined in the SDDS dataset layout and writes its data to the binary file using a non-native byte order. It handles different data types, including strings and numeric types, and supports various compression formats such as gzip and LZMA if enabled.

Depending on the dataset's configuration, the function writes directly to a standard binary file, a gzip-compressed file, or an LZMA-compressed file. It also handles sparse data by only writing rows flagged for inclusion.

Parameters

SDDS_dataset

Pointer to the SDDS_DATASET structure representing the dataset to write.

Returns

• Returns 1 on successful writing of all columns.
• Returns 0 if an error occurs during the writing process.

The function performs the following steps:

• Validates the dataset structure using SDDS_CheckDataset.
• Determines the file format (standard, gzip, LZMA) and initializes the corresponding file pointer.
• Iterates through each column in the dataset:
  • For string columns, writes each string entry individually using the appropriate non-native write function.
  • For numeric columns, writes the entire column data in a buffered manner if all rows are flagged; otherwise, writes individual row entries.
• Handles errors by setting appropriate error messages and aborting the write operation.

Note

• The function assumes that the dataset has been properly initialized and populated with data.
• Compression support (zLib for gzip, LZMA libraries) must be enabled during compilation for handling compressed files.
• The function is not thread-safe and should be called in a synchronized context.

Definition at line 1837 of file SDDS_binary.c.

                                                                     {
  int64_t i, row, rows, size, type;
  SDDS_LAYOUT *layout;
#if defined(zLib)
  gzFile gzfp;
#endif
  FILE *fp;
  struct lzmafile *lzmafp;
  SDDS_FILEBUFFER *fBuffer;
 
  if (!SDDS_CheckDataset(SDDS_dataset, "SDDS_WriteNonNativeBinaryColumns"))
    return (0);
  layout = &SDDS_dataset->layout;
  rows = SDDS_CountRowsOfInterest(SDDS_dataset);
  fBuffer = &SDDS_dataset->fBuffer;
#if defined(zLib)
  if (SDDS_dataset->layout.gzipFile) {
    gzfp = layout->gzfp;
    for (i = 0; i < layout->n_columns; i++) {
      type = layout->column_definition[i].type;
      size = SDDS_type_size[type - 1];
      if (type == SDDS_STRING) {
        for (row = 0; row < SDDS_dataset->n_rows; row++) {
          if (SDDS_dataset->row_flag[row] && !SDDS_GZipWriteNonNativeBinaryString(*((char **)SDDS_dataset->data[i] + row), gzfp, fBuffer)) {
            SDDS_SetError("Unable to write arrays--failure writing string (SDDS_WriteNonNativeBinaryColumns)");
            return (0);
          }
        }
      } else {
        if (rows == SDDS_dataset->n_rows) {
          if (!SDDS_GZipBufferedWrite((char *)SDDS_dataset->data[i], size * rows, gzfp, fBuffer)) {
            SDDS_SetError("Unable to write columns--failure writing values (SDDS_WriteNonNativeBinaryColumns)");
            return (0);
          }
        } else {
          for (row = 0; row < SDDS_dataset->n_rows; row++) {
            if (SDDS_dataset->row_flag[row] && !SDDS_GZipBufferedWrite((char *)SDDS_dataset->data[i] + row * size, size, gzfp, fBuffer)) {
              SDDS_SetError("Unable to write columns--failure writing values (SDDS_WriteNonNativeBinaryColumns)");
              return (0);
            }
          }
        }
      }
    }
  } else {
#endif
    if (SDDS_dataset->layout.lzmaFile) {
      lzmafp = layout->lzmafp;
      for (i = 0; i < layout->n_columns; i++) {
        type = layout->column_definition[i].type;
        size = SDDS_type_size[type - 1];
        if (type == SDDS_STRING) {
          for (row = 0; row < SDDS_dataset->n_rows; row++) {
            if (SDDS_dataset->row_flag[row] && !SDDS_LZMAWriteNonNativeBinaryString(*((char **)SDDS_dataset->data[i] + row), lzmafp, fBuffer)) {
              SDDS_SetError("Unable to write arrays--failure writing string (SDDS_WriteNonNativeBinaryColumns)");
              return (0);
            }
          }
        } else {
          if (rows == SDDS_dataset->n_rows) {
            if (!SDDS_LZMABufferedWrite(SDDS_dataset->data[i], size * rows, lzmafp, fBuffer)) {
              SDDS_SetError("Unable to write columns--failure writing values (SDDS_WriteNonNativeBinaryColumns)");
              return (0);
            }
          } else {
            for (row = 0; row < SDDS_dataset->n_rows; row++) {
              if (SDDS_dataset->row_flag[row] && !SDDS_LZMABufferedWrite((char *)SDDS_dataset->data[i] + row * size, size, lzmafp, fBuffer)) {
                SDDS_SetError("Unable to write columns--failure writing values (SDDS_WriteNonNativeBinaryColumns)");
                return (0);
              }
            }
          }
        }
      }
    } else {
      fp = layout->fp;
      for (i = 0; i < layout->n_columns; i++) {
        type = layout->column_definition[i].type;
        size = SDDS_type_size[type - 1];
        if (type == SDDS_STRING) {
          for (row = 0; row < SDDS_dataset->n_rows; row++) {
            if (SDDS_dataset->row_flag[row] && !SDDS_WriteNonNativeBinaryString(*((char **)SDDS_dataset->data[i] + row), fp, fBuffer)) {
              SDDS_SetError("Unable to write arrays--failure writing string (SDDS_WriteNonNativeBinaryColumns)");
              return (0);
            }
          }
        } else {
          if (rows == SDDS_dataset->n_rows) {
            if (!SDDS_BufferedWrite(SDDS_dataset->data[i], size * rows, fp, fBuffer)) {
              SDDS_SetError("Unable to write columns--failure writing values (SDDS_WriteNonNativeBinaryColumns)");
              return (0);
            }
          } else {
            for (row = 0; row < SDDS_dataset->n_rows; row++) {
              if (SDDS_dataset->row_flag[row] && !SDDS_BufferedWrite((char *)SDDS_dataset->data[i] + row * size, size, fp, fBuffer)) {
                SDDS_SetError("Unable to write columns--failure writing values (SDDS_WriteNonNativeBinaryColumns)");
                return (0);
              }
            }
          }
        }
      }
    }
#if defined(zLib)
  }
#endif
  return (1);
}

SDDS_WriteParameterDefinition()

int32_t SDDS_WriteParameterDefinition ( PARAMETER_DEFINITION * parameter_definition, FILE * fp )

extern

Writes a parameter definition to a standard file.

This function outputs the definition of a single parameter in the SDDS layout. It includes fields such as name, symbol, units, description, format string, data type, and fixed value. The definition is enclosed within &parameter and &end tags.

Parameters

parameter_definition	Pointer to the parameter definition structure.
fp	The file pointer where the parameter definition will be written.

Returns

Returns 1 on success, 0 if the file pointer is NULL or the parameter type is invalid.

Definition at line 411 of file SDDS_write.c.

                                                                                            {
  if (!fp || parameter_definition->type <= 0 || parameter_definition->type > SDDS_NUM_TYPES)
    return (0);
  fputs("&parameter ", fp);
  SDDS_PrintNamelistField(fp, "name", parameter_definition->name);
  SDDS_PrintNamelistField(fp, "symbol", SDDS_BlankToNull(parameter_definition->symbol));
  SDDS_PrintNamelistField(fp, "units", SDDS_BlankToNull(parameter_definition->units));
  SDDS_PrintNamelistField(fp, "description", SDDS_BlankToNull(parameter_definition->description));
  SDDS_PrintNamelistField(fp, "format_string", SDDS_BlankToNull(parameter_definition->format_string));
  SDDS_PrintNamelistField(fp, "type", SDDS_type_name[parameter_definition->type - 1]);
  SDDS_PrintNamelistField(fp, "fixed_value", parameter_definition->fixed_value);
  fputs("&end\n", fp);
  return (1);
}

SDDS_WriteVersion()

int32_t SDDS_WriteVersion ( int32_t version_number, FILE * fp )

extern

Writes the SDDS protocol version to a standard file.

This function outputs the SDDS protocol version string to the provided file pointer. It is crucial that the protocol version remains unchanged to ensure compatibility.

Parameters

version_number	The SDDS protocol version number to write.
fp	The file pointer where the version string will be written.

Returns

Returns 1 on successful write, 0 if the file pointer is NULL.

Definition at line 59 of file SDDS_write.c.

                                                            {
  if (!fp)
    return (0);
  fprintf(fp, "SDDS%" PRId32 "\n", version_number);
  return (1);
}

UnpackLZMAOpen()

void * UnpackLZMAOpen

(

char *

filename

)

Definition at line 438 of file SDDS_lzma.c.

                                     {
  if (!filename)
    return NULL;
  return lzma_open(filename, "rb");
}