Reading and writing time series data¶

The TimeSeries object includes read() and write() methods to enable reading from and writing to files respectively. For example, to read from an ASCII file containing time and amplitude columns:

>>> from gwpy.timeseries import TimeSeries
>>> data = TimeSeries.read('my-data.txt')

TimeSeries.read() will attempt to automatically identify the file format based on the file extension and/or the contents of the file, however, the format keyword argument can be used to manually identify the input file-format.

The read() and write() methods take different arguments and keywords based on the input/output file format, see Built-in file formats for details on reading/writing for each of the built-in formats.

Automatic discovery of GW detector data¶

GW detector data¶

Gravitational-wave detector data, including all engineering diagnostic data as well as the calibrated ‘strain’ data that are searched for GW signals, are archived in GWF files stored at the relevant observatory. These data are available locally to authenticated users of the associated computing centres (typically collaboration members), but are also distributed using CVMFS and are available remotely using nds2. Access to these data is restricted to active collaboration members.

Additionally The Gravitational-Wave Open Science Centre (GWOSC) hosts publicly-accessible ‘open’ data, with event datasets made available at the same time as the relevant result publication and typically including ~1 hour of data around each published event detection, and bulk datasets with the entire observing run data available roughly 18 months after the end of the run.

GWOSC also hosts the Auxiliary Channel Three Hour Release, providing public access to environmental sensor data around GW170814. These data are freely accessible using nds2.

Data discovery methods¶

Built-in file formats¶

ASCII¶

GWpy supports writing TimeSeries (and FrequencySeries) data to ASCII in a two-column time and amplitude format.

Reading¶

To read a TimeSeries from ASCII:

>>> t = TimeSeries.read('data.txt')

See numpy.loadtxt() for keyword argument options.

Writing¶

To write a TimeSeries to ASCII:

>>> t.write('data.txt')

See numpy.savetxt() for keyword argument options.

GWF¶

Additional dependencies: LDAStools.frameCPP or framel or lalframe

The raw observatory data are archived in .gwf files, a custom binary format that efficiently stores the time streams and all necessary metadata, for more details about this particular data format, take a look at the specification document LIGO-T970130.

Reading¶

To read data from a GWF file, pass the input file path (or paths) and the name of the data channel to read:

>>> data = TimeSeries.read('HLV-HW100916-968654552-1.gwf', 'L1:LDAS-STRAIN')

Reading a StateVector uses the same syntax:

>>> data = StateVector.read('my-state-data.gwf', 'L1:GWO-STATE_VECTOR')

Multiple files can be read by passing a list of files:

>>> data = TimeSeries.read([file1, file2], 'L1:LDAS-STRAIN')

When reading multiple files, the nproc keyword argument can be used to distribute the reading over multiple CPUs, which should make it faster:

>>> data = TimeSeries.read([file1, file2, file3, file4], 'L1:LDAS-STRAIN', nproc=2)

The above command will separate the input list of 4 file paths into two sets of 2 files, combining the results into a single TimeSeries before returning.

The start and end keyword arguments can be used to downselect data to a specific [start, end) time segment when reading:

>>> data = TimeSeries.read('HLV-HW100916-968654552-1.gwf', 'L1:LDAS-STRAIN', start=968654552.5, end=968654553)

Additionally, the following keyword arguments can be used:

Reading multiple channels¶

To read multiple channels from one or more GWF files (rather than opening and closing the files multiple times), use the TimeSeriesDict or StateVectorDict classes, and pass a list of data channel names:

>>> data = TimeSeriesDict.read('HLV-HW100916-968654552-1.gwf', ['H1:LDAS-STRAIN', 'L1:LDAS-STRAIN'])

Writing¶

To write data held in any of the gwpy.timeseries classes to a GWF file, simply use:

>>> data.write('output.gwf')

If the output file already exists it will be overwritten.

GWF library availability¶

(Last we checked…) The three GWF library interfaces are available on the following platforms:

HDF5¶

GWpy allows storing data in HDF5 format files, using a custom specification for storage of metadata.

Reading¶

To read TimeSeries or StateVector data held in HDF5 files pass the filename (or filenames) or the source, and the path of the data inside the HDF5 file:

>>> data = TimeSeries.read('HLV-HW100916-968654552-1.hdf', 'L1:LDAS-STRAIN')

As with GWF, the start and end keyword arguments can be used to downselect data to a specific [start, end) time segment when reading:

>>> data = TimeSeries.read('HLV-HW100916-968654552-1.hdf', 'L1:LDAS-STRAIN', start=968654552.5, end=968654553)

Analogously to GWF, you can read multiple TimeSeries from an HDF5 file via TimeSeriesDict.read():

>>> data = TimeSeriesDict.read('HLV-HW100916-968654552-1.hdf')

By default, all matching datasets in the file will be read, to restrict the output, specify the names of the datasets you want:

>>> data = TimeSeriesDict.read('HLV-HW100916-968654552-1.hdf', ['H1:LDAS-STRAIN', 'L1:LDAS-STRAIN'])

Writing¶

Data held in a TimeSeries, TimeSeriesDict, `StateVector, or StateVectorDict can be written to an HDF5 file via:

>>> data.write('output.hdf')

The output argument ('output.hdf') can be a file path, an open h5py.File object, or a h5py.Group object, to append data to an existing file.

If the target file already exists, an IOError will be raised, use overwrite=True to force a new file to be written.

To write a TimeSeries to an existing file, use append=True:

>>> data.write('output.hdf', append=True)

To replace an existing dataset in an existing file, while preserving other data, use both append=True and overwrite=True:

>>> data.write('output.hdf', append=True, overwrite=True)

HDF5 (GWOSC)¶

GWOSC write data in HDF5 using a custom schema that is incompatible with format='hdf5'.

Reading¶

GWpy can read data from GWOSC HDF5 files using the format='hdf5.gwosc' keyword:

>>> data = TimeSeries.read(
...     "H-H1_GWOSC_16KHZ_R1-1187056280-4096.hdf5",
...     format="hdf5.gwosc",
... )

By default, TimeSeries.read() will return the contents of the /strain/Strain dataset, while StateVector.read() will return those of /quality/simple.

As with regular HDF5, the start and end keyword arguments can be used to downselect data to a specific [start, end) time segment when reading.

WAV¶

Any TimeSeries can be written to / read from a WAV file using TimeSeries.read():

Reading¶

To read a TimeSeries from WAV:

>>> t = TimeSeries.read('data.wav')

See scipy.io.wavfile.read() for any keyword argument options.

Writing¶

To write a TimeSeries to WAV:

>>> t.write('data.wav')

See scipy.io.wavfile.write() for keyword argument options.