Skip to content

szcompressor/SZ3

Repository files navigation

SZ3: A Modular Error-bounded Lossy Compression Framework for Scientific Datasets

(C) 2016 by Mathematics and Computer Science (MCS), Argonne National Laboratory. See COPYRIGHT in the top-level directory.

  • Major Authors: Sheng Di, Kai Zhao, Xin Liang, Jinyang Liu
  • Supervisor: Franck Cappello
  • Other Contributors: Robert Underwood, Sihuan Li, Ali M. Gok

Installation

  • mkdir build && cd build
  • cmake -DCMAKE_INSTALL_PREFIX:PATH=[INSTALL_DIR] ..
  • make
  • make install

Then, you'll find all the executables in [INSTALL_DIR]/bin and header files in [INSTALL_DIR]/include

How to run

SZ3 Executable

  • You can use the executable 'tools/sz3/sz3' to do the compression/decompression.

SZ3 C++ API

  • Located in 'include/SZ3/api/sz.hpp'.
  • Requiring a modern C++ compiler.
  • Different with SZ2 API.

SZ3 C API

  • Located in 'tools/sz3c/include/sz3c.h'
  • Compatible with SZ2 API

SZ3 Python API

  • Located in 'tools/pysz/pysz.py'
  • Test file provided ('tools/pysz/test.py')
  • Compatible with both SZ3 and SZ2
  • Requiring SZ2/3 dynamic library

H5Z-SZ3

  • Located in 'tools/H5Z-SZ3'
  • Please add "-DBUILD_H5Z_FILTER=ON" to enable this function for CMake.
  • sz3ToHDF5 and HDF5ToSz3 are provided for testing.

Third-Party APIs

Citations

Version history

Version New features

  • SZ 3.0.0 SZ3 is the C++ version of SZ with a modular and composable design.
  • SZ 3.0.1 Improve the build process.
  • SZ 3.1.0 The default algorithm is now interpolation+Lorenzo.
  • SZ 3.1.1 Add OpenMP support. Works for all algorithms. Please enable it using the config file.
  • SZ 3.1.2 Support configuration file (INI format). An example can be found in 'tools/sz3/sz3.config'.
  • SZ 3.1.3 Support more error control mode: PSNR, L2Norm, ABS_AND_REL, ABS_OR_REL. Support INT32 and INT64 datatype.
  • SZ 3.1.4 Support running on Windows natively with Visual Studio. Please refer to #5 (comment) for details.
  • SZ 3.1.5 Support HDF5 by H5Z-SZ3. Please add "-DBUILD_H5Z_FILTER=ON" to enable this function for CMake.
  • SZ 3.1.6 Support C API and Python API.
  • SZ 3.1.7 Initial MDZ(https://github.com/szcompressor/SZ3/tree/master/tools/mdz) support.
  • SZ 3.1.8 namespace changed from SZ to SZ3. H5Z-SZ3 supports configuration files now.
  • SZ 3.2.0 API reconstructed for FZ. H5Z-SZ3 rewrite. Compression version checking.
  • SZ 3.3.0 Add key QoZ v1 and v2 features to improve compression speed and data quality. The full QoZ is available from a separate branch (https://github.com/szcompressor/SZ3/tree/QoZ).

3rd party libraries/tools

  • Zstandard v1.4.5 will be fetched if libzstd can not be found by pkg-config.
  • The source code of ska_hash is included in SZ3.

About

Error-bounded Lossy Data Compressor (for floating-point/integer datasets)

Resources

Stars

Watchers

Forks

Packages

No packages published

Contributors 14