Skip to content

Use System lz4, zstd, and blosc #569

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 18 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -18,3 +18,12 @@ codecs for use in data storage and communication applications.

.. image:: https://codecov.io/gh/zarr-developers/numcodecs/branch/main/graph/badge.svg
:target: https://codecov.io/gh/zarr-developers/numcodecs

---
If you already have native Blosc, Zstd, and LZ4 installed on your system and want to use these system libraries instead of the vendored sources, you
should set the `NUMCODECS_USE_SYSTEM_LIBS=1` environment variable when building the wheel, like this:

$ NUMCODECS_USE_SYSTEM_LIBS=1 pip install numcodecs --no-binary numcodecs

Blosc, Zstd, and LZ4 are found via the `pkg-config` utility. Moreover, you must build all 3 `blosc`, `libzstd`, and `liblz4`
components. C-Blosc comes with full sources for LZ4, LZ4HC, Snappy, Zlib and Zstd and in general, you should not worry about not having (or CMake not finding) the libraries in your system because by default the included sources will be automatically compiled and included in the C-Blosc library. This effectively means that you can be confident in having a complete support for all the codecs in all the Blosc deployments (unless you are explicitly excluding support for some of them). To compile blosc, see these [instructions](https://github.com/Blosc/c-blosc?tab=readme-ov-file#compiling-the-blosc-library).
1 change: 1 addition & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ requires = [
"Cython",
"py-cpuinfo",
"numpy>2",
"pkgconfig"
]
build-backend = "setuptools.build_meta"

Expand Down
Loading
Loading