External data may arrive from an object in a compressed format. The following are compression types supported for the supported file formats:
Iceberg
| Parquet | ORC | AVRO | |
|---|---|---|---|
| SNAPPY | Yes | Yes | Yes |
| ZSTD (zstandard) | Yes | Yes | Yes |
| ZLIB | No | Yes | No |
| LZ4 | No | Yes | No |
| GZIP (deflate) | Yes | No | Yes |
| LZO | No | No | No |
ZSTD compression is the default for Iceberg - it provides a good compression ratio without compromising the read/write performance. Snappy provides a good read/write performance; however, its compressions ratio is lower than ZSTD or GZIP. If the storage costs permits, Snappy is a better compression choice.
Delta Lake
| Parquet | |
|---|---|
| SNAPPY | Yes |
| ZSTD (zstandard) | No |
| ZLIB | No |
| LZ4 | No |
| GZIP (deflate) | Yes |
| LZO | Yes |