NOS enables you to do the following
- Analyze data stored on an external object store
- Read data in CSV, JSON, or Parquet format on an external object store
- Join or aggregate external data to relational data stored in Advanced SQL Engine
- Query cold data offloaded to an external object store
- Load data from an external object store into the database using one SQL request
There are two ways to read data from an external object store:
- Foreign Tables
- Users with CREATE TABLE privilege can create a foreign table inside the database, point this virtual table to an external storage location, and use SQL to translate the external data into a form useful for business.
- READ_NOS
- READ_NOS allows you to do the following:
- Perform an ad hoc query on CSV and JSON formatted data with the data in-place on an external object store
- List all the objects and path structure of an object store and read the schema of Parquet objects
- Bypass creating a foreign table in the Advanced SQL Engine
Supported External Object Storage Platforms
- Amazon S3
- Microsoft Azure Blob storage
- Azure Data Lake Storage Gen2
- Google Cloud Storage
Supported Compression Formats
External data may also arrive from an object in a compressed format. If that is the case, the data will be decompressed inside the Advanced SQL Engine, but only after decryption has been completed on the object store before being transmitted. GZIP is the only compression format supported for both JSON and CSV. Snappy is supported for Parquet. The database will recognize the ".gz" suffix on the incoming files and performs the decompression automatically. Compression may bring some trade-offs, such as CPU overhead vs. reduced needed Bandwidth amongst others.