Important: - The BigQuery connector is a target connector only and cannot be used as an initiating connector.
- All BigQuery target connector nodes must be driver nodes for communication and data transfer to and from the BigQuery system.
- Install the BigQuery target connector on a host that is able to reach Google BigQuery.
- See Automatically Deploy QueryGrid on Google Cloud Instance Group for additional details.
The following are the minimum requirements for a BigQuery target connector node to install the QueryGrid 3.x software:
- 4GB RAM
- 50GB disk space
- For supported operating system versions, refer to the QueryGrid Compatibility Matrix.
- Memory requirements as defined in the fabric/connector setup
- Latest version of JAVA 8 or 11
When using the BigQuery connector, the system running the QueryGrid driver process handles a greater workload as compared to other connectors. The CPU and memory configuration of the driver node has a significant impact on query performance. The following sample test results are from using different CPU and memory configurations. You can use multiple driver nodes for load balancing.
Throughput
Throughput per query decreases as more concurrent queries are run, but the total throughput of the system increases.
BYNET connection is used between Teradata and the driver node for the following reference data:
2 CPUs, 8GB RAM, 2GB DriverHeapSize, number of threads = 5
| Sessions |
CPU Utilization |
Throughput |
|---|
| Single read session |
9.27% |
9.13MB/sec |
| 5 concurrent read sessions |
33.51% |
8.33MB/sec |
| 20 concurrent read sessions |
70.49% |
7.19MB/sec |
| Sessions |
CPU Utilization |
Throughput |
|---|
| Single write session |
12.07% |
3.26MB/sec |
| 5 concurrent write sessions |
68.62% |
2.24MB/sec |
| 20 concurrent write sessions |
99.01% |
0.35MB/sec |
4 CPUs, 16GB RAM, 2GB DriverHeapSize, number of threads = 5
| Sessions |
CPU Utilization |
Throughput |
|---|
| Single read session |
3.76% |
9.12MB/sec |
| 5 concurrent read sessions |
14.63% |
8.6MB/sec |
| 20 concurrent read sessions |
29.69% |
8.83MB/sec |
| Sessions |
CPU Utilization |
Throughput |
|---|
| Single write session |
5.52% |
3.37MB/sec |
| 5 concurrent write sessions |
50.46% |
2.94MB/sec |
| 20 concurrent write sessions |
97.71% |
0.66MB/sec |