- Specifies the behavior of the target connector component
- Configures how data is transformed
- Configures the underlying link data transportation layer
Links are named configurations that include the target connector. If the same property is set for a link and a connector, the link setting overrides the connector setting.
Basic Tab
| BigQuery Connector | |||
|---|---|---|---|
| Name | Default | Description | Overridable? Property Name |
| Project ID | None | A unique identifier for the
project. This is a required setting. |
|
| Dataset Name | None | The dataset name to use if no
dataset was specified in the query. This is a required setting. |
|
Security Tab
| BigQuery Connector | |||
|---|---|---|---|
| Name | Default | Description | Overridable? Property Name |
| Authentication Mechanism | Application Default | The authentication mechanism used on the cluster. Valid values are Application Default, Service Account, Refresh Token (OAuth). |
|
| Service Account Key Path | None | Path for the service account key json file. The key must be present at the path and accessible on every driver node. QueryGrid must have read access to the key. This property is used when Authentication Mechanism is set to Service Account. This property is mutually exclusive from Service Account Key. |
|
| Service Account Key | None | The contents of the service account key are placed in this property. The key is redacted on Viewpoint and logs. This property is used when Authentication Mechanism is set to Service Account. This property is mutually exclusive from Service Account Key Path |
|
| Client ID | None | ClientID of the client to use with OAuth Refresh Token Authentication when an Authorization Object is not being used. Required when Authentication Mechanism is set to Refresh Token (OAuth). |
|
| Client Secret | None | The client secret to use with
OAuth Refresh Token
Authentication when an Authorization Object is not
being used. Required when Authentication Mechanism is set to Refresh Token (OAuth). |
|
| OAuth Refresh Token | None | Refresh Token to use with OAuth
Refresh Token Authentication when an Authorization Object is not
being used. Required when Authentication Mechanism is set to Refresh Token (OAuth). |
|
| KMS Key Name | False | Resource ID for the cloud KMS key. Provide this value if you do not want Google Cloud to manage the encryption keys for temporary tables created by QueryGrid. | |
Advanced Tab
| BigQuery Connector | |||
|---|---|---|---|
| Name | Default | Description | Overridable? Property Name |
| Temporary Dataset Name | None | QueryGrid needs to create temporary tables when running queries. Setting this property places the temporary tables in the specified BigQuery Dataset within the project. | ● tempDbName |
| Enable Logging | INFO | Logging level for the connector
or link properties. User-level log settings can be set through the
Add or Edit link page in the QueryGrid portlet. Valid values are NONE, WARN, INFO, and DEBUG. |
|
| 16.20+ LOB Support | True | On Teradata systems version 16.20 and later, the STRING and BINARY columns on BigQuery are mapped to CLOB and BLOB by default. Deselect this option to map the STRING and BINARY columns to VARCHAR and VARBYTE, respectively. Disable this option if there are a large number of STRING/BINARY columns in the BigQuery table. |
● lobSupport |
| Default String Size | 32000 characters | The VARCHAR truncation size. The value represents the maximum number of Unicode characters to import, and defaults to 32000 characters. QueryGrid truncates the string columns at the default value set in defaultStringSize if less than the actual column size. Valid values are 1–1048544000 characters. |
● defaultStringSize |
| Default Binary Size | 64000 bytes | The VARBYTE truncation size. Valid values are 1–2097088000 bytes. |
● defaultBinarySize |
| Maximum Number of Transfer Threads | 5 | Maximum number of threads to use for uploading or downloading BigQuery data in a single request. BigQuery has a Storage Write API limitation on concurrent connections that may require reducing the maximum number of threads value if a large number of concurrent queries are expected. |
● numberOfTransferThread |
| Link Buffer Count | 4 | Maximum number of WRITE buffers available on a
single channel at one time. Link
Buffer Count overrides the default internal
fabric property shmDefaultNumMemoryBuffers.
Valid values are 2–16. |
linkBufferCount |
| Response Timeout | 86400000 | The number of milliseconds to wait for the
target query to complete before timing out and stopping the
operation. The fabric stops and releases all resources associated
with queries whose duration exceeds the value set on the target link
properties or the target connector properties. Connectors timeout
when responses from the fabric exceed their response timeout
value. Valid values are 300000–172800000. |
responseTimeout |
| Connection Max Idle Time | 86400 seconds | The maximum idle time for the connection cache
object, after which the object is closed and removed from the cache.
Use this property when there are multiple concurrent users and
queries running on the system that might lead to starvation of
connection objects. Valid values are 1–86400 seconds. |
|
| Connection Pool Size | 100 | Maximum number of connection objects that can be
stored in a connection pool. When acquiring a new connection, the
connector checks for an available space in the pool. If no space is
available in the connection pool, the connection fails after 5
minutes. Only one connection pool and username per connector
configuration is allowed. Valid values are 1–10000. |
|
| Connection Evict Frequency | 30 minutes | Frequency of eviction checks. Connection objects
from the pool are checked, closed, and removed if the idle time
(current time - last time of use) of a connection object is greater
than the Connection Max Idle
Time setting. Reduce the time between checks if there are multiple concurrent users running queries to clear the connections more frequently. Valid values are 1–1440 minutes. |
|