Override Session Properties | Spark SQL Initiator Connector | QueryGrid - Overriding Session Properties (Spark SQL) - Teradata QueryGrid

QueryGridâ„¢ Installation and User Guide - 3.06

Deployment
VantageCloud
VantageCore
Edition
Enterprise
IntelliFlex
Lake
VMware
Product
Teradata QueryGrid
Release Number
3.06
Published
December 2024
ft:locale
en-US
ft:lastEdition
2024-12-07
dita:mapPath
ndp1726122159943.ditamap
dita:ditavalPath
ft:empty
dita:id
lxg1591800469257
lifecycle
latest
Product Category
Analytical Ecosystem
Designating a Spark SQL connector property as overridable in the QueryGrid portlet allows you to override configured Spark SQL connector properties when running queries during an individual processing session.

For the Spark SQL initiator session, setSessionOverride() is used at the Foreign Server level to override configured Spark SQL connector properties. The session override applies to all non-native tables that belong to the Foreign Server.

  1. Under Fabric Configuration, select Fabrics.
  2. Select the fabric.
  3. Perform one of the following:
    Option Description
    To override a Spark SQL connector property in the Connectors tab
    1. Select the Connectors tab.
    2. Select "" next to the connector that has a property to override.
    3. Select Edit.
    4. Select "" next to Connector Software.
    5. Find the Spark SQL connector property to override and select the Overridable check box.
    6. Select OK.
    7. Select Save..
    To override a Spark SQL connector property in the Links tab
    1. Select the Links tab.
    2. Select "" next to the link that has a property to override.
    3. Select Edit.
    4. Select "" next to Initiating Connector or Target Connector.
    5. Find the Spark SQL link property to override and select the Overridable check box.
    6. Select OK.
    7. Select Save..
  4. [Optional] To override the Spark SQL connector property during a Spark SQL session for Foreign Server s1, use the syntax in the following example where the numExecutors and hadoopProperties connector properties are overridden.
    s1.setSessionOverride("linkBufferSize=100000, numExecutors=4, hadoopProperties=name1:value1 | name2:value2")
    When overriding hadoopProperties, you must use the pipe (|) symbol to split Hadoop properties and a colon (:) to split name value pairs, as shown in these examples.
  5. [Optional] To clear Spark SQL connector property overrides without closing a Spark SQL session for Foreign Server S1, use the syntax in the following example:
    s1.setSessionOverride("")