Configure a Foreign Server | Spark SQL-to-Teradata | QueryGrid - Configuring a Foreign Server for Spark SQL-to-Teradata - Teradata QueryGrid

QueryGridâ„¢ Installation and User Guide - 3.06

Deployment
VantageCloud
VantageCore
Edition
Enterprise
IntelliFlex
Lake
VMware
Product
Teradata QueryGrid
Release Number
3.06
Published
December 2024
ft:locale
en-US
ft:lastEdition
2024-12-07
dita:mapPath
ndp1726122159943.ditamap
dita:ditavalPath
ft:empty
dita:id
lxg1591800469257
lifecycle
latest
Product Category
Analytical Ecosystem
The Data Sources API framework is used for the QueryGrid Spark SQL initiator connector. Data Sources API allows Spark SQL to access data stored on other systems. A Spark SQL table created using a Data Source API framework is referred to as a non-native table.

The Data Sources API operates at the table level by default, so one non-native table must be created for each remote table, unlike Teradata Foreign Servers and Presto Catalogs that operate at the database level. However, a Foreign Server Library has been included as part of the Spark SQL connector, which addresses some of the limitations and inconveniences when working with non-native tables. Teradata recommends that the Foreign Server Library be used to interact with the Spark SQL initiator and all Spark SQL initiator examples in this section are based on the Foreign Server Library. For complete details on the foreign server library, see Foreign Server Library API Reference for the Spark SQL Initiator Connector.

The following steps provide an example of configuring a foreign server in order to use it with a Spark SQL-to-TargetConnector (where TargetConnector is any type of target connector):
  1. Set the link properties for the Spark SQL-to-Teradata link in the QueryGrid portlet.
  2. Log on to Scala REPL.
    See Starting Scala REPL for more information.
  3. Import the Foreign Server Library and create a foreign server object, for example:
    scala> import tdqg.ForeignServer
    import tdqg.ForeignServer
     
    scala> val s1 = new ForeignServer("spark_to_teradata_link","active","fs1")
    s1: tdqg.ForeignServer = tdqg.ForeignServer@4eb73cc8
  4. Use the foreign server to show remote schemas and verify the results, for example:
    scala> s1.showSchemas
    +---------------+
    |DATABASE_NAME  |
    +---------------+
    |default	|
    |user1	|
    +---------------+