Presto Target Connector on Kubernetes | QueryGrid - Creating Presto Target Tasks for Presto on Kubernetes (GKE and AKS) - Teradata QueryGrid

QueryGrid™ Installation and User Guide - 3.06

Deployment
VantageCloud
VantageCore
Edition
Enterprise
IntelliFlex
Lake
VMware
Product
Teradata QueryGrid
Release Number
3.06
Published
December 2024
ft:locale
en-US
ft:lastEdition
2024-12-07
dita:mapPath
ndp1726122159943.ditamap
dita:ditavalPath
ft:empty
dita:id
lxg1591800469257
lifecycle
latest
Product Category
Analytical Ecosystem
To deploy QueryGrid on Presto on Kubernetes in Google Cloud (GKE) or Azure (AKS), create the Presto target connector in Viewpoint. Use a cloud-reserved static external IP in the Server property. Refer to Google Cloud or Azure documentation for information on how to reserve a static IP address if you do not already have one. The Google Cloud static IP must be a premium static IP.
  1. If not yet acquired, download the tdqg-presto-k8s-deploy package, see Downloading Required Packages for more information.
  2. Unzip the tdqg-presto-k8s-deploy package, and retrieve the following files:
    • tdqg-node docker image.zip
    • tdqg-init-presto docker image.zip
    • tdqg-presto-node.zip.zip
  3. Create the Presto target connector using a cloud-reserved static external IP.
    • If you previously used the install_tdqg_node_pod.py script for Automatically Deploy QueryGrid on Presto on Kubernetes (GKE or AKS), you can use the same script to create the Presto target connector. Add the fabric and connector details to the tdqg_input.yaml file. Provide the cloud-reserved static external IP in the Server property and run the script.
    • If you did not use the script, create the Presto connector on Viewpoint and use the cloud-reserved static external IP in the Server property.
  4. Create the Presto values.yaml file to deploy Presto using a Helm chart.
    When modifying an existing Presto deployment, you must update the version of the Presto values.yaml file to include the QueryGrid sections. There are two options when modifying: manually or with an install script.
    Action Description
    Adding a new Presto deployment When adding a new Presto deployment, the presto_values.yaml under tdqg-presto-node can be used as the values.yaml file.
    1. Perform standard Presto configuration tasks such as the number of nodes, cpu, memory, and catalogs to point at databases.
    2. Replace the Node software version and Presto connector version under initFile with the actual versions.
    Modifying an existing Presto deployment Add the following sections from the presto_values.yaml file to your existing Presto values.yaml configuration file:
    • initFile.sh – replace node software version and Presto connector version with the actual versions
    • additionalVolumeshostPaths volumes that allow the tdqg-node pod and the Presto pods to exchange files
    • coordinator: affinity and worker: affinity – Presto coordinators and workers must not share the same host
    • catalogs – Add qgremote catalog
  5. Deploy Presto using Helm by adding the Starburst Presto repository by running the helm repo add command.
  6. After the helm repository is added, deploy Presto using the following commands:
    helm upgrade my-presto starburstdata/starburst-enterprise \
    --install \
    --version <Presto_version> \ #e.g. 380.0.1
    --values <path to Presto values.yaml file>
    
  7. Check that the operation was successful:
    kubectl get pods -o wide
    The operations is successful when the following is visible:
    • A tdqg-node pod in the running state for each Kubernetes host
    • A Presto coordinator pod
    • The number of Presto worker pods that were specified earlier

    If the pods are starting and stopping, check the container logs to troubleshoot them.

  8. Expose the Presto server outside the Kubernetes pod network.
    This can be achieved by creating a load balancer with the static IP. Refer to the Starburst Enterprise Presto and Google Cloud/Azure documentation on how to create a load balancer. Redeploy Presto pods if required.
  9. Wait approximately 30 seconds before running kubectl get services to confirm the external IP has been assigned to the Presto Coordinator.
    When upgrading a QueryGrid node or connector software, the Node software version and Presto connector version need to be updated in the Presto configuration yaml file and reapplied, which triggers a restart of all the Presto pods.