To deploy QueryGrid on Presto on Kubernetes running on Google Kubernetes Engine (GKE) or Azure Kubernetes Service (AKS), you must add a tdqg-node pod on every Kubernetes node in the cluster.
This procedure assumes the following prerequisites:
- A Kubernetes cluster has already been created or running.
- The Kubernetes cluster has enough resources to run a QueryGrid Presto node pod. The following are the minimum requirements for QueryGrid Presto node pod on AKS/GKS:
- 4 GB RAM Number of CPU - 1
- The kubectl version is identical to the kubernetes cluster version.
- The helm version is 3.2.4 or later.
- Download the tdqg-presto-k8s-deploy package, see Downloading Required Packages for more information.
- If needed, upload the QueryGrid docker images to the cloud docker image by doing the following:
- Unzip the tdqg-presto-k8s-deploy package and retrieve the following:
- tdqg-node_version.tar.gz and tdqg-init-presto_version under docker-images folder
- tdqg-presto-node-chart-version.tar.gz
- Upload the two docker images under the docker-images folder to the cloud container image repository/registry by following the Azure or Google Cloud documentation as appropriate. This may involve the following tasks:
- Installing Docker on your workstation
- Loading the image
- Tagging and pushing the image to the cloud repository
- Unzip the tdqg-presto-k8s-deploy package and retrieve the following:
- Unzip tdqg-presto-node-chart-version.tar.gz.
- Update values.yaml under tdqg-presto-node folder to to include the path for tdqg-init-presto and tdqg-node docker image as in the following GKE and AKS examples:
registryUrl: gcr.io/gcp-project-id-here tdqgInitPrestoVersion: latest tdqgNodeVersion: latest
registryUrl: tdqgcr.azurecr.io tdqgInitPrestoVersion: latest tdqgNodeVersion: latest
- If not using the default fabric ports for the current fabric and pending fabric upgrades, edit values.yaml to include the correct fabric ports.
- Create system, fabric and deploy tdqg-node pods using one of the following two options:
Using install_tdqg_node_pod.py script If you have QueryGrid Manager credentials you can use the install_tdqg_node_pod.py script found in the tdqg-presto-node folder. Following are the prerequisites to run the script. - Make sure the system running the script can connect to QueryGrid Manager and the Kubernetes cluster.
- Install the following dependency packages where you are running the script:
- python-requests
- pip3
- logging
- pyyaml
- kubectl
- helm (>=3.2.4)
- python 3 or later
- In the tdqg_input.yaml input file, enter details such as the QueryGrid Manager and system information.
The script creates a system then deploys tdqg-node pods on the Kubernetes cluster.
- Run the kubectl get nodes -o wide command from where you plan to run the script to make sure the Kubernetes cluster is up and running.
Check that all nodes have a status of Ready. This is the cluster where the script deploys the QueryGrid node pods. Refer to the README file for more details.
- Run the script as in the following example:.
python3 path_to_install_tdqg_nod_pod.py script
Manual installation Use the manual installation is you do not have QueryGrid Manager credentials. - From Viewpoint, create a QueryGrid system.
- From the system created by QueryGrid Manager, download the tdqg-node.json token file.
- Place the tdqg-node.json file directly under the tdqg-presto-node.
$ ls /tdqg-presto-node/tdqg-node* /tdqg-presto-node/tdqg-node.json
- Deploy tdqg-node pods.
helm install tdqg-presto-node <path_tdqg_node_pod.py_folder>
- Confirm that there is a tdqg-node pod running on each instance with the status of Running:
Status Action Running None. A node is available in Viewpoint for each node in the kubernetes cluster. Not Running Run kubectl get pods -o wide to verify the pods are running. If the kubernetes cluster is resized, then tdqg-node pods appear on each new Kubernetes host.