1.1 - Table Column Mapping - Teradata Vantage

Teradata Vantage™ User Guide

prodname
Teradata Vantage
vrm_release
1.1
created_date
May 2020
category
User Guide
featnum
B700-4002-079K

The following tables detail the mapping of columns between Advanced SQL Engine and ML Engine.

td_mle_db.QueryLog

This is a summary table for each ML Engine analytic query.

Advanced SQL Engine Column Description ML Engine Column Source
CollectTimestamp Time when statistics were collected for this row. nc_system.nc_all_statement_stats.collecttimestamp  
PodName ML Engine pod name. nc_system.nc_all_statement_stats.podname  
UUID Advanced SQL Engine universally unique query ID. nc_system.nc_all_statement_stats.uuid Joined from beehive_statement table.
NumPhases Total ML Engine phases involved in this query. nc_system.nc_all_statement_stats.numphases Joined from beehive_statement_phase table.
ReadIOCount Total read I/O operations (read syscalls). nc_system.nc_all_statement_stats.readiocount Aggregation of /proc/[pid]/io/syscr field value, the number of read I/O operations – that is, system calls such as read(2) and pread(2) value for all arc and runner processes.
WriteIOCount Total write I/O operations (write syscalls). nc_system.nc_all_statement_stats.writeiocount Aggregation of /proc//[pid]/io/syscw field value, the number of write I/O operations – that is, system calls such as write(2) and pwrite(2) value for all arc and runner processes.
ReadBytes Total number of bytes fetched from storage layer (read_bytes). nc_system.nc_all_statement_stats.readbytes

Aggregation of /proc/[pid]/io/read_bytes field value, the number of bytes which a process really did cause to be fetched from the storage layer, for all arc and runner processes.

WriteBytes Total number of bytes sent from storage layer (write_bytes). nc_system.nc_all_statement_stats.writebytes Aggregation of /proc/[pid]/io/write_bytes field value, the number of bytes that a process caused to be sent to the storage layer, for all arc and runner processes.
ReadRateBytesPerSec Number of bytes that storage layer read for each second. nc_system.nc_all_statement_stats.writeatebytespersec Delta of ReadBytes with STATS_UPDATE_INTERVAL_SEC * 100 / STATS_UPDATE_INTERVAL_SEC

where Delta of ReadBytes =

ReadBytes column value at current time - ReadBytes column value at the time before STATS_UPDATE_INTERVAL_SEC

WriteRateBytesPerSec Number of bytes that storage layer wrote for each second. nc_system.nc_all_statement_stats.writeratebytespersec Delta of WriteBytes with STATS_UPDATE_INTERVAL_SEC * 100 / STATS_UPDATE_INTERVAL_SEC

where Delta of WriteBytes =

WriteBytes column value at current time - WriteBytes column value at the time before STATS_UPDATE_INTERVAL_SEC

CpuPercent Total CPU percentage for this query. nc_system.nc_all_statement_stats.cpupercent Delta of stime+utime with STATS_UPDATE_INTERVAL_SEC * 100 / (STATS_UPDATE_INTERVAL_SEC * numCores )

where Delta of stime+utime =

(stime+utime) value from /proc/[pid]/stat at current time - (stime+utime) value from /proc/[pid]/stat stat at the time before STATS_UPDATE_INTERVAL_SEC

CpuSecs Total CPU seconds for vworkers and runners. nc_system.nc_all_statement_stats.cpusecs Aggregation of /proc/[pid]/stat/stime + utime for all arc and runner processes.

/proc/[pid]/stat/stime field is the amount of time that a process has been scheduled in kernel mode, measured in clock ticks (divide by sysconf(_SC_CLK_TCK)).

/proc/[pid]/stat/utime field is the amount of time that a process has been scheduled in user mode, measured in clock ticks divide by sysconf(_SC_CLK_TCK)).

MemoryKb Total memory taken by this query. nc_system.nc_all_statement_stats.memorykb Aggregation /proc/[pid]/stat/rss field value, multiplied by pageSizeKB for all arc and runner processes where pageSizeKB = getpagesize()/1024.

/proc/[pid]/stat/rss field is Resident Set Size: number of pages the process has in real memory. This is just the pages that count toward text, data, or stack space. This does not include pages that have not been demand-loaded in, or that are swapped out.

ErrorCode Query result code:
Code Meaning
1 Canceled
2 Succeeded
3 Error
nc_system.nc_all_statement_stats.errorcode Joined from beehive_statement table.
Cancellable
Code Meaning
1 Canceled
2 Succeeded
3 Error
nc_system.nc_all_statement_stats.cancellable Joined from beehive_statement table.
SessionId Unique sessionid. No correlation to Advanced SQL Engine or Teradata QueryGrid session IDs. nc_system.nc_all_statement_stats.sessionid Joined from beehive_statement table.
ParentSessionId Parent session ID. Empty for nonchild queries. nc_system.nc_all_statement_stats.parentsessionid Joined from beehive_statement table.
QueryId Unique ML Engine query ID. nc_system.nc_all_statement_stats.queryid Joined from beehive_statement table.
StartTime Timestamp when query was submitted. nc_system.nc_all_statement_stats.starttime Joined from beehive_statement table.
EndTime Timestamp when query finished. nc_system.nc_all_statement_stats.endtime Joined from beehive_statement table.
ParentQueryId The parent query id. nc_system.nc_all_statement_stats.parentqueryid Joined from beehive_statement table.
WorkloadName The workload name this query is mapped to. nc_system.nc_all_statement_stats.workloadname Joined from _bee_special.nc_qos_workload_transition table.
ServiceClassName The workload class this query is mapped to. nc_system.nc_all_statement_stats.serviceclassname Joined from _bee_special.nc_qos_workload_transition table.
Priority The workload priority. nc_system.nc_all_statement_stats.priority Joined from _bee_special.nc_qos_workload_transition table.
Weight The workload weight. nc_system.nc_all_statement_stats.weight Joined from _bee_special.nc_qos_workload_transition table.

td_mle_db.queryLogSQL

This table is a summary table for ML Engine query error and text.

Advanced SQL Engine Column Description ML Engine Column Source
CollectTimeStamp Time when statistics were collected for this row. nc_system.nc_all_statement_stats.collectimestamp  
QueryId Unique ML Engine query ID. nc_system.nc_all_statement_stats.queryid Joined from beehive_statement table.
QueryText ML Engine query text. Truncation may occur. nc_system.nc_all_statement_stats.querytext Joined from beehive_statement table.
ErrorText Cumulative number of bytes received over network. nc_system.nc_all_statement_stats.errortext Joined from beehive_statement table.

td_mle_db.ResUsageNetwork

This table contains a row for each time ML Engine collects the networking counts.

Advanced SQL Engine Column Description ML Engine Column Source
CollectTimeStamp Time when statistics were collected for this row. nc_system.nc_all_resource_usage_network_stats.collectimestamp Heapster metrics are collected every 60 seconds.
PodName ML Engine pod name. nc_system.nc_all_resource_usage_network_stats.podname  
NodeId ML Engine node where pod is running. nc_system.nc_all_resource_usage_network_stats.nodeid  
ReadBytes Cumulative number of bytes received over network. nc_system.nc_all_resource_usage_network_stats.readbytes Heapster
WriteBytes Cumulative number of bytes sent over network. nc_system.nc_all_resource_usage_network_stats.writebytes Heapster
ReadRateBytesPerSec Number of bytes received over network for each second. nc_system.nc_all_resource_usage_network_stats.readratebytespersec Heapster
ReadErrorsRateSecs Number of errors while receiving over network for each second. nc_system.nc_all_resource_usage_network_stats.readerrorsratesecs Heapster
WriteRateBytesPerSec Number of bytes sent over network for each second. nc_system.nc_all_resource_usage_network_stats.writeratebytespersec Heapster
WriteErrorsRateSecs Number of errors while sending over network for each second. nc_system.nc_all_resource_usage_network_stats.writeerrorsratesecs Heapster

td_mle_db.ResUsageCpu

This table contains a row for each time ML Engine collects the CPU counts.

Advanced Engine Column Description ML Engine Column Source
CollectTimeStamp Time when statistics were collected for this row. nc_system.nc_all_resource_usage_cpu_stats.collectimestamp  
PodName ML Engine pod name. nc_system.nc_all_resource_usage_cpu_stats.podname  
NodeId ML Engine node identifier. nc_system.nc_all_resource_usage_cpu_stats.nodeid  
CpuIOWaitCentiSecs Time in centiseconds CPU is waiting for I/O completion. nc_system.nc_all_resource_usage_cpu_stats.cpuiowaitcentisecs Aggregated value of Prometheus Node Exporter iowait time for all CPU cores for NodeId.

Retrieved from Prometheus Node Exporter.

Prometheus Node Exporter retrieves its values from /proc/stat/iowait field.

The time waiting for I/O to complete. This value is not reliable for these reasons:

  • The CPU does not wait for I/O to complete; iowait is the time that a task is waiting for I/O to complete. When a CPU goes into idle state for outstanding task I/O, another task will be scheduled on this CPU.
  • On a multi-core CPU, the task waiting for I/O to complete is not running on any CPU, so the iowait on each CPU is difficult to calculate.
  • The value in this field may decrease in certain conditions.
CpuPercent Total CPU percentage for this node nc_system.nc_all_resource_usage_cpu_stats.cpupercent Heapster

td_mle_db.ResUsageDisk

This table is created but is empty.

Advanced SQL Engine Column Description ML Engine Column
CollectTimeStamp Time when statistics were collected for this row. nc_system.nc_all_resource_usage_disk_stats.collectimestamp
PodName ML Engine pod name. nc_system.nc_all_resource_usage_disk_stats.podname
NodeId ML Engine node identifier. nc_system.nc_all_resource_usage_disk_stats.nodeid

td_mle_db.ResUsageMemory

This table contains a row for each time ML Engine collects memory counts.

Advanced SQL Engine Column Description ML Engine Column Source
CollectTimeStamp Time when statistics were collected for this row. nc_system.nc_all_resource_usage_memory_stats.collectimestamp  
PodName ML Engine pod name. nc_system.nc_all_resource_usage_memory_stats.podname  
NodeId ML Engine node identifier. nc_system.nc_all_resource_usage_memory_stats.nodeid  
TotalMemoryUsage Total memory usage. nc_system.nc_all_resource_usage_memory_stats.totalmemoryusage Heapster
PageFaultsRateSec Number of page faults for each second. nc_system.nc_all_resource_usage_memory_stats.pagefaultsrate Heapster