The following tables detail the mapping of columns between SQL Engine and ML Engine.
td_coprocess_db.QueryLog
This is a summary table for each ML Engine analytic query.
| SQL Engine Column | Description | ML Engine Column | Source | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| CollectTimestamp | Time when statistics were collected for this row. | nc_system.nc_all_statement_stats.collecttimestamp | |||||||||
| PodName | ML Engine pod name. | nc_system.nc_all_statement_stats.podname | |||||||||
| UUID | SQL Engine universally unique query ID. | nc_system.nc_all_statement_stats.uuid | Joined from beehive_statement table. | ||||||||
| NumPhases | Total ML Engine phases involved in this query. | nc_system.nc_all_statement_stats.numphases | Joined from beehive_statement_phase table. | ||||||||
| ReadIOCount | Total read I/O operations (read syscalls). | nc_system.nc_all_statement_stats.readiocount | Aggregation of /proc/[pid]/io/syscr field value, the number of read I/O operations – that is, system calls such as read(2) and pread(2) value for all arc and runner processes. | ||||||||
| WriteIOCount | Total write I/O operations (write syscalls). | nc_system.nc_all_statement_stats.writeiocount | Aggregation of /proc//[pid]/io/syscw field value, the number of write I/O operations – that is, system calls such as write(2) and pwrite(2) value for all arc and runner processes. | ||||||||
| ReadBytes | Total number of bytes fetched from storage layer (read_bytes). | nc_system.nc_all_statement_stats.readbytes |
Aggregation of /proc/[pid]/io/read_bytes field value, the number of bytes which a process really did cause to be fetched from the storage layer, for all arc and runner processes. |
||||||||
| WriteBytes | Total number of bytes sent from storage layer (write_bytes). | nc_system.nc_all_statement_stats.writebytes | Aggregation of /proc/[pid]/io/write_bytes field value, the number of bytes that a process caused to be sent to the storage layer, for all arc and runner processes. | ||||||||
| ReadRateBytesPerSec | Number of bytes that storage layer read for each second. | nc_system.nc_all_statement_stats.writeatebytespersec | Delta of ReadBytes with STATS_UPDATE_INTERVAL_SEC * 100 / STATS_UPDATE_INTERVAL_SEC where Delta of ReadBytes = ReadBytes column value at current time - ReadBytes column value at the time before STATS_UPDATE_INTERVAL_SEC |
||||||||
| WriteRateBytesPerSec | Number of bytes that storage layer wrote for each second. | nc_system.nc_all_statement_stats.writeratebytespersec | Delta of WriteBytes with STATS_UPDATE_INTERVAL_SEC * 100 / STATS_UPDATE_INTERVAL_SEC where Delta of WriteBytes = WriteBytes column value at current time - WriteBytes column value at the time before STATS_UPDATE_INTERVAL_SEC |
||||||||
| CpuPercent | Total CPU percentage for this query. | nc_system.nc_all_statement_stats.cpupercent | Delta of stime+utime with STATS_UPDATE_INTERVAL_SEC * 100 / (STATS_UPDATE_INTERVAL_SEC * numCores ) where Delta of stime+utime = (stime+utime) value from /proc/[pid]/stat at current time - (stime+utime) value from /proc/[pid]/stat stat at the time before STATS_UPDATE_INTERVAL_SEC |
||||||||
| CpuSecs | Total CPU seconds for vworkers and runners. | nc_system.nc_all_statement_stats.cpusecs | Aggregation of /proc/[pid]/stat/stime + utime for all arc and runner processes. /proc/[pid]/stat/stime field is the amount of time that a process has been scheduled in kernel mode, measured in clock ticks (divide by sysconf(_SC_CLK_TCK)). /proc/[pid]/stat/utime field is the amount of time that a process has been scheduled in user mode, measured in clock ticks divide by sysconf(_SC_CLK_TCK)). |
||||||||
| MemoryKb | Total memory taken by this query. | nc_system.nc_all_statement_stats.memorykb | Aggregation /proc/[pid]/stat/rss field value, multiplied by pageSizeKB for all arc and runner processes where pageSizeKB = getpagesize()/1024. /proc/[pid]/stat/rss field is Resident Set Size: number of pages the process has in real memory. This is just the pages that count toward text, data, or stack space. This does not include pages that have not been demand-loaded in, or that are swapped out. |
||||||||
| QueryText | ML Engine query text (possibly truncated). | nc_system.nc_all_statement_stats.querytext | Joined from beehive_statement table. | ||||||||
| ErrorCode | Query result code:
|
nc_system.nc_all_statement_stats.errorcode | Joined from beehive_statement table. | ||||||||
| ErrorText | If not null, this field provides error text returned from query execution (possibly truncated). | nc_system.nc_all_statement_stats.errortext | Joined from beehive_statement table. | ||||||||
| SessionId | Unique sessionid. No correlation to SQL Engine or Teradata QueryGrid session IDs. | nc_system.nc_all_statement_stats.sessionid | Joined from beehive_statement table. | ||||||||
| ParentSessionId | Parent session ID. Empty for nonchild queries. | nc_system.nc_all_statement_stats.parentsessionid | Joined from beehive_statement table. | ||||||||
| QueryId | Unique ML Engine query ID. | nc_system.nc_all_statement_stats.queryid | Joined from beehive_statement table. | ||||||||
| StartTime | Timestamp when query was submitted. | nc_system.nc_all_statement_stats.starttime | Joined from beehive_statement table. | ||||||||
| EndTime | Timestamp when query finished. | nc_system.nc_all_statement_stats.endtime | Joined from beehive_statement table. |
td_coprocessor_db.ResUsageNetwork
This table contains a row for each time ML Engine collects the networking counts.
| SQL Engine Column | Description | ML Engine Column | Source |
|---|---|---|---|
| CollectTimeStamp | Time when statistics were collected for this row. | nc_system.nc_all_resource_usage_network_stats.collectimestamp | Heapster metrics are collected every 60 seconds. |
| PodName | ML Engine pod name. | nc_system.nc_all_resource_usage_network_stats.podname | |
| NodeId | ML Engine node where pod is running. | nc_system.nc_all_resource_usage_network_stats.nodeid | |
| ReadBytes | Cumulative number of bytes received over network. | nc_system.nc_all_resource_usage_network_stats.readbytes | Heapster |
| WriteBytes | Cumulative number of bytes sent over network. | nc_system.nc_all_resource_usage_network_stats.writebytes | Heapster |
| ReadRateBytesPerSec | Number of bytes received over network for each second. | nc_system.nc_all_resource_usage_network_stats.readratebytespersec | Heapster |
| ReadErrorsRateSecs | Number of errors while receiving over network for each second. | nc_system.nc_all_resource_usage_network_stats.readerrorsratesecs | Heapster |
| WriteRateBytesPerSec | Number of bytes sent over network for each second. | nc_system.nc_all_resource_usage_network_stats.writeratebytespersec | Heapster |
| WriteErrorsRateSecs | Number of errors while sending over network for each second. | nc_system.nc_all_resource_usage_network_stats.writeerrorsratesecs | Heapster |
td_coprocessor_db.ResUsageCpu
This table contains a row for each time ML Engine collects the CPU counts.
| SQL Engine Column | Description | ML Engine Column | Source |
|---|---|---|---|
| CollectTimeStamp | Time when statistics were collected for this row. | nc_system.nc_all_resource_usage_cpu_stats.collectimestamp | |
| PodName | ML Engine pod name. | nc_system.nc_all_resource_usage_cpu_stats.podname | |
| NodeId | ML Engine node identifier. | nc_system.nc_all_resource_usage_cpu_stats.nodeid | |
| CpuIOWaitCentiSecs | Time in centiseconds CPU is waiting for I/O completion. | nc_system.nc_all_resource_usage_cpu_stats.cpuiowaitcentisecs | Aggregated value of Prometheus Node Exporter iowait time for all CPU cores for NodeId. Retrieved from Prometheus Node Exporter. Prometheus Node Exporter retrieves its values from /proc/stat/iowait field. The time waiting for I/O to complete. This value is not reliable for these reasons:
|
td_coprocessor_db.ResUsageDisk
This table is created but is empty.
| SQL Engine Column | Description | ML Engine Column |
|---|---|---|
| CollectTimeStamp | Time when statistics were collected for this row. | nc_system.nc_all_resource_usage_disk_stats.collectimestamp |
| PodName | ML Engine pod name. | nc_system.nc_all_resource_usage_disk_stats.podname |
| NodeId | ML Engine node identifier. | nc_system.nc_all_resource_usage_disk_stats.nodeid |
td_coprocessor_db.ResUsageMemory
This table contains a row for each time ML Engine collects memory counts.
| SQL Engine Column | Description | ML Engine Column | Source |
|---|---|---|---|
| CollectTimeStamp | Time when statistics were collected for this row. | nc_system.nc_all_resource_usage_memory_stats.collectimestamp | |
| PodName | ML Engine pod name. | nc_system.nc_all_resource_usage_memory_stats.podname | |
| NodeId | ML Engine node identifier. | nc_system.nc_all_resource_usage_memory_stats.nodeid | |
| TotalMemoryUsage | Total memory usage. | nc_system.nc_all_resource_usage_memory_stats.totalmemoryusage | Heapster |
| PageFaultsRateSec | Number of page faults for each second. | nc_system.nc_all_resource_usage_memory_stats.pagefaultsrate | Heapster |