Things to Consider When Logging DBQL Data - Teradata Database
Teradata Database Administration
Product
Teradata Database
Release Number
15.10
Language
English (United States)
Last Update
2018-10-06
Product Category
Software
Preface
Audience
Supported Software Releases and Operating Systems
Prerequisites
Changes to This Book
Additional Information
Product Safety Information
Teradata Database Optional Features
Chapter 1 Setting Up Your Administrative Environment: All DBAs
Logging in to the Operating System
About Root Logons
About Tdtrusted Logons
About User DBC
Setting Up Teradata Administrator
Set Up the Data Directory for Teradata Administrator
Setting Up the Database Administrator User
Creating the DBADM Profile
Creating User DBADMIN
Granting Database Privileges to User DBADMIN
Creating a Spool Reserve Database
Guidelines for Reserving Minimum Spool Space
Using Viewpoint Alerts
Setting Up Alert Actions
Creating Alerts
Setting Up Teradata Viewpoint Alerts for Space Usage
About the Global Default Parameters
Chapter 2 About Databases and Users in Teradata: All DBAs
About Databases and Users
Space Used by Databases and Users
Creating Versus Owning Objects in the Hierarchy
About Creating a Database or User
The System Users
System User DBC
System User SYSTEMFE
System User SYSADMIN
Other System Databases and Users
Recommended Hierarchy
Secure Zones Overview
Chapter 3 Working with Databases: All DBAs
About Database Creation
Best Practices for Database Creation
Creating the Tables Database
Creating the Views Database
Working with Table Access Privileges for Views
Dropping a Database or User
Transferring Ownership with GIVE
Chapter 4 Working with Tables and Views: Application DBAs
Choosing a Primary Index
Guidelines for Choosing Primary Index Columns
Unique and Nonunique Primary Indexes
Row Partitioning
No Primary Index (NoPI) Tables
Relationship of Primary Indexes, Primary Keys, and Foreign Keys
Creating Tables
Considerations for Defining Tables
Creating Tables in Teradata
Copying a Table
Copying Statistics From a Base to a Target Table
Dropping a Table
Recreating a Table
Using INSERT…SELECT
Specifying Fallback Tables
Working with Views
Creating Views Procedure
Using BTEQ Scripts to Create Database Objects
Chapter 5 Working with Stored Procedures and User-defined Functions: Application DBAs
Creating Stored Procedures
Example 2: Creating a Stored Procedure to Update Accrued Interest
Creating User-defined Functions
Chapter 6 Working with Users, Roles, and Profiles: Operational DBAs
Overview of Establishing Users
User Access Methods
Types of Users
Assessing Database User Needs
Best Practices for Creating Users
Creating User Accounts
Creating Account Strings
Determining the Account for a User Session
Creating User Profiles
Creating Profiles
Dropping Profiles
Working with Database Users
Creating Users
Creating Temporary Passwords for First Login
Using Roles to Manage User Privileges
User Types and Minimum Required Privileges
Types of Privileges
Privileges That Must Be Explicitly Granted
Creating User Roles
Granting Privileges to a Role
Granting User Membership in a Role
Defining the Default Role for a User
Granting Privileges Directly to Users
Example: Granting Privileges on Sys_Calendar to All Users
Granting Privileges to a User
Automatic Privileges for Creators
Automatic Privileges for Created Users and Databases
Working with Table Access Privileges for Views
About Viewpoint Monitoring Privileges
Setting Up Teradata Viewpoint Users
Configuring User Dashboards
Creating a Dashboard
Chapter 7 Loading and Exporting Data: Application DBAs
About Teradata Parallel Transporter
About the Load Operator
About the Update Operator
About the Export Operator
About Basic Teradata Query for Loading Tables
Updating Tables
Loading Data into Teradata Database
Restarts and Aborts on BTEQ Jobs with Identity Column
About Reading Committed Data While Loading to the Same Table
About Cleaning up Load Isolated Tables
About FastLoad
About FastExport
About MultiLoad
Maximum Number of Concurrent FastLoad, MultiLoad, and FastExport Jobs
Utility Management
Teradata Parallel Data Pump
Restarts on TPump Jobs with Identity Column
Loading Geospatial Data
Interpreting LOAD Utility Status
Choosing the Best Utility for Your Purpose
Definitions
Guidelines for Inserting Rows
Guidelines for Deleting Rows
Guidelines for Other Batch Operations
Utility Job Performance Analysis and Capacity Planning
Examples of Using the DBQL Utility Job Log Table
Chapter 8 Working with Sessions and Accounts: Operational DBAs
About Session Modes and Transaction Processing
Transaction Processing in ANSI Session Mode
Transaction Processing in Teradata Session Mode
Setting the Session Mode
Checking for Logged On Sessions
Obtaining Session Information
Troubleshooting Problems with Sessions
Working with User Accounts
Working with System Accounting Views
About DBC.AccountInfoV
About the DBC.AMPUsage View
Logging Resource Usage Data with Account String Variables
Enabling Account String Expansion
Setting the Default Account String to Use ASE
About the ASE Substitution Variables
Restrictions and Usage Rules
Considerations for Assigning ASE Variables to Different Workloads
About Using ASE With Client Utilities
Chapter 9 Managing Space: Operational DBAs
About Types of Space
Capacity Planning
About Identifying and Correcting System-level Space Problems
Using Viewpoint to Find System Space Issues
Identifying Space Problems Using Viewpoint
Identifying Space Issues by Querying System Views
Querying the TableSizeV View
Warning About AllSpaceV View
Finding Space Issues
Transferring Ownership and Permanent Space
Reallocating Perm Space to Resolve Space Issues
Increasing the Spool Space Limit for a User
Finding and Fixing Skewed Tables Using Viewpoint
Finding Skewed Tables by Querying the TableSizeV View
Defining Temporary Space Limits
Protecting Transactions by Reserving Cylinders for Perm Space
Creating a Macro for Space Usage Reporting
Viewing Space Usage Trends for Databases and Users
Other Ways to View Space Utilization
The Ferret Utility
SQL Macros
Managing Data Blocks
Setting Data Block Sizes Using SQL
Setting Data Block Sizes Using the DBS Control Utility
Setting Data Block Merging Limits
Managing Cylinders
Finding Loosely Packed Tables with the SHOWFSP Command
Freeing Cylinders Using PACKDISK
Giving One User to Another
Adjusting Perm Space Limits Using SQL
Adjusting Perm Space Limits Using Viewpoint
Chapter 10 Maintaining the Database: Operational DBAs
About Database Maintenance Tasks
Managing Accumulated Log Data
Deleting Old Log Data with Teradata Viewpoint
Deleting Old Log Data Manually from System Tables
Managing Vproc Status and Initializing AMP Disks with Vproc Manager
Maintaining Data Dictionary Tables and Logs
Purging the System Logs
Clearing Out Values in the DBC.Acctg Table
Archiving and Resetting Accumulators and Peak Values
Correcting DBC.DataBaseSpace and DBC.DBase Values
Collecting Statistics on Data Dictionary Tables
Data Dictionary Tables That Require Maintenance
Housekeeping on an Ad-Hoc Basis
Updating Software (Applying Proper Patches)
Chapter 11 Archiving, Restoring, and Recovering Data: Operational DBAs
Methods of Archiving, Restoring, and Recovering Data
About the Archive and Recovery Utility
Common Uses for ARC
Best Practices
Archiving the Data Dictionary
Archiving and Restoring Tables That Use Block-Level Compression
Archiving and Restoring Tables Using Temperature-based Compression
Archiving and Restoring Tables Inside Secure Zones
Archiving and Restoring Column-Partitioned Tables and Databases
Selective Backup and Restore Rules
SYSUDTLIB System Database
Troubleshooting
Managing Space Availability Issues
Archiving After Recovering a Down AMP
Archiving or Restoring a Down Table
Recovering Data
Starting Rollbacks
Canceling Rollbacks
Recovering Data with Rollforward
Using Recovery Control Views
Chapter 12 Managing Database Resources: Operational DBAs
About Managing I/O with Cylinder Read
About Tracking Cylinder Read Resource Usage
Managing the Database Workload with Teradata Active System Management
Determining Workload Management Needs
Defining a Ruleset
About Ensuring a Consistent Response Time for a Workload
Managing Sessions and Transactions with Query Banding
About Defining Name-Value Pairs for Session and Transaction Query Bands
About Defining Name-Value Pairs for Profile Query Bands
About Query Bands and Proxy Users
Finding the Origin of a Query Using Query Bands
Other Uses for Query Bands
Tracking Object Use and Table Changes
Objects That Can Be Tracked
Table Changes That Can Be Tracked
Enabling Object Use Counts
Controlling How Often Counts Are Recorded
Views That Report Count Data
Resetting Use Counts
Usage Recommendations
About Resource Usage Trend Analysis
Enabling RSS Logging
Chapter 13 Managing Queries: Operational DBAs
Redrive Protection for Queries
Enabling Redrive Protection
Disabling Redrive Protection
Setting Redrive Protection for Users, Accounts, and Profiles
Recommendations for Common Query Problems
Identifying Poorly Performing Queries
Controlling Poorly Performing Queries
Finding and Fixing Skewed Tables
Finding and Resolving Lock Contentions
About Lock Contentions
Investigating Query Blocks and Delays
Chapter 14 Improving Query Performance Using COLLECT STATISTICS: Application DBAs
About Statistics Collection
Required Privileges
Resource Considerations
About Automated Statistics Management
When To Collect Statistics
Collecting Statistics
Recommendations for Large Tables
About Stale Statistics
About SUMMARY Statistics
Skipping Unneeded Statistics Recollection with the THRESHOLD Option
Sampling Statistics with the USING SAMPLE Option
Working with Special Cases
Collecting Statistics on Null
Chapter 15 Tracking Query Behavior with Database Query Logging: Operational DBAs
DBQL Overview
When to Use DBQL
How to Use DBQL
Overview of Collection Options
Best Fit Rule Hierarchy
Rules Validation
SQL Statements That Should Be Captured
SQL Logging Statements
SQL Logging Considerations
XML Query Plan Logging
Capturing DBQL Through the Account String
SQL Logging by Workload Type
SQL Statements to Control Logging
BEGIN/REPLACE/END QUERY LOGGING Statements
WITH Logging Options
LIMIT Logging Options
Mode Logging Option
DBQL Tables, Views, and Macros
About the Presentation of Tables and Views in This Book
About Querying DBQL Tables
Main Log Table: DBQLogTbl
View of the Main Log Table: QryLog[V]
Teradata Dynamic Workload Management View of the Main Log Table: QryLogTDWM[V]
Explain Log Table: DBQLExplainTbl
View of the EXPLAIN Log Table: QryLogExplainDoc[V]
Improving Explain Text Readability
View of the EXPLAIN Log Table: QryLogExplain[V]
Object Log Table: DBQLObjTbl
ObjectType
View of the Object Log Table: QryLogObjects[V]
Parameterized Query Log Table: DBQLParamTbl
View of the Parameterized Query Log Table: QryLogParam[V]
View of the Parameterized Query Log Table: QryLogParamJSON[V]
SQL Log Table: DBQLSQLTbl
When SQL is Not Recorded
View of the SQL Log Table: QryLogSQLDoc[V]
Improving SQL Text Readability
View of the SQL Log Table: QryLogSQL[V]
Query Step Information Log Table: DBQLStepTbl
Step Descriptions
View of the Query Step Information Log Table: QryLogSteps[V]
Summary Log Table: DBQLSummaryTbl
Rows Generated by the THRESHOLD Option
Rows Generated by the SUMMARY Option
View of the Summary Log Table: QryLogSummary[V]
Utility Job Log Table: DBQLUtilityTbl
View of the Utility Job Log Table: QryLogUtility[V]
XML Lock Log Table: DBQLXMLLockTbl
Shredding the Lock Plan Information in the XMLTextInfo Column
Output of Shredding
Shredded_LockTbl: Blocking Transaction Lock Shredding Target Table
Shredded_TransTbl: Blocked Transaction Lock Shredding Target Table
View of the XML Lock Log Table: QryLockLogXML[V]
XML Plan Log Table: DBQLXMLTbl
View of the XML Plan Log Table: QryLogXMLDocV
Improving XML Text Readability
View of the XML Plan Log Table: QryLogXML[V]
Query Data Storage and Protection
Protection for a Committed Row
Things to Consider When Logging DBQL Data
Logging Overhead
Options for Flushing the DBQL Cache
Flushing the DBQL Cache Manually
Logging Scenarios
Scenarios of Logging Accounts
Scenarios of Detailed Logging
Scenarios of Threshold and Summary Options
Examining the Logged Data
Comparing CollectTimeStamp Value Between Tables
Using QueryID for Joins on DBQL Tables
Zone-level Query Logging
About Query Logging Privileges in a Zoned System
Relationship Between DBQL Temporary Tables and DBQL History Tables
Daily Maintenance Process
Monthly Maintenance Process
Maintaining the Logs
Methods for Minimizing Log Table Size
Archiving DBQL Data
Reviewing or Ending Current Rules
Reviewing Rules
Ending DBQL Logging for Specific Users and/or Accounts
Ending All DBQL Logging
Effects of Dynamically Enabling/Replacing/Disabling Logging on Current Rules
Granting DBQL Administrative Privileges to Other Users
Chapter 16 Analyzing Requests and Request Plans: Application DBAs
Using EXPLAIN to Analyze Request Plans
Generating an EXPLAIN with SQL Assistant
Generating an EXPLAIN with BTEQ
Capturing Request Plan Steps
About Capturing Request Plans Using DBQL
Relationship Between BEGIN QUERY CAPTURE and BEGIN QUERY LOGGING
DBS Control Parameters for Tuning the Workload Cache Limits for Index and Partition Analysis
About the Query Capture Database
Deleting Query Plans from a QCD
Creating the Query Capture Database Tables
Procedure Using the Visual EXPLAIN Utility
Procedure Using BTEQ
Dropping Query Capture Database Tables
Procedure Using Visual EXPLAIN
Procedure Using BTEQ
About Querying QCD Tables
About QCD Request Macros and Views
QCD Macro and View Versions
User Categories
Specific User Category Privileges
Building Baseline Transaction Profiles
Chapter 17 Testing Query Performance Using Target-Level Emulation: Application DBAs
About Target Level Emulation
Benefits
About the Two Forms of Target Level Emulation
About Related Tools and Utilities
Enabling Target Level Emulation
Target Level Emulation and Cost Profiles
Capturing Target System Optimizer Cost Values and Dynamic AMP Statistical Samples
Procedure Using the Teradata System Emulation Tool
Procedure Using SQL DIAGNOSTIC Statements
Target Level Emulation and Cost Profiles
Capturing Profile Information
Teradata System Emulation Tool Import Function
Installing Cost Segments at a Particular Emulation Level
Capturing and Installing Costing Information Including the Active Cost Profile
Procedure Using SQL DIAGNOSTIC Statement Requests
About Mapping Target System Files to a Test System
Chapter 18 Working with System Information and Global Defaults: Operational DBAs
Viewing the Software Release and Version
Reviewing or Changing System-Wide Parameters
About International Character Set Settings and Defaults
International Language Support Mode
Default Server Character Sets
Client Character Sets
Changing Character Set Defaults
Changing Collation Defaults
Changing Time Zones, Currencies, and Formats for Date, Time, and Numbers
Changing Time Zone Strings and Rules
Client Configuration Overview
Teradata Director Program for Mainframe-Attached Clients
Network-Attached Clients
Communicating with Teradata Database
Controlling Session Defaults
Client-Server Applications Connectivity
Mainframe Environment
CP and CUA
Software Components
Mainframe Sessions
Session Pools
TDP Functionality
TDP Exits
Memory Management
Using TDP Commands
Network Environment
Functionality
Software Components
The Teradata Gateway
Displaying Network and Session Information
Controlling Network Sessions
Controlling Trace Logs
Chapter 19 Troubleshooting: Operational DBAs
Tools for Troubleshooting and Administrating
Tools for Troubleshooting Client Connections
Network Connection Tools
Troubleshooting Spool Space Problems
Using the Update Space Utility
Adjusting for Low Available Free Memory
Solving Partitioning and RI Validation Errors
Incident Information Checklist
Monitoring Transaction Recovery
Starting Recovery Manager
Stopping Recovery Manager
Recovery Manager Commands
Resolving Orphan or Phantom Spool Issues
Repairing Data Corruption
Appendix A How to Read Syntax Diagrams: All DBAs
Syntax Diagram Conventions
Appendix B Teradata System Limits: All DBAs
System Limits
Miscellaneous System Limits
Message Limits
Storage Limits
Gateway and Vproc Limits
Hash Bucket Limits
Interval Histogram Limits
Database Limits
Name and Title Size Limits
Table and View Limits
Spool Space Limits
BLOB, CLOB, XML, and Related Limits
User-Defined Data Type, ARRAY Data Type, and VARRAY Data Type Limits
Macro, UDF, SQL Procedure, and External Routine Limits
Query and Workload Analysis Limits
Secondary, Hash, and Join Index Limits
Reference Index Limits
SQL Request and Response Limits
Row-Level Security Constraint Limits
Session Limits
Appendix C Performance Management: All DBAs
Implementing Performance Management
About Performance Management Setup
Monitoring the System
Using Alerts to Monitor the System
Suggested Alerts and Thresholds
Weekly and/or Daily Reports
Collecting Statistics
Collecting Statistics on Skewed Data
Collecting Statistics on Join Index Columns
About PARTITION Statistics
Setting Up Database Query Logging
Performance Effects of Query Logging
Performance Information in DBQL Tables
Recommended DBQL Logging Scenarios
DBQL Collection Recommendations
Using DBQL Tables
Collecting Useful DBQL Historical Data
Managing Query Performance
Finding and Fixing Problem Queries
Alternate Query Investigation Methods
Managing Skewed Processing
Skewed Query Processing Across AMPs
Identifying and Fixing Skewed Tables
Identifying Skewed Distribution of Table Data
System Tables Subject to Skewing
Using Teradata Viewpoint to Find Skewed Tables
Finding Skewed Tables in DBC.TableSizeV
Finding Uneven Distribution Using Hash Functions
Primary Index and Skewed Row Distribution
Adjusting Skew Allowance
Identifying and Managing Blocked Queries
Locking Overview
Investigating Query Blocks and Delays
Reducing Deadlocks
Handling Blocked Internal DBQL Requests
Working with Transaction Rollbacks
Effects on Performance
RollbackPriority
Minimizing or Avoiding Rollbacks
Detecting a Rollback in Progress
Canceling Rollbacks
Noticing a Slow Rollback of an ALTER TABLE Statement
Managing the Database Workload
Evaluating Peak Utilization Periods
Assessing a Busy System
Job Scheduling Around Peak Utilization
Using Canary Queries
ASE Impact on PE and AMP Performance
Account Strings and Performance Groups
Charge Back
Priority Scheduler
Managing Database Resources to Enhance Performance
Performance and Space Management
Managing System Disk Space
Managing Spool Space
Managing Cylinders
Freeing Space on Cylinders
Managing I/O with Cylinder Read
Managing Free Space
Determining a Value for FSP
PACKDISK and FreeSpacePercent
Data Compression and Performance
Managing Data Block Usage
Performance and Memory Management
About Memory-Consuming Features
Investigating the Need to Add Memory
ResUsage and Cache Hit Rates
Using Memory Effectively
Managing Free Memory
Managing FSG Cache
Using DBS Control Settings to Manage Memory
Periodic Maintenance and Performance
DefragLowCylProd
Capacity Planning
Monitoring Disk Utilization
Monitoring Traffic
Performance Considerations when Expanding or Upgrading the Database System
Adding Disk Arrays
Adding Vprocs
Adding Memory
Reconfiguring the Database
Scaling
Checking Performance when Upgrading
Appendix D Handling Teradata Crashdumps: Operational DBAs
Teradata Vital Infrastructure and Crashdumps
TSET File Pack and the DBC.TSETQueryText Table
Crashdumps Overview
Dump Types
Teradata Crashdumps Processes
Teradata Crashdump Formats
Teradata Crashdump Sizes
Time Considerations
Viewing Teradata Crashdump Messages
Locating Crashdumps
Configuring the Crashdumps Database
Ad-Hoc Creation of DBC.Crashdumps with DIPCRASH
Checking DBC.Crashdumps Space Allocation and Usage
Allocating More Space to DBC.Crashdumps Database
Automatic Versus Forced Crashdump
Automatic Teradata Crashdumps
Forced Teradata Crashdumps
Commands for Forcing a Teradata Crashdump
Handling Teradata Crashdumps
Save Considerations
Listing Teradata Crashdumps Using CSP
Manually Saving Teradata Crashdumps
Manually Saving Crashdumps to Another Teradata System
Handling an Interrupted Dump Save Operation
Error Logging
Saving Crashdumps to Disk or Tape
Sending Crashdumps to Teradata
Debugging Crashdumps
Attaching a Raw Dump and Displaying the Backtrace
Deleting Teradata Crashdumps
Fault Isolation Diagnostics
Troubleshooting Crashdumps
Appendix E Considerations for Administering a System with Unity Director: All DBAs
Creating a Database or User
Creating Roles and Profiles
Creating Tables and Views
Loading and Exporting Data
Querying the AMPUsage and AccountInfoV Views
Managing Space
Restarting Teradata Database
Investigating Lock Contentions
Analyzing Queries and Query Plans
Making Global and User-Level Configurations Consistent
You must have EXECUTE privilege on the DBC.DBQLAccessMacro macro to log DBQL data.
To grant the EXECUTE privilege to an administrative user, see “Granting DBQL Administrative Privileges to Other Users” on page 390 .
Teradata recommends that you use BTEQ script files to enable DBQL. You can define
the rules for logging in these scripts. Just be sure to keep the scripts up to date
and saved for future use. You can also create files that disable query logging for
your convenience.