Open Analytics Framework uses structured logging in JSON format to output any warnings, information, debug and error messages. Log entries are structured in a way that helps track and trace which database query or user request produced the corresponding log message. The framework has the capabilities to process all logs and metrics generated for further log visualization, monitoring and detailed analysis.
Open Analytics Framework empowers users to independently retrieve and examine their Python or R script execution logs, obviating the need for technical support staff intervention. The script logs are accessible to the user anytime within 14 days of its creation.
- Download script logs executed using Apply Table Operator.
- Debug the scripts based on information (debug statements, error traces, and so on.) available in log files.
- log_type set to "apply"
- query_id
- log_dir
This optional argument specifies the location where the log files are to be written in the user’s local machine .
The default value is C:\\local_repo\\AppData\\Local\\Temp\\tmp00kuxlgu\\{query_id}.
For each Apply query, the script log is associated with a corresponding database user and a query_id for the Apply query. query_id refers to the ID of the query for which logs are to be retrieved. This query_id is part of the error message received when Apply class or DataFrame apply method calls fail to execute the Apply table operator query.
With this capability, users can independently view the script logs, including the output of print statements within Python and R scripts, for the purpose of monitoring and troubleshooting the script execution.
Example 1: Download the Apply query logs to a default temp folder
This example uses query id from the error messages returned by Apply class.
view_log(log_type="apply", query_id='307161028465226056')
Logs for query_id "307191028465562578" is stored at "C:\\local_repo\\AppData\\Local\\Temp\\tmp00kuxlgu\\307161028465226056"
Example 2: Download the Apply query logs to a specific folder
This example uses query id from the error messages returned by Apply class.
view_log(log_type="apply", query_id='307161028465226056',log_dir='C:\\local_repo\\workspace')
Logs for query_id "307191028465562578" is stored at "C:\\local_repo\\workspace\\307161028465226056"