- Open Table Format (OTF) table support from teradataml is supported on DB 20.00.24.xx and earlier versions.
- You must complete the OTF setup on Vantage to work with OTF. More details can be found in different sections under Apache Iceberg and Delta Lake Open Table Format on VantageCloud Lake.
- Refer to the Installation and Setup section for installation and setup of catalogs.
- Refer to the Creating a DATALAKE Object section for datalake creation and authorization.
- OTF does not support creating a view on a table; you must set the configuration option in teradataml to create volatile tables.
Once the teradataml DataFrame is created, use the DataFrame just like a regular teradataml DataFrame to perform any operations such as dataframe manipulation or for machine learning functions except the ones mentioned in Teradata Package for Python Limitations and Considerations.
Prerequisite: Create a teradataml DataFrame on an OTF table
Set configure.temp_object_type to VT:
>>> configure.temp_object_type = "VT"
When creating the DataFrame, you must provide the datalake name, database name, and table name. Use one of the two approaches.
Approach 1: Using in_schema()function and DataFrame()
>>> from teradataml.dataframe.dataframe import in_schema
Create an in_schema object to provide additional information about datalake.
>>> in_schema_tbl = in_schema(schema_name="datalake_db",
... table_name="datalake_table_name",
... datalake_name="datalake")
>>> otf_df = DataFrame(in_schema_tbl)
>>> otf_df
Feb Jan Mar Apr datetime
accounts
Alpha Co 210.0 200.0 215.0 250.0 04/01/2017
Blue Inc 90.0 50.0 95.0 101.0 04/01/2017
Jones LLC 200.0 150.0 140.0 180.0 04/01/2017
Orange Inc 210.0 NaN NaN 250.0 04/01/2017
Yellow Inc 90.0 NaN NaN NaN 04/01/2017
Red Inc 200.0 150.0 140.0 NaN 04/01/2017
Approach 2: Using DataFrame.from_table() method
>>> otf_df = DataFrame.from_table(table_name = "datalake_table_name",
... schema_name="datalake_db",
... datalake_name="datalake")
>>> otf_df
Feb Jan Mar Apr datetime
accounts
Alpha Co 210.0 200.0 215.0 250.0 04/01/2017
Blue Inc 90.0 50.0 95.0 101.0 04/01/2017
Jones LLC 200.0 150.0 140.0 180.0 04/01/2017
Orange Inc 210.0 NaN NaN 250.0 04/01/2017
Yellow Inc 90.0 NaN NaN NaN 04/01/2017
Red Inc 200.0 150.0 140.0 NaN 04/01/2017
Verify the name of underlying DB object by accessing db_object_name property
>>> otf_df.db_object_name '"datalake"."datalake_db"."datalake_table_name"'