fastload() | Teradata Package for Python - fastload() - Teradata Vantage

Required Arguments:

• df: Specifies the pandas or teradataml DataFrame object to be saved in Vantage.
• table_name: Specifies the name of the table to be created in Vantage.

Optional Arguments:

• schema_name: Specifies the name of the SQL schema in Vantage to write to.

  The default value is None, which means use the default Vantage schema.

• if_exists: Specifies the action to take when table already exists in the database.
  Possible values are:

  • 'fail': If table exists, do nothing;
  • 'replace': If table exists, drop it, recreate it, and insert data;
  • 'append': If table exists, insert data. Create if does not exist.

  The default value is 'replace'.

• index: Specifies whether to save pandas DataFrame index as a column or not. Possible values are True or False.

  The default value is False.

• index_label: Specifies the column labels for pandas DataFrame index columns.

  The default value is None.

• primary_index: Specifies the column(s) to use as primary index while creating tables in Vantage. When None (default value), No Primary Index (NOPI) tables are created.
  For example:

  • primary_index = 'my_primary_index'
  • primary_index = ['my_primary_index1', 'my_primary_index2', 'my_primary_index3']
• types: Specifies required data types for requested columns to be saved in Vantage.

  The default value is None.

  This argument accepts a dictionary of column names and their required teradatasqlalchemy types as key-value pairs (({column_name1: type_value1, ... column_nameN: type_valueN})), allowing to specify a subset of the columns of a specific type.

  • When only a subset of all columns are provided, the rest are defaulted to appropriate types.
  • When types argument is not provided, all column types are appropriately assigned and defaulted. The column types are assigned as listed in the following table.

    pandas/NumPy Type	teradatasqlalchemy Type
    int32	INTEGER
    int64	BIGINT
    bool	BYTEINT
    float32/float64	FLOAT
    datetime64/datatime64[ns]	TIMESTAMP
    datatime64[ns,time_zone]	TIMESTAMP(timezone=True)
    Any other data type	VARCHAR(configure.default_varchar_size)

  • This argument does not have any effect when the table specified using table_name and schema_name exists and if_exists = 'append'.
• batch_size: Specifies the number of rows to be loaded in a batch. The value of batch_size must be a positive integer.

  For better performance, Teradata recommends batch size to be at least 100,000.

  If this argument is None, which is the default value, there are two cases based on the number of rows, say N, in the dataframe 'df':

  • If N is greater than 100,000, the rows are divided into batches of equal size with each batch having at least 100,000 rows (except the last batch which might have more rows).
  • If N is less than 100,000, the rows are inserted in one batch after notifying the user that insertion happens with degradation of performance.

  If this argument is not None, the rows are inserted in batches of size given in the argument, irrespective of the recommended batch size.

  The last batch will have rows less than the batch size specified, if the number of rows is not an integral multiples of the argument batch_size.

• save_errors: Specifies whether to persist the error and warning information in Vantage or not.
  • If set to False, which is the default value, error and warnings are not persisted as tables.
  • If set to True, the error and warnings information are persisted and names of error and warning tables are returned. Otherwise, the function returns None for the names of the tables.
• open_sessions: Specifies the number of Teradata data transfer sessions to be opened for fastload operation.

  If this argument is not provided, the default value is the smaller of 8 or the number of AMPs available.

  See the FastLoad section of https://pypi.org/project/teradatasql/ for additional information about number of Teradata data transfer sessions opened during fastload.

FastLoad returns a dict containing the following attributes:

• errors_dataframe: It is a Pandas DataFrame containing error messages thrown by fastload.

  DataFrame is empty if there are no errors.

• warnings_dataframe: It is a Pandas DataFrame containing warning messages thrown by fastload.

  DataFrame is empty if there are no warnings.

• errors_table: Name of the table containing errors.

  It is None, if argument save_errors is set to 'False'.

• warnings_table: Name of the table containing warnings.

  It is None, if argument save_errors is set to 'False'.

teradatasql version 16.20.00.48 or later is required for fastload() function to work properly. If you have a lower version installed, then teradatasql raises OperationalError and fastload() call ends with the following error:

[Teradata Database] [Error 3706] Syntax error: expected something between the beginning of the request and the word 'teradata_require_fastloadINSERT'.

pandas version 0.24 or later is required for fastload() API to work properly. If you have a lower version, fastload() API will fail with following error:

AttributeError: 'Index' object has no attribute 'to_list'

Install pandas >= 0.24 to solve this issue.