Example 1: Working with Python Script with APPLY table operator | VantageCloud Lake - Example 1: Working with Python Script - Teradata Vantage

In this example, the Python script mapper.py reads in a line of text input ("Old Macdonald Had A Farm") from a CSV file, splits the line into individual words, and emits a new row for each word.

• Load example data.

  >>> load_example_data("Script", ["barrier"])

• Create teradataml DataFrame objects.

  >>> barrierdf = DataFrame.from_table("barrier")

• Create remote user environment.

  >>> testenv = create_env('testenv', 'python_3.7.13', 'Demo environment')

  User environment testenv created.

  >>> import os, teradataml

  >>> teradataml_dir = os.path.dirname(teradataml.__file__)

• Create an APPLY object that allows user to run script.

  >>> apply_obj = Apply(data=barrierdf,
                        script_name='mapper.py',
                        files_local_path= os.path.join(teradataml_dir, 'data', 'scripts'),
                        apply_command='python3 mapper.py',
                        data_order_column="Id",
                        is_local_order=False,
                        nulls_first=False,
                        sort_ascending=False,
                        returns={"word": VARCHAR(15), "count_input": VARCHAR(10)},
                        env_name=testenv,
                        delimiter='\t')

• Install the script file in remote user environment.

  >>> apply_obj.install_file(file_name=os.path.join(teradataml_dir, 'data', 'mapper.py'))
  File 'mapper.py' installed successfully in the remote user environment 'demo_env'.

• Run the user script in Open Analytics Framework.

  >>> apply_obj.execute_script()

          word count_input
  0  Macdonald           1
  1          A           1
  2       Farm           1
  3        Had           1
  4        Old           1
  5          1           1

• Remove the installed file from remote user environment.

  >>> apply_obj.remove_file(file_name='mapper.py')

  File 'mapper.py' removed successfully from the remote user environment 'demo_env'.