PySpark API Supportability Matrix | SparkSession APIs | pyspark2teradataml - SparkSession APIs - Teradata Package for Python

Teradata® pyspark2teradataml User Guide

Deployment
VantageCloud
VantageCore
Edition
VMware
Enterprise
IntelliFlex
Product
Teradata Package for Python
Release Number
20.00
Published
December 2024
ft:locale
en-US
ft:lastEdition
2026-01-07
dita:mapPath
oeg1710443196055.ditamap
dita:ditavalPath
zuq1752009390153.ditaval
dita:id
oeg1710443196055
Product Category
Teradata Vantage
Function Name Supported Notes
active  
builder.appName  
builder.config Configure options for PySpark are not supported. The API is made available only to maintain parity and it is not significant.
builder.getOrCreate getOrCreate accepts Vantage connection parameters. Refer create_context function parameters in teradataml.
builder.master  
builder.remote  
catalog  
conf Configure options for PySpark are not supported. The API is made available only to maintain parity and it is not significant.
createDataFrame It also accepts the name of the table that resides in Vantage .
  • If the input is a pandas.DataFrame and its column name is of type integer, then 'col_' prefix will be added to the column name.
  • If the input is a list of lists/tuples and does not have explicit column names, then 'col' prefix is added to the column name.
getActiveSession  
newSession Returns same session instead of new session as teradataml does not support multiple simultaneous sessions.
read
  • If the file is a CSV file, the API can read the file from either local file system or from cloud storage.
    • API uses teradataml read_csv to read the file if file exists in local file system. teradataml read_csv arguments are accepted as options.
  • If the file is not a CSV file, the API does not read the file from local file system and it can read only from cloud storage.
  • API uses teradataml ReadNos for reading the data from cloud storage. teradataml ReadNoS Arguments are accepted as options.
sparkContext  
sql  
stop  
table  
version  
builder.create  
conf.set  
conf.unset  
range