PySpark API Supportability Matrix | SparkSession APIs | pyspark2teradataml - SparkSession APIs - Teradata VantageCloud Lake

Lake - Analyze Your Data with ClearScape Analytics™

Deployment
VantageCloud
Edition
Lake
Product
Teradata VantageCloud Lake
Release Number
Published
February 2025
ft:locale
en-US
ft:lastEdition
2026-02-20
dita:mapPath
tcl1683670667798.ditamap
dita:ditavalPath
pny1626732985837.ditaval
dita:id
tcl1683670667798
Function Name Supported Notes
active  
builder.appName  
builder.config Configure options for PySpark are not supported. The API is made available only to maintain parity and it is not significant.
builder.getOrCreate  
builder.master  
builder.remote  
catalog  
conf Configure options for PySpark are not supported. The API is made available only to maintain parity and it is not significant.
createDataFrame Creates DataFrame from table which resides in VantageCloud Lake. Argument data accepts table name and creates DataFrame on it.
getActiveSession  
newSession Returns same session instead of new session as teradataml does not support multiple simultaneous sessions.
read
  • If the file is a CSV file, the API can read the file from either local file system or from cloud storage.
    • API uses teradataml read_csv to read the file if file exists in local file system. teradataml read_csv arguments are accepted as options.
  • If the file is not a CSV file, the API does not read the file from local file system and it can read only from cloud storage.
  • API uses teradataml ReadNos for reading the data from cloud storage. teradataml ReadNoS Arguments are accepted as options.
sparkContext  
sql  
stop  
table  
version  
builder.create  
conf.set  
conf.unset