PySpark API Supportability Matrix | SparkSession APIs | pyspark2teradataml - SparkSession APIs - Teradata Package for Python

Teradata® pyspark2teradataml User Guide

Deployment
VantageCloud
VantageCore
Edition
Enterprise
IntelliFlex
VMware
Product
Teradata Package for Python
Release Number
20.00
Published
December 2024
ft:locale
en-US
ft:lastEdition
2024-12-18
dita:mapPath
oeg1710443196055.ditamap
dita:ditavalPath
ayr1485454803741.ditaval
dita:id
oeg1710443196055
Product Category
Teradata Vantage
Function Name Supported Notes
active  
builder.appName  
builder.config Configure options for PySpark are not supported. The API is made available only to maintain parity and it is not significant.
builder.getOrCreate getOrCreate accepts Vantage connection parameters. Refer create_context function parameters in teradataml.
builder.master  
builder.remote  
catalog  
conf Configure options for PySpark are not supported. The API is made available only to maintain parity and it is not significant.
createDataFrame Creates DataFrame from table which resides in Vantage . Argument data accepts table name and creates DataFrame on it.
getActiveSession  
newSession Returns same session instead of new session as teradataml does not support multiple simultaneous sessions.
read If the file is a CSV file, the API can read the file from either local file system or from cloud storage.
  • API uses teradataml read_csv to read the file if file exists in local file system. teradataml read_csv arguments are accepted as options.

If the file is not a CSV file, the API does not read the file from local file system and it can read only from cloud storage.

API uses teradataml ReadNos for reading the data from cloud storage. teradataml ReadNoS Arguments are accepted as options.

sparkContext  
sql  
stop  
table  
version  
builder.create  
conf.set  
conf.unset