join | teradatamlspk | pyspark2teradataml - join - Teradata Vantage

Teradata® VantageCloud Lake

Deployment
VantageCloud
Edition
Lake
Product
Teradata Vantage
Published
January 2023
ft:locale
en-US
ft:lastEdition
2024-12-11
dita:mapPath
phg1621910019905.ditamap
dita:ditavalPath
pny1626732985837.ditaval
dita:id
phg1621910019905

When using join in teradatamlspk, if join is of the following types: inner, cross, outer, full, fullouter, full_outer, left, leftouter, left_outer, right, rightouter, right_outer, and if both input DataFrames share common column names, then column names for those columns are prefixed with 'l' and 'r' in the output DataFrame.

Also the order of the columns varies.

PySpark

>>> df1.join(df, ["dept_name"]).show()
+---------+-------+-------+
|dept_name|dept_id|dept_id|
+---------+-------+-------+
|  Finance|     10|     10|
|       IT|     40|     40|
|Marketing|     20|     20|
|    Sales|     30|     30|
+---------+-------+-------+

teradatamlspk

>>> df1.join(df, ["dept_name"]).show()
+-----------+---------+---------+
|l_dept_name|l_dept_id|r_dept_id|
+-----------+---------+---------+
|      Sales|       30|       30|
|         IT|       40|       40|
|  Marketing|       20|       20|
|    Finance|       10|       10|
+-----------+---------+---------+