| Function Name | Supported | Notes |
|---|---|---|
| col | ||
| column | ||
| asc | ||
| asc_nulls_last | ||
| desc | ||
| desc_nulls_first | ||
| desc_nulls_last | ||
| avg | ||
| any_value | ||
| row_number | ||
| count | ||
| rank | ||
| cume_dist | ||
| dense_rank | ||
| percent_rank | ||
| max | ||
| mean | ||
| min | ||
| sum | ||
| std | ||
| stddev_pop | ||
| stddev_samp | ||
| stddev | ||
| var_pop | ||
| var_samp | ||
| variance | ||
| lag | ||
| lead | ||
| count_distinct | ||
| countDistinct | ||
| corr | ||
| covar_pop | ||
| covar_samp | ||
| first | ||
| first_value | ||
| last | ||
| last_value | ||
| regr_avgx | ||
| regr_avgy | ||
| regr_count | ||
| regr_intercept | ||
| regr_r2 | ||
| regr_slope | ||
| regr_sxx | ||
| regr_sxy | ||
| regr_syy | ||
| sum_distinct | ||
| sumDistinct | ||
| lit | ||
| broadcast | ||
| coalesce | ||
| isnan | ||
| isnull | ||
| monotically_increasing_id | ||
| nanvl | ||
| rand | ||
| randn | ||
| spark_partition_id | ||
| when | ||
| bitwise_not | ||
| bitwiseNOT | ||
| expr | ||
| greatest | ||
| least | ||
| sqrt | ||
| abs | ||
| acos | ||
| asin | ||
| asinh | ||
| atan | ||
| atanh | ||
| atan2 | ||
| bin | ||
| cbrt | ||
| ceil | ||
| ceiling | ||
| input_file_name | ||
| named_struct | ||
| conv | ||
| cos | ||
| cosh | ||
| cot | ||
| csc | ||
| e | ||
| exp | ||
| expm1 | ||
| factorial | ||
| floor | ||
| hex | ||
| unhex | ||
| hypot | ||
| ln | ||
| log | ||
| log10 | ||
| log1p | ||
| log2 | ||
| negate | ||
| negative | ||
| pi | ||
| pmod | ||
| positive | ||
| pow | ||
| power | ||
| rint | ||
| round | ||
| bround | ||
| shiftleft | ||
| shiftright | ||
| shiftrightunsigned | ||
| sign | ||
| signum | ||
| sin | ||
| sinh | ||
| tan | ||
| tanh | ||
| toDegrees | ||
| try_add | ||
| try_avg | ||
| try_divide | ||
| try_multiply | ||
| try_subtract | ||
| try_sum | ||
| try_to_number | ||
| degrees | ||
| toRadians | ||
| radians | ||
| width_bucket | ||
| add_months | ||
| convert_timezone | ||
| curdate | ||
| current_date | ||
| current_timestamp | ||
| current_timezone | ||
| date_add | ||
| date_diff | ||
| date_format | ||
| date_from_unix_date | ||
| date_trunc | ||
| dateadd | ||
| datediff | ||
| day | ||
| date_part | ||
| datepart | ||
| dayofmonth | ||
| dayofweek | ||
| dayofyear | ||
| extract | ||
| second | ||
| weekofyear | ||
| year | ||
| quarter | ||
| month | ||
| last_day | ||
| localtimestamp | ||
| make_dt_interval | ||
| make_interval | ||
| make_timestamp | time zone values in column support only teradataml timezone strings. teradatamlspk timezone strings don't consider Daylight Saving Time(DST). Teradata recommends using Vantage timezone strings for DST consideration. |
|
| make_timestamp_ltz | time zone values in column support only teradataml timezone strings. teradatamlspk timezone strings don't consider Daylight Saving Time(DST). Teradata recommends using Vantage timezone strings for DST consideration. |
|
| make_timestamp_ntz | ||
| make_ym_interval | ||
| minute | ||
| months_between | ||
| next_day | ||
| hour | ||
| make_date | ||
| now | ||
| from_unixtime | ||
| unix_timestamp | ||
| to_unix_timestamp | ||
| to_timestamp | time zone values in column support only teradataml timezone strings. teradatamlspk timezone strings don't consider Daylight Saving Time(DST). Teradata recommends using Vantage timezone strings for DST consideration. |
|
| to_timestamp_ltz | time zone values in column support only teradataml timezone strings. teradatamlspk timezone strings don't consider Daylight Saving Time(DST). Teradata recommends using Vantage timezone strings for DST consideration. |
|
| to_timestamp_ntz | time zone values in column support only teradataml timezone strings. teradatamlspk timezone strings don't consider Daylight Saving Time(DST). Teradata recommends using Vantage timezone strings for DST consideration. |
|
| to_date | ||
| trunc | ||
| weekday | ||
| unix_date | ||
| concat | ||
| from_utc_timestamp | time zone values in column supports only teradataml timezone strings. teradatamlspk timezone strings don't consider Daylight Saving Time(DST). Teradata recommends using Vantage timezone strings for DST consideration. |
|
| to_utc_timestamp | time zone values in column supports only teradataml timezone strings. teradatamlspk timezone strings don't consider Daylight Saving Time(DST). Teradata recommends using Vantage timezone strings for DST consideration. |
|
| window | ||
| session_window | ||
| timestamp_micros | time zone values in column support only teradataml timezone strings. teradatamlspk timezone strings don't consider Daylight Saving Time(DST). Teradata recommends using Vantage timezone strings for DST consideration. |
|
| timestamp_millis | time zone values in column support only teradataml timezone strings. teradatamlspk timezone strings don't consider Daylight Saving Time(DST). Teradata recommends using Vantage timezone strings for DST consideration. |
|
| timestamp_seconds | time zone values in column support only teradataml timezone strings. teradatamlspk timezone strings don't consider Daylight Saving Time(DST). Teradata recommends using Vantage timezone strings for DST consideration. |
|
| try_to_timestamp | time zone values in column support only teradataml timezone strings. teradatamlspk timezone strings don't consider Daylight Saving Time(DST). Teradata recommends using Vantage timezone strings for DST consideration. |
|
| unix_micros | time zone values in column support only teradataml timezone strings. teradatamlspk timezone strings don't consider Daylight Saving Time(DST). Teradata recommends using Vantage timezone strings for DST consideration. |
|
| unix_millis | time zone values in column support only teradataml timezone strings. teradatamlspk timezone strings don't consider Daylight Saving Time(DST). Teradata recommends using Vantage timezone strings for DST consideration. |
|
| unix_seconds | time zone values in column support only teradataml timezone strings. teradatamlspk timezone strings don't consider Daylight Saving Time(DST). Teradata recommends using Vantage timezone strings for DST consideration. |
|
| window_time | ||
| array | ||
| array_contains | ||
| array_overlap | ||
| array_join | ||
| create_map | ||
| slice | ||
| array_position | ||
| element_at | ||
| array_append | ||
| array_size | ||
| array_sort | ||
| array_insert | ||
| array_remove | ||
| array_prepend | ||
| array_distinct | ||
| array_intersect | ||
| array_union | ||
| array_except | ||
| array_compact | ||
| transform | ||
| exists | ||
| forall | ||
| filter | ||
| aggregate | ||
| zip_with | ||
| transform_keys | ||
| tranform_values | ||
| map_filter | ||
| map_from_arrays | ||
| map_zip_with | ||
| explode | ||
| explode_outer | ||
| pos_explode | ||
| posexplode_outer | ||
| inline | ||
| inline_outer | ||
| get | ||
| get_json_object | ||
| json_tuple | ||
| from_json | ||
| schema_of_json | ||
| to_json | ||
| json_array_length | ||
| json_object_keys | ||
| size | ||
| cardinality | ||
| struct | ||
| sort_array | ||
| array_max | ||
| array_min | ||
| shuffle | ||
| reverse | DataFrameColumn.reverse is supported on string type columns and not supported on array type columns. | |
| flatten | ||
| sequence | ||
| array_repeat | ||
| map_contains_key | ||
| map_keys | ||
| map_values | ||
| map_entries | ||
| map_from_entries | ||
| arrays_zip | ||
| map_concat | ||
| from_csv | ||
| schema_of_csv | ||
| str_to_map | ||
| to_csv | ||
| try_element_at | ||
| years | ||
| months | ||
| days | ||
| hours | ||
| bucket | ||
| ascii | ||
| base64 | ||
| btrim | ||
| char | ||
| character_length | ||
| char_length | ||
| concat_ws | ||
| contains | ||
| decode | ||
| elt | ||
| encode | ||
| endswith | ||
| find_in_set | ||
| format_number | ||
| format_string | ||
| ilike | ||
| initcap | ||
| instr | ||
| lcase | ||
| length | ||
| like | ||
| lower | ||
| left | ||
| levenshtein | ||
| locate | ||
| lpad | ||
| ltrim | ||
| mask | ||
| octet_length | ||
| parse_url | ||
| position | ||
| printf | ||
| rlike | Regex pattern varies from pyspark to teradatamlspk. As teradata uses posix for regex match. | |
| regexp | ||
| regexp_like | ||
| regexp_count | ||
| regexp_extract | ||
| regexp_extract_all | ||
| regrexp_replace | ||
| regexp_substr | ||
| regexp_instr | ||
| replace | ||
| right | ||
| ucase | ||
| unbase64 | ||
| rpad | ||
| repeat | ||
| rtrim | ||
| soundex | ||
| split | ||
| split_part | ||
| startswith | ||
| substr | ||
| substring | ||
| substring_index | ||
| overlay | ||
| sentences | ||
| to_binary | ||
| to_char | Format specifiers for PySpark is different from Teradata if format specifiers lies in a Column. Make sure to update the format specifiers in the corresponding column accordingly. |
|
| to_number | Format specifiers for PySpark is different from Teradata if format specifiers lies in a Column. Make sure to update the format specifiers in the corresponding column accordingly. | |
| to_varchar | Format specifiers for PySpark is different from Teradata if format specifiers lies in a Column. Make sure to update the format specifiers in the corresponding column accordingly. | |
| translate | ||
| trim | ||
| upper | ||
| url_decode | ||
| url_encode | ||
| bit_count | ||
| bit_get | ||
| getbit | ||
| call_function | ||
| call_udf | Columns passed to the UDF should be present in the corresponding DataFrame. | |
| pandas_udf | ||
| udf | Columns passed to the UDF should be present in the corresponding DataFrame. | |
| udtf | ||
| unwrap_udt | ||
| aes_decrypt | ||
| bitmap_bit_position | ||
| bitmap_bucket_number | ||
| bitmap_construct_agg | ||
| bitmap_count | ||
| bitmap_or_agg | ||
| current_catalog | ||
| current_database | ||
| current_schema | ||
| current_user | ||
| input_file_block_length | ||
| input_file_block_start | ||
| md5 | ||
| sha | ||
| sha1 | ||
| sha2 | ||
| crc32 | ||
| hash | ||
| xxhash64 | ||
| assert_true | ||
| raise_error | ||
| reflect | ||
| hll_sketch_estimate | ||
| hll_union | ||
| java_method | ||
| stack | ||
| try_aes_decrypt | ||
| typeof | ||
| user | ||
| version | Returns Teradata database version. | |
| equalnull | ||
| ifnull | ||
| isnotnull | ||
| nullif | ||
| nvl2 | ||
| nvl | ||
| xpath | ||
| xpath_boolean | ||
| xpath_double | ||
| xpath_float | ||
| xpath_int | ||
| xpath_long | ||
| xpath_number | ||
| xpath_short | ||
| xpath_string |