detect_language() Parameters | Teradata Package for Generative AI - detect_language Parameters - Teradata VantageCloud Lake

Lake - Analyze Your Data with ClearScape Analytics™
Deployment
VantageCloud
Edition
Lake
Product
Teradata VantageCloud Lake
Release Number
Published
February 2025
ft:locale
en-US
ft:lastEdition
2026-02-20
dita:mapPath
tcl1683670667798.ditamap
dita:ditavalPath
pny1626732985837.ditaval
dita:id
tcl1683670667798

These parameters are supported when 'api_type' is set to aws, azure, gcp, or nim.

Name Required or Optional Description Type Default Value Permitted Values
column Required Specifies the column of the teradataml DataFrame. string N/A N/A
data Required Specifies the teradataml DataFrame containing the column specified in "column". teradataml DataFrame N/A N/A
persist Optional Specifies whether to persist the output or not. When set to True, results are stored in permanent tables, otherwise in volatile tables. boolean False True, False
accumulate Optional Specifies the names of input teradataml DataFrame columns to copy to the output. By default, the method copies no input teradataml DataFrame columns to the output. string or list of strings All columns Valid column from 'data' DataFrame.
data_partition_column Optional Specifies the column used to partition the the 'data' for processing. string Non Valid column from 'data' DataFrame.
is_debug Optional Specifies whether to enable error logging. boolean False True/False
volatile Optional Specifies whether to put the results of the function in a volatile table or not. When set to True, results are stored in a volatile table, otherwise not. boolean False True/False
lang Optional Specifies the languages for detection. If no specific language is provided, the method will attempt to automatically detect the language of the text to the best of its ability. It can also detect languages that are not specified in the parameter. string N/A N/A
show_num_tokens Optional Specifies whether to display token information in the query output. If set to True, it includes input and output token counts in the result. boolean False True/False
refresh_credential_time Optional Specifies the refresh interval, in seconds, for credentials used by AWS and Azure VMs when connecting to AWS Bedrock and Azure AI without requiring external credentials. Applicable only if "api_type" is 'azure' or 'aws'. integer 0  

These parameters are supported when 'api_type' is set to hugging_face.

Name Required or Optional Description Type Default Value Permitted Values
column Required Specifies the column of the teradataml DataFrame. string N/A N/A
data Required Specifies the teradataml DataFrame containing the column specified in "column". teradataml DataFrame N/A N/A
persist Optional Specifies whether to persist the output or not. When set to True, results are stored in permanent tables, otherwise in volatile tables. boolean False True, False
returns Optional
  • Specifies the returns clause for the apply query in a user-defined script for inferencing. It contains a dict which specifies the column name as key and datatype as the value.
  • For example, the script returns two columns ‘text’ and ‘sentiment of VARCHAR datatype, then the "returns" argument looks like this: {"text": VARCHAR(10000), "sentiment": VARCHAR(10000)}
 dictionary N/A N/A
script Optional Specifies the user defined script for inferencing. Use this argument to configure the model to process the input and output.
The sample script uses the following mechanism to load and use the model:
  • It uses 'AutoTokenizer.from_pretrained()' function from the transformers library that automatically detects the correct tokenizer class based on the model's configuration.
  • The "transformer_class" provided in the "model_args" argument of TeradataAI class is used to load the given pre-trained model.

    Example: 'AutoModelForSequenceClassification',

    'AutoModelForTokenClassification' etc.

  • It then uses the 'pipeline' module for using the model. Pipeline makes use of "task" argument which defines which pipeline to be used for processing.

    Example: 'token-classification', 'summarization' etc.

  • The following command line arguments should be handled by the user-defined script:
    • 0th argument: script_name.
    • 1st argument: string containing extra parameters in dict format. Using json.loads() will convert this to dict format.
    • The following arguments are passed if supplied by the respective function:
      • classify_labels = "labels" argument for classify_text()
      • target_lang = "target_lang" argument for translate()
      • entity_groups = "entity_groups" argument for recognize_entities()
      • pipeline_kwargs = "pipeline_kwargs" for all functions
      • delimiter = "delimiter" for all functions
      • func_name = "func_name" for all functions
 string N/A N/A
output_labels Optional
  • Specifies the output labels which are used in the "returns" argument for the apply query. Use this in the default script when you want specific columns from the output.
  • For example, if the model outputs text: [{'label': 'anger', 'score': 0.9979689717292786}], in order to extract 'label' and 'score' as separate columns, "output_labels" can be specified as follows: output_labels={'label': str, 'score': float}
 dictionary N/A N/A
delimiter Optional Specifies a delimiter to use when reading columns from a row and writing result columns. Delimiter must be a valid Unicode code point.
  • "quotechar" cannot be the same as "delimiter".
  • The value of "delimiter" cannot be an empty string, newline, and carriage return.
 string comma (,) N/A
quotechar Optional Specifies the character used to quote all input and output values for the script.
"quotechar" value cannot be the same as "delimiter".
string double quote (") N/A
task Optional string N/A N/A
libs Optional Specifies the add-on Python library names to be installed. string or list of strings N/A N/A
pipeline_kwargs Optional Specifies any extra parameters which needs to be supplied to the 'pipeline' function of transformers module. This can be used in both sample script and user defined script. Refer to the notes in "script" parameter on usage. dictionary N/A N/A