Teradata Package for R Function Reference | 17.20 - WordEmbeddings - Teradata Package for R - Look here for syntax, methods and examples for the functions included in the Teradata Package for R.

Teradata® Package for R Function Reference

Deployment
VantageCloud
VantageCore
Edition
Enterprise
IntelliFlex
VMware
Product
Teradata Package for R
Release Number
17.20
Published
March 2024
Language
English (United States)
Last Update
2024-05-03
dita:id
TeradataR_FxRef_Enterprise_1720
Product Category
Teradata Vantage

WordEmbeddings

Description

The td_word_embeddings_sqle() function produces vectors for each piece of text and finds the similarity between the texts.
Word embedding is the representation of a word in multi-dimensional space such that words with similar meanings have similar embedding.
Each word is mapped to a vector of real numbers that represent the word.

The function contains training and prediction using models. The models contain each possible token and its corresponding vectors. The closer the distance between the vectors the more the similarity. Function operations are token-embedding, doc-embedding, token2token-similarity, and doc2doc-similarity.

Usage

  td_word_embeddings_sqle (
      data = NULL,
      model = NULL,
      id.column = NULL,
      model.text.column = NULL,
      model.vector.columns = NULL,
      primary.column = NULL,
      secondary.column = NULL,
      operation = "token-embedding",
      accumulate = NULL,
      convert.to.lowercase = TRUE,
      remove.stopwords = FALSE,
      stem.tokens = FALSE,
      ...
  )

Arguments

data

Required Argument.
Specifies the input tbl_teradata.
Types: tbl_teradata

model

Required Argument.
Specifies the tbl_teradata which contains the model vectors for all the possible tokens.
Types: tbl_teradata

id.column

Required Argument.
Specifies the input data column name that has the unique identifier for each row in the input.
Types: character

model.text.column

Required Argument.
Specifies the column that contains the token in the "model".
Types: character

model.vector.columns

Required Argument.
Specifies range of columns in the "model" data that contains real value vector.
Types: character OR vector of Strings (character)

primary.column

Required Argument.
Specifies name of the input data column that contains the text.
Types: character

secondary.column

Optional Argument.
Specifies name of the input data column that contains the text.
Note:

  • This field is applicable for the 'token2token-similarity' and 'doc2doc-similarity' operations only.

Types: character

operation

Optional Argument.
Specifies the operation to be performed on the data.
Permitted Values: token-embedding, doc-embedding, token2token-similarity, doc2doc-similarity Default Value: 'token-embedding'

Types: character

accumulate

Optional Argument.
Specifies the name(s) of input tbl_teradata column(s) to copy to the output.
Note:

  • This is not applicable with the 'token-embedding' operation.

Types: character OR vector of Strings (character)

convert.to.lowercase

Optional Argument.
Specifies whether to convert input data to lower case or not.
When set to TRUE, input data is converted to lower case.
Otherwise, input data is not converted to lower case.
Default Value: TRUE
Types: logical

remove.stopwords

Optional Argument.
Specifies whether to remove stop words from the input data.
Note:

  • This is not applicable with the 'token2token-similarity' operation.

Default Value: FALSE
Types: logical

stem.tokens

Optional Argument.
Specifies whether to convert word to its root word in the input data.
For example, convert 'going' to 'go'.
Default Value: FALSE
Types: logical

...

Specifies the generic keyword arguments SQLE functions accept. Below
are the generic keyword arguments:

persist:
Optional Argument.
Specifies whether to persist the results of the function in a table or not. When set to TRUE, results are persisted in a table; otherwise, results are garbage collected at the end of the session.
Default Value: FALSE
Types: logical

volatile:
Optional Argument.
Specifies whether to put the results of the function in a volatile table or not. When set to TRUE, results are stored in a volatile table, otherwise not.
Default Value: FALSE
Types: logical

Function allows the user to partition, hash, order or local order the input data. These generic arguments are available for each argument that accepts tbl_teradata as input and can be accessed as:

  • "<input.data.arg.name>.partition.column" accepts character or vector of character (Strings)

  • "<input.data.arg.name>.hash.column" accepts character or vector of character (Strings)

  • "<input.data.arg.name>.order.column" accepts character or vector of character (Strings)

  • "local.order.<input.data.arg.name>" accepts logical

Note:
These generic arguments are supported by tdplyr if the underlying SQL Engine function supports, else an exception is raised.

Value

Function returns an object of class "td_word_embeddings_sqle" which is a named list containing object of class "tbl_teradata".
Named list member(s) can be referenced directly with the "$" operator using the name(s):result

Examples

  
    
    # Get the current context/connection.
    con <- td_get_context()$connection
    
    # Load the example data.
    loadExampleData("tdplyr_example", "word_embed_model",
                    "word_embed_input_table1","word_embed_input_table2")
    
    # Create tbl_teradata object.
    model_input <- tbl(con, "word_embed_model")
    data_input1 <- tbl(con, "word_embed_input_table1")
    data_input2 <- tbl(con, "word_embed_input_table2")
    
    # Check the list of available analytic functions.
    display_analytic_functions()
    
    # Example 1 : Generate vectors for each words present in column 'doc1'
    #             using the word embedding model and 'token-embedding' operation.
    #             Each word is assigned with vectors and closer the distance
    #             between two vectors greater the similarity between two words.
    WordEmbeddings_out1 <- td_word_embeddings_sqle(
                            data = data_input1,
                            model = model_input,
                            id.column = "doc_id",
                            model.text.column = "token",
                            model.vector.columns = c("v1", "v2", "v3", "v4"),
                            primary.column = "doc1",
                            operation = "token-embedding")
    
    # Print the result.
    print(WordEmbeddings_out1$result)
    
    # Example 2 : Generate vectors for each row present in column 'doc1'
    #             using the word embedding model and 'doc-embedding' operation.
    #             Each row with para or sentence is assigned with vectors and
    #             closer the distance between two vectors greater the similarity
    #             between two para or sentence.
    WordEmbeddings_out2 <- td_word_embeddings_sqle(
                            data = data_input1,
                            model = model_input,
                            id.column = "doc_id",
                            model.text.column = "token",
                            model.vector.columns = c("v1", "v2", "v3", "v4"),
                            primary.column = "doc1",
                            operation = "doc-embedding",
                            accumulate = "doc1")
    
    # Print the result.
    print(WordEmbeddings_out2$result)
    
    # Example 3 : Find the similarity between two columns 'token1' and 'token2' in 'data_input2',
    #             using the word embedding model and 'token2token-similarity' operation.
    WordEmbeddings_out3 <- td_word_embeddings_sqle(
                            data = data_input2,
                            model = model_input,
                            id.column = "token_id",
                            model.text.column = "token",
                            model.vector.columns = c("v1", "v2", "v3", "v4"),
                            primary.column = "token1",
                            secondary.column = "token2",
                            operation = "token2token-similarity",
                            accumulate = c("token1", "token2"))
    
    # Print the result.
    print(WordEmbeddings_out3$result)
    
    # Example 4 : Find the similarity between two columns 'doc1' and 'doc2' in 'data_input1',
    #             using the word embedding model and 'doc2doc-similarity' operation.
    WordEmbeddings_out4 <- td_word_embeddings_sqle(
                            data = data_input1,
                            model = model_input,
                            id.column = "doc_id",
                            model.text.column = "token",
                            model.vector.columns = c("v1", "v2", "v3", "v4"),
                            primary.column = "doc1",
                            secondary.column = "doc2",
                            operation = "doc2doc-similarity",
                            accumulate = c("doc1", "doc2"))
    
    # Print the result.
    print(WordEmbeddings_out4$result)