Teradata R Package Function Reference | 17.00 - 17.00 - TextMorph - Teradata R Package

Teradata® R Package Function Reference

prodname
Teradata R Package
vrm_release
17.00
created_date
September 2020
category
Programming Reference
featnum
B700-4007-090K

Description

Lemmatization is a basic text analysis tool that determines the lemmas (standard forms) of words, so that all forms of a word can be grouped together, improving the accuracy of text analysis.

The TextMorph function implements a lemmatization algorithm based on the WordNet 3.0 dictionary, which is packaged with the function. If an input word is in the dictionary, the function outputs its morphs with their parts of speech; otherwise, the function outputs the input word itself and sets its part of speech to None.

When an input word has multiple morphs, the function outputs them by the order of precedence of their parts of speech: noun, verb, adj, and adv. That is, if an input word has a noun form, then it is listed first. If the same word has a verb form, then it is listed next, and so on.

Usage

  td_text_morph_mle (
      data = NULL,
      word.column = NULL,
      postag.column = NULL,
      single.output = TRUE,
      pos = NULL,
      accumulate = NULL,
      data.sequence.column = NULL,
      data.order.column = NULL
  )

Arguments

data

Required Argument.
Specfies the input tbl_teradata that contains the input words or phrases.

data.order.column

Optional Argument.
Specifies Order By columns for "data".
Values to this argument can be provided as a vector, if multiple columns are used for ordering.
Types: character OR vector of Strings (character)

word.column

Required Argument.
Specifies the name of the input tbl_teradata column that contains the words.
Types: character

postag.column

Optional Argument.
Specifies the name of the input tbl_teradata column that contains the part-of-speech (POS) tags of the words, generated by the function POSTagger td_pos_tagger_mle.
If you specify this argument, the function outputs each morph according to its POS tag.
Types: character

single.output

Optional Argument.
Specifies whether to output only one morph for each word. If you specify FALSE, the function outputs all morphs for each word.
Default Value: TRUE
Types: logical

pos

Optional Argument.
Specifies the parts of speech to output. Specification order is irrelevant; the order of precedence is: "noun", "verb", "adj", "adv". By default, the function outputs all parts of speech. If you specify this argument and the "single.output" argument is set to TRUE, then the function outputs only the first POS.
Note: The function does not determine the part of speech of the word from its context, it uses all possible parts of speech for the word in the dictionary.
Permitted Values: NOUN, VERB, ADJ, ADV
Types: character OR vector of characters

accumulate

Optional Argument.
Specifies the names of the input columns to copy to the output tbl_teradata.
Types: character OR vector of Strings (character)

data.sequence.column

Optional Argument.
Specifies the vector of column(s) that uniquely identifies each row of the input argument "data". The argument is used to ensure deterministic results for functions which produce results that vary from run to run.
Types: character OR vector of Strings (character)

Value

Function returns an object of class "td_text_morph_mle" which is a named list containing object of class "tbl_teradata".
Named list member can be referenced directly with the "$" operator using the name: result.

Examples

    # Get the current context/connection
    con <- td_get_context()$connection
    
    # Load example data.
    loadExampleData("pos_tagger_example", "paragraphs_input")
    loadExampleData("text_morph_example", "words_input")
    
    # Create object(s) of class "tbl_teradata".
    # The input table "words_input" contains different words to be
    # morphed by the function based on the parts-of-speech(POS).
    words_input <- tbl(con, "words_input")
    
    # Example 1 - This example outputs only one morph for each word as
    # "single.output" argument is set to TRUE.
    td_text_morph_out1 <- td_text_morph_mle(data=words_input,
                                            word.column='word',
                                            accumulate=c('id','word'),
                                            single.output=TRUE
                                            )
                              
    # Example 2 - This example outputs all morphs for each word as
    # "single.output" argument is set to FALSE.
    td_text_morph_out2 <- td_text_morph_mle(data=words_input,
                                            word.column='word',
                                            accumulate=c('id','word'),
                                            single.output=FALSE
                                            )
                              
    # Example 3 - When "single.output" argument is set to FALSE and "pos" is set to
    # [noun,verb], the words, better and father, in the "data" appear in
    # the output tbl_teradata object as both nouns and verbs.
    td_text_morph_out3 <- td_text_morph_mle(data=words_input,
                                            word.column='word',
                                            accumulate=c('id','word'),
                                            pos = c("noun","verb"),
                                            single.output=FALSE
                                            )
                              
    # Example 4 - When "single.output" argument is set to TRUE, the words, better
    # and father, in the "data" appear in the output tbl_teradata only as nouns.
    td_text_morph_out4 <- td_text_morph_mle(data=words_input,
                                            word.column='word',
                                            accumulate=c('id','word'),
                                            pos = c("noun","verb"),
                                            single.output=TRUE
                                            )
                          
    # Create object(s) of class "tbl_teradata".
    paragraphs_input <- tbl(con, "paragraphs_input")
    
    # Example 5 - This example uses the output of td_pos_tagger_mle() function as input.
    pos_tagger_out <- td_pos_tagger_mle(data=paragraphs_input,
                                        text.column='paratext',
                                        accumulate='paraid'
                                        )
                              
    td_text_morph_out5 <- td_text_morph_mle(data=pos_tagger_out$result,
                                            word.column='word',
                                            postag.column='pos_tag',
                                            accumulate=c('word_sn', 'word', 'pos_tag')
                                            )