Description
Lemmatization is a basic text analysis tool that determines the lemmas (standard forms) of words, so that all forms of a word can be grouped together, improving the accuracy of text analysis.
The TextMorph function implements a lemmatization algorithm based on the WordNet 3.0 dictionary, which is packaged with the function. If an input word is in the dictionary, the function outputs its morphs with their parts of speech; otherwise, the function outputs the input word itself and sets its part of speech to None.
When an input word has multiple morphs, the function outputs them by the order of precedence of their parts of speech: noun, verb, adj, and adv. That is, if an input word has a noun form, then it is listed first. If the same word has a verb form, then it is listed next, and so on.
Usage
td_text_morph_mle ( data = NULL, word.column = NULL, postag.column = NULL, single.output = TRUE, pos = NULL, accumulate = NULL, data.sequence.column = NULL, data.order.column = NULL )
Arguments
data |
Required Argument. |
data.order.column |
Optional Argument. |
word.column |
Required Argument. |
postag.column |
Optional Argument. |
single.output |
Optional Argument. |
pos |
Optional Argument. |
accumulate |
Optional Argument. |
data.sequence.column |
Optional Argument. |
Value
Function returns an object of class "td_text_morph_mle" which is a
named list containing Teradata tbl object.
Named list member can be referenced directly with the "$" operator
using name: result.
Examples
# Get the current context/connection con <- td_get_context()$connection # Load example data. loadExampleData("pos_tagger_example", "paragraphs_input") loadExampleData("text_morph_example", "words_input") # Create remote tibble objects. # The input table "words_input" contains different words to be # morphed by the function based on the parts-of-speech(POS). words_input <- tbl(con, "words_input") # Example 1 - This example outputs only one morph for each word as # "single.output" is set to TRUE. td_text_morph_out1 <- td_text_morph_mle(data=words_input, word.column='word', accumulate=c('id','word'), single.output=TRUE) # Example 2 - This example outputs all morphs for each word as # "single.output" is set to FALSE. td_text_morph_out2 <- td_text_morph_mle(data=words_input, word.column='word', accumulate=c('id','word'), single.output=FALSE) # Example 3 - With "single.output" set to FALSE and "pos" set to # [noun,verb], the words, better and father, in the "data" appear in # the output tbl_teradata object as both nouns and verbs. td_text_morph_out3 <- td_text_morph_mle(data=words_input, word.column='word', accumulate=c('id','word'), pos = c("noun","verb"), single.output=FALSE) # Example 4 - With "single.output" set to TRUE, the words in "data", better # and father, appear in the output table only as nouns. td_text_morph_out4 <- td_text_morph_mle(data=words_input, word.column='word', accumulate=c('id','word'), pos = c("noun","verb"), single.output=TRUE) # Create remote tibble objects. paragraphs_input <- tbl(con, "paragraphs_input") # Example 5 - This example uses the output of td_pos_tagger_mle as input. pos_tagger_out <- td_pos_tagger_mle(data=paragraphs_input, text.column='paratext', accumulate='paraid') td_text_morph_out5 <- td_text_morph_mle(data=pos_tagger_out$result, word.column='word', postag.column='pos_tag', accumulate=c('word_sn', 'word', 'pos_tag'))