7.00.02.01 - aa.text.tokenizer() - Aster R

Teradata Aster® R User GuideUpdate 3

prodname
Aster R
vrm_release
7.00.02.01
created_date
December 2017
category
Programming Reference
User Guide
featnum
B700-1033-700K

Use the aa.text.tokenizer() function to extract tokens (or text segments) from text. Examples of tokens are words, punctuation marks, and numbers. This function supports the following languages: English, Chinese, and Japanese.