1.0 - 8.00 - TextParser Example 2: StemmingExceptions without StopWords - Teradata Vantage

Teradata® Vantage Machine Learning Engine Analytic Function Reference

Product
Teradata Vantage
Release Number
1.0
8.00
Release Date
May 2019
Content Type
Programming Reference
Publication ID
B700-4003-098K
Language
English (United States)

Input

complaints_mini
doc_id text_data category
1 consumer was driving approximately 45 mph hit a deer with the front bumper and then ran into an enbankment head-on passenger's side air bag did deploy hit windshield and deployed outward. driver's side airbag cover opened but did not inflate it was still folded causing injuries. crash
2 when vehicle was involved in a crash totalling vehicle driver's side/ passenger's side air bags did not deploy. vehicle was making a left turn and was hit by a ford f350 traveling about 35 mph on the front passenger's side. driver hit his head-on the steering wheel. hurt his knee and received neck and back injuries. crash

The stemming exceptions table, stemmingexception.text, contains:

consumer customer
enbankment embankment

SQL Call

SELECT * FROM TextParser (
  ON complaints_mini
  USING
  TextColumn ('text_data')
  ToLowerCase ('true')
  Stemming ('true')
  OutputByWord ('false')
  Punctuation ('\[.,?\!\]')
  Accumulate ('doc_id', 'category')
  StemmingExceptions ('stemmingexception.txt')
) AS dt ORDER BY doc_id;

Output

doc_id category tokens
1 crash customer was drive approxim 45 mph hit a deer with the front bumper and then ran into an embankment head-on passeng side air bag did deploy hit windshield and deploy outward driver side airbag cover open but did not inflat it was still fold caus injuri
2 crash when vehicl was involv in a crash total vehicl driver side/ passeng side air bag did not deploy vehicl was make a left turn and was hit by a ford f350 travel about 35 mph on the front passeng side driver hit his head-on the steer wheel hurt his knee and receiv neck and back injuri