ChunkText Example: Loading Tokenizer with TDStudio - Teradata Vantage

Teradata Vantage™ - Bring Your Own Model User Guide

Deployment
VantageCloud
VantageCore
Edition
VMware
Enterprise
IntelliFlex
Lake
Product
Teradata Vantage
Release Number
20.00
Published
May 2026
ft:locale
en-US
ft:lastEdition
2026-06-18
dita:mapPath
fee1607120608274.ditamap
dita:ditavalPath
ayr1485454803741.ditaval
dita:id
fee1607120608274
-- load  model using Teradata Studio/client
-- distilbert is the tokenizer_id of the tokenizer to be installed
insert into chunk_tokenizers_tst values ('bge-small-en-v1.5', ?);

--The '?' will open a GUI to browse the tokenizer file that needs to be inserted