ChunkText Syntax Elements | Vantage BYOM - ChunkText Syntax Elements - Teradata Vantage

Teradata Vantage™ - Bring Your Own Model User Guide
Deployment
VantageCloud
VantageCore
Edition
VMware
Enterprise
IntelliFlex
Lake
Product
Teradata Vantage
Release Number
20.00
Published
May 2026
ft:locale
en-US
ft:lastEdition
2026-06-18
dita:mapPath
fee1607120608274.ditamap
dita:ditavalPath
ayr1485454803741.ditaval
dita:id
fee1607120608274
MaxTokens
Specify the maximum number of tokens per chunk.
Enter an integer value greater than 2.
MaxOverlapWords
[Optional] Specify the number of words to carry over from the previous chunk.

Default: 0

Enter an integer that is greater than or equal to 0 and less than MaxTokens/2. 0 specifies no overlap.
FirstNChunks
[Optional] Specifies the number of chunks to return from the beginning of the output.

Default: 0

Enter an integer value greater than or equal to 0. 0 returns all chunks.
OutputLength
[Optional] Specifies the length of the output txt column.

Default: 6000

Enter an integer value from 1 to 32000.
The output column is always a VARCHAR that supports Unicode.
SplittingStrategy
[Optional] Specifies the method used to split the input text.

Valid values: 'WORDS', 'SENTENCES', and 'PARAGRAPHS'

Default: 'WORDS'

  • 'WORDS': Splits text by words. Groups words so the total tokens in each chunk do not exceed MaxTokens.
  • 'SENTENCES': Splits text by sentences using language boundaries. Groups complete sentences within the token limit.
  • 'PARAGRAPHS': Splits text by paragraphs. Creates chunks with one or more complete paragraphs within the token limit.
IsDebug
[Optional] When true, debug statements are added to a trace table that must be created beforehand. See Error Logs.

Only available with BYOM version 3.00.00.02 and later.

Default: False
Unicode values are not supported in argument name-value pairs.