16.20 - Memory Limitation - Advanced SQL Engine - Teradata Database

Teradata Vantage™ - SQL External Routine Programming

Product
Advanced SQL Engine
Teradata Database
Release Number
16.20
Release Date
April 2020
Content Type
Programming Reference
Publication ID
B035-1147-162K
Language
English (United States)

R table operators use the udfGPL secure server. The GPLUDFServerMemSize field in the cufconfig GDO (Globally Distributed Object) is used to limit the memory of the udfGPL server. This limit is set to 3.5 GB by default, but you can set the limit from 0 to 3.5 GB. To use the maximum available memory, set GPLUDFServerMemSize to 0; however, a nonzero value is recommended so that there is a limit to the amount of memory that can be used.

For information about the cufconfig utility, see Teradata Vantage™ - Database Utilities , B035-1102 .

If you set a limit over 2 GB, you get a warning message:
Warning: Although the memory limit is set, it is very high. Your system has high risk of running out of virtual memory. Please consider to set it to a lower value.

If you set a limit over 3.5 GB, you get an "out of range" error:

Out of range error. Field GPLUDFServerMemSize: 0 - 3758096384

The maximum amount of memory that can be used by an R table operator is determined by the GPLUDFServerMemSize limit. It includes user data and other metadata used by R for various maintenance activities. Any attempt to use more than the limit results in evaluation errors while running the program. The error may be caused by the memory request from either user program, R internal calls, or operating system calls.

The udfGPL server may seem to consume more memory than the specified limit. This is expected because various resources are required to start and run the user programs. The GPLUDFServerMemSize limit is in addition to whatever the server uses. For example, assume that GPLUDFServerMemSize is set to 32 MB. When the udfGPL server starts, it may need 103 MB. The table operator would have 32 MB in addition to the 103 MB for usage. Therefore, in total, about 135 MB of memory may be consumed by the user program or operator. The memory may be split evenly between different streams (input and output) of the table operator when required depending on the memory usage requirements of the operator.

A GPL server might not be started if the total of GPLUDFServerMemSize plus the memory space for the R server itself after initialization is bigger than UINT_MAX at runtime. In this case, you will get an error:
Error 7583 The secure mode processes had a set up error.

Note that when an R table operator is executed, there is some memory held by the R interpreter. R generally does not reclaim all memory used by the program objects. It often tries to reclaim such unused memory by running garbage collector automatically. Therefore, in the next run of another R table operator, the available memory may be less than the GPLUDFServerMemSize limit. You can overcome this by forcefully releasing the memory used by different objects in the environment. For example, rm(list=ls()).