7.00.02 - Total Number of vWorkers Calculation - Aster Execution Engine

Aster Instance Installation Guide for Aster-on-Hadoop Only

Aster Execution Engine
July 2017
The output from this calculation can be used when determining the number of vWorkers under Setting HDP 2.3.4 YARN Configuration Changes, and as the NUM_PARTITIONS parameter when you configure the /home/beehive/.cluster_config file.

The conclusion from the above calculations is that one vWorker with 32 GB memory and four CPUs can handle 500 GB of data and ten concurrent users. Using this information, the formula to calculate the total number of vWorkers is:

N = y/10
TN = N * (x/500)

where x is the database size in GB and y is the concurrency

For example, if the user requirement is 10 TB and 40 users:

N = 40/10 = 4
TN = 4 * (10 TB / 0.5 TB) = 4 * 20 = 80 total vWorkers

Using this information and calculations for vWorker per container, you can calculate the number of YARN containers needed by Aster. Aster requests 1 container per node, so this estimation will help you determine the number of nodes.