Total Number of vWorkers Calculation - Aster Execution Engine

Aster Instance Installation Guide for Aster-on-Hadoop Only

Product
Aster Execution Engine
Release Number
7.00.02
Published
July 2017
Language
English (United States)
Last Update
2018-04-17
dita:mapPath
cng1496682698441.ditamap
dita:ditavalPath
Generic_no_ie_no_tempfilter.ditaval
dita:id
zjc1495153304403
lifecycle
previous
Product Category
Software
The output from this calculation can be used when determining the number of vWorkers under Setting HDP 2.3.4 YARN Configuration Changes, and as the NUM_PARTITIONS parameter when you configure the /home/beehive/.cluster_config file.

The conclusion from the above calculations is that one vWorker with 32 GB memory and four CPUs can handle 500 GB of data and ten concurrent users. Using this information, the formula to calculate the total number of vWorkers is:

N = y/10
TN = N * (x/500)

where x is the database size in GB and y is the concurrency

For example, if the user requirement is 10 TB and 40 users:

N = 40/10 = 4
TN = 4 * (10 TB / 0.5 TB) = 4 * 20 = 80 total vWorkers

Using this information and calculations for vWorker per container, you can calculate the number of YARN containers needed by Aster. Aster requests 1 container per node, so this estimation will help you determine the number of nodes.