CPU RAM requirements by the size

Hi All,

Is there a way we can estimate the RAM needed based on the amount of data.
In the hardware requirements it was given in comparison with no. of records.

Is there a way we can get the approximate amount of RAM needed for 200GB hot data.

Regards,
Sireesha

1 Like

Hi @sireesha,

to get an approximate amount of COU/GPU RAM needed by your project, you should need to know at least how many records and which columns (and the datatype used) of them that you need to have into CPU and GPU tiers

How have you calculated the 200GB? are they 200GB when in text format? which datatypes have to be conveted into?

I suggest you to download the OS version of OmnisciDB, create tables and load a subste of datam then optimize tables (you can use the \o omnisql command), load the same data on memory ooptmized tables, then run a set of significant query and collect the emount of caches used by the system (\memory_summary command of omnisql)

as an example, I loaded a TPCH schema with SF of 100, so 100gb of uncompressed data (the data is designed to don’t be compressible), run some queries to get how much memory (CPU and GPU) have been used

User admin connected to database omnisci
omnisql> \memory_summary
OmniSci Server CPU Memory Summary:
            MAX            USE      ALLOCATED           FREE
    51433.94 MB    24413.38 MB    24576.00 MB      162.61 MB

OmniSci Server GPU Memory Summary:
[GPU]            MAX            USE      ALLOCATED           FREE
  [0]    10635.44 MB     8774.19 MB    10635.44 MB     1861.25 MB
  [1]    10634.12 MB     8271.92 MB    10634.12 MB     2362.20 MB

User admin connected to database omnisci
omnisql> \memory_summary
OmniSci Server CPU Memory Summary:
MAX USE ALLOCATED FREE
51433.94 MB 24413.38 MB 24576.00 MB 162.61 MB

OmniSci Server GPU Memory Summary:
[GPU] MAX USE ALLOCATED FREE
[0] 10635.44 MB 8774.19 MB 10635.44 MB 1861.25 MB
[1] 10634.12 MB 8271.92 MB 10634.12 MB 2362.20 MB

So running all the queries I needed used just 24GB of CPU Ram (and just 16GB of GPU RAM, but I know some queries fallen back to CPU for execution).

To run the same queries on a SF of 5000 I would need 50 times the memory (more or less 1200GB); this is a decent extimation

@candido.dessanti,

Thank you so much .This information helps.
I tried the memory summary command and it gave the RAM allocated and used .
I will do some test runs and estimate the size required for our data .