MapD Community Edition: Data Compressions


#1

What level/kind of compressions are allowed in mapD ce ?


#2

Are you referring to importing compressed files?


#3

You basically get a compression thru text dictionary encoding; you can save memory and disk space using fixed encodings on integer datatypes


#4

Thx @aznable , Is dict encoding being implemented by mapd at the data ingestion time? Are there any settings we need to get this encoding enabled in Community edition?


#5

You have to define a text column with the dictionary encoding on table creation; the encoding has to be defined accordingly the cardinality of distinct values of the column itself ( 8 bits if the dv are less than 256, 16 bit it they are are less tha 64k and 32 otherwise). At ingestion time they will be encoded and you will be able to filter, join and aggregate using the encoded column.

Anyway don’t overestimate the compression factor because mapd is incredibly fast performing joins, so a balanced design of tables is important as usual