Loading data using pymapd error

While loading data into a table I am getting this error :
TOmniSciException: TOmniSciException(error_msg=‘Exception: Expected string type’)

I wrote the following query to create the table:
query= “CREATE TABLE IF NOT EXISTS t1( time_stamp TIMESTAMP NOT NULL ENCODING FIXED(32),port_name TEXT NOT NULL ENCODING DICT ,counter_type TEXT NOT NULL ENCODING DICT,group_id SMALLINT,counter_name TEXT NOT NULL ENCODING DICT, switch_id SMALLINT,port_type TEXT ENCODING DICT,port_id SMALLINT,counter_value FLOAT) with (max_rows=110, fragment_size=1);”

And used this command while trying to load the table:
con.load_table(“t1”, df)

Any idea what this error may indicate.

-Dipanwita.

Hi,

My best guess is the max_rows and fragment size are set with extremely low values.

You can also try to run the load_table method into the same table with default values (so without the with clause) and also on a different table name (if I am not wrong the load table would create a table if it’s not exists, iferring datatypes from the data frame)

Candido

1 Like

Thank you, it worked !!! I increased the fragment size and max row size like you mentioned and it seems to be working now.

For most use cases the default fragment size is better than anything you’d be able to set. Our default fragment size is 32 million rows, is there a reason you’re setting this so low? You can use this thumb rule to configure it

(number of rows in your table)/(number of CPU cores)

Hi,

My intention was to store streaming data (for, say, 15 minutes from the current time which means data beyond 15 will be automatically deleted ). One way to do this is to restrict the max number of rows, I also found frag_size as an option so tried that as well. We might just fix the number of rows and not touch the fragment size, at this point.