I am using the Pentaho 8.1 ETL tool to copy data from mapd 4.0 to postgres 10.4. I can get everything to work as long as the table in mapd is small.
But if I try to copy an absolutely identical table (only difference is that it has more rows - about 100 million). I cannot copy the table out of mapd. I get the error message:
TSocket::write_partial() send() <Host: localhost Port: 9091>Broken pipe
Also, if I do a simple “select count(*) from my_table” I also get the same error message - but, it does actually eventually return the actual number of rows (about 100 million).
I find the various hard coded limits in mapd frustrating. I don’t care how long it takes the query to return - I just want to do a simple 100 million row copy from mapd to postgres.
Are there configuration options you can set in mapd to cause it to allow large queries to run to completion even if the amount of time they take is large?