I have been trying to optimize my workflow between Python & Omnisci Core Db using pymapd & cudf but running into some roadblocks (use case = export to cudf dataframe, export to csv):
1- Is pymapd the right library? (Conscious there is pyomnisci available though running through compatibility issues when trying to conda install with cudf in a dedicated conda environment.)
2- Using the select_ipc_gpu method (or select_ipc), it throws out an exception due to text columns being included (“TOmniSciException(error_msg=‘TEXT is not supported in Arrow result sets.’)”).
3- When trying another export method by using execute(“COPY (SELECT * FROM table) TO ‘/home/extract.csv’;”), date fields aren’t showing as date, is there an option to force this in the “WITH” clause?
Current workflow works fine using pandas.read_sql and pandas.to_csv though it’s on the slow(ish) side, what would be your recommendation(s) to improve this type of workflow?