I am building an app that requires a GPU database and Omni-sci seems to fit the bill quite well, I’ve only just started looking into Omni-Sci and am still at the stage of feeling overwhelmed with all the new information. I’m looking for some advice to help focus my efforts in the right direction.
I have a C++ Server and am looking for a way to interact with Omni Sci using C++
Ideally, I would like my C++ server to completely control all interactions with OmniSci. There will be no direct user interaction with the database. I would like the server to be able to:
setup schemas/indexes(if these are used),
extract query results
I would also like to send functions as part of the queries, I am led to believe that the LLVM compiler might allow this but I’m not sure how exactly that would work yet.
My server also sends workloads to a C++ PyTorch API, so I would like to be able to access data from OmniSci to send to my PyTorch library. The simple way to do this would be to query for some data and then send the query results to my ML libraries, but I get the feeling there might be a way to leverage Arrow and do a non-copy transfer of data to my ML Libraries.
My question is how much of the above requirements could be achieved by using pre-existing APIs or connectivity like Thrift. And if so what do I need to look into. Do you have any dedicated C++ API’s
What if any of the above would require me to work directly with the OmniSci source code, which id like to avoid if I can.
Any information that helps clear up my thoughts would be greatly appreciated.