Ruby support via Thrift


Anyone out there using MapD with Ruby (via Thrift)? I generated the Ruby classes from the mapd.thrift IDL file (thrift --gen rb mapd.thrift) and found that the “mapd_types.rb” file it generated has types with circular references, and Ruby just can’t handle it.

2.3.3 :002 > require './app/models/map_d/mapd_types'
NameError: uninitialized constant TDatum
Did you mean?  TDatumVal
	from app/models/map_d/mapd_types.rb:80:in `<class:TDatumVal>'

If I change the order so that TDatum is defined first, we run into a similar problem where TDatumVal is now undefined. There may be other circular references in this file, but this is the first one I tripped on.

Is this a solved problem, or are we breaking new ground here?


This is something we’ve hit in the Python world as well. The issue is that we currently represent arrays as recursive types, which some languages don’t like.

For Python I have a script at which circumvents the issue by delaying the initializing a few things until after all the types have been defined.

Alternatively you can simply remove array support. Not desirable, but it will at least let you continue without further hacks:

Modify mapd.thrift to remove these two lines:
4: list<TDatum> arr_val from TDatumVal (~line 43)
4: list<TColumn> arr_col from TColumnData (~line 81)

and then regenerate the ruby bindings with thrift -gen ...


Thanks for the quick reply. Are there plans to change how arrays are represented to avoid this problem, or are you sort of stuck with doing it this way?

I ended up exploiting Ruby’s feature of re-opening classes to simply define “skeleton” constants before where they were being referenced, and then let the real definition come later. For example:

# Hack to avoid circular reference
class TDatum

class TDatumVal
  ARR_VAL => {:type => ::Thrift::Types::LIST, :name => 'arr_val', :element => {:type => ::Thrift::Types::STRUCT, :class => ::TDatum}}

class TDatum
  ....real definition....

Thanks again.


One other question related to this. The MapD service in mapd.thrift has this:

  TSessionId connect(1: string user, 2: string passwd, 3: string dbname) throws (1: TMapDException e)

How do I specify the hostname and port to connect to given this API definition? Is that embedded in dbname somehow?


Ah, nevermind, I see that I have to create the thrift transport and protocol objects which define this stuff.

Thrift newbie. :slight_smile: