Discussions

Expand all | Collapse all

TApplicationException: String too long for dictionary encoding.

  • 1.  TApplicationException: String too long for dictionary encoding.

    Posted 25 days ago
    Hi there

    I am getting the error

    TApplicationException: String too long for dictionary encoding.
    I have encoded all string variables as "text encoding dict(32)" when building a create table statement.

    What should I use as the format for my long string variables?

    Please can you tell me the maximum number of characters I can put into a field defined as text encoding dict(32)?

    Many thanks 

    Michael
    #Core


  • 2.  RE: TApplicationException: String too long for dictionary encoding.

    Posted 25 days ago
    Hi @Michael Kay,

    The max length of a string is 32767 whatever the encoding you are using, since 3.2.0 release as you can read here

    https://www.omnisci.com/docs/latest/7_0_release.html#id84

    If you try to inject a longer string, an error occurs as you can observe into the log file

    E0426 17:23:03.750990 64661 Importer.cpp:2005] Input exception thrown: String too long for column c1 was 32820 max is 32767. Row discarded.
    you can try splitting a string on more text fields or using an array of texts (never tried)



  • 3.  RE: TApplicationException: String too long for dictionary encoding.

    OmniSci Employee
    Posted 25 days ago
    Hi @Michael Kay, in addition to what @Candido Dessanti suggested, if you don't need to group by or join on the string (just project or filter on it), you might try making it `Encoding None`, i.e. non-dictionary encoded.


  • 4.  RE: TApplicationException: String too long for dictionary encoding.

    Posted 25 days ago
    Edited by Candido Dessanti 25 days ago
    Hi @Todd Mostak,

    I tried texts without a dictionary encoding, but at least, with copy command is failing on Omnisci 4.5 and @Michael Kay is loading Arrow's data with pymapd driver, that has a bug loading text encoding none, so if the bug hasn't been corrected, using unencoded texts isn't a viable option for him.


    ​​


  • 5.  RE: TApplicationException: String too long for dictionary encoding.

    Posted 24 days ago
    Thanks Guys
    I truncated any text field > 32767 and used text encoding dict(32) and all good