Discussions

Expand all | Collapse all

Error launching the GPU kernel: an illegal memory access was encountered

  • 1.  Error launching the GPU kernel: an illegal memory access was encountered

    Posted 10 days ago
    Hi everybody,

    I was testing the DB with a large dataset and few complex queries.
    While doing so I encountered the following error message:
    I0312 10:35:41.545658 71650 Catalog.cpp:3074] Time to load Dictionary 1_2 was 56ms
    I0312 10:35:41.550999 71664 Calcite.cpp:395] Time in Thrift 1 (ms), Time in Java Calcite server 53 (ms)
    I0312 10:35:41.551895 71664 RelAlgOptimizer.cpp:755] (RelProject<140184021675344> (RexInput 8 140184782011392) (RexLiteral 1) (RexInput 0 140184782011392) (RexInput 2 140184782011392)) eliminated 1 columns.
    F0312 10:35:42.319502 45530 Execute.cpp:2210] Error launching the GPU kernel: an illegal memory access was encountered
    I0312 10:35:42.320631 45530 Calcite.cpp:432] Shutting down Calcite server
    I0312 10:35:42.328037 45530 Calcite.cpp:441] shut down Calcite
    I0312 10:35:42.328122 45530 MapDServer.cpp:102] Interrupt signal (11) received.


    My setup is:
    • Tesla K80 with NVIDIA-SMI 410.79       Driver Version: 410.79       CUDA Version: 10.0;
    • Ubuntu 16.04.5 LTS xenial;

    • OmniSci Server Version: 4.5.0-20190221-e41be43ff0 OS from  the repositories.

    The disk space usage looks like this:

    124K /var/lib/omnisci/data/mapd_catalogs
    
    2.4M /var/lib/omnisci/data/mapd_log
    
    147G /var/lib/omnisci/data/mapd_data/table_1_1
    
    8.1M /var/lib/omnisci/data/mapd_data/DB_1_DICT_2
    
    8.1M /var/lib/omnisci/data/mapd_data/DB_1_DICT_4
    
    2.0G /var/lib/omnisci/data/mapd_data/DB_1_DICT_1
    
    8.1M /var/lib/omnisci/data/mapd_data/DB_1_DICT_3
    
    12K /var/lib/omnisci/data/mapd_data/table_0_0
    
    149G /var/lib/omnisci/data/mapd_data
    
    4.0K /var/lib/omnisci/data/mapd_export
    
    149G /var/lib/omnisci/data
    
    149G /var/lib/omnisci/


    Does anyone have any idea about why this is occurring? Is there a way for me to have a better understanding of why is this happening?


    Best,
    Francesco.
    #Core


  • 2.  RE: Error launching the GPU kernel: an illegal memory access was encountered

    OmniSci Employee
    Posted 10 days ago
    Hi @Francesco Ferretti. ​Can you please provide the verbose logs and the query memory descriptor output?

    ------------------------------
    Eric Grant
    Director of Community
    OmniSci
    Seattle WA
    ------------------------------



  • 3.  RE: Error launching the GPU kernel: an illegal memory access was encountered

    Posted 9 days ago
    Edited by Candido Dessanti 9 days ago
    Ciao @Francesco Ferretti,

    to get a QMD output, you have to set to true the verbose option.

    If you can, please include the query you


    ------------------------------
    Candido Dessanti
    Dba
    consulting
    Rome
    ------------------------------