Question about MapD compiling & profiling


Q1) I am currently profiling MapD using TPC-H data. And we’re looking at where the botleneck comes from. But there’s not much information. Have you ever used CUPTI to apply to a java file about MapDJDBC? If you did, I would like to know what kind of CPTI metric you used. And I would like to know if there is a source code related to CUPTI and MapD.

Q2) I am trying to compile MapDServer.cpp. However, the error occurs because “MapDRelease.h” was not found. How do I create that file?

Q3) When compiling MapDServer.cpp, can I change the slab size within MapdServer.cpp code?

I’m sorry for asking too many questions. Is there a way to get in touch with MapD developers quickly other than this community?


Hey thanks for trying MapD out.

A1) CUPTI might be a level higher for profiling you may first want to look into nvprof or nsight from nvidia to generally understand how the database works. For query profiling you can run many tools which may not be GPU specific. We have a CPU profile option which enables gperf tools ENABLE_PROFILER. As the bottlenecks could be on CPU as well.

A2) To compile mapd first, please carefully read the Read Me on our repository. There we specify the tools you will need to compile mapd. From the pictures you have provided I can see your are using a very old version of gcc. We have specified the steps on how to compile mapd_server exactly on the same page. From the logs I can also see that you have not invoked cmake in build directory. Lastly since the cmake was not fired in correct directory the tools cannot find MapDRelease.h. That file is in the repository.

We also have shell scripts to build all the dependencies that you will need to compile mapd in scripts directory:

A3) Sure. But we don’t guarantee that it will always work.

Feel free to ask more questions here.



A2 -> I’ve read Documentation but I don’t understand it. MapDRelease.h file does not seem to be created even after following installation instructions. Can you explain yourself?

A3 -> It’s okay if it doesn’t always work. I’d like to refer to it.


Attempting to recompile MapD-Core failed due to library installation issue. It was the same when I tried to run the files in the mapd-core/scripts folder.
If i still use cmake, i’m have library problems.
So when I tried to install the library, I had problems installing the library, so I installed some of them. But there are a lot of things that can’t be installed. Is there a solution?



hi @Hyuck,

it looks like you haven’t run the script /usr/local/mapd-deps/ before invoking cmake

the script it’s created by the script used to compile follymthrifht and so on.


In ubuntu is different but I think the result is the same

looking to the centos scripts, at the end I can find this command
cp mapd-deps-$ mapd-deps-$SUFFIX.modulefile $PREFIX

so you can try to run /usr/local/mapd-deps/[some date]/mapd-deps-[some date].sh before use cmake.


Q1. I am recompiling to change the size of the cpu slap or gpu slap in the mapd_server. Can I change them just by using the configuration parameter when running Mapd_server? Then, I don’t think I need a recompile.

Q2. So once I tried the configuration parameter for changing the gpu map size, it was analyzed as nvprof and ended. Why does it end immediately when I don’t even run the query?

image.png1234x289 15.6 KB

Q3. We will try to program java using the ‘nvprof --unified-memory-profiling off mapd_server --config [your storage directory]/mapd.conf’ you taught last time. But this is only possible by pressing Ctrl+C. Do you think Java programming is possible?


1-2) do you mean changing --gpu-buffer-mem-bytes or --CPU-buffer-mem-bytes parameters?

those parameters contain just the max memory (in bytes) the system can allocate for CPU (system) or GPU.

limit the max memory available for GPU to 12GB (13221225472 bytes)

mapd_server  --config /opt/mapd_storage/mapd.conf --gpu-buffer-mem-bytes=13221225472

mapdql> \memory_gpu
MapD Server Detailed GPU Memory Usage:
Maximum Bytes for one page: 512 Bytes
Maximum Bytes for node: **12608 MB**
Memory allocated: **2048 MB**
GPU[0] Slab Information:
   0           0    62500      0 USED    1,29,1,0,
   0       62500    62500      1 USED    1,29,1,1,
   0      125000    62500      2 USED    1,29,1,2,
   0      187500    62500      3 USED    1,29,1,3,
   0      250000    62500      4 USED    1,29,1,4,
   0      312500    62500      5 USED    1,29,1,5,
   0      375000    62500      6 USED    1,29,1,6,
   0      437500    62500      7 USED    1,29,1,7,
   0      500000    62500      8 USED    1,29,1,8,
   0      562500    62500      9 USED    1,29,1,9,
   0      625000    62500     10 USED    1,29,1,10,
   0      687500    62500     11 USED    1,29,1,11,
   0      750000    62500     12 USED    1,29,1,12,
   0      812500    62500     13 USED    1,29,1,13,
   0      875000    62500     14 USED    1,29,1,14,
   0      937500    62500     15 USED    1,29,1,15,
   0     1000000     9433     16 USED    1,29,1,16,
   0     1009433  3184871     46 FREE

As you can see from memory_summary command, the maximum memory available to GPU is 12GB, the Slab 0 is of 2GB

So as I already said you have to recompile to change the Slab’s size.

The system cannot start with --gpu-buffer-mem-bytes set to 5000, because of 5000 bytes arent enough :wink:

  1. I don’t know if it’s possible to profiles without exiting from the program; I’ll check later, but you can use other switches and configure the profiler to save an output file with the -o option; in this way you can have a boinary files that you can open with nvvp graphical command

as example

nvprof --print-gpu-trace --unified-memory-profiling off -o /home/mapd/430_2.nvvp -f /opt/mapd/mapd-ce-4.3.0-20181119-b7f85d00bd-Linux-x86_64-render/bin/mapd_server --config /opt/mapd_storage/mapd.conf

after exiting from program you will be able to open the output file with nvvp program.

It’s easier to understand than the text output and you can get more ifnormations


Last time, we checked which function operates through nvprof.

This time, I’d like to see what functions within the MapD work when I run the query.
I heard that you can do it through a ‘gperf’. What should I do?

I want a file like ‘mapd_server.out’ (?)…


Hi @Hyuck,

profiling an executable with gperf is quite straightforward.

You have to install the package (but i guess everything is already installed on ubuntu), then

export LD_PRELOAD=/usr/lib/
export CPUPROFILE=/tmp/mapd_server.out
then run the mapd_server command as you did with nvprof.

When the server stop you should find a file called mapd_server.out.12345 (the number is the pid of the process), and you willl be ablt to analyze it with google-pprof command.


What I want to do is analyze which MapD functions after executing the query.

I confirmed that the out file was released.


and this is result…Is this right?

However, to use ‘gprof’, the -pg option should be given when compiling, but it seems impossible to automatically create a mapd_server file due to ‘make’.
The ‘google-pprof’ tool you mentioned was able to analyze the .prop file. Is there a way to analyze the out file with this tool?
Or is there another way?

I would appreciate it if you could write down the example command.


@aznable Can I use the ‘-pg’ option to build a ‘mapd_server’ to enable gprof? If so, what do I do?


Hi @Hyuck,

AFAIK the -pg option of gcc should force the creation of a .out file with profiling information everytime you run the executable.

Have you tried to use the -DENABLE_PROFILER option on cmake as @VrajPandya suggested in an earlier post ? I’m not able to profile the mapd_server anyway, because the file is empty an that’s strange; I’ll look at the repository in GitHub to figure what’s the problem


Thank you, @aznable
So how does ‘google-pprof’ apply to ‘mapd_server’?