Importance of PCIe Bandwidth in a Multiple GPU env?


Hi Everyone,
We plan to setup a workstation to perform Machine Learning & MapD activities in a dual GPU environment (Dual Titan X Or 1080Ti).
We actually evaluate several CPU/MB configurations,so the question is the following one: From a MapD Point of View, and based on your experience,is there a significant difference in terms of query/load/transfer performance between PCIe x16/x16 and PCIe 8x/8x?

Thanks in advance !



First general motherhood statement: As an in memory GPU database wherever possible we try to minimize the back and forth over the PCIe bus as we would like data to live on the GPU as much as possible (shout out to great blog here talking about GPU Open Analytics Initiative (GOAI) which is leveraging this even further in the wider community)

With all that said we obviously transfer lots of data over the PCIe bus when we are initially loading the appropriate pieces of the DB to the GPU, and when we need to swap data off in the case of needing to bring new data to the GPU. So we try to have as fast a connection to the GPU is possible. I have not tested recently but I recall using a machine with 8x GPU connection and the transfer speed was basically halved from around 8 to 4 gig per second actual.

So the speed of the PCIe is crucial for a good users experience especially when loading large new datasets to GPU but maybe not as large as your initial thoughts on a steady state DB.

Did you mean a Titan Xp? i would recommend a Titan Xp, As far as Titan X to 1080Ti, unless you really need the extra 1GB of vram on a titan X I would recommend the 1080Ti over an older Titan X.



Hi & thanks for your quick reply!Yes,you were right I meant Titan XP in my initial post.So yes the bandwidth will be really a key factor for the swap/initial load and I guess that having the data stored on a NVME drive (PCIe gen3 x4) will also help to handle those kind of activities.
FYI, we plan to attack datasets which size will be between 10 and 50/70Go max per country,so given the GPU memory size (11/12Go) there will be some Data round trips hereā€¦

By the way,I had a look at an article where a 500Gb CSV dataset has been attacked with 4 Titan X with impressive response times - I quickly computed the ratios between the Dataset size and System Ram & GPU Ram involved,and here are the estimation for a 70Go dataset :

Do you think that this kind of estimation makes sense?

Thanks again !