It's been a while that I have not looked into building a PC piece by piece and I had to refresh my rusty knowledge. First you'd want to take a look at these: Which GPU(s) to Get for Deep Learning, which suggests for GTX 980 and GTX Titan and Which GPU to use for deep learning? that sugegsts GTX 780TI as a top mid-range. There is also this, which is a good quick guide on building a PC. So, here it boils down to two main goals: take the biggest memory on a cuda-enabled GPU which means NVIDIA. AMD on the other hand offers OpenCL library but I would say you won't find any deep learning library using OpenCL. So stick to NVIDIA, and any recent NVIDIA will do.
Such a system will be single threaded deep learning algorithm, so CPU does not play any roles here. The faster matrix computations the farther you'll get.
NVIDIA offers several categories of GPUs (Geforce, Tesla, Quadro).
Tesla and Quadro are for rendering where a double precision is key. They are usually very expensive (several thousand dollars). For deep learning purposes double precision is not important as further precision digits roles are not significant in overal network performance. So we stick to GTX series.
If you look at Theano-users group, and search for "GTX", you'll see that recent activities (posts since 2013) This boils us down to 10 GTX Titans, 12 GTX 770, 6 GTX 780, and 3 GTX 980. So overall we are left to choose between either Titan or 980. It was so surprising for me that you won't be able to find either Titan or Titan black, while being only a year old cards they're just not there! Also note that when looking in here, the more memory bandwidth GB/s a graphics card has the better for deep learning (you'll see that 980 and Titan X are pretty much the same 336). Also worthwhile to see here. it suggests that a next generation of 780 Ti (i.e. 980) will be a good competitor. Its just that 780 is 3GB but 980 is 4GB. See also the video card benchmark comparing GTX 980, another benchmark, and another.
How about multiple GPUs? In practice it is usually advised against multiple GPUs as the gain will not be as much as going for a newer more capable GPU. In NVIDIA jargon this is called SLI and in AMD it is called Crossfire: link. We focus on SLI. See SLI Performance Scaling - Gaming at 4K Resolution.
So we know about GPU, now let's look at CPU and build the system from there. All together there have been two top notch choices with reasonable prices. Core i7 5820k and Core i7 5930k. The only difference lies in the PCI lanes available, the rest are the same (28 vs 40), see here and here. Look into here for a benchmarking of both: PCIe Lanes - PCIe 8x vs 16x in SLI.So there is not much difference and we stick to 5280 to same some for the GPU.
Now is time for mother board: We go for X99 series which offers plenty for us. There is ASRock, MSI and Gigabyte. Based on price and features, I'd go for MSI.
Memory: DDR3 vs DDR4. I suggest 16GB or 32 GB.
Power is 750+ W. 1000W or 1500W if considering SLI (using multiple GPUs)
liquid-based cooler is also needed.
Finally: Assembly guide. Another deep learning hardware guide.
Very helpful!
ReplyDelete