2

According the following table (from this paper), numpy's np.dot performance is comparable to a CUDA implementation of matrix multiplication, in experiments with 320x320 matrices. And I did replicate this Speedup in my machine for np.dot with enough precision. Their code for CUDA with Numba ran much slower though, with a Speedup of about 1200 instead of the 49258 reported.

Why is numpy's implementation so fast?

https://link.springer.com/article/10.1007/s11227-017-2213-5

Edit: here's the code taken from the paper. I just added the timeit calls. I ran it in the following laptop.

enter image description here

bwdm
  • 690
  • 7
  • 15
  • 2
    I'm voting to close this question as off-topic because there is no particular code problem, with example code, to solve – mtrw Oct 16 '19 at 00:39
  • 4
    So.... many... variables here.. What code was used? In what machine? What are the specifications? Does the person that wrote the code know what they're doing (i.e. are they fluent in python/numpy/CUDA) ? What is the test objectively? It is so easy to fall in the trap of running and measuring _an implementation that you built yourself_ and then generalizing the results as a behavior for the whole programming language itself.. – rafaelc Oct 16 '19 at 01:26
  • Ok, I added the code, system info and changed the title slightly. That generalization was unintentional. – bwdm Oct 16 '19 at 10:47
  • Possible duplicate of [How does BLAS get such extreme performance?](https://stackoverflow.com/questions/1303182/how-does-blas-get-such-extreme-performance) – norok2 Oct 16 '19 at 12:10
  • I don't think this can be a duplicate of that question because the subjects are different, unless you happen to know how implementation details of Numpy. But a valid answer could state that Numpy uses BLAS and then point to that question. – bwdm Oct 16 '19 at 13:29

2 Answers2

2

Aside from what @norok2 links to, there is the large overhead of transferring the data to the GPU. This becomes significant in several cases:

  • it is comparably expensive to what you do on the GPU when compared to data transfer overhead, i.e. you only do one operation on less than a MB of data.
  • The size of your problem doesn't scale extremely well. This is the case if your data size or your underlying problem don't allow the GPU to use its parallel processing sufficiently.
  • There are too many branches in your parallel code. This usually means a large set of parallel processors needs to wait on each branch (branching hardware is usually grouped per X number of arithmetic processors on a GPU), slowing down the whole computation.

Both points apply here. 320x320 is not extremely large, and a multiplication is the only thing you're doing. CPUs aren't obsoleted by GPUs by far, and let this type of thing prove exactly that.

rubenvb
  • 72,003
  • 32
  • 177
  • 319
0

NumPy is so fast because it uses a highly optimized BLAS library which is likely to be using the SIMD instructions of your CPU.

norok2
  • 21,682
  • 3
  • 59
  • 88