How do I use TensorFlow GPU?

Question

How do I use TensorFlow GPU version instead of CPU version in Python 3.6 x64?

import tensorflow as tf

Python is using my CPU for calculations.
I can notice it because I have an error:

Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2

I have installed tensorflow and tensorflow-gpu.

How do I switch to GPU version?

Have you tried uninstalling `tensorflow` and just keep the `tensorflow-gpu` installed? — Jorge Leitao, Jul 12 '18 at 13:27
That's just a warning, if you have an NVIDIA GPU, Tensorflow-gpu will automatically use that. To know more and how to disable the warning: https://stackoverflow.com/a/47227886/4892874 To check that you're using the GPU: `sess = tf.Session(config=tf.ConfigProto(log_device_placement=True))` — iacolippo, Jul 12 '18 at 13:29
Are you using Anaconda ? if you are you have to switch the interpreter of the particular environment in PyCharm. — Mohan Radhakrishnan, Jul 12 '18 at 13:30
NVIDIA 940mx, it is relative new Nvidia card. @JorgeLeitão yes, then I have no TensorFlow — Guruku, Jul 12 '18 at 13:36

score 68 · Accepted Answer · edited Mar 20 '21 at 11:08

68

Follow this tutorial Tensorflow GPU I did it and it works perfect.

Attention! - install version 9.0! newer version is not supported by Tensorflow-gpu

Steps:

Uninstall your old tensorflow
Install tensorflow-gpu pip install tensorflow-gpu
Install Nvidia Graphics Card & Drivers (you probably already have)
Download & Install CUDA
Download & Install cuDNN
Verify by simple program

from tensorflow.python.client import device_lib 
print(device_lib.list_local_devices())

edited Mar 20 '21 at 11:08

Ynjxsjmh

16,448
3
17
42

answered Jul 12 '18 at 13:48

Ashwel

1,094
9
16

So, CUDA 9.2 can be the problem? Should I install 9.0 then? Yesterday I had an error after installing CUDA 9.2. It said that I need DLL from CUDA 9.0 and I knew it was using GPU version, today all the time I see the error about CPU, so it is using CPU. I will try it step by step, thanks. – Guruku Jul 12 '18 at 13:52
Yea, i had same problem about 2 month ago. I tried CUDA 9.1 and Tensorflow doesn't support it..Then i uninstall everything and repeat all steps with CUDA 9.0 and now it works without problems. – Ashwel Jul 12 '18 at 14:06
Maybe I should ask in another question, but is tensorflow on GPU is faster than CPU at your PC? I tried with linear regression model and for 10000 epos and step 0.01 my CPU can calculate it in 13 second and when I use GPU it takes 39 second. – Guruku Jul 12 '18 at 17:13
Ofc i have CPU i7 8700k and GPU nVidia GTX 1060 STRING 6GB and on GPU is faster 20 times aproximate. – Ashwel Jul 13 '18 at 06:25
3

Note that newer versions of tensorflow combine the cpu and gpu packages together into a single package. https://www.tensorflow.org/install/gpu – Paul Bendevis Jul 27 '20 at 12:44
I did the same, and does tensors need to be uploaded on gpu, like they need in pytorch? – dato nefaridze Aug 01 '20 at 09:45

kkica · Answer 2 · 2021-08-10T08:34:08.207

8

The 'new' way to install tensorflow GPU if you have Nvidia, is with Anaconda. Works on Windows too. With 1 line.

conda create --name tf_gpu tensorflow-gpu

This is a shortcut for 3 commands, which you can execute separately if you want or if you already have a conda environment and do not need to create one.

Create an anaconda environment conda create --name tf_gpu
Activate the environment conda activate tf_gpu
Install tensorflow-GPU conda install tensorflow-gpu

You can use the conda environment.

edited Aug 10 '21 at 08:34

answered Jul 31 '19 at 00:07

kkica

3,964
1
19
40

1

In 2: ``conda activate tf_gpu`` – Rodrigo Hjort Aug 09 '21 at 18:44

Paul Bendevis · Answer 3 · 2020-07-27T13:38:57.917

7

Follow the steps in the latest version of the documentation. Note: GPU and CPU functionality is now combined in a single tensorflow package

pip install tensorflow

# OLDER VERSIONS pip install tensorflow-gpu

https://www.tensorflow.org/install/gpu

This is a great guide for installing drivers and CUDA if needed: https://www.quantstart.com/articles/installing-tensorflow-22-on-ubuntu-1804-with-an-nvidia-gpu/

edited Jul 27 '20 at 13:38

answered Jul 27 '20 at 12:42

Paul Bendevis

1,806
2
26
38

Hazarapet Tunanyan · Answer 4 · 2019-05-07T15:17:38.697

6

First you need to install tensorflow-gpu, because this package is responsible for gpu computations. Also remember to run your code with environment variable CUDA_VISIBLE_DEVICES = 0 (or if you have multiple gpus, put their indices with comma). There might be some issues related to using gpu. if your tensorflow does not use gpu anyway, try this

edited May 07 '19 at 15:17

answered Mar 22 '19 at 18:20

Hazarapet Tunanyan

2,691
25
27

mrk · Answer 5 · 2019-02-08T18:57:03.883

I tried following the above tutorial. Thing is tensorflow changes a lot and so do the NVIDIA versions needed for running on a GPU. The next issue is that your driver version determines your toolkit version etc. As of today this information about the software requirements should shed some light on how they interplay:

NVIDIA® GPU drivers —CUDA 9.0 requires 384.x or higher.
CUDA® Toolkit —TensorFlow supports CUDA 9.0.
CUPTI ships with the CUDA Toolkit.
cuDNN SDK (>= 7.2) Note: Make sure your GPU has compute compatibility >3.0
(Optional) NCCL 2.2 for multiple GPU support.
(Optional) TensorRT 4.0 to improve latency and throughput for inference on some models.

And here you'll find the up-to-date requirements stated by tensorflow (which will hopefully be updated by them on a regular basis).

score 2 · Answer 6 · answered Mar 30 '22 at 14:22

For conda environment.

conda search tensorflow to search the available versions of tensorflow. The ones that have mkl are optimized for CPU. You can choose the ones with gpu.
Then check the version of your cuda using nvcc --version and find the proper version of tensorflow in this page, according to your version of cuda.
For example, for cuda/10.1,and python3.8, you can use
- conda install tensorflow=2.2.0=gpu_py38hb782248_0

score 1 · Answer 7 · answered Mar 28 '19 at 07:57

Strangely, even though the tensorflow website 1 mentions that CUDA 10.1 is compatible with tensorflow-gpu-1.13.1, it doesn't work so far. tensorflow-gpu gets installed properly though but it throws out weird errors when running.

So far, the best configuration to run tensorflow with GPU is CUDA 9.0 with tensorflow_gpu-1.12.0 under python3.6.

Following this configuration with the steps mentioned in https://stackoverflow.com/a/51307381/2562870 (the answer above), worked for me :)

score 1 · Answer 8 · edited Jan 30 '20 at 15:25

Uninstall tensorflow and install only tensorflow-gpu; this should be sufficient. By default, this should run on the GPU and not the CPU. However, further you can do the following to specify which GPU you want it to run on.

If you have an nvidia GPU, find out your GPU id using the command nvidia-smi on the terminal. After that, add these lines in your script:

os.environ["CUDA_DEVICE_ORDER"] = "PCI_BUS_ID"
os.environ["CUDA_VISIBLE_DEVICES"] = #GPU_ID from earlier

config = tf.ConfigProto()
sess = tf.Session(config=config)

For the functions where you wish to use GPUs, write something like the following:

with tf.device(tf.DeviceSpec(device_type="GPU", device_index=gpu_id)):

How do I use TensorFlow GPU?

8 Answers8

Linked

Related