nvidia-smi does not display memory usage

Question

I want to use nvidia-smi to monitor my GPU for my machine-learning/ AI projects. However, when I run nvidia-smi in my cmd, git bash or powershell, I get the following results:

$ nvidia-smi
Sun May 28 13:25:46 2017
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 376.53                 Driver Version: 376.53                    |
|-------------------------------+----------------------+----------------------+
| GPU  Name            TCC/WDDM | Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  GeForce GTX 1070   WDDM  | 0000:28:00.0      On |                  N/A |
|  0%   49C    P2    36W / 166W |   7240MiB /  8192MiB |      4%      Default |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID  Type  Process name                               Usage      |
|=============================================================================|
|    0      7676  C+G   ...ost_cw5n1h2txyewy\ShellExperienceHost.exe N/A      |
|    0      8580  C+G   Insufficient Permissions                     N/A      |
|    0      9704  C+G   ...x86)\Google\Chrome\Application\chrome.exe N/A      |
|    0     10532    C   ...\Anaconda3\envs\tensorflow-gpu\python.exe N/A      |
|    0     11384  C+G   Insufficient Permissions                     N/A      |
|    0     12896  C+G   C:\Windows\explorer.exe                      N/A      |
|    0     13868  C+G   Insufficient Permissions                     N/A      |
|    0     14068  C+G   Insufficient Permissions                     N/A      |
|    0     14568  C+G   Insufficient Permissions                     N/A      |
|    0     15260  C+G   ...osoftEdge_8wekyb3d8bbwe\MicrosoftEdge.exe N/A      |
|    0     16912  C+G   ...am Files (x86)\Dropbox\Client\Dropbox.exe N/A      |
|    0     18196  C+G   ...I\AppData\Local\hyper\app-1.3.3\Hyper.exe N/A      |
|    0     18228  C+G   ...oftEdge_8wekyb3d8bbwe\MicrosoftEdgeCP.exe N/A      |
|    0     20032  C+G   ...indows.Cortana_cw5n1h2txyewy\SearchUI.exe N/A      |
+-----------------------------------------------------------------------------+

The column GPU Memory Usage shows N/A for every single process. Also, there are a lot more processes listed than I found for most examples on the Internet. What could be the reason for this?

I am running a Nvidia GTX 1070 by ASUS, my OS is Windows 10 Pro.

Solution on Windows: Run Sysinternals Process Explorer as Administrator, then switch on columns "GPU Dedicated" and "GPU Committed" to see per-process GPU memory usage. — Contango, Jan 11 '21 at 09:55

score 20 · Accepted Answer · edited Jun 02 '18 at 20:19

20

If you perform the following : nvidia-smi -q you will see the following:

Processes
        Process ID                  : 6564
            Type                    : C+G
            Name                    : C:\Windows\explorer.exe
            Used GPU Memory         : Not available in WDDM driver model

Not available in WDDM driver model => WDDM stand for Windows Display Driver Model. You can switch to TCC and obtain the information with the command: nvidia-smi -dm 1, however this operation can only performed if you do not have any display attached to the GPU. So... It's mostly impossible...

By the way, don't worry about a high memory usage, Tensorflow reserve as much GPU memory as it can to speed up its processes. If you prefer a finer grained control on the memory taken use the following (it may slow down a little bit your computations):

config = tf.ConfigProto()
config.gpu_options.allow_growth = True
sess = tf.Session(config=config)

You can create a dual boot on Ubuntu or just forget about this.

edited Jun 02 '18 at 20:19

CodesInChaos

106,488
23
218
262

answered May 28 '17 at 14:22

Jonathan DEKHTIAR

3,456
1
21
42

Thank you so much for the answer, that explains a lot! – Paul Fauth-Mayer May 28 '17 at 14:57
Exactly the answer I was looking for. – Tyson Hilmer Apr 18 '18 at 08:58
Solution on Windows: Run Sysinternals Process Explorer as Administrator, then switch on columns "GPU Dedicated" and "GPU Committed" to see per-process GPU memory usage. – Contango Jan 11 '21 at 09:52
If windows disable Nvidia compute card, could we run cuda program in terminal? Or is there a way to make it? Thanks! – roachsinai Aug 16 '21 at 01:41

nvidia-smi does not display memory usage

1 Answers1

Linked