2

I am running Ubuntu 16.04 on an EC2 p2.xlarge shared instance. enter image description here

The P2.xlarge instance gives access to a single GPU (1/2 of an NVidia K80 GPU?) to my shared instance.

How is this GPU shared between other Amazon EC2 instances on the same physical machine?

I was under the impression that 100% of the GPU was allocated to my instance. But this is clearly not the case. When my instance is running nothing on the GPU:

$ nvidia-smi
Tue Feb 21 00:11:16 2017       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 375.39                 Driver Version: 375.39                    |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  Tesla K80           Off  | 0000:00:1E.0     Off |                    0 |
| N/A   39C    P0    55W / 149W |      0MiB / 11439MiB |     63%      Default |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID  Type  Process name                               Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+

A moment later:

|   0  Tesla K80           Off  | 0000:00:1E.0     Off |                    0 |
| N/A   40C    P0    57W / 149W |      0MiB / 11439MiB |     97%      Default |

And the next moment:

|   0  Tesla K80           Off  | 0000:00:1E.0     Off |                    0 |
| N/A   40C    P0    56W / 149W |      0MiB / 11439MiB |    100%      Default |

And pretty much stays there...

|   0  Tesla K80           Off  | 0000:00:1E.0     Off |                    0 |
| N/A   41C    P0    56W / 149W |      0MiB / 11439MiB |    100%      Default |


|   0  Tesla K80           Off  | 0000:00:1E.0     Off |                    0 |
| N/A   41C    P0    56W / 149W |      0MiB / 11439MiB |     99%      Default |

What are the rules for GPU allocation for GPU instances?

talonmies
  • 70,661
  • 34
  • 192
  • 269
B Faust
  • 234
  • 1
  • 6
  • 1
    You're not sharing. That may be a deceptive indication. See [nvidia-smi Volatile GPU-Utilization explanation?](http://stackoverflow.com/a/40938696/1695906) If you want new hardware, stopping and starting your insrance will move it to a new machine (hardware is deallocated when instances are stopped and reallocated on the next start, even though the instance-id remains unchanged -- that's why they aren't billed when stopped). – Michael - sqlbot Feb 21 '17 at 01:44
  • 1
    Thank you for taking the time to share. Yes, stopping and then starting *may* assign the instance to different hardware. But if you create a shared instance, it is a fraction of the price of running on dedicated hardware, because it runs along with other instances on the same physical machine. (There is a choice when you start an instance in the EC2 Dashboard.) Maybe a dedicated CPU is assigned? A dedicated GPU? But `nvidia-smi` seems to indicate otherwise. I've been using it for years locally and it reports 0-2% when not in use. It does the same on g2 instances. Is it accurate on p2? – B Faust Feb 21 '17 at 04:12
  • Actually, that's not what dedicated vs. shared means. The two are technically identical, but with a dedicated instance, you are the first tenant on the machine -- but if *you* create a second instance -- dedicated or shared -- it may be the second tenant on the machine as well as the first: you're sharing it with yourself. ["Dedicated instances may share hardware with other instances from the same AWS account that are not Dedicated instances."](https://aws.amazon.com/ec2/purchasing-options/dedicated-instances/) This is primarily for regulatory and licensing reasons, not technical ones. – Michael - sqlbot Feb 21 '17 at 10:19
  • @Michael-sqlbot Ah, yes. I see where you're coming from.. You're thinking "dedicated instance," and that's a good point you make about _dedicated instances_, while I'm thinking "dedicated host", where you really do get the hardware to yourself and thus, where the real expense lies. – B Faust Feb 22 '17 at 00:38
  • The description of dedicated hosts: https://aws.amazon.com/ec2/dedicated-hosts/ – B Faust Feb 22 '17 at 00:45
  • Right, but still fundamentally similar: from the link you posted, *"You have the option to launch instances onto a specific Dedicated Host, or you can let Amazon EC2 place the instances automatically."* If you have only one dedicated instance, it is *still* going to be launched on a physical host system that has only your instance as a guest. If you have more than one dedicated instance, they may share a singe host system with each other. The only material difference is that with dedicated hosts, you can control which specific host a given instance resides on. – Michael - sqlbot Feb 22 '17 at 00:54

0 Answers0