MIG: Multi-Instance GPU

Enabling MIG

MIG can be enabled on all GPUs or on a per GPU basis using the nvidia-smi tool.

To enable on all GPUs

root@sdgx-server:~# nvidia-smi -mig 1

Alternatively, MIG can be enabled on one GPU only

root@sdgx-server:~# nvidia-smi -i 4 -mig 1

Listing Available GPU Instance Profiles

root@sdgx-server:~# nvidia-smi mig -i 4 -lgip
+--------------------------------------------------------------------------+
| GPU instance profiles:                                                   |
| GPU   Name          ID    Instances   Memory     P2P    SM    DEC   ENC  |
|                           Free/Total   GiB              CE    JPEG  OFA  |
|==========================================================================|
|   4  MIG 1g.5gb     19     7/7        4.75       No     14     0     0   |
|                                                          1     0     0   |
+--------------------------------------------------------------------------+
|   4  MIG 2g.10gb    14     3/3        9.75       No     28     1     0   |
|                                                          2     0     0   |
+--------------------------------------------------------------------------+
|   4  MIG 3g.20gb     9     2/2        19.62      No     42     2     0   |
|                                                          3     0     0   |
+--------------------------------------------------------------------------+
|   4  MIG 4g.20gb     5     1/1        19.62      No     56     2     0   |
|                                                          4     0     0   |
+--------------------------------------------------------------------------+
|   4  MIG 7g.40gb     0     1/1        39.50      No     98     5     0   |
|                                                          7     1     1   |
+--------------------------------------------------------------------------+

root@sdgx-server:~# nvidia-smi mig -i 4 -lgipp
GPU  4 Profile ID 19 Placements: {0,1,2,3,4,5,6}:1
GPU  4 Profile ID 14 Placements: {0,2,4}:2
GPU  4 Profile ID  9 Placements: {0,4}:4
GPU  4 Profile ID  5 Placement : {0}:4
GPU  4 Profile ID  0 Placement : {0}:8

Creating GPU Instances

GPU instances are created using the '-cgi' option. the syntax for the command is

root@sdgx-server:~# nvidia-smi mig -i <gpu_id> -cgi profile_id,<profile_id>,...

For example to create 4 GPU instances of the MIG 1g.5gb profile on GPU ID 4

root@sdgx-server:~# nvidia-smi mig -i 4 -cgi 19,19,19,19
Successfully created GPU instance on GPU  4 using profile ID 19
Successfully created GPU instance on GPU  4 using profile ID 19
Successfully created GPU instance on GPU  4 using profile ID 19
Successfully created GPU instance on GPU  4 using profile ID 19

root@sdgx-server:~# nvidia-smi mig -i 4 -lgip +--------------------------------------------------------------------------+ | GPU instance profiles: | | GPU Name ID Instances Memory P2P SM DEC ENC | | Free/Total GiB CE JPEG OFA | |==========================================================================| | 4 MIG 1g.5gb 19 3/7 4.75 No 14 0 0 | | 1 0 0 | +--------------------------------------------------------------------------+ | 4 MIG 2g.10gb 14 1/3 9.75 No 28 1 0 | | 2 0 0 | +--------------------------------------------------------------------------+ | 4 MIG 3g.20gb 9 0/2 19.62 No 42 2 0 | | 3 0 0 | +--------------------------------------------------------------------------+ | 4 MIG 4g.20gb 5 0/1 19.62 No 56 2 0 | | 4 0 0 | +--------------------------------------------------------------------------+ | 4 MIG 7g.40gb 0 0/1 39.50 No 98 5 0 | | 7 1 1 | +--------------------------------------------------------------------------+ root@sdgx-server:~# nvidia-smi mig -i 4 -lgipp GPU 4 Profile ID 19 Placements: {0,1,2,3,4,5,6}:1 GPU 4 Profile ID 14 Placements: {0,2,4}:2 GPU 4 Profile ID 9 Placements: {0,4}:4 GPU 4 Profile ID 5 Placement : {0}:4 GPU 4 Profile ID 0 Placement : {0}:8

root@sdgx-server:~# nvidia-smi mig -i 5 -cgi 19,19,19,19 Successfully created GPU instance on GPU 5 using profile ID 19 Successfully created GPU instance on GPU 5 using profile ID 19 Successfully created GPU instance on GPU 5 using profile ID 19 Successfully created GPU instance on GPU 5 using profile ID 19

root@sdgx-server:~# nvidia-smi mig -lgi +----------------------------------------------------+ | GPU instances: | | GPU Name Profile Instance Placement | | ID ID Start:Size | |====================================================| | 4 MIG 1g.5gb 19 7 4:1 | +----------------------------------------------------+ | 4 MIG 1g.5gb 19 8 5:1 | +----------------------------------------------------+ | 4 MIG 1g.5gb 19 9 6:1 | +----------------------------------------------------+ | 4 MIG 1g.5gb 19 11 0:1 | +----------------------------------------------------+ | 5 MIG 1g.5gb 19 7 0:1 | +----------------------------------------------------+ | 5 MIG 1g.5gb 19 11 4:1 | +----------------------------------------------------+ | 5 MIG 1g.5gb 19 12 5:1 | +----------------------------------------------------+ | 5 MIG 1g.5gb 19 13 6:1 | +----------------------------------------------------+ | 6 MIG 1g.5gb 19 7 0:1 | +----------------------------------------------------+ | 6 MIG 1g.5gb 19 11 4:1 | +----------------------------------------------------+ | 6 MIG 1g.5gb 19 12 5:1 | +----------------------------------------------------+ | 6 MIG 1g.5gb 19 13 6:1 | +----------------------------------------------------+ | 7 MIG 1g.5gb 19 7 4:1 | +----------------------------------------------------+ | 7 MIG 1g.5gb 19 8 5:1 | +----------------------------------------------------+ | 7 MIG 1g.5gb 19 9 6:1 | +----------------------------------------------------+ | 7 MIG 1g.5gb 19 11 0:1 | +----------------------------------------------------+ root@sdgx-server:~# nvidia-smi mig -lci