Difference between revisions of "MIG: Multi-Instance GPU"
(Created page with "== Enabling MIG == MIG can be enabled on all GPUs or on a per GPU basis using the nvidia-smi tool. To enable on all GPUs <syntaxhighlight> root@sdgx-server:~# nvidia-smi -mig 1 </bash...") |
|||
| Line 17: | Line 17: | ||
<syntaxhighlight> | <syntaxhighlight> | ||
| − | |||
root@sdgx-server:~# nvidia-smi mig -i 4 -lgip | root@sdgx-server:~# nvidia-smi mig -i 4 -lgip | ||
+--------------------------------------------------------------------------+ | +--------------------------------------------------------------------------+ | ||
| Line 24: | Line 23: | ||
| Free/Total GiB CE JPEG OFA | | | Free/Total GiB CE JPEG OFA | | ||
|==========================================================================| | |==========================================================================| | ||
| − | | 4 MIG 1g.5gb 19 | + | | 4 MIG 1g.5gb 19 7/7 4.75 No 14 0 0 | |
| 1 0 0 | | | 1 0 0 | | ||
+--------------------------------------------------------------------------+ | +--------------------------------------------------------------------------+ | ||
| − | | 4 MIG 2g.10gb 14 | + | | 4 MIG 2g.10gb 14 3/3 9.75 No 28 1 0 | |
| 2 0 0 | | | 2 0 0 | | ||
+--------------------------------------------------------------------------+ | +--------------------------------------------------------------------------+ | ||
| − | | 4 MIG 3g.20gb 9 | + | | 4 MIG 3g.20gb 9 2/2 19.62 No 42 2 0 | |
| 3 0 0 | | | 3 0 0 | | ||
+--------------------------------------------------------------------------+ | +--------------------------------------------------------------------------+ | ||
| − | | 4 MIG 4g.20gb 5 | + | | 4 MIG 4g.20gb 5 1/1 19.62 No 56 2 0 | |
| 4 0 0 | | | 4 0 0 | | ||
+--------------------------------------------------------------------------+ | +--------------------------------------------------------------------------+ | ||
| − | | 4 MIG 7g.40gb 0 | + | | 4 MIG 7g.40gb 0 1/1 39.50 No 98 5 0 | |
| 7 1 1 | | | 7 1 1 | | ||
+--------------------------------------------------------------------------+ | +--------------------------------------------------------------------------+ | ||
| Line 46: | Line 45: | ||
GPU 4 Profile ID 5 Placement : {0}:4 | GPU 4 Profile ID 5 Placement : {0}:4 | ||
GPU 4 Profile ID 0 Placement : {0}:8 | GPU 4 Profile ID 0 Placement : {0}:8 | ||
| + | </syntaxhighlight> | ||
| + | |||
| + | == Creating GPU Instances == | ||
| + | |||
| + | GPU instances are created using the '-cgi' option. the syntax for the command is | ||
| + | <syntaxhighlight> | ||
| + | root@sdgx-server:~# nvidia-smi mig -i <gpu_id> -cgi profile_id,<profile_id>,... | ||
| + | </syntaxhighlight> | ||
| + | |||
| + | For example to create 4 GPU instances of the MIG 1g.5gb profile on GPU ID 4 | ||
| + | <syntaxhighlight> | ||
| + | root@sdgx-server:~# nvidia-smi mig -i 4 -cgi 19,19,19,19 | ||
| + | Successfully created GPU instance on GPU 4 using profile ID 19 | ||
| + | Successfully created GPU instance on GPU 4 using profile ID 19 | ||
| + | Successfully created GPU instance on GPU 4 using profile ID 19 | ||
| + | Successfully created GPU instance on GPU 4 using profile ID 19 | ||
</syntaxhighlight> | </syntaxhighlight> | ||
Latest revision as of 16:41, 3 November 2020
Enabling MIG
MIG can be enabled on all GPUs or on a per GPU basis using the nvidia-smi tool.
To enable on all GPUs
root@sdgx-server:~# nvidia-smi -mig 1Alternatively, MIG can be enabled on one GPU only
root@sdgx-server:~# nvidia-smi -i 4 -mig 1Listing Available GPU Instance Profiles
root@sdgx-server:~# nvidia-smi mig -i 4 -lgip
+--------------------------------------------------------------------------+
| GPU instance profiles: |
| GPU Name ID Instances Memory P2P SM DEC ENC |
| Free/Total GiB CE JPEG OFA |
|==========================================================================|
| 4 MIG 1g.5gb 19 7/7 4.75 No 14 0 0 |
| 1 0 0 |
+--------------------------------------------------------------------------+
| 4 MIG 2g.10gb 14 3/3 9.75 No 28 1 0 |
| 2 0 0 |
+--------------------------------------------------------------------------+
| 4 MIG 3g.20gb 9 2/2 19.62 No 42 2 0 |
| 3 0 0 |
+--------------------------------------------------------------------------+
| 4 MIG 4g.20gb 5 1/1 19.62 No 56 2 0 |
| 4 0 0 |
+--------------------------------------------------------------------------+
| 4 MIG 7g.40gb 0 1/1 39.50 No 98 5 0 |
| 7 1 1 |
+--------------------------------------------------------------------------+
root@sdgx-server:~# nvidia-smi mig -i 4 -lgipp
GPU 4 Profile ID 19 Placements: {0,1,2,3,4,5,6}:1
GPU 4 Profile ID 14 Placements: {0,2,4}:2
GPU 4 Profile ID 9 Placements: {0,4}:4
GPU 4 Profile ID 5 Placement : {0}:4
GPU 4 Profile ID 0 Placement : {0}:8Creating GPU Instances
GPU instances are created using the '-cgi' option. the syntax for the command is
root@sdgx-server:~# nvidia-smi mig -i <gpu_id> -cgi profile_id,<profile_id>,...For example to create 4 GPU instances of the MIG 1g.5gb profile on GPU ID 4
root@sdgx-server:~# nvidia-smi mig -i 4 -cgi 19,19,19,19
Successfully created GPU instance on GPU 4 using profile ID 19
Successfully created GPU instance on GPU 4 using profile ID 19
Successfully created GPU instance on GPU 4 using profile ID 19
Successfully created GPU instance on GPU 4 using profile ID 19root@sdgx-server:~# nvidia-smi mig -i 4 -lgip +--------------------------------------------------------------------------+ | GPU instance profiles: | | GPU Name ID Instances Memory P2P SM DEC ENC | | Free/Total GiB CE JPEG OFA | |==========================================================================| | 4 MIG 1g.5gb 19 3/7 4.75 No 14 0 0 | | 1 0 0 | +--------------------------------------------------------------------------+ | 4 MIG 2g.10gb 14 1/3 9.75 No 28 1 0 | | 2 0 0 | +--------------------------------------------------------------------------+ | 4 MIG 3g.20gb 9 0/2 19.62 No 42 2 0 | | 3 0 0 | +--------------------------------------------------------------------------+ | 4 MIG 4g.20gb 5 0/1 19.62 No 56 2 0 | | 4 0 0 | +--------------------------------------------------------------------------+ | 4 MIG 7g.40gb 0 0/1 39.50 No 98 5 0 | | 7 1 1 | +--------------------------------------------------------------------------+ root@sdgx-server:~# nvidia-smi mig -i 4 -lgipp GPU 4 Profile ID 19 Placements: {0,1,2,3,4,5,6}:1 GPU 4 Profile ID 14 Placements: {0,2,4}:2 GPU 4 Profile ID 9 Placements: {0,4}:4 GPU 4 Profile ID 5 Placement : {0}:4 GPU 4 Profile ID 0 Placement : {0}:8
root@sdgx-server:~# nvidia-smi mig -i 5 -cgi 19,19,19,19 Successfully created GPU instance on GPU 5 using profile ID 19 Successfully created GPU instance on GPU 5 using profile ID 19 Successfully created GPU instance on GPU 5 using profile ID 19 Successfully created GPU instance on GPU 5 using profile ID 19
root@sdgx-server:~# nvidia-smi mig -lgi +----------------------------------------------------+ | GPU instances: | | GPU Name Profile Instance Placement | | ID ID Start:Size | |====================================================| | 4 MIG 1g.5gb 19 7 4:1 | +----------------------------------------------------+ | 4 MIG 1g.5gb 19 8 5:1 | +----------------------------------------------------+ | 4 MIG 1g.5gb 19 9 6:1 | +----------------------------------------------------+ | 4 MIG 1g.5gb 19 11 0:1 | +----------------------------------------------------+ | 5 MIG 1g.5gb 19 7 0:1 | +----------------------------------------------------+ | 5 MIG 1g.5gb 19 11 4:1 | +----------------------------------------------------+ | 5 MIG 1g.5gb 19 12 5:1 | +----------------------------------------------------+ | 5 MIG 1g.5gb 19 13 6:1 | +----------------------------------------------------+ | 6 MIG 1g.5gb 19 7 0:1 | +----------------------------------------------------+ | 6 MIG 1g.5gb 19 11 4:1 | +----------------------------------------------------+ | 6 MIG 1g.5gb 19 12 5:1 | +----------------------------------------------------+ | 6 MIG 1g.5gb 19 13 6:1 | +----------------------------------------------------+ | 7 MIG 1g.5gb 19 7 4:1 | +----------------------------------------------------+ | 7 MIG 1g.5gb 19 8 5:1 | +----------------------------------------------------+ | 7 MIG 1g.5gb 19 9 6:1 | +----------------------------------------------------+ | 7 MIG 1g.5gb 19 11 0:1 | +----------------------------------------------------+ root@sdgx-server:~# nvidia-smi mig -lci