Implement GPU-as-a-Service with Kueue and NVIDIA MIG
GPU-as-a-Service (GPUaaS) addresses the common problem of expensive, underutilized GPUs in organizations by enabling self-service reservation of GPU slices. The approach uses Red Hat OpenShift with Kueue (a Kubernetes queueing and quota system) and NVIDIA Multi-Instance GPU (MIG) technology. MIG allows a single physical GPU to be partitioned into isolated slices of varying sizes, while Kueue manages resource pools, fair sharing, and quota enforcement via ClusterQueues. A custom OpenShift web console plug-in lets developers book GPU time slots through a calendar UI without writing YAML, generating native Kueue resources under the hood. Once a reservation is made, developers can deploy models from the OpenShift AI model catalog using preconfigured hardware profiles that tie deployments to their reserved MIG slice. This enables long-running inference workloads and batch jobs like fine-tuning to share GPU resources elastically under defined access policies.