Skip to content

Introduction

GPU Manager optimizes and orchestrates the management of GPUs and workloads to accelerate AI development and deployment. The manager provides several key capabilities:

  1. Resource Management: GPU Manager enables efficient management of GPU and compute resources across on-premises, cloud, and hybrid environments. It allows for dynamic resource allocation based on workload needs, improving utilization and reducing bottlenecks.
  2. Workload Orchestration: Using Kubernetes-based software, GPU Manager orchestrates containerized AI workloads, including training, inference, and development tasks. It supports various workloads, including KubeFlow, third-party integrations, and typical Kubernetes workloads.
  3. Scheduling and Optimization: Maximizing resource utilization and allowing multiple users to share GPU clusters effectively, GPU Manager offers advanced scheduling features like fair-share scheduling, GPU pooling, and fractional GPU allocation.
  4. Visibility and Control: GPU Manager provides dashboards and analytics for monitoring resource usage, workload performance, and overall system health. It also offers tools for administrators to set quotas, priorities, and access controls.
  5. Integration and Compatibility: Designed to work seemlessly with NVIDIA's AI Infrastructure (including DGX systems), GPU Manager integrates with popular AI tools, frameworks, and Kubernetes variants.
  6. Scalability: GPU Manager is built to handle data-center-scale GPU clusters, supporting enterprises as they scale their AI initiatives.
  7. Cost Optimization: By improving resource utilization and providing better visibility into usage patterns, GPU Manager helps organizations reduce their AI infrastructure costs.
  8. Billing: Usage tracking of infrastructure resources, such as GPUs, allows accurate billing and cross-charging across multiple departments. This efficiently allows unutilized resources from one department to be temporarily used by another and the costs reclaimed.

Overall, the GPU Manager aims to simplify GPU management, improve resource efficiency, and accelerate AI development cycles for organizations of various sizes across multiple industries.