Bridging Cloud and HPC for AI: Intelligent Workload Placement and Orchestration
November 23 – November 24 CET
From Kubernetes and Slurm to portable, resource-aware AI execution across Cloud and High-Performance Computing infrastructures
AGENDA
1. Cloud and HPC in Context
Introduction to Cloud Computing and High-Performance Computing (HPC), their main characteristics, typical use cases, benefits, and limitations.
2. Technology Stacks and Execution Models
Overview of the main technologies used in Cloud and HPC environments, including Kubernetes, containers, OpenStack, Slurm, Apptainer, Message Passing Interface (MPI), and the differences between Cloud-native and HPC execution models.
3. AI Workloads Across Cloud and HPC
How AI training, inference, preprocessing, and distributed workloads can benefit from Cloud and HPC infrastructures, and the factors that influence workload placement decisions.
4. Cloud and HPC Infrastructure Used in Our Work
A focused overview of the OpenStack Cloud, Kubernetes, and MareNostrum 5 (MN5) HPC environments used in our work, highlighting the infrastructure characteristics that directly affect workload placement and execution, including CPU/GPU resources, memory, storage, networking, resource allocation, Kubernetes, and Slurm.
5. The Cloud–HPC Integration Challenge
Why moving or adapting workloads between Cloud and HPC environments is not straightforward, including differences in scheduling, resource semantics, storage, networking, containers, and execution models.
6. Existing Approaches and the Need for Smarter Orchestration
A brief overview of current Cloud–HPC and hybrid orchestration approaches, what they already solve, and the remaining challenges related to workload understanding, semantic mapping, feasibility, placement, and verification.
7. Experimental Validation, Lessons Learned and Future Directions
Presentation of the Kubernetes–MareNostrum 5 experimental workflow, reference AI workloads, execution strategies, measured results, key lessons, current limitations, future developments, and final conclusions.
8. MINERVA Support for Cloud–HPC Integration in Industry
A closing perspective on how MINERVA can support companies in adopting Cloud–HPC workflows best practices, and pathways
—————————————————————————————————————–
Day 1 — Cloud, HPC and the Integration Challenge
@10:30 – 12:00
45 min presentation — Agenda Points 1–5
Q&A and Discussion — 20 min
Day 2 — Orchestration and Experimental Validation
@10:30 – 12:00
45 min presentation — Agenda Points 5–8
Q&A and Discussion — 20 min
