Posted 1 month ago
HPC Engineer
Sunnyvale, CARemoteFull-time
AI Summary
Provides operational coverage for overnight hours as a primary point of contact, monitoring and maintaining large-scale GPU clusters, performing incident triage, supporting researchers, and executing runbooks.
About this role
About MBZUAI
The Institute for Foundation Models (IFM) operates some of the world's largest AI supercomputing environments.
Position Summary
This role provides operational coverage during Abu Dhabi overnight hours and serves as a primary point of contact for infrastructure monitoring, incident triage, researcher support, and production operations.
The Institute for Foundation Models (IFM) operates some of the world's largest AI supercomputing environments.
Position Summary
This role provides operational coverage during Abu Dhabi overnight hours and serves as a primary point of contact for infrastructure monitoring, incident triage, researcher support, and production operations.
Responsibilities
• Monitor health, performance, and availability of large-scale GPU clusters.
• Respond to incidents and perform first-level triage.
• Support researchers and troubleshoot job failures.
• Execute operational runbooks and recovery procedures.
• Validate cluster deployments, upgrades, and maintenance activities.
• Track infrastructure utilization and operational metrics.
• Develop automation and monitoring tools.
• Contribute to documentation and reporting.
• Respond to incidents and perform first-level triage.
• Support researchers and troubleshoot job failures.
• Execute operational runbooks and recovery procedures.
• Validate cluster deployments, upgrades, and maintenance activities.
• Track infrastructure utilization and operational metrics.
• Develop automation and monitoring tools.
• Contribute to documentation and reporting.
Education
Bachelor's degree in Computer Science, Computer Engineering, Software Engineering, Information Technology, Electrical Engineering, Mathematics, Physics, or related disciplines.
Experience
• 2+ years in Linux systems administration, SRE, DevOps, cloud operations, HPC, or infrastructure operations.
• Strong Linux troubleshooting skills.
• Experience with scripting using Python or Bash.
• Strong Linux troubleshooting skills.
• Experience with scripting using Python or Bash.
Preferred Qualifications
• Slurm.
• GPU infrastructure.
• AWS, Azure, or GCP.
• Grafana, Prometheus, Datadog, or similar tools.
• Containers and Kubernetes.
• AI/ML infrastructure exposure.
• Research computing environments.
• GPU infrastructure.
• AWS, Azure, or GCP.
• Grafana, Prometheus, Datadog, or similar tools.
• Containers and Kubernetes.
• AI/ML infrastructure exposure.
• Research computing environments.
Skills
AWSAzureBashDataDogGCPGPUGrafanaHPCKubernetesLinuxPrometheusPythonSlurmSRE
Explore related jobs
More jobs at Institute of Foundation Models
- Inference Optimization Intern – Performance ModelingSunnyvale, CA
- Eval360 - Error Analysis EngineerSunnyvale, CA
- AI Research Internship - WMSunnyvale, CA
- Research Scientist, Agentic Data & BenchmarkingSunnyvale, CA
- Research Scientist - Vision Language ModelSunnyvale, CA
- Finance Operations SpecialistSunnyvale, CA
