Staff Software Engineer (Cloud Infrastructure)

You will diagnose, maintain, and repair high-performance GPU compute clusters. You will automate hardware fault diagnosis, develop software for NVIDIA and AMD GPU platforms, perform component repairs and validation testing, manage firmware upgrades, document maintenance, create troubleshooting procedures, and investigate systemic failures.

Responsibilities

• Automate diagnosis and troubleshooting of hardware faults within GPU racks
• Develop software to troubleshoot and support GPU platforms
• Execute component-level diagnosis and remediation for failed or degraded hardware
• Perform field-replaceable unit repairs for GPUs, power supplies, cooling systems, interconnects, and networking hardware
• Conduct post-repair validation, burn-in, Torch, and NVIDIA NCCL testing
• Implement preventative maintenance procedures
• Perform firmware and BIOS upgrades across the GPU fleet
• Document maintenance activities, failures, and resolutions
• Develop and update standard operating procedures for troubleshooting, repair, and validation
• Identify root causes of systemic failures and implement preventative solutions
• Participate in a rotating infrastructure on-call schedule

Requirements

• Ability to code in Golang
• Experience diagnosing and repairing high-density rack-mounted compute hardware in production environments
• Deep understanding of GPU architectures and hands-on experience with GPU-based systems
• Experience supporting NVIDIA A100, H200, GB200, B200, and AMD 350X and 355X platforms
• Familiarity with InfiniBand, NVLink, and RoCE
• Strong Linux experience with Ubuntu, Rocky Linux, and CentOS
• Proficiency with NVIDIA DCGM and NVIDIA field diagnostic utilities
• Experience with enterprise server hardware, power delivery, and cooling systems
• Experience working with hardware vendors and escalations
• Background in large-scale GPU fleet operations or hyperscale data center environments

Benefits

• Hybrid work schedule
• Industry competitive pay
• Restricted Stock Units
• Health insurance options including HDHP and PPO
• Vision insurance
• Dental insurance
• Employer contributions to HSA accounts
• Paid parental leave
• Paid life insurance
• Short-term and long-term disability insurance
• Teladoc
• 401(k) with 100% match up to 4% of salary
• Generous paid time off and holiday schedule
• Cell phone reimbursement
• Tuition reimbursement
• Subscription to the Calm app
• MetLife Legal
• Company-paid commuter benefit of $300 per pay period