MLOps Engineer (LLM Infrastructure)

Senior MLOps Engineer with 7+ years of experience needed for an AI-powered construction software company.

Responsibilities:
Design, build, and maintain infrastructure for deploying, serving, and monitoring machine learning models at scale
Deploy and scale large language models (7B-376B parameters) across varied sizes, optimizing for latency, throughput, and cost
Operate and enhance a multi-provider gateway for routing requests between self-hosted models and external APIs
Develop and manage automated pipelines for model training, fine-tuning, evaluation, and continuous delivery
Monitor production ML services, implement observability, and respond to incidents
Implement evaluation and verification systems to detect quality and performance regressions
Collaborate with cross-functional teams to integrate ML models into production environments
Deliver infrastructure-as-code and CI/CD workflows for repeatable, secure deployments
Optimize model and infrastructure efficiency through quantization, pruning, and distributed inference
Provide self-serve tooling and documentation for internal engineering teams

Must-have:
Deep expertise in LLM deployment and scaling (7B to 376B parameters)
Strong Python proficiency, with C/C++ experience for performance-critical components
Hands-on experience with inference serving engines (vLLM, Triton, TGI)
Advanced knowledge of distributed training frameworks (DeepSpeed, FSDP, Accelerate)
Proficiency with MLOps platforms and pipelines (MLflow, Kubeflow, SageMaker Pipelines)
Cloud infrastructure and orchestration (AWS, Docker, Kubernetes, EKS, EC2, Lambda)
GPU/cloud resource optimization and cost efficiency management
Monitoring, logging, alerting, and incident response for ML services
Experience with quantization, pruning, and multi-GPU/distributed inference
Experience building or operating multi-provider model gateways

Conditions:
B2B Freelance contract
Full Remote
12+ months duration
ASAP start date

Project:
AI-powered construction software startup. Pioneer in artificial intelligence focused on developing secure, enterprise-grade solutions bridging advanced AI research with real-world applications.