Staff / Principal Software Engineer
You'll join the Inworld platform team to help build and scale a set of newly launched consumer AI products. You will work on the Inworld Router, an intelligent routing layer giving developers a single API to access 200+ LLMs, owning core systems for multi-provider failover, cost/latency-based routing, live A/B experimentation, and real-time observability at massive scale. You'll also contribute to the Realtime API, API-based model services such as custom TTS/STT with instant voice cloning, and new large-scale products launching later this year. You'll work on services for control and optimization, as well as infrastructural projects like platformization, developer tooling, and system-wide billing. As an IC-focused Staff/Principal engineer, you'll establish significant scope by collaborating with PMs, engineers and leads, operate with technical autonomy to bring in new technical dependencies or standards where appropriate, collaborate and deliver on the core building loop balancing speed and quality, and advocate for and drive system improvements. You'll be working on-site in the South Bay office a few days a week alongside sibling ML teams.
Responsibilities
- Own core systems for multi-provider failover, cost/latency-based routing, live A/B experimentation, and real-time observability at massive scale
- Build and evolve API-based model services including custom TTS/STT models with instant voice cloning
- Develop new large-scale products for launch later this year
- Build services for control and optimization
- Work on infrastructural projects such as platformization, development tooling integration, and system-wide billing
- Collaborate with PMs, engineers and leads to determine biggest product needs to focus on
- Suggest and bring in new technical dependencies or standards
- Advocate for and drive system improvements related to and independent of key features
Requirements
- Excellent programming skills and experience in a statically typed backend programming language, preferably Go, Python, C++ or Rust
- Experience developing and deploying cloud-based services to at least hundreds of qps
- Experience with relational databases (PostgreSQL or MySQL)
- Hands-on experience with caching (Redis or Memcached), pubsub/queues, data pipelines (Flink, Beam), and cloud storage
- Excellent verbal and written communication skills
- Experience building API gateways, routing/proxy layers, or multi-provider orchestration systems (bonus)
- Experience with analytics or timeseries databases (ClickHouse, Timescale, InfluxDB) (bonus)
- Experience with OpenTelemetry (bonus)
- Experience with C++ (bonus)
- Must be based in the SF Bay Area or willing to relocate
Benefits
- Equity