Machine Learning Infrastructure Engineer

About the company

Our client is a software company.

The role

Raydar is recruiting for this role on behalf of our client. Own the systems that run and train language models, from early experimentation through live production use. The focus is on speed, dependability and efficient hardware spend, working side by side with senior technical leaders and researchers in a small team.

What you'll do

• Take wide ownership of the compute platform used to train and serve models, covering both research and production needs.
• Measure and reduce response times, processing capacity and spend on accelerators under realistic load.
• Refine how requests are grouped, how results are stored for reuse and how memory is managed, and test compression and faster decoding approaches where they pay off.
• Support model inference for evaluation runs, synthetic data creation and reinforcement learning workloads.
• Build dependable release and observability workflows for models.
• Work with founders and researchers to split GPU capacity between training and serving, and to roll out models in customer environments.

What we're looking for

• Professional experience at a company building or tuning GPU-based infrastructure for large language models in production; coursework or personal projects alone are not enough.
• Deep expertise in diagnosing GPU performance limits, tuning model serving and setting up monitoring.
• Solid debugging and low-level systems skills, with a good grasp of accelerator memory and compute limits.
• Track record of owning an infrastructure system from start to finish, ideally launching one from scratch at a startup or in a small team; strong infrastructure engineers from large technology companies are also welcome.
• Hands-on work with an LLM serving framework or a comparable in-house engine, using PyTorch or JAX.
• Sound judgment in balancing model accuracy against speed, scale and cost to pick pragmatic solutions.

Bonus points

• Experience at an AI-focused startup that trains its own models.
• Low-level optimization experience, such as custom GPU kernels, quantization, speculative decoding, or distributed and multi-model serving.
• Technical degree from a leading program.
• Publications at top machine learning conferences or public writing about systems work.

Location and work model

• California, United States
• On-site, 5 days a week in the office
• Full-time

Compensation, as the posting describes it

• Equity