ML Engineer, Large Language Models (LLM Training & Inference Optimization)

The role

This role is for Nebius AI R&D, a team focused on applied research and the development of AI-heavy products. Examples of applied research that we have recently published include:

• investigating how test-time guided search can be used to build more powerful agents;
• dramatically scaling task data collection to power reinforcement learning for SWE agents;
• maximizing efficiency of LLM training on agentic trajectories.

One example of an AI product that we are deeply involved in is Nebius AI Studio — an inference and fine-tuning platform for AI models.

We are currently in search of senior and staff-level ML engineers to work on optimizing training and inference performance in a large-scale multi-GPU multi-node setups. 

This role will require expertise in distributed systems and high-performance computing to build, optimize, and maintain robust pipelines for training and inference.

Your responsibilities will include: 

• Architecting and implementing distributed training and inference pipelines that leverage techniques such as data, tensor, context, expert (MoE) and pipeline parallelism. 
• Implementing various inference optimization techniques, e.g. speculative decoding and its extensions (Medusa, EAGLE, etc.), CUDA-graphs, compile-based optimization.
• Implementing custom CUDA/Triton kernels for performance-critical neural network layers.

We expect you to have:

• A profound understanding of theoretical foundations of machine learning
• Deep understanding of performance aspects of large neural networks training and inference (data/tensor/context/expert parallelism, offloading, custom kernels, hardware features, attention optimizations, dynamic batching etc.)
• Expertise in at least one of those fields:
• Implementing custom efficient GPU kernels in CUDA and/or Triton
• Training large models on multiple nodes and implementing various parallelism techniques
• Inference optimization techniques - disaggregated prefill/decode, paged attention, continuous batching, speculative decoding, etc.
• Strong software engineering skills (we mostly use python)
• Deep experience with modern deep learning frameworks (we use JAX & PyTorch)
• Proficiency in contemporary software engineering approaches, including CI/CD, version control and unit testing
• Strong communication and ability to work independently

Nice to have:

• Familiarity with modern LLM inference frameworks (vLLM, SGLang, TensorRT-LLM, Dynamo)
• Familiarity with important ideas in LLM space, such as MHA, RoPE, ZeRO/FSDP, Flash Attention, quantization
• Bachelor’s degree in Computer Science, Artificial Intelligence, Data Science, or a related field Master’s or PhD preferred
• Track record of building and delivering products (not necessarily ML-related) in a dynamic startup-like environment
• Experience in engineering complex systems, such as large distributed data processing systems or high-load web services
• Open-source projects that showcase your engineering prowess
• Excellent command of the English language, alongside superior writing, articulation, and communication skills