Storage Protocols Engineering Manager

Work set-up: 
Full Remote
Contract: 
Experience: 
Senior (5-10 years)
Work from: 

Offer summary

Qualifications:

10+ years of software development experience., At least 5+ years in a management or lead role in storage software engineering., Extensive experience designing and implementing distributed storage systems., Knowledge of storage protocols like S3, NFS, iSCSI, and programming languages such as C++, Go, Rust, or Python..

Key responsibilities:

  • Lead and mentor a team of storage and distributed systems engineers.
  • Drive the technical strategy for distributed storage protocols and systems.
  • Collaborate with research, product, and infrastructure teams to meet storage needs.
  • Stay current with trends in distributed systems and storage technologies.

Lambda logo
Lambda SME https://lambdalabs.com/
51 - 200 Employees
See all jobs

Job description

Lambda is the #1 GPU Cloud for MLAI teams training, finetuning and inferencing AI models, where engineers can easily, securely and affordably build, test and deploy AI products at scale. Lambda’s product portfolio includes onprem GPU systems, hosted GPUs across public & private clouds and managed inference services – servicing government, researchers, startups and Enterprises worldwide.


If youd like to build the worlds best deep learning cloud, join us.


*Note: This position requires presence in our San Francisco office location 4 days per week; Lambda’s designated work from home day is currently Tuesday.


Engineering at Lambda is responsible for building and scaling our cloud offering. Our scope includes the Lambda website, cloud APIs and systems as well as internal tooling for system deployment, management and maintenance.

In the world of distributed AI, raw GPU and CPU horsepower is just a part of the story. Highperformance networking and storage are the critical components that enable and unite these systems, making groundbreaking AI training and inference possible.

The Lambda Infrastructure Engineering organization forges the foundation of highperformance AI clusters by welding together the latest in AI storage, networking, GPU and CPU hardware.

Our expertise lies at the intersection of:

  • HighPerformance Distributed Storage Solutions and Protocols: We engineer the protocols and systems that serve massive datasets at the speeds demanded by modern clustered GPUs.

  • Dynamic Networking: We design advanced networks that provide multitenant security and intelligent routing without compromising performance, using the latest in AI networking hardware.

  • Compute Virtualization: We enable cuttingedge virtualization and clustering that allows AI researchers and engineers to focus on AI workloads, not AI infrastructure, unleashing the full compute bandwidth of clustered GPUs.

    • About the Role:

      We are seeking an experienced Software Engineering Manager with a history in the development of storage protocols and distributed storage systems to lead a team of Storage Software Engineers and Distributed Systems Engineers in the design, development, and optimization of cuttingedge distributed storage solutions. Your team will be responsible for building highperformance, scalable, and reliable implementations of object, block, and file protocols, specifically tailored to serve performance demanding AI training and inference workloads. This is a unique opportunity to work at the intersection of largescale distributed systems and the rapidly evolving field of artificial intelligence infrastructure. You will be building the foundational infrastructure that powers some of the most advanced AI research and products in the world.

      What You’ll Do

      • Team Leadership & Management:

        • GrowHire, lead, and mentor a toptalent team of highperforming software engineers focused on delivering distributed storage protocols.

        • Foster a highvelocity culture of innovation, technical excellence, and collaboration.

        • Conduct regular oneonone meetings, provide constructive feedback, and support career development for team members.

        • Drive outcomes by managing project priorities, deadlines, and deliverables using Agile methodologies.

          • Technical Strategy & Execution:

            • Drive the technical vision and strategy for our distributed storage protocols (e.g., S3, NFS, iSCSI) and their underlying distributed systems.

            • Oversee the development of highly optimized storage solutions designed to meet the performance demands of AIML workloads (e.g., high throughput, low latency, optimization for AI workload access patterns).

            • Lead the team in tackling complex distributed systems challenges, including concurrency, consistency, fault tolerance, and data durability across multiple data centers.

            • Guide engineering team in problem identification, requirements gathering, solution ideation, and stakeholder alignment on engineering RFCs.

            • Deeply understand the performance bottlenecks of existing storage systems and guide the team in developing innovative solutions to overcome them.

            • Lead the team in supporting customers.

              • CrossFunctional Collaboration:

                • Work closely with AIML research and products teams to understand customers storage needs and translate them into technical requirements.

                • Work closely with the product engineering team to deliver high quality products to customers to meet their unique needs.

                • Collaborate with product management to define the product roadmap and prioritize features.

                • Work closely with HPC Architecture, Networking, Compute, and Storage Engineering teams to deploy highperformance distributed storage protocols to serve AIML workloads.

                • Partner with fleet engineering and platforms teams to ensure seamless deployment, monitoring, and maintenance of the distributed storage protocols.

                • Work in lockstep with the Storage Engineering team to provide reliable storage products on top of a variety of physical storage solutions.

                  • Innovation & Research:

                    • Stay current with the latest trends and research in distributed systems, storage technologies, and AIML hardwaresoftware advancements.

                    • Work with the Lambda product team to uncover new trends in the AI inference and training product category.

                    • Encourage and support the team in exploring new technologies and approaches to improve system performance and efficiency.

                        • You

                          • Experience:

                            • 10+ years of experience in software development, with at least 5+ years in a management or lead role in storage software engineering.

                            • Demonstrated experience leading a team of software engineers on complex, crossfunctional projects in a fastpaced startup environment.

                            • Extensive handson experience in designing and implementing distributed storage systems.

                            • Experience with storage protocols serving storage volumes at a scale greater than 20PB.

                            • Experience developing and tuning distributed storage protocols across scaling challenges using namespacing, sharding, and caching strategies.

                            • Familiarity with deploying and running applications on Kubernetes or other container orchestration systems (e.g., AWS ECS, Hashicorp Nomad).

                            • Strong project management skills, leading highconfidence planning, project execution, and delivery of team outcomes on schedule.

Required profile

Experience

Level of experience: Senior (5-10 years)
Industry :
Spoken language(s):
English
Check out the description to know which languages are mandatory.

Other Skills

  • Team Leadership
  • Mentorship
  • Problem Solving
  • Collaboration

Engineering Manager Related jobs