Staff Software Engineer - Infinia Tensor Interface

Overview:

This is an incredible opportunity to be part of a company that has been at the forefront of AI and high-performance data storage innovation for over two decades. DataDirect Networks (DDN) is a global market leader renowned for powering many of the world's most demanding AI data centers, in industries ranging from life sciences and healthcare to financial services, autonomous cars, Government, academia, research and manufacturing.

"DDN's A3I solutions are transforming the landscape of AI infrastructure." – IDC 

“The real differentiator is DDN. I never hesitate to recommend DDN. DDN is the de facto name for AI Storage in high performance environments” - Marc Hamilton, VP, Solutions Architecture & Engineering | NVIDIA 

DDN is the global leader in AI and multi-cloud data management at scale. Our cutting-edge data intelligence platform is designed to accelerate AI workloads, enabling organizations to extract maximum value from their data. With a proven track record of performance, reliability, and scalability, DDN empowers businesses to tackle the most challenging AI and data-intensive workloads with confidence. 

Our success is driven by our unwavering commitment to innovation, customer-centricity, and a team of passionate professionals who bring their expertise and dedication to every project. This is a chance to make a significant impact at a company that is shaping the future of AI and data management. 

Our commitment to innovation, customer success, and market leadership makes this an exciting and rewarding role for a driven professional looking to make a lasting impact in the world of AI and data storage. 

We're seeking an experienced Staff Storage Platform Engineer to develop a high-performance, distributed file system optimized for AI/ML workloads. You'll build a storage control plane that provides both S3-compatible and POSIX-compliant interfaces, delivering enhanced performance characteristics specifically designed for compute-intensive AI applications. This dual-interface storage solution will help organizations maximize their AI/ML capabilities through flexible access patterns and efficient storage utilization.

What you will be doing:

• Design and implement a storage platform with both S3-compatible APIs and POSIX-compliant file system access optimized for AI/ML workloads
• Develop a storage SDK that abstracts between object and file access patterns, allowing customers to use whichever paradigm best suits their application needs
• Engineer intelligent data consistency and caching mechanisms that maintain proper semantics across both access methods
• Build scalable storage services capable of handling exabyte-scale deployments across multi-region environments
• Create integration points with popular AI frameworks to enable seamless data pipelines through both S3 and POSIX interfaces
• Design and implement high-performance data access layers, including specialized APIs and other acceleration techniques that radically reduce data access latency for AI workloads
• Pioneer innovative storage acceleration techniques that break traditional storage access bottlenecks for AI/ML operations
• Collaborate with AI researchers and customers to understand evolving storage requirements for next-generation AI applications

What we need to see:

• Strong experience with both S3 API implementation and POSIX-compliant file systems
• Expertise in distributed storage systems that support hybrid access patterns
• Track record of building high-throughput, low-latency data access solutions
• Experience optimizing storage systems for specialized workloads, particularly AI/ML data patterns
• Proficiency in Go, Python, and infrastructure automation technologies
• Familiarity with Kubernetes-native storage solutions and cloud-native architectures
• Understanding of consistency challenges between object storage and file system semantics
• Bachelor's degree in Computer Science or related field with 8+ years of relevant industry experience

Ways to stand out from the crowd:

• Strong experience with both S3 API implementation and POSIX-compliant file systems
• Experience developing unified storage systems that bridge object and file access paradigms
• Background implementing POSIX semantics on distributed storage backends
• Knowledge of performance optimization techniques for both sequential and random-access patterns
• Experience with GPU-accelerated computing storage optimizations (NVIDIA GPU Direct Storage, RDMA)
• Familiarity with AI/ML frameworks and their specific storage access patterns and requirements
• Track record of designing storage SDKs that successfully abstract complexity from application developers
• Contributions to open-source storage projects that implement S3 or POSIX interfaces
• Understanding of consistency challenges between object storage and file system semantics
• Deep understanding of storage I/O patterns and how to optimize them for specific access requirements
• Demonstrated ability to create innovative performance optimizations that significantly accelerate data-intensive workloads
• Proficiency in C, C++, Python, GO, and infrastructure related technologies 

This position requires participation in an on-call rotation to provide after-hours support as needed.

We're looking for someone who can think beyond traditional storage paradigms to create breakthrough performance improvements. Our ideal candidate will develop creative solutions—such as specialized KV store APIs, tensor-aware data layouts, or predictive data prefetching mechanisms—that dramatically reduce the data access bottleneck for AI workloads. Your innovations should help our customers achieve training and inference speeds that were previously impossible with conventional storage approaches.

Our solution aims to give AI/ML developers the best of both worlds: the scalability and simplicity of S3 object storage combined with the familiar, high-performance local access of POSIX file systems, enhanced with specialized access methods that eliminate traditional I/O bottlenecks, enabling organizations to achieve faster training times and more efficient infrastructure utilization.

Join our dynamic and driven team, where engineering excellence is at the heart of everything we do. We seek individuals who love to challenge themselves and are fueled by curiosity. Here, you'll have the opportunity to work across various areas of the company, thanks to our flat organizational structure that encourages hands-on involvement and direct contributions to our mission. Leadership is earned by those who take initiative and consistently deliver outstanding results, both in their work ethic and deliverables, making strong prioritization skills essential. Additionally, we value strong communication skills in all our engineers and researchers, as they are crucial for the success of our teams and the company as a whole.

Interview Process: After submitting your application, one of our recruiters will review your resume. If your application passes this stage, you will be invited to a 30-minute interview during which a member of our team will ask some basic questions. If you clear the interview, you will enter the main process, which can consist of up to four interviews in total:

• Coding assessment: Often in a language of your choice.
• Systems design: Translate high-level requirements into a scalable, fault-tolerant service (depending on role).
• Real-time problem-solving: Demonstrate practical skills in a live problem-solving session.
• Meet and greet with the wider team.
• Our goal is to finish the main process in 2-3 weeks at most.

DataDirect Networks (DDN) is an Equal Opportunity/Affirmative Action employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, gender, gender identity, gender expression, transgender, sex stereotyping, sexual orientation, national origin, disability, protected Veteran Status, or any other characteristic protected by applicable federal, state, or local law.

#LI-Remote