Member of Technical Staff (Data): World Models

Your Charter

• Data at Scale: Own the pipelines and storage systems that feed petabyte-scale multimodal datasets into model training.

• Sustainable Platforms: Build tooling and systems that are automated and efficient, enabling processing at scale and handling many small heterogeneous datasets.

Required Skillsets

• Data Engineering: Knowledge of Python ETL pipelines and supporting infrastructure, data formats, and storage systems at scale.

• ML Data Ops: Experience managing datasets, annotations, and data versioning for model training.

• Basic ML Knowledge: Solid grasp of ML fundamentals is essential to collaborate effectively with researchers and make sound data platform decisions.

• Agentic Engineering: Skilled at writing high-quality specifications for AI agents, while maintaining effective human review of AI-generated work.

Responsibilities

• Design, automate, maintain, and optimize Python ETL pipelines (Spark/Ray) for large-scale multimodal data.

• Build and maintain data cataloging, lineage, quality tooling, integrity verification, access controls, and lifecycle management systems.

• Provide guidance, internal tools, and documentation to colleagues on data best practices.

• Serve as a custodian of the company’s datasets, ensuring overall data health, quality, and discoverability.

Challenges You'll Tackle

• Implement high-performance, multimodal data pipelines capable of processing petabyte-scale datasets on 10,000s of CPUs and 100s of GPUs.

• Evolve data formats, storage, and processing to keep pace with cutting-edge AI advancements, while maintaining backward compatibility.

• Scale data infrastructure to handle the next order of magnitude in growth.

• At the same time, ensure the data platform flexible to rapidly handle many small heterogeneous datasets and ad hoc analytics queries.

Traits of the Ideal Candidate

• High agency and ownership: proactively picks up new work according to priority, manages their own backlog, and escalates early when priorities are unclear or deadlines are at risk.

• Takes responsibility for validating inputs end-to-end: spot-checks data, understands upstream preprocessing, and speaks up when something doesn't add up.

• Takes responsibility for ensuring outputs are correct and handed over: actively seeks sign-off from downstream consumers, communicates caveats, and ensures relevant stakeholders are aware of changes and breaking impacts.

• Cares about continuously improving pipelines, tooling, and processes so that each iteration makes the next one faster, more reliable, and easier for the team.

• Comfortable with rapid, pragmatic solutions when needed, but committed to high-quality, long-term solutions.