Lead Data Pipeline Engineer

Would you like to use your engineering skills and your experience with machine learning (ML) and large language models (LLMs) to help policy makers, corporate leaders, and operators make sense of increasingly complex information spaces? Are you excited by the possibilities of generative and agentic artificial intelligence? Because Two Six Technologies uses these technologies to provide Information Advantage to Government, Military, and private sector customers tasked with mission critical decision making. 

Our products collect, synthesize, and enrich huge quantities of publicly available data, then serve that data to be queried by our expert analysts and to be used in machine learning analytics. As a data pipeline engineer you will work with product engineering teams, data scientists, devops, and architects to integrate data science into our products. 

As a top candidate for this role you are an experienced server-side Python engineer with a history of building products or platforms with integrated ML capabilities. You are very comfortable working with AWS, Linux, and using technologies like Docker and Kubernetes. You also have worked as a member of remote engineering teams in Agile software development environments. You are a good fit if you want to continue to grow your career in the ML space by taking data science and making it available to end users in fast, scalable, and secure applications. This role is right for you if you have managed before, or if you have been a team lead or senior engineer before and are interested in managing a small team of data pipeline engineers while also being a hands-on data pipeline engineer yourself. 

Most importantly you share our passion for applying technology to solve hard problems and finding innovative ways to derive insights from high-volume data. 

Responsibilities:

• Developing and enhancing high-volume data pipelines where data is enriched using ML and NLP functionality, and then used to feed additional machine learning and data science
• Integrating models that data science creates into applications that are fast and scalable
• Building and enhancing user-facing applications that use generative AI and LLMs to help our customers gain insights from data faster
• Operating in a collaborative, agile environment, with a focus on taking action and working closely with peers to enable team success
• Creating proofs-of-concept and prototypes to quickly test ideas, as well as designing and building scalable, production-ready solutions
• Considering security best practices at every phase of software development
• Continuously learning and improving
• Working in a fully remote team with a diverse set of skills and experiences
• Independently identifying and solving problems, and questioning assumptions in pursuit of the right solutions 
• Actively participating in the peer code review process
• Creating and executing both manual and automated testing
• Managing a small team of data pipeline engineers 

Qualifications:

• Experience building scalable server-side applications and data pipelines using Python
• Experience working with ML, LLMs, and collaborating closely with data science teams
• Experience with Elastic or similar data stores, as well as with SQL database technologies, ideally Postgres
• Experience with AWS or similar cloud-based infrastructure
• Experience with the design, development, testing, and support of scalable, data-driven applications deployed to production SaaS environments
• Experience working independently and as a part of an Agile team 

Optional Skills / Domain Experience

• Prior experience as a platform engineer or ML Ops engineer 
• Experience with processing, storaging, and querying of data in multiple natural languages
• Elastic certification
• AWS certification

Clearance/Citizenship:

• Preferred TS Cleared
• Must be eligible to obtain a US government clearance