Principle Data Engineer - Tech Lead

As a Principal Data Engineer, you will lead the development of a cutting-edge data platform to support the next generation learning experience platform (LXP) powered by Generative AI. Leveraging our unique proprietary data, advanced learning science techniques and the power of LLMs, you will lead the team to design and implement a full-scale data platform from robust data pipelines, to combined SQL/NoSQL/Vector DB infrastructure, and with ETL data indexing, and data processing engines. The ideal candidate is a hands-on principal level data engineer with leadership experience for a small and focused team. He/she is passionate about leveraging data to drive business decisions, thrives in a fast-paced environment, and is eager to contribute to the success of our company.

• Designing and implementing scalable data solutions that enhance our learning platforms, utilizing AWS technologies such as DMS, S3, OpenSearch/Elasticsearch, KMS, and Athena.
• Lead a three to five data engineer team to develop a full data platform, with first priority to store and process digital course data to support the development of an AI Agent framework.
• Communicate with various stakeholders, such as data scientists, business analysts, and executives, to understand their requirements and translate them into technical solutions.
• Establish and maintain the data pipeline and infrastructure. Collaborate with Quality Engineers to develop data validation strategies.
• Identify and troubleshoot issues in the back-end infrastructure and applications.
• Optimize applications for maximum speed and scalability.
• Motivate and lead data engineering development team. 

• Bachelor's degree or higher in Computer Science, Engineering, or related field; equivalent experience will be considered.
• Proven track record in data engineering roles and experience in leading a team.
• Strong foundation in data engineering concepts, including data modeling, database design, ETL processes, and data warehousing.
• Experience with Data Management and Data Governance processes and standards.
• Exceptional expertise of system design and application architecture for data platforms with hands-on experience with the popular data lake technology stacks.
• Programming expertise in Python, C#, and SQL to build and maintain efficient, reusable, and reliable code.
• AWS proficiency, ensuring optimized infrastructure deployment and maintenance for high availability and performance. AWS certification is highly desirable.
• Experience with recognized industry patterns, methodologies, and techniques.
• Excellent Problem-solving skills with keen attention to detail.
• Ability to thrive in a collaborative team environment and effectively communicate complex technical concepts to non-technical stakeholders.
• Familiarity with Agile development methodologies.