Partner with internal operations teams to identify, collect, and integrate data from various business systems, ensuring comprehensive and accurate data capture
Design, implement, and maintain robust data pipelines that feed data into our Data Platform, ensuring high performance, scalability, and reliability
Ensure data pipelines adhere to best practices and are optimized for performance and scalability
Conduct thorough testing of data pipelines to validate data accuracy and integrity
Monitor data pipelines, troubleshoot any issues that arise, and make improvements to these issues where applicable
Establish and track SLAs for data processing and delivery, ensuring timely and reliable access to data for all users
Become a mentor for less experienced team members, and establish patterns and practices that can be followed to increase quality, accuracy, and efficiency of solutions produced by the team
Work with other teams in order to ensure access to data corresponds with company policies, and ensure data access, processing, and storage is in compliance with regulatory (e.g. GDPR, CCPA, etc.) requirements
QUALIFICATIONS
4+ years experience with AWS data processing tools (storage, processing, etc.)
5+ years of Java development experience.
Experience with Kafka and Kafka Connect.
Experience creating ETL pipelines using AWS tools (Lambda, Glue, Redshift, S3).
Experience with orchestration tools such as Apache Airflow.
Experience working with data visualizations and dashboards.
Experienced with JDK 17+, Spring Boot.
Experience in creating and implementing REST API in microservices components
Experience with both relational and non-relational data stores.
Experience with Docker.
Excellent verbal and written communication skills.
Excellent time management and organizational skills.
Ability to keep current and do independent research when needed.
Diabolocom
Sigma IT Poland
Capgemini Engineering
Sigma Software Group
eXperience IT Solutions