Offer summary

Qualifications:

4+ years of hands-on experience as a Data Engineer, BA/BSc in a related field such as CS, engineering, or information systems, High proficiency in SQL & Python, Experience with cloud-hosted data warehouses and ETL/ELT processes..

Key responsibilities:

Manage end-to-end data pipelines from operational systems to the data lake

Complete migration from Hadoop to a Spark-based environment on AWS

Develop and implement ETL/ELT processes using Spark and SQL

Collaborate with team members to ensure data quality and integrity.

Globaldev Group provides end-to-end software development services and builds skillful teams of specialists to fuel your business’ growth. Leveraging our extensive 12-year background and deep industry knowledge, we have established ourselves as a reliable partner for numerous enterprises, SMEs, and startups. ▪️️ Global Teams ▪️ Globaldev offers R&D team extensions to both emerging technology companies and global corporations. Empowered by our global talent pool, we excel at connecting our clients with high-skilled specialists. Hiring, onboarding, and continuous management are all included in our comprehensive set of services, enabling our clients to quickly scale their teams and achieve their product development and implementation goals. ▪️ Global Engineering ▪️ - Obtain a custom-designed solution from scratch - Scale or enhance your current software infrastructure - Stay ahead with the latest cutting-edge technologies - Access a comprehensive range of services from a single vendor As a full-stack development company, we are equipped to fulfill all your business requirements. Whether it's product discovery, proof of concept, UX design, or development, Globaldev serves as your all-in-one solution provider.

Job description

Responsibilities:

Manage the end to end data pipelines that pull data from operational system and from different end points, transform and load the data to the company data lake
Current environment is set up on hadoop, but the team will be responsible to complete migration to a new Spark based environment on AWS

Qualifications:

4+ years of hands-on experience as a Data Engineer
BA/BSc in a related field such as CS, engineering, information systems, or equivalent.
High proficiency in SQL & Python
Experience working with cloud-hosted data warehouses (Hive, Snowflake)
Hands-on experience with developing end-to-end ETL/ELT processes (Spark and SQL)
Experience with the Hadoop framework and parallel computing (EMR, HIVE, Presto/Trino/Athena, Glue)
Proven experience with data warehousing, modeling paradigms, and architectures, proficient in DWH methodologies and best practices