Important Information
Location: Brazil
Job Mode: Full-time
Work Mode: Work from home
Job Summary
As a Senior Data Engineer, you will be responsible for designing, developing, and maintaining high-quality software solutions. You will collaborate with cross-functional teams to understand business requirements and translate them into scalable and efficient software applications. Your role will involve leading technical projects, mentoring junior engineers, and continuously improving software development practices to ensure the delivery of robust and reliable software systems.
Responsibilities and Duties
- Architect and build scalable ETL/ELT pipelines using Azure Data Factory (ADF) and Azure Synapse;
- Implement complex logic to convert time-indexed sensor data into depth-indexed formats, handling rig states (e.g., pausing while drilling);
- Create robust ingestion frameworks for standard oilfield file formats (LAS, DLIS) and real-time streams (WITSML);
- Standardize sporadic data incoming from different vendors (Halliburton, Schlumberger, etc.) into a common schema;
- Implement stream processing solutions for live sensor data ingestion;
- Build automated checks within the pipeline to flag gaps, spikes, or flat-lines in sensor data;
- Tune Spark jobs and SQL queries in Synapse for performance and cost efficiency;
- Organize and manage the Azure Data Lake Storage (ADLS Gen2) folder structures (Bronze/Silver/Gold);
- Implement Row-Level Security (RLS) and manage access via Azure AD;
- Work closely with the Physics Owner (BA) to understand log mnemonics and conversion rules.
Essential Skills
- Strong experience designing and implementing data pipelines using Azure Data Factory and Azure Synapse Analytics;
- Advanced proficiency in SQL and experience working with large scale data processing;
- Strong programming skills in Python with experience using PySpark or Pandas for data processing;
- Experience working with big data technologies such as Spark for distributed data processing;
- Experience managing and organizing data in Azure Data Lake Storage Gen2;
- Knowledge of common data formats such as JSON, Parquet, Delta Lake and XML based formats;
- Understanding of ETL and ELT patterns and modern data engineering best practices;
- Experience processing time series or real time streaming data;
- Strong problem solving and analytical skills to handle complex data transformation logic;
- Experience optimizing data pipelines, Spark jobs or SQL queries for performance and scalability;
- Familiarity with data security practices such as access management and data governance in cloud environments;
- Microsoft Certified Azure Data Engineer Associate (DP 203) certification is a plus.
About Encora
Encora is the preferred digital engineering and modernization partner of some of the world’s leading enterprises and digital native companies. With over 9,000 experts in 47+ offices and innovation labs worldwide, Encora’s technology practices include Product Engineering & Development, Cloud Services, Quality Engineering, DevSecOps, Data & Analytics, Digital Experience, Cybersecurity, and AI & LLM Engineering.
At Encora, we hire professionals based solely on their skills and qualifications, and do not discriminate based on age, disability, religion, gender, sexual orientation, socioeconomic status, or nationality.
