Roles & Responsibilities

Strong experience with Python, PL/SQL, DBT, Airflow, and BigQuery on Google Cloud Platform.
Proficiency in data transformation, ETL scripting, and data manipulation with libraries such as pandas and NumPy.
Hands-on experience with orchestration and data processing tools including Airflow, Dataflow, Cloud Composer, and data ingestion methods (Cloud Functions, Cloud Run, SFTP; Falcon for DB connections).
Familiarity with cloud-native data processing, data integration from multiple sources, incremental loading, and partitioning strategies.

Requirements:

Design, build, and maintain scalable data pipelines using Python, DBT, and Airflow on GCP, ensuring data quality and lineage.
Develop data transformations, create views and tables, and implement ETL logic using pandas and NumPy.
Collaborate with data scientists and stakeholders to implement householding, data segmentation, and support streaming and batch data workloads.
Optimize data architecture for performance and cost, including incremental loading and partitioning strategies across multiple sources.

Job description

Job Title: Senior Data Engineer

Location: Remote role

Salary: $135-145K Max salary.

Mandatory Skills: Python, PL/SQL, DBT, Airflow, BigQuery, do not send resume if candidate is not meeting the requirements.

Technical Skills:

Python: Programming, ETL scripting, business logic implementation, and experience with data manipulation libraries (pandas, NumPy).
PL/SQL: Data transformation, view/table creation, and persistence layer interactions.
Data Transformation Tools: DBT (Data Build Tool) for creating models and documentation.

Orchestration and Data Processing Tools
Orchestration: Airflow, Dataflow, and Cloud Composer.
Data Ingestion: Cloud Functions, Cloud Run, SFTP mechanisms, and Falcon (for database connections).

Cloud Platform:

Google Cloud Platform (GCP) and BigQuery experience.
Experience with cloud-native data processing.

Additional Desired Skills:

Understanding of householding and data segmentation logic.
Ability to work with streaming and batch data.
Experience with data integration from multiple sources (e.g., Reltio, external databases).
Familiarity with incremental data loading and partitioning strategies.

Diverse Lynx LLC is an Equal Employment Opportunity employer. All qualified applicants will receive due consideration for employment without any discrimination. All applicants will be evaluated solely on the basis of their ability, competence and their proven capability to perform the functions outlined in the corresponding role. We promote and support a diverse workforce across all levels in the company.

Ready to apply?

APPLY

Share ·