Data Engineer Team Lead

Description

Bigabid is an innovative technology company led by data scientists and engineers who are devoted to mobile app growth. Our proprietary ad platforms, powered by machine learning, are the outcome of that devotion.

We use elite programmatic user acquisition and retargeting technologies to deliver valuable results and insights for a fastgrowing clientele of major app developers.

Our everevolving, stateoftheart machine learning technology analyzes tens of TB of raw data per day to produce millions of ad recommendations in real time. This data is used to power our machine learning predictions, businesscritical metrics, and analytics, which in turn power our decisionmaking.

At Bigabid, you will own your tasks from top to bottom, with the freedom to learn, innovate, and sharpen your skills alongside an exceptionally talented team. We value a hunger for growth, hard work, and collaboration.

We are looking for a data centric quality driven team leader focusing on data process observability. The person is passionate about building highquality data products and processes as well as supporting production data processes and adhoc data requests.

As a Data OPS TL at Bigabid, you will be in charge of the quality of service as well as quality of the data and knowledge platform for all Bigabids data processes. You’ll be coordinating with stakeholders and play a major role in driving the business by promoting the quality and stability of the data performance and lifecycle and giving the Operational groups immediate abilities to affect the daily business outcomes.

Responsibilities:

• Process monitoring managing and monitoring the daily data processes; troubleshooting server and process issues, escalating bugs and documenting data issues.
• Adhoc operation configuration changes Be the extension of the operation side into the data process; Using Airflow and python scripting alongside SQL to extract specific client relevant data points and calibrate certain aspects of the process.
• Data quality automation Creating and maintaining data quality tests and validations using python code and testing frameworks.
• Metadata store ownership Creating and maintaining the metadata store; Managing the metadata system which holds meta data of tables, columns, calculations and lineage. Participating in the design and development of the knowledge base metastore and UX. In order to be the pivotal point of contact when needing information on tables, columns and how they are connected. I.e., What is the data source? What is it used for? Why are we calculating this field in this manner?
• 
  

  Requirements

  • Minimum of 2 years in any leadership role within a data team or IT.
  • Over 3 years of handson experience as a Data Engineer, with strong proficiency in Python and Airflow.
  • Solid background in working with both SQL and NoSQL databases and data warehouses, including but not limited to MySQL, Presto, Athena, Couchbase, MemSQL, and MongoDB.
  • Bachelor’s degree or higher in Computer Science, Mathematics, Physics, Engineering, Statistics, or a related technical discipline.
  • Previous experience as a NOC or DevOps engineer is a plus.
  • Familiarity with PySpark is considered an advantage.

  • Preferred Attributes:

    • Highly organized with a proactive mindset.
    • Strong service orientation and a collaborative approach to problemsolving.
    • 
      

      Excerpt

      As a DataOps TL, you will be in charge of the quality of service as well as quality of the data and knowledge platform for all Bigabids data processes.