Logo for Bitful Consulting

Senior Data Engineer

Role overview

Summary

This role focuses on building and maintaining data pipelines that support various business functions at the company. You will collaborate with analysts, architects, developers, and stakeholders to deliver data solutions that enhance reporting and decision-making.

Qualifications

  • 7+ years of experience in data engineering, including building and supporting production data pipelines.
  • Strong SQL skills and a solid understanding of data modeling, database design and data warehousing.
  • Hands-on experience with Databricks, Apache Spark and cloud data platforms.
  • Strong proficiency in Python and PySpark.
  • Experience preparing reliable datasets for Tableau or comparable business intelligence tools.

Responsibilities

  • Build and maintain data pipelines and datasets within a Databricks lakehouse environment
  • Ingest, validate, clean and transform structured and unstructured data using Python, PySpark and SQL
  • Integrate established data warehouse and reporting systems with modern cloud analytics platforms
  • Provide technical guidance, review code and mentor other data engineers

Key facts

Hard skills

Key skills for this role

Other skills

  • Communication
  • Collaboration

About the company

Bitful Consulting logo

Bitful Consulting

  • IT Services & Consulting
  • Startup
  • 11–50
Bitful is a new kind of technology consulting firm: One that puts humans first. Together, we build future-proof, open, explorable data systems and perform cloud-computing marvels. We profoundly solve app challenges, and engineer game-changing digital products. And we do all of the above to create and fund opportunities for underserved communities who deserve to be every bit as supported, connected, and inspired as we are at Bitful. Better tech for the greater good: That’s Bitful.

Your match analysis

See how your profile stacks up against this role.

We compared the job requirements to your profile to show where you're strong and where you fall short.

Job description

  • You’ll build and maintain data pipelines and datasets that support reporting, visualization, machine learning and data-driven decision making across finance, sales, supply chain, marketing and other business functions.
  • Your responsibilities will include:
    • Developing and maintaining scalable data pipelines and reusable datasets within a Databricks lakehouse environment
    • Ingesting, validating, cleaning and transforming structured and unstructured data from multiple sources using Python, PySpark and SQL
    • Building data models and reporting datasets that support Tableau and established analytics tools such as Spotfire
    • Integrating established data warehouse and reporting systems with modern cloud analytics platforms
    • Improving pipeline reliability, data quality and processing performance
    • Implementing automated testing, monitoring, documentation and CI/CD practices
    • Partnering with analysts, architects, developers and business stakeholders to deliver practical data solutions
    • Providing technical guidance, reviewing code and mentoring other data engineer

Requirements

    • 7+ years of experience in data engineering, including building and supporting production data pipelines
    • Strong SQL skills and a solid understanding of data modeling, database design and data warehousing
    • Hands-on experience with Databricks, Apache Spark and cloud data platforms
    • Strong proficiency in Python and PySpark
    • Experience developing ETL/ELT pipelines and integrating data from multiple source systems
    • Experience preparing reliable datasets for Tableau or comparable business intelligence tools
    • Familiarity with source control, automated testing and CI/CD practices
    • Ability to troubleshoot complex data issues and optimize pipeline performance
    • Strong communication skills and experience collaborating with technical teams and business stakeholders
    • Experience with Spotfire, Oracle, Informatica, R or data virtualization preferred
    • Databricks certification preferred

Related jobs

See all Data Engineer jobs