Offer summary

Qualifications:

Experience with large and complex data sets., Proficiency in PySpark, SQL, and Azure Databricks., Knowledge of database design using Microsoft SQL Server., Ability to develop data pipelines and optimize data flow..

Key responsibilities:

Build and optimize data and data pipeline architecture.

Support cross-functional teams with data initiatives.

Ensure consistent and efficient data delivery across projects.

Work independently to support multiple data needs.

Job description

This is a remote position.

To work in a variety of settings to build systems that collect, manage, and convert raw data into usable information for data scientists and business analysts to interpret.
Their ultimate goal is to make data accessible so that organizations can use it to evaluate and optimize their performance.
They will be responsible for expanding and optimizing our data and data pipeline architecture, as well as optimizing data flow and collection for cross functional teams.
They will support our software developers, database architects, data analysts and data scientists on data initiatives and will ensure optimal data delivery architecture is consistent throughout ongoing projects.
They must be self-directed and comfortable supporting the data needs of multiple teams, systems and products.

Requirements

Demonstrated experience working with large and complex data sets as well as experience analyzing volumes of data.
Using Data frames to convert for PySparkSql. Working in Azure Databricks and DATAFACTORY to migrate the data.
Creating Databricks notebooks using SQL, Pyspark for data validation.
Experience in database design and development using Microsoft SQL Server, Stored Procedure, and Functions.
Involved in development of code from scratch for the given requirement.