Key Facts

Remote From:

Full time

Senior (5-10 years)

English

Hard Skills

Data Modeling Snowflake (Data Warehouse) Data Architecture Databricks Cloud Database Data Warehouse Systems Databricks Spark Framework Data Warehouse Architectures ETL (Extract Transform Load) Relational Databases AWS Glue Semi-Structured Data Apache Airflow Apache Kafka Unstructured Data Systems Design Data Governance MLflow Data Quality Assessment Snowflake Schema Python (Programming Language) Apache Spark Talend Technology Assessment Apache NiFi Data Collection Performance Systems Analysis Data Mining Information Privacy Data Strategy Database Application SQL Server Integration Services (SSIS) Predictive Modeling Amazon S3 Informatica Product Roadmap Management Amazon Redshift Data Profiling Data Quality Assessment Data Grid Luigi (Python Package) Computer Systems Apache Beam Architectural Design Engineering Documentation Collaborative Filtering Standards Organization Stakeholder Communications Software Architecture

Other Skills

•
Problem Solving
•
Collaboration
•
Leadership
•
Communication
•
Strategic Planning
•
Problem Solving

Roles & Responsibilities

15+ years of overall experience
10+ years of data architecture, data platform, or data warehouse experience
Hands-on experience with Snowflake (4+ years) and Databricks (4+ years); 5+ years combined experience
Proficiency with Delta Lake, MLflow, and Databricks SQL; experience managing Spark clusters and ML workflows

Requirements:

Own design and maintenance of data solutions including modeling, development, technical documentation, data diagrams and data dictionaries
Provide expertise in standards, architectural governance, design patterns, and evaluation of best solutions for different use cases
Lead the data strategy and own the vision and roadmap of data products, coordinating with stakeholders on sensitive data requirements and governance
Develop, maintain, and optimize data infrastructure using Delta Lake, MLflow, and Databricks SQL to enhance data management and analytics

NIR-YU

Human Resources, Staffing & Recruiting

About NIR-YU

Nir-Yu specializes in helping Small and Medium Enterprises unlock the potential of their business. Our typical customer is worried about not being able to make enough progress on their product(s) or project(s) with their current team; they need additional team-members to deliver work on-time and with the required quality. Tight budgets also make it necessary to lower average wage costs, making it necessary to consider hiring internationally to lower costs. We help these companies by providing a bespoke service that finds and hires remote workers from Mexico, Colombia, Argentina, Chile, Brazil and other countries in LatAm based on the company’s specific job descriptions to work as an extension of their team and perform the required functions. Unlike many other companies, we don’t have a bench of contractors to be hired out, instead each team is hired specifically to meet specific requirements and to work exclusively as a remote employee.

Company type: Scaleup

Industry: Human Resources, Staffing & Recruiting

Founded: 2018

Company size: 201 - 500

Website LinkedIn See all jobs →

Job description

The Role:

We are looking for hands-on Principal Data-Architect for a challenging and fun filled work of building a data architecture that is future proof for one of the customers in the financial domain.

Responsibilities:

Own design and maintenance of all aspects of data solutions including modeling, developing, technical documentation, data diagrams and data dictionaries.
Provide expertise in the development of standards, architectural governance, design patterns, and practices, evaluate best applicable solutions for different use cases
Determines and develops architectural approaches and solutions, conducts business reviews, documents current systems, and develops recommendations
Lead the data strategy and own the vision and roadmap of data products
Work with stakeholders to ensure that data related business requirements for protecting sensitive data are clearly defined, communicated, and well understood and considered as part of operational prioritization and planning
Develop, maintain, and optimize data infrastructure using Delta Lake, MLflow, and Databricks SQL to enhance data management, processing, and analytics.
Utilize Snowflake’s features such as data sharing, zero-copy cloning, and automatic scaling to optimize data storage, accessibility, and performance. Ensure effective management of both semi-structured and structured data within Snowflake’s architecture.
Implement and manage data storage solutions using Amazon S3, perform data warehousing with Amazon Redshift.
Design and implement data integration workflows using AWS Glue to orchestrate and automate data movement and transformation.
Design and implement scalable data pipelines using tools like Apache Kafka or Apache Airflow to facilitate real-time data processing and batch data workflows.
Apply advanced analytics techniques, including predictive modeling and data mining, to uncover insights and drive data-driven decision-making.

Requirements:

Overall experience of 15+ years
Data architecture, data platform, data warehouse related experience of 10+ years
Hands on experience with snowflake - 4+ years experience, Data bricks 4+ years
Between snowflake and Data bricks (5+ years experience)
Proficiency in features such as Delta Lake, MLflow, and Databricks SQL. Experience in managing Spark clusters and implementing machine learning workflows.
Solid experience in emerging and traditional data stack components such as: batch and real time data ingestion, ETL, ELT, orchestration tools, on-prem and cloud DW, Python, structured, semi and unstructured databases
Knowledge of features like Snowflake’s data sharing, zero-copy cloning, and automatic scaling. Experience in working with Snowflake’s architecture for semi-structured and structured data.
Experience with services like Amazon S3, Amazon Redshift, and AWS Glue.
Proficiency in tools such as Apache NiFi, Talend, Informatica, or Microsoft SQL Server Integration Services (SSIS).
Experience in designing and implementing data pipelines using tools like Apache Kafka or Apache Airflow.
Ability to perform data profiling, data quality assessments, and performance tuning.
Experience in comparing and evaluating different data technologies based on criteria like performance, scalability, and cost.
Skills in applying advanced analytics techniques, including predictive modeling and data mining.
Expert with industry standard data practices, data strategies and data concepts
Demonstrated experience in architecting/re-architecting complex data systems and data models.
Demonstrated experience in overall system design, including database selection and solutioning.

Nice-to-Have Skills:

Experience with data governance tools such as Collibra or Alation.
Knowledge of data quality frameworks and standards, such as Data Quality Dimensions (completeness, consistency, etc.).
Familiarity with tools like Apache Beam or Luigi for managing complex data workflows.
Awareness of emerging data technologies such as data mesh, data fabric, and real-time data processing frameworks.