Education Requirements, Skills, and Knowledge
- Good problem solving and decision-making skills; ability to understand and analyze complex issues
- Self-motivated, detail orientated, highly organized and able to handle a variety of tasks and responsibilities in an efficient manner with a high level of quality
- Exceptional analytical and conceptual thinking skills.
- The ability to influence stakeholders and work closely with them to determine acceptable solutions.
- Bachelor's Degree in Computer Science, Information Systems, Engineering, or a related field (or equivalent experience).
- 5+ years of experience as a Data Engineer or in a similar role, with a strong focus on Microsoft Azure technologies.
- Proficiency with Azure Data Tools: Azure Data Factory, Azure SQL, Azure Data Lake, Azure Synapse Analytics.
- Expertise in SQL for data manipulation, optimization, and query building.
- Knowledge of programming languages such as Python or .NET for data processing and automation tasks.
- Experience with cloud-based infrastructure management and infrastructure-as-code (e.g., ARM templates, Terraform).
- Familiarity with CI/CD pipelines and version control tools (e.g., Git, Azure DevOps).
- Strong understanding of data modeling, data architecture, and data warehousing concepts.
- Knowledge of Azure security best practices and experience with data governance in cloud environments.
- Excellent problem-solving skills and a keen attention to detail.
Responsibilities:
· Data Pipeline Development: Design, develop, and maintain scalable and reliable ETL (Extract, Transform, Load) processes using Azure Data Factory, Data Lake, and other Microsoft tools.
· Data Warehousing: Implement and optimize data storage solutions using Azure SQL Data Warehouse, Azure Synapse Analytics, and/or Azure Data Lake Storage.
· Data Integration: Collaborate with other teams to integrate data from various sources, including on-premise systems, cloud services, and third-party APIs into Azure cloud infrastructure.
· Performance Optimization: Monitor and optimize the performance of data pipelines and databases to ensure minimal downtime, reduced latency, and efficient resource use.
· Data Governance & Security: Ensure data governance standards, including data quality, security, and privacy, are adhered to by utilizing Azure security tools such as Azure Key Vault and Azure Active Directory.
· Collaboration: Work with cross-functional teams, including data scientists, analysts, and software engineers, to understand business needs and deliver tailored data solutions.
· Automation & Monitoring: Set up automated data workflows, monitoring dashboards, and alerts using Azure Monitor, Azure Logic Apps, and PowerShell scripts.
· Documentation: Maintain thorough documentation of data models, pipeline configurations, and operational processes.
