Logo for Awign

Lead Data Engineer

Role overview

Qualifications

  • 10+ years of experience in Data Engineering
  • Proficiency in T-SQL and Databricks
  • Basic knowledge of C# and SQL
  • Bachelors’s degree in Computer Science

Responsibilities

  • Design and implement data ingestion pipelines using Azure Data Factory
  • Analyze and optimize performance metrics for data ingestion pipelines
  • Provide technical leadership and mentorship to data engineering teams
  • Collaborate with cross-functional teams including architects and analysts

Key facts

Hard skills

Key skills for this role

Other skills

  • Problem Solving
  • Collaboration

About the company

Awign logo

Awign

  • Outsourcing & BPO
  • Scaleup
  • 201–500
Awign is India's largest on-demand work fulfilment platforms, helping enterprises run their business at scale through end-to-end management and outcome-based execution of core business functions. This is done through our mobile and distributed community of 1.3mn+ gig partners across 12,000+ pin codes in India. Our tech-driven solution allows enterprises to variabilize their fixed costs, optimize toplines and focus on business profitability. We work with enterprises to take up recurring business functions, such as auditing, due diligence, invigilation, proctoring, last-mile delivery, new business development, telecalling along with content & data operations. Our division for high-skill gig talent, Awign Expert helps enterprises hire high-skill talent on a contractual basis and manage the entire lifecycle of those professionals.We are on a mission to uplift lives of 100mn people in India by providing them a platform to leverage and grow their skills. Awign enables gig partners to expand their income potential beyond the bounds of time, location and skills. Gig partners can choose between multiple gig opportunities across varied skill sets according to their own time and location-based flexibility. One platform, infinite avenues. #AwignEkMaukeAnek

Your match analysis

See how your profile stacks up against this role.

We compared the job requirements to your profile to show where you're strong and where you fall short.

Job description

This is a remote position.

Screening Checklist:
Proficiency in interpreting data transformation logic written in T-SQL and implementing equivalent processes within Databricks
Ability to design and implement data ingestion pipelines using Azure Data Factory (from source to RAW layer)
Basic knowledge of C# and Sql(atleast read the coding, no need to write)
Experience in collecting and analyzing performance metrics to optimize data ingestion pipelines
Competence in performing performance optimizations for Databricks read/write queries as needed
Strong motivation and the ability to provide guidance to other data engineers within the team

Job Overview
We are currently looking for highly accomplished Senior Data Engineers (10+ years of experience)
with deep expertise in Databricks and PySpark to play a key role in an ongoing, large-scale data
transformation initiative. The ideal candidates will bring extensive hands-on experience, strong
architectural understanding, and the ability to lead and influence data engineering practices across
teams.
Key capabilities and expectations include:
• Extensive experience in analyzing, interpreting, and modernizing complex data
transformation logic written in T-SQL, and implementing optimized, scalable equivalents within
Databricks using PySpark.
• Proven expertise in architecting and implementing end-to-end data ingestion frameworks
using Azure Data Factory, managing data flows from multiple source systems through the RAW
and subsequent data layers, with a strong focus on reliability and scalability.
• Strong experience in defining, capturing, and analyzing performance metrics, enabling
proactive monitoring, bottleneck identification, and continuous optimization of data ingestion
pipelines.
• Demonstrated ability to perform advanced performance tuning and optimization of Databricks
read/write operations, including partitioning strategies, caching, file formats, and query
optimization techniques.
• Capability to provide technical leadership and mentorship to data engineering teams, establish
best practices, conduct design reviews, and drive engineering excellence.
• Strong problem-solving skills, a results-oriented mindset, and the ability to collaborate effectively
with cross-functional teams, including architects, analysts, and stakeholders.
This role demands a senior engineering mindset with a balance of deep technical expertise,
architectural ownership, and people leadership to deliver scalable, high-performance data solutions in an
enterprise environment.
Educational qualifications:
Bachelors’s degree or Computer CIENCE

Related jobs

See all Data Engineer jobs

Other jobs at Awign