Logo for CodersBrain

Bigdata_shailja

CodersBrain

Roles & Responsibilities

  • Proficiency with Hadoop ecosystem technologies such as HDFS and Spark.
  • Good knowledge of Big Data querying tools like Hive and Impala.
  • Experience with data ingestion using Spark for various file formats including JSON, CSV, and Parquet.
  • Strong programming skills in Python and experience with Azure DataBricks and related tools.

Requirements:

  • Support and implement Hadoop production systems.
  • Translate complex requirements into detailed data designs.
  • Load and process data from various sources to meet business needs.
  • Monitor and optimize data performance while maintaining security and privacy.

Job description

Big Data + Azure DataBricks

Primary Skills:

Proficiency with Hadoop ecosystem technologies HDFS and Spark.

Good knowledge on Big Data querying toolsHive and Impala

Working Knowledge of data ingestion using spark for supporting various file formats like JSON, CSV and parquet with its compression techniques.

Perform analysis of vast data stores and uncover insights

Experience in Shell Script

Data Engineer with hand on experience in execution of data project on Azure (Migration or build)

Azure Databricks, ADF and Power BI, Git, Azure Devops must have skill

Secondary Skills:

Strong programming skills in Python

Roles and Responsibilities:

Hadoop production support and implementation.

Translate complex functional and technical requirements into detailed design.

Loading from disparate data sets and processing the data as per business need

Organizing data into tables, performing transformations, performance tuning and simplifying complex queries with Pyspark.

Monitoring data performance and modifying infrastructure as needed

Maintain security and data privacy.

Propose best practicesstandards.

Work on transformation roadmap Hadoop stack to Azure Databricks
Ready to apply?
APPLY
·

Related jobs

India

Other jobs at CodersBrain

We help you get seen. Not ignored.

We help you get seen faster — by the right people.

🚀

Auto-Apply

We apply for you — automatically and instantly.

Save time, skip forms, and stay on top of every opportunity. Because you can't get seen if you're not in the race.

✨

AI Match Feedback

Know your real match before you apply.

Get a detailed AI assessment of your profile against each job posting. Because getting seen starts with passing the filters.

Upgrade to Premium. Apply smarter and get noticed.

Upgrade to Premium

Join thousands of professionals who got noticed and hired faster.