Hands-on experience with Apache Spark for design, development, and deployment.
Preferred BFSI domain experience.
Requirements:
Translate business needs into technical specifications.
Design, build, and deploy large-scale data processing solutions.
Collaborate with teams to integrate systems and develop analyses.
Create and update technical documentation.
Job description
Apache spark
Skillset : Apache Spark Designer
Education BE, MCA, MSc or equivalent( valid degree)
8 or more years of professional IT experience, including the majority of the following (specific experiences can be concurrent):
BFSI domain experience is preferred
JD :
Tech Mahindra Ltd., Mumbai looking for a Apache Spark Designer to create and manage large scale data processing solutions. In this role, Candidate should have a background in data and business analysis. Should have hands on Apache Spark where you have worked on design, development and deployment on cluster of servers. Also have data related processing problemsolving aptitude.
Responsibilities
Translate business needs to technical specifications
Architecting, Design, build and deploy data processing solutions
Hands on tools to process, stage, store data Conduct recon of data
Collaborate with teams to integrate systems Develop and execute Spark Queries, database queries and conduct analyses
Create dataset, RDD, Data Frame for requested projects Develop and update technical documentation
Sound knowledge about effort estimation methods for data processing solutions requirements Proven experience as Apache Spark Designer.
Handson coding experience for Apache Spark designing
Background in data processing design (e.g. large scale data processing with API based status) and data curation
Indepth understanding of database management systems, Spark clustering, Spark executor, Driver, real time data processing platform, Java 1.8, and ETL (Extract, transform, load) framework
Knowledge of SQL queries, Spark Engine and Spark Services, Experience of Spark query tuning and performance optimization
Soft skills
Proven abilities to take initiative and be innovative
