Java Spark Engineer

**Will prioritize candidates in New York but will consider remote candidates**



About the Job:

As a member of the development team, you will help define, establish, and evolve our development capacities. You will be tasked in creating innovative solutions that advance our clients businesses. You'll join a strong and inspiring team of technologists dedicated to improving the design, analytics, development, coding, testing and application programming that goes into creating high-quality software and new products. You'll be tasked with keeping the team and other key stakeholders up to speed on the progress of what's being developed.

What You'll Get to Do:

You will contribute to creating one of our most ambitious and critical technology projects. You will participate in technical and architectural conversations with peers and experts to design the platform together. Additionally, will work with a team of other highly motivated data engineers, product managers, SDETs, etc.

• Work collaboratively across the organization
• Solve challenging problems in an elegant but pragmatic way
• Write high quality, reusable code and adhere to coding standards
• Write professional, clear, and comprehensive documentation
• Apply best development and DevOps practices 
• Be aware of on-going technology trends and know how to  leverage their benefits

What you'll bring with You:

• 5+ years experience using Java and/or Scala
• 4+ years of Apache Spark is a must
• Microsoft Azure is a must / ADF experience is a plus 
• Solid understanding of Spark/Databricks batch and streaming solutions (Delta Lake, Lakehouse) is a plus
• Implemented robust products, services and capabilities for the data platform
• Knowledge of Azure Data Factory
• Knowledge of Kafka, Event hubs and ADLS Gen2 on Microsoft Azure
• Knowledge of TDD, BDD and how to integrate within the technologies described above