Databricks Architect

At Accellor, we are a trusted consulting partner that uses best-of-breed Cloud technology to deliver superior customer engagement and business effectiveness for clients. We bring a deep understanding of Financial, Retail, High Tech, Healthcare, and Retail verticals.

We’ve created an atmosphere that encourages curiosity, constant learning, and persistence. We encourage our employees to grow and explore their interests. We cultivate an environment of collaboration, autonomy, and delegation – we know our people have a strong work ethic and a sense of pride and ownership over their work. They are passionate, eager, and motivated – focused on building the perfect solution but never losing sight of the bigger picture.

We are seeking an experienced and highly skilled Databricks Architect to join our team. The ideal candidate will have a deep understanding of Databricks and cloud-based data architectures and possess hands-on experience in designing, implementing, and maintaining Databricks solutions for data processing, analytics, and machine learning. The role involves collaborating with cross-functional teams to drive the development and optimization of scalable and efficient data platforms using Databricks.

Responsibilities

• Design, implement, and maintain scalable and high-performance data architectures on Databricks, leveraging Spark, Delta Lake, and other cloud technologies.
• Collaborate with data engineers, data scientists, and business stakeholders to define data requirements and ensure Databricks-based solutions meet business needs.
• Develop and optimize ETL pipelines, ensuring data is processed efficiently and accurately.
• Provide technical leadership and guidance on best practices for Databricks, Spark, and cloud-based data solutions.
• Monitor and troubleshoot performance and scalability issues within Databricks environments, providing recommendations for improvements.
• Ensure the implementation of data governance, security, and compliance standards in all Databricks solutions.
• Work with cloud platforms (AWS, Azure, GCP) to integrate Databricks with other data services and storage solutions.
• Mentor junior team members and promote a culture of continuous learning and improvement.
• Stay up-to-date with the latest Databricks and cloud technologies, applying new techniques to enhance platform capabilities.

Requirements

• Proven experience as a Databricks Architect or similar role in data engineering or data architecture.
• Strong hands-on experience with Databricks, Apache Spark, Delta Lake, and cloud technologies (AWS, Azure, or GCP).
• Expertise in designing and optimizing data pipelines, data lakes, and cloud-based data architectures.
• Strong programming skills in languages such as Python, Scala, or SQL.
• Experience with distributed systems and big data processing frameworks.
• Knowledge of data governance, security, and privacy standards.
• Familiarity with machine learning workflows and tools (MLflow, TensorFlow, etc.) within Databricks.
• Strong communication and collaboration skills, with the ability to work across multiple teams and business units.
• Experience with CI/CD pipelines and automation tools in a cloud-based environment is a plus.
• Bachelor's degree in Computer Science, Engineering, or related field (or equivalent work experience).

Benefits

We strive to offer benefits that support the diverse needs of our employees.

Our package includes perks like flexible and discretionary time off, healthcare coverage for you and your loved ones, and a 401k plan with a company match to help you plan for the future.

Additionally, we offer access to flexible spending and health savings accounts, life and AD&D insurance, and opportunities for professional development.