Logo for KATBOTZ

Databricks SME (Subject Matter Expert)

Role overview

Qualifications

  • 10+ years of experience in Data Engineering and Big Data technologies
  • Strong hands-on expertise in Databricks and Apache Spark
  • Experience with Python, PySpark, SQL, and Scala
  • Expertise in Delta Lake, Unity Catalog, and Databricks Workflows

Responsibilities

  • Lead the design, development, and optimization of Databricks-based data platforms and pipelines
  • Provide technical expertise on Databricks architecture, performance tuning, and best practices
  • Build scalable ETL/ELT workflows using Apache Spark and Databricks
  • Collaborate with business stakeholders, architects, and engineering teams to gather requirements and deliver solutions

About the company

KATBOTZ logo

KATBOTZ

Business Consulting & Services

Welcome to KATBOTZ: Empowering Business Transformation & Innovation Stay steps ahead of the competition with our cutting-edge business consulting services, expertly optimized for the latest technologies. Ignite growth and knowledge with our transformative education services, while securing top-tier talent for your organization through our unrivaled recruiting solutions. Business Consulting: We specialize in serving enterprise and mid-market customers in the life sciences, hi-tech, and consumer goods industries. Our core offerings include finance transformation, solution integration, data management, reporting and analytics, and program management. Additionally, we have strategic partnerships to support our clients in additional areas. Solutions: We are investing in cutting-edge solutions and technology innovations to bring rapid time-to-value for our customers. By leveraging automation, AI, and machine learning, we help address our clients' most pressing business challenges. Education: We share our expertise to empower the new workforce and business users with the latest knowledge in technology. Through our educational initiatives, we equip them with the skills and insights necessary for success in today's rapidly evolving business landscape. Talent: Finding top-tier talent can be challenging, but with our comprehensive recruiting solutions, you can trust that you'll access the best professionals in the industry. Furthermore, we offer coaching and mentoring services to support the growth and development of the new workforce. Our passion for making a real impact drives us to serve our customers with dedication and expertise. We differentiate ourselves by offering competitive prices, quality resources, and minimal overheads to provide our clients with the best value. Join us as we work to make business transformation a reality for our clients. Let's create a brighter future together. We value feedback, please submit here: https://forms.gle/vpj5qi3HxrQK1Aee6

Company details

Company typeSmall startup
IndustryBusiness Consulting & Services
Company size2 - 10

Your match analysis

See how your profile stacks up against this role.

We compared the job requirements to your profile to show where you're strong and where you fall short.

Job description

This is a remote position.

KATBOTZ is seeking an experienced Databricks SME to support enterprise-level data engineering and analytics initiatives. The ideal candidate will have deep expertise in the Databricks ecosystem, big data technologies, cloud platforms, and modern data architecture. The candidate will work closely with cross-functional teams to design, optimize, and implement scalable data solutions for global clients.

Key Responsibilities

  • Lead the design, development, and optimization of Databricks-based data platforms and pipelines.

  • Provide technical expertise on Databricks architecture, performance tuning, and best practices.

  • Build scalable ETL/ELT workflows using Apache Spark and Databricks.

  • Collaborate with business stakeholders, architects, and engineering teams to gather requirements and deliver solutions.

  • Implement Delta Lake, data governance, and data quality frameworks.

  • Support migration of legacy data platforms to Databricks.

  • Optimize data processing workflows for performance and cost efficiency.

  • Mentor development teams on Databricks, Spark, and cloud-native solutions.

  • Troubleshoot production issues and provide timely resolutions.

  • Ensure adherence to security, compliance, and coding standards.

Required Skills & Qualifications

  • 10+ years of experience in Data Engineering and Big Data technologies.

  • Strong hands-on expertise in Databricks and Apache Spark.

  • Experience with Python, PySpark, SQL, and Scala.

  • Expertise in Delta Lake, Unity Catalog, and Databricks Workflows.

  • Strong understanding of data warehousing and modern lakehouse architecture.

  • Experience with cloud platforms such as Microsoft Azure, Amazon AWS, or Google GCP.

  • Experience integrating Databricks with BI and reporting tools.

  • Knowledge of CI/CD pipelines and DevOps practices.

  • Strong analytical and problem-solving skills.

  • Excellent communication and stakeholder management abilities.


Preferred Qualifications

  • Databricks Certified Professional certification.

  • Experience in real-time streaming using Kafka or Structured Streaming.

  • Exposure to AI/ML workflows within Databricks.

  • Prior experience working with global remote teams.

Benefits
  • Competitive compensation package 
  • Opportunities for professional development and career advancement.
  • Flexible working conditions, with remote options available.
  • Dynamic and supportive work environment.
Equal Employment Opportunity

KATBOTZ LLC is an Equal Opportunity Employer. We provide equal employment opportunities to all qualified individuals, regardless of race, religion, gender, gender identity, age, marital status, national origin, sexual orientation, citizenship status, veteran status, disability, or any other legally protected status. As an organization, we are unwavering in our commitment to maintaining a discrimination-free work environment, and fostering a culture of inclusivity, belonging and equal opportunity for all employees and applicants.




Apply once. Then go straight to the hiring manager.

After you apply, unlock the direct contact details of the people who actually make the call. A quick follow-up makes you 5x more likely to land an interview.

MR

Marcus Rivera

Chief Revenue Officer

m.rivera@company.com
linkedin.com/in/marcusrivera
Unlocked after you apply
·

Related jobs

Other jobs at KATBOTZ

Premium

Reach out to the hiring manager directly.

Gain access to the contact details of the hiring managers who actually decide, and reach out to network with them directly. That, plus more when you upgrade:

  • Full match report with fit score and gaps
  • Career diagnostics on how recruiters read you
  • Curated company matches and warm intros
  • 48h early access to new roles

Cancel anytime.