Scala Data Engineer

Index Analytics, LLC, is a rapidly growing Baltimore-based small business providing health related consulting services to the federal government. At the center of our company culture is a commitment to instilling a dynamic and employee-friendly place to work.  We place a priority on promoting a supportive and collegial team environment and enhancing staff’s experience through career development and educational opportunities.

Index Analytics is seeking a Scala Data Engineer to support Government clients in the Baltimore and Washington D.C. Metro Area. The Data Engineer will provide O&M support and contribute to building new implementations for thousands of users spread across the country.

Responsibilities

• Responsible for Data engineering tasks focused on AWS analytics products (such as Redshift and Hive) and open-source data engineering technologies (such as Apache Spark and Hadoop) using strong SQL and Scala programming skills.
• Design and build ETL pipelines to automate the ingestion of structured and unstructured data using Scala and Spark. Design and build data processing pipelines using tools and frameworks in the Hadoop and AWS ecosystem.
• Orchestrate data workflows for reuse and sharing using Apache Airflow, Nifi and Jenkins 
• Optimize ETL processes for scalability and efficiency.
• Identify and resolve performance bottlenecks in data processing systems.
• Ensure data governance and compliance with industry standards and regulations.
• Analyze requirements and architecture specifications to create a detailed design document.

Qualifications

Qualifications

• US citizen or Authorized and lived in the US for 3 of the last 5 years.
• Must be able to obtain a U.S. Federal government client badge and pass a government Public Trust background investigation.
• Bachelor’s degree or higher in computer science or relevant discipline required.
• 5 + years of overall work experience.
• 3 years’ experience with developing and operating data applications within AWS using Scala required.
• Experience with programming and database scripting.
• Experience in Big Data and working in a data engineering role.
• Experience building a proper path to production leveraging multiple lifecycles, testing, integration, and CI/CD pipelines.
• Experience running, deploying, and maintaining production cloud infrastructure in AWS.
• Experience with configuration management tools.
• Experience with ETL/ELT design and implementations in the context of large, disparate, and complex datasets.
• Demonstrated experience with a variety of relational database and data warehousing technology such as AWS Redshift and Databricks -high priority. 
• Familiarity with Oracle, MySQL, and PostgreSQL to support Commercial Off-the-Shelf (COTS) products, but not required.
• Other tools used are Tivoli Work Scheduler (TWS) to load data from Databricks into Redshift, Cognos, and SAS interaction with the data infrastructure.

Index Analytics provides equal employment opportunities to all employees and applicants for employment and prohibits discrimination and harassment of any type without regard to race, color, religion, age, sex, national origin, disability status, genetics, protected veteran status, sexual orientation, gender identity or expression, or any other characteristic protected by federal, state or local laws.

This policy applies to all terms and conditions of employment, including recruiting, hiring, placement, promotion, termination, layoff, recall, transfer, leaves of absence, compensation and training.

Attention Candidates

We're dedicated to ensuring a safe and transparent recruitment process for all candidates and have implemented robust measures to protect your personal information. Please be aware that all employment-related communications will originate from a secure portal (NAME@msg.paycomonline.com) or a corporate email address (NAME@index-analytics.com). If you have any concerns, please don't hesitate to reach out to us at recruiting@index-analytics.com.