Location: Washington, DC 100% Onsite
Clearance: Top Secret/SCI
Job Type: Full-time
Target Salary Range*:$105,000 - $125,000
*This represents the potential salary range for this position depending on education level, years of experience and/or certifications in addition to other position specific requirements which may impact salary
Position Overview
The Data Engineering Software Developer (Junior) supports the design, development, integration, and maintenance of scalable data systems and pipelines used for analytics, reporting, and operational insight. This role works with enterprise data sources, cloud and container platforms, data-processing technologies, and DevOps tooling to help ingest, transform, secure, and deliver reliable data across complex DoD and DISA environments.
The position supports solutions using Confluent Kafka, Elastic Stack, AWS, Red Hat OpenShift/Kubernetes, data warehouses, and data lakes while collaborating with software developers, analysts, visualization engineers, and other technical stakeholders.
Key Responsibilities
Data Pipeline Development and Integration
- Design, build, and maintain scalable and reliable data systems and pipelines to ingest, process, and transform large, complex, and disparate data sources for analytics and reporting.
- Unify and integrate data from multiple network operations systems into a consistent and accessible platform.
- Create and maintain Kafka, Elastic, and Logstash pipelines supporting enterprise data ingestion and processing.
- Automate ETL/ELT processes, data validation, cleansing, monitoring, and related data-processing activities.
Data Platforms and Software Development
- Implement data solutions using Confluent Kafka, Elastic Stack, AWS, and Red Hat OpenShift/Kubernetes.
- Develop and manage data warehouses and data lakes supporting analytics, reporting, and operational dashboards.
- Support Kibana visualizations and dashboards using React, JavaScript, and HTML.
- Apply automation and development best practices to improve platform scalability, maintainability, and reliability.
Performance and Troubleshooting
- Analyze and optimize ingestion throughput, query performance, indexing strategies, and storage efficiency across Elastic and Kafka platforms.
- Troubleshoot and resolve issues related to data ingestion, processing, storage, memory, partitioning, and cluster performance.
- Support infrastructure optimization for performance, scalability, and cost effectiveness.
Security, Compliance, and DevOps
- Ensure data quality, accuracy, security, and compliance with applicable DoD and DISA requirements.
- Support CI/CD pipelines for automated build, test, and deployment activities.
- Apply version control and DevOps practices to data-engineering workflows.
- Support secure data access and platform configuration in accordance with applicable security requirements.
Collaboration and Documentation
- Collaborate with data analysts, visualization engineers, software developers, and other stakeholders to deliver integrated and actionable data solutions.
- Document data architecture, integration patterns, platform configurations, and operational procedures.
- Develop and maintain technical documentation and runbooks supporting data platforms and pipelines.
Qualifications
Education
Experience
- Hands-on experience designing, building, and maintaining ETL/ELT data pipelines and integrating multiple disparate data sources. [Required]
- Experience with data warehouse and/or data lake technologies such as AWS Redshift, Amazon S3, Hadoop, Snowflake, or similar platforms. [Required]
- Experience with Elastic Stack, including Elasticsearch, Logstash, and Kibana, and/or Kafka for data ingestion and processing. [Required]
- Experience with Linux/UNIX system administration and automation. [Required]
- Experience with CI/CD pipelines, containerized pipelines, version control technologies such as Git and Bitbucket, and DevOps practices. [Required]
- Familiarity with network operations data, including logs, metrics, events, ticketing, CMDB data, and related data models.
Skills
- Proficiency with data engineering tools and languages such as Python, Java, SQL, and shell scripting. [Required]
- Knowledge of data security, access controls, and compliance frameworks, including DoD, DISA, RMF, and STIG requirements. [Required]
- Strong communication and documentation skills supporting collaboration and knowledge sharing. [Required]
Certifications
Clearance
Other Requirements
Preferred Qualifications
- Experience supporting DISA, DISN, or other DoD network environments.
- Experience developing and deploying software applications that meet DoD security standards, including applicable STIGs.
- Knowledge of network security, encryption, and access controls within classified environments.
- Experience with big data technologies and platforms such as Kafka, Spark, NiFi, Hadoop, Elasticsearch, Logstash, Databricks, and ELK Stack.
- Experience applying big data technologies to text mining, summarization, search, and entity extraction.
- Experience with cloud-based data platforms, including AWS, Azure, GCP, or AWS GovCloud.
- Experience with cloud-integrated platforms and Infrastructure as Code, including networking and security policies.
- Familiarity with Kubernetes deployment, platform upgrades, and patching.
- Experience with configuration management technologies such as Ansible, Puppet, or Chef.
- Experience with monitoring, alerting, and observability technologies such as Prometheus, Grafana, and Elastic.
- Experience with business intelligence and visualization tools such as Kibana.
- Experience developing and maintaining technical documentation and operational runbooks.
- Strong understanding and practical experience with Agile methodologies, including Scrum and SAFe.
- Experience with Atlassian Jira and Confluence.
- Confluent Developer certification.
- Elastic Certified Engineer certification.
- Experience working remotely with geographically dispersed teams and within matrixed organizations.
- Experience combining software development, integration, and data engineering practices.
- Experience applying knowledge of system architecture, networking, and centralized logging using ELK to support data transformation initiatives.
