GSA Data Engineer

Job Description

·       Engineerdata solutions in support of Sustainability reporting and analytics initiatives.

·       Engagewith product owner, analysts, visualization developers, and business partnersto understand capability requirements, and to develop and support datasolutions based on product backlog priorities.

Responsibilities
       General Purpose PythonProgramming:

·       Python hasbeen your primary coding language (daily use) for at least 3 years.

·       You have authoreddistributable Python packages (packages which can be built, installed, and distributedusing setuptools, pip, and twine)

·       You have asolid understanding of how pip dependency resolution works.

·       You are proficientin authoring and automating unit and integration tests for python packages using(minimally) unittests, pytest, and tox.

·       You are meticulousabout code quality, including readability, know your PEP8 guidelines inside andout, and are capable of authoring code which will pass validation by commonly usedstatic analysis tools including mypy and flake8.
Database Design and SQL

·       You areproficient in authoring readable, well-structured, SQL SELECT statements usingISO/ANSI-standard SQL.

·       You havehands-on professional experience in data warehouse design and modeling,including authoring DDL statements.

VersionControl and CI/CD

·       You have experiencewith trunk-based development (feature branching) using git for version control,with fully automated deployments (CI/CD).

Required Skillsets

GeneralPurpose Python Programming:

·       You have adeep understanding of python’s standard library and python internals. You understandpython memory management, how CPython implements built-in data structures, and whichdata structures are best suited for different scenarios.

·       You understandand can compare/contrast CPython’s built-in concurrency models, when to use each,and what obstacles might prevent the use of each mechanism.

DatabaseDesign, SQL, and Object Relational Models:

·       You are adeptat performance-tuning SQL queries for both OLAP and OLTP databases.

·       Youunderstand and are prepared to discuss how and when/where to utilize more esotericand/or modern SQL features such as window functions and common table expressions.

·       You understandand are prepared to discuss the performance implications of columnar vs relational databases.

·       You have firsthandexperience in managing database schema migrations (ideally using SQLAlchemy’s ORM+ Alembic).

Version Controland CI/CD

·       You have experiencewith trunk-based development (feature branching) using git for version control,with fully automated deployments (CI/CD).

Cloud Infrastructureand Amazon Web Services

·       You have firsthandexperience using boto3 to interact with Amazon Web Services’ resource APIs, particularlyAmazon S3 (Simple Storage Service).

·       You have hands-onexperience authoring unit and integration tests utilizing localstack to emulateAWS resources.

·       You have firsthandexperience using HashiCorp Terraform to manage cloud infrastructure.

·       You have firsthandexperience developing serverless ASGI applications using AWS lambda and AWS APIGateway.

Web API Serverand Client Development:

·       You haveexperience planning and executing the design and development of web APIs using amodern python ASGI framework (preferably FastAPI).

·       You have authored,validated, and maintained OpenAPI documents describing your web APIs accurately.
You have experience developing and testing python web API client librariesbased on an OpenAPI document.
Distributed Computing and Apache Spark

·       You haveexperience using Apache Spark for ingestion and manipulation of data sets whichare too large to process efficiently in-memory.

·       You have firsthandexperience translating algorithms and procedures designed by topical subject matterexperts, having varying levels of engineering experience, into well-designed datapipelines.

·       You have experienceconfiguring and tuning Spark clusters to optimize use of computing resources forvarying workloads.

·       You understandand can discuss when and why to use distributed computing frameworks, such asApache Spark, versus alternate concurrency models such as asyncio or multiprocessing.

DatabaseDesign, SQL, and Object Relational Models.

·       You have experiencemodeling databases using SQLAlchemy’s ORM framework.

Required Soft Skills

·       You are proficientin communicating effectively and efficiently within a hybrid remote/in-person teamstructure:

·       You are meticulousabout managing your calendar to accurately reflect your free/busy hours.

·       You respectand seek to learn digital communications etiquette—including region-specific, industry-specific,and organization-specific etiquette.

·       You proactivelyinitiate constructive discussions while curating and targeting your communicationswith respect for your colleagues’ time and schedules.

·       You are adeptat discovering and navigating the complex bureaucratic resources of a large organization.

Top 3 Skills

·       Python

·       SQL

·       Spark