Bachelor’s degree in Computer Science or equivalent work experience
Minimum 10+ years’ experience in IT
8+ years Proficiency in Python libraries (Pandas, NumPy, etc.)
4+ years Strong SQL skills
2+ years Strong AWS skills
2+ years’ experience in Data Warehouse technologies
2+ years ETL tools experience
1+ year in Hadoop, Hive
Requirements:
Implement data pipelines, data curation, and data modeling
Design and develop data pipelines using Python frameworks
Collaborate with engineers and Data Scientists
Diagnose and solve issues in existing data pipelines
Job description
This is a remote position.
Responsibilities
Python programmers/developers who done extensive hands-on work in data engineering space
Willingness to quickly learn and adapt.
Experience in solutioning and implementing data pipelines, data curation, data modeling and implementing data solutions.
Strong understanding of different type of data and the lifecycle of data.
Design, develop, and launch extremely efficient and reliable data pipelines using Python frameworks to move data and to provide intuitive analytics to our partner teams.
Collaborate with other engineers and Data Scientists to Client for the best solutions.
Diagnose and solve issues in our existing data pipelines and envision and build their successors.
Requirements
Required Qualifications
Bachelor’s degree in Computer Science or equivalent work experience
Minimum 10+ years’ experience in IT
8+ years Proficiency working with Python specifically related to data processing with proficiency in Python Libraries - Pandas, NumPy, PySpark, PyOdbc, PyMsSQL, Requests, Boto3, Simple Salesforce, Json.
4+ years Strong SQL (SQL, performance, Stored Procedures, Triggers, schema design) skills and knowledge of one of more RDBMS like MSSQL/MySQL.
2+ years Strong AWS skills using AWS Data Exchange, Athena, Cloud Formation, Lambda, S3, AWS Console, IAM, STS, EC2, EMR
2+ years’ experience in Data Warehouse technologies like Snowflake/Spark/Databricks/ Informatica
2+ years ETL tools like Apache Airflow/AWS Glue/Azure Data Factory/Talend/Alteryx
1+ year in Hadoop, Hive
Excellent verbal communication skills.
Knowledge of DevOps/Git for agile planning and code repository