POS-4028
We are seeking a Senior Data Engineer to join our team. In this role, you will work with various external data sources, design and maintain data pipelines, and ensure the efficiency and security of our data processes. Your contributions will help HubSpot leverage data-driven insights for effective decision-making.
This role resides within our Employee Technology group which comprises Business Systems Analysts, System Engineers, Analytic Engineers and administrators responsible for HubSpot's employee-facing application infrastructure.
In this role you will:
Data Acquisition and Ingestion
- Acquire data from multiple sources using pre-built connectors and custom solutions.
- Develop data ingestion pipelines to effectively manage internal and external API interactions, addressing challenges such as rate limiting, pagination, and retry mechanisms.
- Process and securely ingest large datasets, ensuring encrypted data transfer and runtime credential safety.
- Implement Snowpipe for data ingestion into Snowflake and utilize Snowflake’s external tables for accessing S3 data.
AWS Infrastructure and File Management
- Manage and optimize data workflows within the AWS ecosystem, with a focus on S3 operations.
- Manipulate S3 files upon arrival, archive older files to Glacier, and do light cleansing/alteration of files, as needed, for various workflows.
- Automate AWS processes using tools like AWS Lambda, Step Functions, or custom scripts to improve efficiency and reliability.
Data Migrations and Utility Development
- Develop scripts for data migrations between Snowflake instances and automate tasks using Looker’s APIs.
- Create utility scripts for business needs, such as disabling inactive users, purging broken/unused Looker content, and simplifying otherwise burdensome/manual processes.
Data Modeling and Development
- Build and maintain data models using dbt, following best SQL practices and data warehousing principles to ensure schema design and data integrity.
- Develop data pipelines in a DAG structure, ensuring clear dependencies and performing data validation and testing.
- Implement CI/CD workflows with GitHub actions/hooks for automated testing and deployment.
We are looking for people who have:
- Proven experience in developing custom data ingestion pipelines and integrating with APIs, effectively managing complexities such as rate limiting and pagination.
- Proficient in automating workflows using AWS Lambda, Step Functions, or comparable tools.
- Extensive knowledge of databases (especially Snowflake.) Experience with Snowflake technologies such as Snowpipe, Snowpark, Data Sharing, Cloning etc., and Snowpipe usage.
- Strong background in SQL for data warehouse transformations. Experience with dbt is a plus.
- Solid understanding of data warehousing principles, DAG structures, and comprehensive data validation/testing frameworks.
- Skilled in Python for creating utility scripts and automated workflows.
- Experience with data manipulation libraries such as Pandas, PySpark, or similar technologies.
- Familiarity with streaming data solutions, including Kafka, AWS Kinesis, RabbitMQ, etc
Cash compensation range: 157600-236400 USD Annually
This resource will help guide how we recommend thinking about the range you see. Learn more about HubSpot’s
compensation philosophy.
The cash compensation above includes base salary, on-target commission for employees in eligible roles, and annual bonus targets under HubSpot’s bonus plan for eligible roles. In addition to cash compensation, some roles are eligible to participate in HubSpot’s equity plan to receive restricted stock units (RSUs). Some roles may also be eligible for overtime pay. Individual compensation packages are based on a few different factors unique to each candidate, including their skills, experience, qualifications and other job-related reasons.
We know that benefits are also an important piece of your total compensation package. To learn more about what’s included in total compensation, check out some of the
benefits and perks HubSpot offers to help employees grow better.
At HubSpot, fair compensation practices isn’t just about checking off the box for legal compliance. It’s about living out our value of transparency with our employees, candidates, and community.
We are seeking a Senior Data Engineer to join our team. In this role, you will work with various external data sources, design and maintain data pipelines, and ensure the efficiency and security of our data processes. Your contributions will help HubSpot leverage data-driven insights for effective decision-making.
This role resides within our Employee Technology group which comprises Business Systems Analysts, System Engineers, Analytic Engineers and administrators responsible for HubSpot's employee-facing application infrastructure.
In this role you will:
Data Acquisition and Ingestion
- Acquire data from multiple sources using pre-built connectors and custom solutions.
- Develop data ingestion pipelines to effectively manage internal and external API interactions, addressing challenges such as rate limiting, pagination, and retry mechanisms.
- Process and securely ingest large datasets, ensuring encrypted data transfer and runtime credential safety.
- Implement Snowpipe for data ingestion into Snowflake and utilize Snowflake’s external tables for accessing S3 data.
AWS Infrastructure and File Management
- Manage and optimize data workflows within the AWS ecosystem, with a focus on S3 operations.
- Manipulate S3 files upon arrival, archive older files to Glacier, and do light cleansing/alteration of files, as needed, for various workflows.
- Automate AWS processes using tools like AWS Lambda, Step Functions, or custom scripts to improve efficiency and reliability.
Data Migrations and Utility Development
- Develop scripts for data migrations between Snowflake instances and automate tasks using Looker’s APIs.
- Create utility scripts for business needs, such as disabling inactive users, purging broken/unused Looker content, and simplifying otherwise burdensome/manual processes.
Data Modeling and Development
- Build and maintain data models using dbt, following best SQL practices and data warehousing principles to ensure schema design and data integrity.
- Develop data pipelines in a DAG structure, ensuring clear dependencies and performing data validation and testing.
- Implement CI/CD workflows with GitHub actions/hooks for automated testing and deployment.
We are looking for people who have:
- Proven experience in developing custom data ingestion pipelines and integrating with APIs, effectively managing complexities such as rate limiting and pagination.
- Proficient in automating workflows using AWS Lambda, Step Functions, or comparable tools.
- Extensive knowledge of databases (especially Snowflake.) Experience with Snowflake technologies such as Snowpipe, Snowpark, Data Sharing, Cloning etc., and Snowpipe usage.
- Strong background in SQL for data warehouse transformations. Experience with dbt is a plus.
- Solid understanding of data warehousing principles, DAG structures, and comprehensive data validation/testing frameworks.
- Skilled in Python for creating utility scripts and automated workflows.
- Experience with data manipulation libraries such as Pandas, PySpark, or similar technologies.
- Familiarity with streaming data solutions, including Kafka, AWS Kinesis, RabbitMQ, etc
Cash compensation range: 157600-236400 USD Annually
This resource will help guide how we recommend thinking about the range you see. Learn more about HubSpot’s
compensation philosophy.
The cash compensation above includes base salary, on-target commission for employees in eligible roles, and annual bonus targets under HubSpot’s bonus plan for eligible roles. In addition to cash compensation, some roles are eligible to participate in HubSpot’s equity plan to receive restricted stock units (RSUs). Some roles may also be eligible for overtime pay. Individual compensation packages are based on a few different factors unique to each candidate, including their skills, experience, qualifications and other job-related reasons.
We know that benefits are also an important piece of your total compensation package. To learn more about what’s included in total compensation, check out some of the
benefits and perks HubSpot offers to help employees grow better.
At HubSpot, fair compensation practices isn’t just about checking off the box for legal compliance. It’s about living out our value of transparency with our employees, candidates, and community.
We know the confidence gap and imposter syndrome can get in the way of meeting spectacular candidates, so please don’t hesitate to apply — we’d love to hear from you.
If you need accommodations or assistance due to a disability, please reach out to us using this form. This information will be treated as confidential and used only for the purpose of determining an appropriate accommodation for the interview process.
At HubSpot, we value both flexibility and connection. Whether you’re a Remote employee, or work from the Office, we want you to start your journey here by building strong connections with your team and peers.
If you are joining our Engineering team in a full-time role, you will be required to attend a regional HubSpot office for in-person onboarding. If you join our broader Product team, you’ll also attend other in-person events such as HubSpot’s annual PEER week, your Product Group Summit, and other in-person gatherings to continue building on those connections.
If you require an accommodation due to travel limitations or other reasons, please inform your recruiter during the hiring process. We are committed to supporting candidates who may need alternative arrangements.
Germany Applicants: (m/f/d) - link to HubSpot's Career Diversity page here.
India Applicants: link to HubSpot India's equal opportunity policy here.
About HubSpot
HubSpot (NYSE: HUBS) is a leading customer relationship management (CRM) platform that provides software and support to help businesses grow better. We build marketing, sales, service, and website management products that start free and scale to meet our customers’ needs at any stage of growth. We’re also building a company culture that empowers people to do their best work. If that sounds like something you’d like to be part of, we’d love to hear from you.
You can find out more about our company culture in the HubSpot Culture Code, which has more than 5M views, and learn about our commitment to creating a diverse and inclusive workplace, too. Thanks to the work of every employee globally, HubSpot was named the #2 Best Place to Work on Glassdoor in 2022 and has been recognized for its award-winning culture by Great Place to Work, Comparably, Fortune, Entrepreneur, Inc., and more.
Headquartered in Cambridge, Massachusetts, HubSpot was founded in 2006. Today, thousands of employees across the globe work remotely and in HubSpot offices. Visit our careers website to learn more about the culture and opportunities at HubSpot.
By submitting your application, you agree that HubSpot may collect your personal data for recruiting, global organization planning, and related purposes. HubSpot's Privacy Notice explains what personal information we may process, where we may process your personal information, our purposes for processing your personal information, and the rights you can exercise over HubSpot’s use of your personal information.
