Logo for AWISEE

Senior Data Infrastructure Engineer (Scraping & Scale)

AWISEE

Role overview

Qualifications

  • 4+ years of experience in high-volume web scraping, data engineering, or reverse engineering
  • Deep experience defeating advanced anti-bot providers (Cloudflare, Akamai, PerimeterX)
  • Mastery of Python or Go, with deep knowledge of Playwright, Puppeteer, Scrapy, or Selenium
  • Experience with distributed systems and task queues (Temporal, Ray, Kafka, Redis)

Responsibilities

  • Build and manage stealth scraping clusters using residential proxy networks, TLS fingerprinting, headful/headless browser farms
  • Build fault-tolerant, scalable web-scraping pipelines that extract data from various platforms
  • Design distributed queues and workflow engines to manage millions of asynchronous scraping tasks daily
  • Architect structured and unstructured storage environments for downstream AI modeling

Key facts

Hard skills

About the company

AWISEE logo

AWISEE

Our link building and SEO agency is dedicated to helping businesses improve their online visibility and attract more traffic to their websites. With our team of experts, we provide a range of services that can help businesses boost their search engine rankings and drive more conversions. Our link building services involve creating high-quality backlinks from authoritative websites in your industry. Experts in International SEO and expanding presence in multiple geographical locations Our goal is to help you build a strong backlink profile that can help you rank higher in search results and attract more traffic to your website. In addition to link building, we also provide comprehensive SEO services that include on-page optimization, keyword research, and content strategy. Our team will work closely with you to identify the keywords and topics that are most relevant to your business, and develop a content strategy that can help you rank for those terms. We've established network of websites and media outlets in over 20 markets and 30 languages: Europe: Germany, France, Spain, Netherlands, Italy & more. Latin America: Brazil, Mexico, Columbia, Argentina, Chile. North America: USA & Canada. APAC: Australia, New Zealand, China, Japan, Korea & more. At our link building and SEO agency, we believe in delivering results for our clients. We use proven strategies and tactics to help you achieve your online marketing goals, and we provide regular reporting to help you track your progress. Contact us today to learn more about how we can help you improve your online visibility and drive more traffic to your website.

Company details

Links

LinkedIn See all jobs

Your match analysis

See how your profile stacks up against this role.

We compared the job requirements to your profile to show where you're strong and where you fall short.

Job description

This is a remote position.

We are looking for a Senior Data Infrastructure Engineer specializing in web scraping, anti-bot evasion, and large-scale data ingestion. In this role, you will build and maintain the core ingestion engine powering our social media data lab. You will overcome complex platform defenses to deliver millions of profile, post, and video records daily with near-zero downtime.

Responsibilities
  • Anti-Bot Evasion Architecture: Build and manage stealth scraping clusters using residential proxy networks, TLS fingerprinting, headful/headless browser farms (Playwright, Puppeteer), and session rotation.
  • High-Throughput Pipelines: Build fault-tolerant, scalable web-scraping pipelines that extract data from Instagram, TikTok, YouTube, X, and web sources.
  • Pipeline Orchestration: Design distributed queues and workflow engines (Temporal, Ray, Apache Kafka, Celery) to manage millions of asynchronous scraping tasks daily.
  • Storage & Data Lake Management: Architect structured and unstructured storage environments (Parquet, Apache Iceberg, Snowflake, S3) for downstream AI modeling.
  • Monitoring & Evasion Recovery: Implement automated alerts for platform UI/API changes, blocking patterns, and proxy failures.


Requirements

  • 4+ years of experience in high-volume web scraping, data engineering, or reverse engineering.
  • Deep experience defeating advanced anti-bot providers (Cloudflare, Akamai, PerimeterX) via TLS impersonation, browser automation, and proxy management.
  • Mastery of Python or Go, with deep knowledge of Playwright, Puppeteer, Scrapy, or Selenium.
  • Experience with distributed systems and task queues (Temporal, Ray, Kafka, Redis).
  • Strong SQL skills and experience with modern analytical data lakes / data warehouses.
Preferred Qualifications
  • Direct experience extracting short-form video content and user metadata from TikTok, Instagram, and YouTube.
  • Experience integrating scraping outputs directly into vector databases and real-time AI processing queues.


Apply once. Then go straight to the hiring manager.

After you apply, unlock the direct contact details of the people who actually make the call. A quick follow-up makes you 5x more likely to land an interview.

MR

Marcus Rivera

Chief Revenue Officer

m.rivera@company.com
linkedin.com/in/marcusrivera
Unlocked after you apply
·

Infrastructure Engineer Related jobs

WorldwideInfrastructure Engineer

Other jobs at AWISEE

Premium

Reach out to the hiring manager directly.

Gain access to the contact details of the hiring managers who actually decide, and reach out to network with them directly. That, plus more when you upgrade:

  • Full match report with fit score and gaps
  • Career diagnostics on how recruiters read you
  • Curated company matches and warm intros
  • 48h early access to new roles

Cancel anytime.