Data Engineer – Web Scraping, LLM Pipelines and Scalable Data Infrastructure

The Role: You’ll help build the data foundation of our product: high‑volume web scraping systems, structured datasets and LLM‑driven processing pipelines. The role combines hands‑on engineering with architectural thinking and suits someone who enjoys turning messy web data into reliable, scalable outputs. Key Responsibilities: Build new structured datasets, including scraping accelerators, Form D filings and dynamic web sources. Develop automated ETL pipelines that parse, clean and transform content using LLMs. Define and maintain database schemas in Supabase or PostgreSQL. Create evaluation frameworks to measure and compare LLM performance across pipeline components. Contribute to the design of scalable data architectures using GCP services.…