Lead Data Engineer

This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Lead Data Engineer based in India.

• As a Lead Data Engineer, you’ll architect and own the end-to-end data sourcing and ETL infrastructure powering a large-scale B2B data platform.
• You’ll transform complex, unstructured web data into accurate, structured datasets across multiple international markets.
• The role combines advanced data engineering, web extraction, LLM infrastructure, agentic workflows, and production-grade orchestration.
• You’ll establish the technical standards and frameworks that enable other engineers to build reliable, scalable extraction systems.
• A significant part of the role is hands-on, tackling difficult problems such as anti-bot defenses, JavaScript-heavy sources, schema drift, and cost optimization.
• You’ll also shape how AI is evaluated and operated in production through versioning, observability, labelled datasets, and precision and recall measurement.
• Working within a lean, senior, AI-native environment, you’ll have substantial autonomy and direct ownership of architecture, quality, performance, and outcomes.