X, bigly labs
Lead Data Engineer
Hybrid (Melrose Arch)
hirly Pro
Get in front of the people hiring at X, bigly labs
- Reach the hiring manager at X, bigly labs on LinkedIn, from your own account
- hirly applies to this job for you, with unlimited applications
- A resume and cover letter written for this job, unlimited
- The hiring manager and recruiter at X, bigly labs, with contact details
$49 a month · ₹1,999 for 30 days in India · cancel any time
Get past the screening software and onto a recruiter's desk
hirly rewrites your resume for this job — matching the keywords and skills in the posting, moving your most relevant experience to the top, and writing a cover letter to fit. About 30 seconds.
- Keywords matched to this posting
- Fit score before you apply
- Cover letter included
Matched against 2.8M live jobs from 220,000+ employers in 200+ countries.
Tailor my resume for this job →Apply from your AI assistant
Connect hirly to Claude or ChatGPT and ask it to apply to this job. hirly tailors your resume, fills the employer’s form and asks before sending.
Some employer sites stop an application at a CAPTCHA or sign-in and hand it back with a link. Applying needs a paid plan. Works with any assistant that supports MCP.
hirly's read of this role
- Role family
- Data & ML
- Seniority
- Lead / management
- Country
- ZA
- Work mode
- On-site / unstated
- First seen by hirly
- 8 Nov 2026
Derived automatically from the posting. Upload your resume above to see how the role scores against it.
the posting
We are X, bigly labs, Dis-Chem’s high-performance innovation hub; a place where bold ideas meet data, design, and radical customer focus to reimagine what healthcare can be.
Our mission is to power the future of healthcare by lowering costs, unlocking new possibilities, and improving access to healthcare for all South Africans. We do this through cutting-edge digital solutions that deliver smarter, more human, and truly patient-focused experiences. Here, your work is not limited to a whiteboard; it becomes real. It drives decisions, shapes experiences, and moves healthcare forward. We are driven by one big question: How do we use data and today’s technology to build breakthroughs to better health?
There’s only one question left: are you ready to make healthcare smarter, and actually mean it?
We are seeking a Lead D ata E ngineer to lead the develop ment of a modern, scalable, and secure data platform that supports enterprise-wide analytics, machine learning, and AI initiatives. The Lead Data Engineer will be responsible for managing data delivery teams as well as design, build, maintain and optimis e data p latforms while enabling governed data sharing with external partners. This role will work closely with data analysts, data scientists, and AI engineers to ensure the scalable, secure, and high-performance data platforms that power analytics, AI, machine learning and operational excellence required by business teams .
WHAT WE ARE LOOKING FOR?
Minimum
Bachelor’s degree in Computer Science, Data Engineering, Information Systems, or a related field.
Certification in cloud platform: AWS, Azure Data Engineer (DP-203/DP-700), or Databricks Certified Data Engineer.
8+ years of experience in data engineering or software development.
4+ years of experience in leading data teams or data program management.
3+ years of hands-on experience with Databricks Intelligence Platform and implementing Medallion Architecture.
Strong proficiency in Python, SQL, Spark, and cloud-native data services (e.g., Azure Data Lake, AWS S3, GCP BigQuery).
Familiarity with cloud-native data integration tools (e.g. SAP Datasphere, Snowflake Airflow).
Experience in delivery and collaboration tools: GitHub, JIRA, Confluence and MIRO
Proven experience in developing data platforms that support data science, data analytics and AI workloads.
Extensive experience in ETL development and Data Pipelines with CI/CD, as well as data governance, and data quality frameworks.
Solid understanding of data science workflows, model lifecycle, and AI concepts.
Preferred Qualifications
Certification in Machine Learning or AI (e.g., Azure AI Engineer Associate).
Experience in healthcare, retail, or insurance data ecosystems
Working knowledge of SAP BW, S4 HANA, CRM, ERP, and Data Business Cloud
WHAT YOU WILL BE DOING?
Lead the development and maintenance of a scalable and secure data platform using the Medallion Architecture to support analytics, data science, and AI workloads with high performance and reliability.
Build robust, reusable, and modular data pipelines leveraging Python, SQL, Spark, and cloud-native services to ensure consistent, timely, and accurate data delivery across the organisation.
Enable secure and governed inbound and out bound data sharing with external partners by applying access controls, encryption, and data contracts to ensure compliance, trust, and seamless collaboration.
Collaborate with data analysts, data scientists, AI engineers and data quality anal ysts through regular engagement, feedback sessions, and shared development environments to ensure the platform meets analytical and modelling needs effectively.
Ensure data integrity, privacy, and compliance by implementing automated data quality checks, lineage tracking, and governance frameworks to maintain trust and meet regulatory standards (e.g., POPIA, GDPR).
Establish coding and architectural best practices by creating reusable templates, guidelines, and peer review processes to improve code quality and team efficiency.
Monitor platform performance and pipeline health using observability tools and alerting mechanisms to proactively resolve issues and maintain SLAs.
WHO YOU ARE?
Have an advanced knowledge of AWS, Databricks and Apache Spark.
Be an expert in Medallion Architecture (Bronze, Silver, Gold layers).
Be able to balance multiple initiatives and deadlines with structured focus
The ability to help coordinate timelines, deliverables, and interdependencies across teams
Have work delivery experience using agile principles and collaboration tools: GitHub, JIRA, Confluence and MIRO
Be able to adapt effectively to shifting priorities, evolving goals, and dynamic business contexts.
Our values show up in how we think, build, and make choices that make an impact. We are building a culture that drives progress; one that fixes friction, chases what changes, and owns the outcome. We design with tomorrow in mind, ask sharper questions, and answer them with care, urgency, and systems that scale.
Think you’ve got energy, curiosity, and guts? Come hold the space, make it matter, and own a breakthrough.
Listed on hirly, a job board. hirly is not the employer: X, bigly labs is hiring for this role.
Similar jobs
- Senior Data EngineerVmlenterprisesolutions · Cape Town, South AfricaFirst seen 25d ago
- Data EngineerTFG · Western Cape, South AfricaFirst seen 39d ago
- Data EngineerWundermanthompson · Cape Town, Western Cape, South AfricaFirst seen 49d ago
- Data EngineerVmlenterprisesolutions · Cape Town, South AfricaFirst seen 49d ago
- Data Engineerooba · South AfricaFirst seen 50d ago
Browse similar roles
Want this one?
Upload your resume and hirly rewrites it for this job and writes the cover letter — in about thirty seconds, before you sign up.
Tailor my resume for this job