Senior Data Engineer

Description

• Equal Experts is an innovative consultancy specialising in the delivery of custom software solutions for
• blue-chip enterprise and public sector clients across a range of industry sectors. We work on important
• projects for our clients, deliver market-leading propositions across the digital, online and mobile
• channels, and are recognised for our leadership in the application of agile and lean delivery methods to
• assure delivery. We have offices in Australia, India, South Africa, the UK, the EU and the USA.

• We’re looking for Data Engineers with proven senior-level experience to join the Equal Experts
• network.

• Our Data Engineers are highly skilled in creating data pipelines for ingest, transformation and storage of
• data, usually at scale, and are familiar with standing-up analytics clusters.
• This role involves using modern agile technical practices - including Continuous Integration and
• Deployment, TDD, and XP - to deliver timely and pragmatic solutions, as well as helping others to do
• their jobs in a more effective way.

• This role involves working in teams that use modern agile technical practices - including continuous
• integration and deployment and fast feedback loops - to deliver timely and pragmatic solutions, as well
• as helping others to do their jobs in a more effective way.

Responsibilities

• ● Working with the client and the team to determine data processing and access needs and how
• best to deliver them.
• ● Creation and support of highly available data pipelines and storage solutions (e.g. data
• warehouses).
• ● Delivery of software using agile (pair programming, TDD, CI/CD, etc) practices.
• ● Automation of data infrastructure and deployments.
• ● Advocating agile practices to the client organisation.
• ● Mentoring client team members.
• ● Helping improve the team and clients’ data capabilities.

Required Skills

• ● Significant experience working on data pipelines, platforms and projects.
• ● Experience in developing solutions in one or more of the main Cloud providers (AWS, GCP,
• Azure).
• ● Strong background in at least one programming language, Python or Scala preferable.
• ● Working with a variety of databases, data warehouses and distributed file systems in production
• environments.
• ● Strong experience in SQL and data modeling.
• ● Exposure with data processing at scale.
• ● Knowledge about GDPR and data security. Working with source control and infrastructure as code (Git/Github, Terraform, ansible etc.) and CI/CD tools.
• ● Understanding and being able to support data scientists or business intelligence teams - in data
• prep, mentoring and support functions.
• ● Strong collaboration and communication skills, including the ability to communicate effectively
• while working remotely with globally distributed teams.