Quality Engineer

Role: Quality Engineer

Location: New York, NY

Position Overview

We are looking for an experienced Data Quality Engineer with strong expertise in cloud-based data platforms, data pipeline testing, API automation, performance testing, and database validation. The ideal candidate will be responsible for ensuring the quality, reliability, accuracy, and performance of large-scale data platforms and pipelines, with a strong focus on GCP and Python-based test automation.

Key Responsibilities

• •Design and execute comprehensive data quality and testing strategies for cloud-based data platforms, pipelines, APIs, and applications.
• •Validate end-to-end data pipelines, including data ingestion, transformation, processing, storage, and downstream consumption.
• •Perform data validation across streaming and batch-processing pipelines, ensuring data accuracy, completeness, consistency, and integrity.
• •Develop and maintain Python-based automation frameworks for backend, REST API, and data validation testing.
• •Develop UI automation tests using Playwright and perform manual validation where required.
• •Perform RESTful API testing, including functional, integration, negative, regression, and end-to-end validation.
• •Design and execute performance, load, stress, latency, and throughput testing using tools such as k6.
• •Validate data across relational, document, analytical, and columnar databases, including PostgreSQL/AlloyDB, MongoDB, BigQuery, and optionally ClickHouse.
• •Work extensively within the Google Cloud Platform (GCP) ecosystem, including Pub/Sub, GCS, Dataflow, Dataproc, and Cloud Composer.
• •Troubleshoot data discrepancies, pipeline failures, API issues, and performance bottlenecks, collaborating closely with Data Engineering, Development, DevOps, and Product teams.
• •Build automated reconciliation and data-validation checks to identify missing, duplicate, inconsistent, incorrect, or delayed data.
• •Contribute to CI/CD processes by integrating automated data, API, and application quality checks into deployment pipelines.
• •Support data profiling, governance, cataloging, lineage, and automated data-quality controls, preferably using Dataplex.

Required Skills

• •Strong experience as a Data Quality Engineer, SDET, Data Test Engineer, or QA Automation Engineer working with modern data platforms.
• •Strong hands-on experience with Python-based test automation.
• •Strong experience testing RESTful APIs and backend services.
• •Experience with Playwright or similar UI automation frameworks.
• •Hands-on experience with GCP data technologies, particularly Pub/Sub, GCS, Dataflow, Dataproc, Cloud Composer, and BigQuery.
• •Strong SQL and data-validation skills with experience working across large and complex datasets.
• •Experience with PostgreSQL/AlloyDB, MongoDB, and BigQuery.
• •Hands-on performance testing experience using k6 or comparable modern load-testing tools.
• •Strong understanding of ETL/ELT, batch and streaming pipelines, data reconciliation, schema validation, and data-quality principles.
• •Strong analytical, debugging, problem-solving, and cross-functional communication skills.

Nice to Have

• •Experience with ClickHouse or other high-performance columnar databases.
• •Experience with Dataplex for data profiling, cataloging, governance, and automated data-quality controls.
• •Experience integrating automated tests into CI/CD pipelines.
• •Knowledge of data observability, data lineage, metadata management, and cloud-native monitoring