GCP BQ/ GCP python

Responsibilities

GCP BigQuery Development

Design, develop, and optimize data warehouses using Google BigQuery

Build and maintain data models (fact/dimension, star/snowflake schemas)

Develop complex SQL queries for analytics and reporting

Optimize queries for cost, performance, and scalability

Implement partitioning, clustering, and materialized views

Python Development

Develop data pipelines and workflows using Python

Build reusable scripts for data ingestion, transformation, and automation

Integrate Python applications with GCP services and APIs

Ensure code quality, modularity, and scalability

Data Engineering & ETL

Design and build ETL/ELT pipelines using GCP tools

Ingest data from multiple sources (APIs, databases, streaming platforms)

Process structured and unstructured data efficiently

Ensure data quality, validation, and governance

GCP Cloud Services

Work with GCP services such as:

BigQuery (mandatory)

Cloud Storage (GCS)

Dataflow / Dataproc

Pub/Sub (streaming)

Cloud Composer (Airflow)

Design scalable and secure cloud-native architectures

Performance Optimization

Monitor and optimize BigQuery cost and performance

Troubleshoot data pipeline failures and bottlenecks

Implement best practices for efficient data processing

Collaboration & Leadership

Collaborate with data scientists, analysts, and business stakeholders

Provide technical guidance and mentorship to junior engineers

Participate in architecture and design discussions

Work in Agile/Scrum environments

Technical requirements

Primary skills:Technology->Functional Testing->Mainframe testing->Proterm,Technology->OpenSystem->Python - OpenSystem

Additional responsibilities

Required Skills & Qualifications

✅ Core Skills

5–9 years of experience in data engineering / big data development

Strong hands-on experience with GCP BigQuery (mandatory)

Strong proficiency in Python (mandatory)

Expertise in SQL and query optimization

Experience in building ETL/ELT pipelines

✅ Technical Skills

Deep knowledge of BigQuery architecture and data modeling

Experience with Airflow / Cloud Composer for orchestration

Strong understanding of data warehousing concepts

Familiarity with Git and CI/CD processes

Knowledge of REST APIs and integrations

Preferred Skills

Experience with streaming (Pub/Sub, Kafka equivalent)

Exposure to Dataproc / Spark / PySpark

Familiarity with data lakes and lakehouse architectures

Knowledge of Docker/Kubernetes

Experience with BI tools (Looker, Tableau, Power BI)

Education

MCA,MSc,MTech,BCA,BSc,BTech