Python, Spark Scala Developer

oin a high-impact engineering team where you’ll build and optimize large-scale data processing solutions using Python, Apache Spark, and Scala. This role is ideal for someone who enjoys solving complex data challenges, improving performance, and delivering reliable pipelines that power analytics and downstream applications. You’ll collaborate closely with data engineers, platform teams, and stakeholders to translate business needs into scalable implementations, while promoting clean code, testing discipline, and continuous improvement. If you’re motivated by ownership, enjoy mentoring others, and want to work in a culture that values collaboration, learning, and measurable outcomes, this opportunity offers the space to make a visible difference—both through technical contributions and by raising the bar for engineering excellence.

Responsibilities

Data Engineering & Development

Design, develop, and maintain scalable batch/stream data pipelines using Spark with Scala and Python.

Implement efficient transformations, joins, aggregations, and data quality checks for large datasets.

Build reusable frameworks/utilities to standardize pipeline patterns and reduce delivery time.

Performance & Reliability

Tune Spark jobs (partitioning, caching, shuffles, memory/executor settings) to improve performance and cost efficiency.

Troubleshoot production issues, perform root-cause analysis, and implement preventive fixes.

Ensure reliability through robust logging, monitoring hooks, and failure-handling strategies.

Collaboration & Delivery

Work with stakeholders to refine requirements and deliver well-documented, production-ready solutions.

Conduct code reviews, enforce best practices, and mentor team members on Spark/Scala/Python patterns.

Contribute to CI/CD-friendly development practices including testing, version control, and release readiness.

Technical requirements

Technology->Analytics - Packages->Python - Big Data,Technology->Big Data - Data Processing->Spark,Technology->Big Data - Data Processing->Scala

Additional responsibilities

Bachelor’s degree in Computer Science, Engineering, or a related field (or equivalent practical experience).

5–9 years of experience in data engineering or backend development with strong hands-on delivery ownership.

Strong proficiency in Python and Apache Spark for large-scale data processing.

Solid experience with Scala for Spark-based development and production-grade implementations.

Working knowledge of Hive and SQL-based data querying/processing concepts.

Education

Bachelor of Engineering