Spark Scala Developer

Responsibilities

We are seeking a skilled Spark Scala Developer to design and build scalable big data processing applications. The ideal candidate will have strong experience in Apache Spark, Scala, and distributed data processing, and will work on building high-performance data pipelines for analytics and data engineering use cases.

Key Responsibilities

Big Data Development

Develop and maintain data pipelines using Apache Spark (Scala)

Process large-scale datasets in distributed environments

Implement batch and real-time data processing solutions

Spark & Scala Development

Write efficient and scalable code using Scala

Work extensively with Spark Core, Spark SQL, and Spark Streaming

Optimize Spark jobs for performance and resource utilization

Data Engineering & Integration

Ingest data from various sources:

Databases (RDBMS, NoSQL)

APIs

File systems (HDFS, S3)

Build ETL/ELT pipelines and data transformation workflows

Performance Optimization

Tune Spark applications (partitioning, caching, memory tuning)

Optimize queries using Spark SQL

Improve job execution time and reduce resource consumption

Collaboration & Design

Work with data engineers, data scientists, and business analysts

Translate business requirements into scalable data solutions

Participate in code reviews and architecture discussions

Monitoring & Troubleshooting

Monitor data pipelines and Spark jobs

Debug job failures and performance bottlenecks

Ensure reliability and data quality in pipelines

Technical requirements

Primary skills:Technology->Big Data - Data Processing->Spark,Technology->Functional Programming->Scala

Education

MCA,MSc,MTech,Bachelor of Engineering,BCA,BSc,BTech