Pyspark developer

Pyspark Expertise

Expert proficiency in Pyspark

Ability to design and implement efficient data processing workflows

Experience with Pyspark SQL and DataFrames

Good exposure to Big Data architectures and good understanding of Big Data eco system

Experience with some framework building experience on Hadoop

Good with DB knowledge with SQL tuning experience.

Good to have experience with Python, APIs and exposure to Kafka.

Responsibilities

Team Management: Lead and manage a team of technical professionals – Data Engineer and Application Developers, ensuring effective collaboration and productivity.

Client Interaction: Serve as the primary point of contact for clients, understanding their needs and ensuring successful project delivery.

Status Reporting: Prepare and present detailed status reports on ongoing projects to stakeholders, highlighting progress, risks, and mitigation plans.

Issue Resolution: Proactively identify and resolve any technical challenges or roadblocks faced by the team, ensuring smooth project execution.

Project Management: Oversee the entire project lifecycle, from planning to execution, ensuring timely delivery within scope and budget.

Technical requirements

Pyspark, Bigdata, Spark, Python, Databricks, AWS

Additional responsibilities

Ability to develop value-creating strategies and models that enable clients to innovate, drive growth and increase their business profitability

Good knowledge on software configuration management systems

Awareness of latest technologies and Industry trends

Logical thinking and problem solving skills along with an ability to collaborate

Understanding of the financial processes for various types of projects and the various pricing models available

Ability to assess the current processes, identify improvement areas and suggest the technology solutions

One or two industry domain knowledge

Client Interfacing skills

Project and Team management

Education

Bachelor of Engineering,BBA,BCom