hirly

DataPelago

Data Processing Engineer - I/O

Remote

hirly Pro

Get in front of the people hiring at DataPelago

  • Reach the hiring manager at DataPelago on LinkedIn, from your own account
  • hirly applies to this job for you, with unlimited applications
  • A resume and cover letter written for this job, unlimited
  • The hiring manager and recruiter at DataPelago, with contact details
Get Pro for this job →

$49 a month · ₹1,999 for 30 days in India · cancel any time

See how you match this job — and similar ones. Free.

Upload your resume and hirly scores it against this role at DataPelago first, then against similar open jobs, and shows where you fit and why.

PDF or DOCX, up to 12MB. No sign-up to see your matches.

Get past the screening software and onto a recruiter's desk

hirly rewrites your resume for this job — matching the keywords and skills in the posting, moving your most relevant experience to the top, and writing a cover letter to fit. About 30 seconds.

  • Keywords matched to this posting
  • Fit score before you apply
  • Cover letter included

Matched against 2.8M live jobs from 220,000+ employers in 200+ countries.

Tailor my resume for this job →

Apply from your AI assistant

Connect hirly to Claude or ChatGPT and ask it to apply to this job. hirly tailors your resume, fills the employer’s form and asks before sending.

Some employer sites stop an application at a CAPTCHA or sign-in and hand it back with a link. Applying needs a paid plan. Works with any assistant that supports MCP.

hirly's read of this role

Seniority
Mid level
Work mode
Remote-friendly
First seen by hirly
1 Sept 2026

Derived automatically from the posting. Upload your resume above to see how the role scores against it.

the posting

  • Data Processing Engineer - I/O
  • Mountain View, CA / Hyderabad, IN / Remote

About DataPelago:

DataPelago is at the forefront of revolutionizing data processing for traditional analytics and cutting-edge GenAI preprocessing. We are building an innovative data processing engine that is transforming how Apache Spark, Apache Flink, Ray and others operate on diverse, large-scale data. Our team of engineers drive and adopt advances in hardware-accelerated computing, parallel processing of large-scale data, query optimization, distributed systems, compilers, machine learning, and cloud-native computing. We are looking for specialists to join our engineering team and shape the future of accelerated data processing.

  • The Opportunity:
  • As a Data Processing Engineer - I/O, you will be a key individual contributor in advancing data
  • read and write capabilities of DataPelago’s data processing engine. You will enhance functional
  • breadth, performance, scale, and reliability of the DataPelago engine in reading and writing large scale data of various data types from diverse data sources and data sinks. This is a unique opportunity to make a significant impact on a category-defining product and work with a talented team of engineers.
  • What You'll Do:
  • Architect: Influence the architecture of how our data processing engine interfaces with data
  • sources and sinks, catalogs, data formats.
  • Design: Lead design of functional and performance enhancements to adapters/connectors,
  • data representations, data filtering, caching and more in our data processing engine.

Core Development: Individually design, implement, test, optimize, and maintain components of the data processing engine.

  • Innovation and Differentiation: Analyze technology roadmap of existing and emerging data
  • formats and libraries, open table formats, catalog services, and more (e.g., Apache Arrow,

Apache Parquet, Apache Iceberg) and identify opportunities for our engine to enhance technology and product leadership.

  • Collaboration: Partner effectively with engineering and product management in defining and
  • realizing the data I/O roadmap of our product..
  • Continuous Improvement: Foster best practices in design and code reviews, testing, CI/CD,
  • and issue resolution to maintain highest product quality, security, efficiency, & productivity.

What You'll Bring:

Bachelor's degree in Computer Science or a related field with 7+ years of relevant experience OR a Master's degree in Computer Science or a related field with 5+ years of relevant

experience.

3+ years of deep technical experience in developing and optimizing data read and write interfaces for large-scale data processing, particularly related to Apache Parquet, Apache

ORC, Apache Iceberg, Apache Spark, and similar technologies.

  • Demonstrated experience in instrumenting, analyzing, and optimizing the performance of
  • data processing engine components on benchmark and customer workloads.

Demonstrated experience in the design, development, and successful release of high-performance data processing engine features for large production deployments.

  • Good knowledge of the architecture of one or more of Apache Spark, Apache Flink, Presto/
  • Trino.
  • Exceptional programming skills in C, C++. Rust experience preferred.
  • Extensive development experience in Linux environments.
  • Strong analytical and problem-solving skills with a passion for performance optimization.

Location Considerations:

We value face-to-face collaboration, but recognize that talent can be found anywhere. Our engineering team works at our headquarters in Mountain View, CA, at our India office in Hyderabad, and at remote locations.

  • Why Join DataPelago?
  • Technology Leadership: Shape the architecture and development of how our core engine
  • works with advanced data store platforms.
  • Cutting-Edge Innovation: Work on challenging problems at the forefront of accelerated
  • computing and data processing.
  • Significant Impact: Your contributions will directly impact the performance and scalability
  • of our mission-critical platform.
  • Growth: Expand your technical expertise and scope of responsibilities working with other
  • talented engineers and with a growing product.
  • Competitive compensation, stock options, comprehensive benefits package, leadership de-
  • velopment opportunities.
Original posting on DataPelago's site ↗

Listed on hirly, a job board. hirly is not the employer: DataPelago is hiring for this role.

Browse similar roles

Want this one?

Upload your resume and hirly rewrites it for this job and writes the cover letter — in about thirty seconds, before you sign up.

Tailor my resume for this job