Senior Data Engineer

Job Description Overview:

Designs, develops, tests, deploys and supports complex data pipelines, data products and data management solutions across modern cloud, warehouse and lakehouse platforms. Applies software-engineering, data-modeling and distributed-processing practices to deliver secure, reliable, maintainable and cost-conscious solutions. Works independently on defined outcomes, collaborates across technology and data teams, and contributes to reusable engineering patterns and continuous improvement.

Primary Responsibilities:

• •Design, develop, test, deploy and support complex production data pipelines and data products with guidance on broader architecture where needed.
• •Develop maintainable data-processing and transformation solutions using SQL and programming languages such as Python.
• •Implement solutions using Snowflake, Databricks, dbt and comparable modern data platforms and frameworks.
• •Build data solutions across relational databases, cloud object storage, warehouse and lakehouse architectures, including open table formats such as Apache Iceberg where applicable.
• •Develop batch, micro-batch, CDC and streaming pipelines using approved enterprise patterns and technologies such as Apache Kafka or comparable platforms.
• •Integrate data from relational and mainframe systems, APIs, event streams, SaaS platforms, files and object storage.
• •Apply normalized, dimensional and denormalized data-modeling techniques appropriate to analytical and data-product requirements.
• •Tune SQL, Snowflake and Databricks/Spark workloads for performance and cost using appropriate partitioning, parallelism and compute practices.
• •Implement data quality, reconciliation, metadata, lineage, logging, monitoring and alerting controls.
• •Build pipelines that support error handling, restartability, idempotency, retry, replay and recovery using established patterns.
• •Participate in production support, incident resolution and root-cause analysis; implement corrective actions to prevent recurrence.
• •Use Git, peer review, automated testing, CI/CD and environment promotion practices for data engineering delivery.
• •Develop and reuse shared components, templates and patterns; contribute improvements to engineering standards and documentation.
• •Participate in technical design and code reviews, identify implementation risks and escalate architectural concerns appropriately.
• •Partner with product owners, architects, governance, cybersecurity, platform and consuming teams to clarify requirements and deliver supported solutions.
• •Contribute to technical epics and stories, estimate assigned work, identify dependencies and deliver through Agile practices.
• •Provide day-to-day technical assistance and informal mentoring to less-experienced engineers. This role does not have direct people-management responsibility.
• •Understand and adhere to the Company’s risk and regulatory standards, policies and controls in accordance with the Company’s Risk Appetite.
• •Design, implement, maintain and enhance internal controls to mitigate risk on an ongoing basis. Identify risk-related issues requiring escalation to management.
• •Maintain internal control standards, including timely implementation of internal and external audit points and issues raised by external regulators, as applicable.
• •Promote an environment that supports belonging and reflects the Company brand.
• •Complete other related duties as assigned.

Education and Experience Required:

• •Associate’s degree and a minimum of 5 years’ systems analysis and/ or application development work experience or Bachelor's degree and a minimum of 3 years’ systems analysis and/ or application development work experience. In lieu of degree, a combined minimum of 7 years’ education and/or relevant work experience, including a minimum of 3 years’ systems analysis and/or application development work experience
• •Strong SQL skills, including development of complex transformations and analytical queries.
• •Experience with Python or another general-purpose programming language used for data engineering.
• •Hands-on experience with Snowflake, Databricks or a comparable cloud data platform.
• •Experience with ELT/ETL and transformation practices, including dbt or a comparable framework.
• •Experience with relational data stores and warehouse or lakehouse data-modeling concepts.
• •Experience with distributed processing using Apache Spark/PySpark or a comparable framework.
• •Experience with Git-based source control, automated testing and CI/CD fundamentals.
• •Experience with batch data processing and working knowledge of streaming or event-driven concepts.
• •Knowledge of cloud and object-storage concepts, preferably in Microsoft Azure.
• •Experience working with structured and semi-structured data using formats such as Parquet, JSON, CSV, Avro and XML.
• •Experience with Linux/Unix environments and scripting.
• •Demonstrated ability to independently deliver complex data pipeline components and data products within an established architecture.
• •Demonstrated ability to troubleshoot data-processing, data-quality and production issues across multiple technologies.
• •Demonstrated ability to apply established engineering patterns while identifying opportunities for improvement.
• •Demonstrated ability to communicate implementation choices, risks and dependencies to technical and product partners.
• •Demonstrated ability to manage assigned work from design through production support with limited day-to-day supervision.

Education and Experience Preferred:

• •Production experience with Snowflake, Databricks, Apache Spark/PySpark and dbt.
• •Experience with Apache Iceberg or another open table format.
• •Experience with Apache Kafka, CDC or event-driven data processing.
• •Experience with orchestration technologies such as Apache Airflow, Azure Data Factory, Databricks Workflows or dbt orchestration.
• •Experience with Terraform or another Infrastructure as Code technology.
• •Experience with data catalogs, metadata, lineage, observability or automated data-quality tooling.
• •Experience delivering governed enterprise data products.
• •Experience working in a regulated environment such as financial services.

M&T Bank is committed to fair, competitive, and market-informed pay for our employees. The pay range for this position is $97,100.00 - $161,800.00 Annual (USD). The successful candidate’s particular combination of knowledge, skills, and experience will inform their specific compensation.

Location

Buffalo, New York, United States of America