Data Engineer / Platform Administrator (Azure Databricks)

Discover Vanderbilt Health : With eight hospitals and over 180 clinics, we are the region’s largest comprehensive academic health system. Vanderbilt Health has a long tradition of providing world-class care to the people of the Mid-South, living out our mission of moving medicine forward through preeminent programs in patient care, education and research. Here you’ll find a spirit of collaboration and a supportive community where your expertise is valued, your knowledge expanded and your uniqueness celebrated. We are committed to fostering an environment where talented, compassionate people are united in bringing the most advanced care to our neighbors throughout the region.

Organization:

HealthIT Data Platform Services

Job Summary:

The Data Platform Services team is looking for a Data Engineer / Platform Administrator with an inquisitive, analytical mindset who wants to deepen their technical skills. About 80% of your time will go to administering our Azure Databricks platform of 100+ workspaces: building automation, monitoring performance and costs, managing configuration, testing new features, and promoting best practices. The remaining 20% will be spent as a data engineer, building and optimizing data pipelines alongside the rest of the team.

You’ll own defined areas of the platform from the start, with support from senior engineers and room to take on broader scope as you grow.

Note: This role can be performed fully remote from most US locations

.

KEY RESPONSIBILITIES

Platform Administration ( ~80% )

Administer our Azure Databricks environment of 100+ workspaces, including compute policies, access controls, and governance, working with senior team members on larger architectural decisions

Build automation and Infrastructure as Code to provision and manage workspaces at scale

Build solutions for proactive monitoring of platform health, performance, and costs

Test new Databricks features and promote best practices to teams across the organization

Develop tools and services that other teams use to create, test, and deliver data-related assets

Data Engineering ( ~20% )

Design and develop performance-focused data pipelines that support a variety of source systems (flat files, APIs, databases, and more)

Troubleshoot issues with production pipelines and work with internal stakeholders to deploy fixes

Build reusable solutions for recurring problems and help ensure data accuracy and integrity

Work with stakeholders to identify data needs and look for opportunities to improve platform scalability

Technical Qualifications

Minimum Qualifications

Bachelor’s degree

1+ year of experience with Databricks (certifications are a plus)

2+ years of Python in an object-oriented paradigm

2+ years implementing cloud solutions with Azure, AWS, or GCP

1+ year of SQL experience

Preferred Qualifications

Experience with Infrastructure as Code solutions such as Terraform

Experience with CI/CD automation in GitLab, GitHub, or Azure DevOps

Experience with REST data services, APIs, and microservices

Experience with containerization using Docker or Podman

Bonus Qualifications

Experience with PowerShell and/or Bash

A link to your GitHub profile or other personal code examples

TECHNICAL CAPABILITIES

• •CORPORATE SYSTEMS ARCH STANDARDS (PROFICIENT): Has basic knowledge of the architecture of the systems which support more than one core business area. Demonstrates knowledge of how the core business areas' applications and support systems are structured, how various components and parts interact, as well as what protocols and interfaces are used for communications and cooperation between modules and components. Participates in the overall planning and design of the specific core business systems. Able to train and educate others in the structure and components of systems architecture for those core business areas.
• •BUSINESS ARCHITECTURE (PROFICIENT): Has basic knowledge of the architecture of the business systems which support more than one core business area. Demonstrates knowledge of how the core business areas' applications and support systems are structured, how various components and parts interact, as well as what protocols and interfaces are used for communications and cooperation between components.
• •CLOUD ARCHITECTURE (PROFICIENT): Has basic knowledge of leveraging cloud-based architecture to support the life cycle of a data lake. Demonstrates an understanding of when and how to scale cloud components to provide the required level of performance and support of a data lake.
• •DATA LAKES (PROFICIENT): Demonstrates a basic understanding of the effective application, operation, or practice in complex data lakes. Experienced in all core features and functions. Demonstrates a thorough understanding of advanced concepts. Applies concepts and tools consistently in delivering data lakes. May have successfully completed training in advanced techniques, features, and tools.
• •DATA MARTS (PROFICIENT): Proficient understanding of fundamental data mart concepts, encompassing their purpose, significance within data warehousing, and the advantages they provide in supporting data analytics efforts.
• •DATA WAREHOUSING (PROFICIENT): Demonstrates a foundational understanding of data warehousing and its role within an organization. Possesses knowledge of basic corporate data warehouse models and their purposes. Understands the core components of data warehousing systems. Familiar with fundamental data warehouse structures both in theory and basic practical application. Capable of basic performance considerations for data warehousing components.

About the Department:

Health IT

HealthIT provides the best health information technology tools that support Vanderbilt University Medical Center’s mission of:

· Delivering distinctively personalized care

· Improving the health care of individuals and communities regionally, nationally and internationally

· Providing transformative learning programs

· Supporting compelling discoveries

Our tools, which form the digital arteries of VUMC, are either developed in-house by our innovative product teams or selected from the most cutting-edge solutions available in today’s ever-changing marketplace. Our 500 colleagues provide ongoing support over each product’s entire lifespan, ensuring that the tools are meeting the evolving needs of the Medical Center’s 24,000 colleagues.

Our solutions are driven by the incredible work and research of our colleagues throughout Vanderbilt and supported through a close partnership with VUMC Information Technology (VUMC IT) . The strong collaboration among our teams means that VUMC can respond to clinical and operational issues with agility and innovation. Together, we ensure VUMC remains a leader in its pioneering use of healthcare information technology.

Underscoring our entire department are our core values of accountability, transparency and execution , delivered with a strong Partner Promise .

Our professional administrative functions include critical supporting roles in information technology and informatics, finance, administration, legal and community affairs, human resources, communications and marketing, development, facilities, and many more.

At our growing health system, we support each other and encourage excellence among all who are part of our workforce. High-achieving employees stay at Vanderbilt Health for professional growth, appreciation of benefits, and a sense of community and purpose.

Core Accountabilities:

Organizational Impact: Executes job responsibilities with the understanding of how output would affect and impact other areas related to own job area/team with occasional guidance. Problem Solving/ Complexity of work: Analyzes moderately complex problems using technical experience and judgment. Breadth of Knowledge: Has expanded knowledge gained through experience within a professional area. Team