Lead AI Engineer (Agentic Systems)

About the Role:

Grade Level (for internal use):

11 Lead AI Engineer (Agentic Systems)

Role Summary

As the Lead AI Engineer (Agentic Systems), you will help architect and build the organization’s next generation of autonomous AI workflows. This is a multidisciplinary technical role operating at the intersection of Software Engineering, Data Engineering, and Machine Learning Engineering . You will move beyond simple "chatbots" to design production-grade Agentic Systems: intelligent applications capable of reasoning, planning, and executing complex tasks autonomously.

Responsibilities

Agentic Systems Architecture & Core Engineering

Architect & Build Multi-Agent Workflows: Lead the hands-on design and coding of stateful, production-grade agentic systems using Python and orchestration frameworks like LangGraph , CrewAI , or AutoGen .

Agent-to-Agent (A2A) Communication: Design and implement robust A2A protocols enabling autonomous agents to collaborate, hand off sub-tasks, and negotiate execution paths dynamically within multi-agent environments.

State Management & Orchestration: Engineer robust control flows for non-deterministic agents; implement complex message passing, memory persistence, and interruptible state handling to support long-running autonomous tasks.

Tool Interface Design (MCP): Implement and standardize the Model Context Protocol (MCP) to create universal interfaces between agents, data sources, and operational tools, ensuring modularity and scalability.

Model Integration & Optimization: Utilize proxy services ( i.e. LiteLLM ) to manage model routing and fallback strategies; optimize context windows and inference costs across proprietary and open-source models.

Production Deployment: Containerize agentic workloads using Docker and orchestrate deployments on Kubernetes; leverage AWS AgentCore or similar cloud-native services for scalable infrastructure.

Data Engineering & Operational Real-Time Integration

Build Agent Data Pipelines: Write and maintain high-throughput ingestion pipelines (using Databricks or Python-based ETL) that transform raw operational signals into structured context for agents.

Real-Time Context Injection: Ensure agents have access to "operational real-time" data (seconds/minutes latency) by optimizing retrieval architectures and vector store performance.

Cross-Functional Engineering: Act as the technical bridge between Data Engineering and AI teams; translate complex agent requirements into concrete data schemas and pipeline specifications, while stepping in to resolve hands-on bottlenecks in data availability.

Observability, Governance & Human-in-the-Loop

LLMOps & Tracing: Implement comprehensive observability using tools like Langfuse to trace agent reasoning steps, monitor token usage, and debug latency issues in production.

Safety & Control Frameworks: Design hybrid execution modes ranging from Human-in-the-Loop (HITL) for sensitive operations to fully autonomous execution; build "break-glass" mechanisms and guardrails for automated decision-making.

Evaluation & Reliability: Establish technical standards for testing non-deterministic outputs; automate evaluation pipelines to measure agent accuracy, hallucination rates, and drift before deployment.

Technical Leadership & Strategy

Technical Roadmap Definition: Partner with Product and Engineering leadership to scope feasibility for autonomous projects; define the "Agentic Architecture" roadmap.

Mentorship & Standards: Define code quality standards, architectural patterns, and PR review processes for the AI engineering team; upskill team members on the latest agentic frameworks and methodologies.

Innovation: Proactively prototype with emerging tools (e.g., new reasoning models, graph-based RAG) to solve high-value business problems, moving successful experiments into the production roadmap.

Qualifications

Required

Experience: 7+ years of total technical experience in Software Engineering, Data Engineering, or Machine Learning.

GenAI Specialization: 2+ years of specific experience building and deploying LLM-based applications or Agentic Systems in production.

Database & Lakehouse Mastery: E xperience architecting storage layers for AI, including Vector Databases (e.g., Pinecone, Weaviate , Qdrant ), NoSQL/Relational Databases (PostgreSQL, DynamoDB), and modern Data Lakehouses (specifically Databricks or Snowflake).

Cloud & Infrastructure: E xpertise in cloud architecture and container orchestration (AWS, GCP, or Azure) using Kubernetes and Docker. You must be comfortable deploying and scaling your own applications.

LLM Ecosystem: F amiliarity with common LLM frameworks and orchestration libraries (e.g., LangGraph , LangChain , CrewAI , AutoGen ). You understand the mechanics of RAG, embeddings, and context window management.

Hybrid Engineering Skillset: A unique blend of Data Science (understanding model behavior, probability, and prompting) and Software Engineering (CI/CD, API design, asynchronous programming, and system reliability).

Language Proficiency: Advanced proficiency in Python for systems engineering, capable of writing modular, testable, and maintainable production code.

Education: Bachelor’s degree in Computer Science , Engineering, Mathematics, or a related technical field.

Preferred

Advanced Education: Master’s degree or PhD in Computer Science, Artificial Intelligence, or a related quantitative field.

NLP Expertise: 5+ years of hands-on experience in Natural Language Processing (NLP), ranging from foundational techniques (e.g., text processing, embeddings, classification) to modern architectures.

Graph Technologies: Experience with Knowledge Graphs (e.g., Neo4j, AWS Neptune), Graph Databases, and GraphML (Graph Machine Learning) to support complex reasoning and relationship modeling.

Agentic Tooling: Specific experience with LangGraph , LiteLLM , Langfuse , AWS AgentCore , or implementing the Model Context Protocol (MCP).

Advanced Architectures: Proven track record of implementing Agent-to-Agent (A2A) communication, swarm intelligence, or multi-modal agent workflows.

Real-Time Operations: Experience working in environments requiring operational real-time processing (e.g., FinTech, Energy, Logistics).

Why This Role Matters

You won't just be building chatbots here; you will be architecting the organization’s "central nervous system." As the Lead AI Engineer for Agentic Systems, you are bridging the gap between static data models and active decision-making. The autonomous workflows you design—capable of planning, collaborating (A2A), and executing tasks—will fundamentally change how we operate , moving us from human-dependent processes to self-healing, intelligent systems. This is a rare opportunity to define the standards for Agentic AI in a production environment, working with a stack that represents the absolute cutting edge of the industry.

• About S&P Global Energy
• At S&P Global Energy, our comprehensive view of global energy and commodities markets enables our customers to make superior decisions and create long-term, sustainable value. Our four core capabilities are: Platts for news and pricing; CERA for research and advisory; Horizons for energy expansion and sustainability solutions; and Events for industry collaboration.

S&P Global Energy is a division of S&P Global (NYSE: SPGI). S&P Global enables businesses, governments, and individuals with trusted data, expertise, and technology to make decisions with conviction. We are Advancing Essential Intelligence through world-leading benchmarks, data, and insights that customers need in order to plan confidently, act decisively, and thrive economically in a rapidly changing global landscape. Learn more at www.spglobal.com/energy .

What’s In It For You?

Our Mission:

Advancing Essential Intelligence.

Our People:

We're more than 35,000 strong world