Senior Test Automation Engineer - Data & AI Team (m/f/d)

At SOFICO , we're building the future of mobility through Miles, our market-leading platform for leasing, fleet, and mobility operations worldwide. As an international team, we combine innovation, collaboration, and ownership to deliver real impact for our customers, in a culture of continuous learning, mutual support, and shared success.

From our office in Ghent , we are scaling our Data & AI organization , the team behind SOFICO AI Cloud and the Miles Enterprise Agentic Platform (AutoPilot & AutoRoute). Our platform combines AI, cloud-native technologies, and enterprise software, leveraging Java/Quarkus services, Python/LangGraph agents, Azure AI Foundry, Temporal workflows, and Langfuse observability.

As a Senior Test Automation Engineer , you will drive the quality and reliability of this platform by building and owning a robust test automation strategy across the entire AI ecosystem . Working closely with engineers across AI Runtime, Cloud Ops, API & Saga, Knowledge, Governance, and Data teams, you will ensure that critical business flows, integrations, and AI-powered capabilities are validated through scalable, automated testing.

What you’ll do ...

Deterministic testing (classic, pass/fail quality gates)

Own and evolve the test automation strategy for the Data & AI platform — unit, integration, contract, regression, and end-to-end — designing, building, and maintaining automated frameworks and suites integrated into CI/CD pipelines.

Validate end-to-end business processes within sandboxed Miles environments, ensuring correctness, reliability, and safe failure handling.

Develop API and contract testing approaches for distributed services and integrations.

Non-deterministic testing (AI and agentic behavior)

Design and own evaluation approaches for AI-powered and agentic capabilities where outcomes are probabilistic rather than pass/fail: LLM-as-a-judge setups, baseline/benchmark datasets, and process/quality metrics for agent behavior.

Establish baselines and regression tracking for model and agent output quality over time, this alongside the deterministic CI gates.

Platform-wide

Collaborate across engineering domains to ensure platform-wide quality and release confidence.

Drive continuous improvement in test automation practices and tooling, and act as a mentor and quality advocate, helping teams adopt effective automated testing approaches.

What you bring ...

We're looking for a strong overall profile , without expecting you will tick every box.

Please indicate in your application your current experience level (e.g. none / familiar / proficient / expert) for each of the areas below . This helps us understand your profile and have a more focused conversation.

Test automation engineering: unit/integration/e2e frameworks, CI/CD integration

Testing APIs, backend services, microservices, and distributed systems

Programming in Java and/or Python, including typed schemas, mocking, and test-data setup/teardown

Designing scalable automation frameworks from the ground up, and coaching others in their adoption

Non-deterministic / AI evaluation testing: LLM-as-a-judge, benchmarking, quality metrics for agentic systems ( plus )

Workflow orchestration platforms, cloud-native applications, and LLM/agentic systems ( plus )

Tooling such as Playwright, Kubernetes, and modern DevOps practices ( advantage )

An analytical mindset, attention to detail, and a genuine drive to improve software quality matter as much as any single line above.

What's in it for you…

Competitive salary package.

Meal allowance provided per working day.

Group and hospitalisation insurance.

Hybrid working and flexible hours, giving you the freedom to shape your workday.

A modern co-working space offering a comfortable working environment.

A structured onboarding program from day one, with a dedicated coach to help you get up to speed and grow in your role.

Regular opportunities to connect beyond day-to-day work — casual drinks, team activities, and company-wide events.

Ready to build the secure and deterministic capabilities behind the next generation of mobility software? Join SOFICO and let's shape what's next together!