Building a Real-Time Data Mesh Platform for Insurance

Designed and built a production-grade data mesh supporting real-time and batch workloads under strict regulatory and operational constraints.

The Client

Global specialty insurance group operating across multiple markets and lines of business, with a strong reliance on data for underwriting, claims, risk management, and regulatory reporting. The organisation runs complex, high-volume workloads under strict governance, security, and compliance constraints.

10+

Countries of operation

20+

Lines of business

1000+

Employees worldwide

The Challenge

The organisation had an established on-premise data architecture supporting production workloads, but it was not designed for the scale, speed, and delivery model now required. Modernisation was slowed by architectural complexity, legacy constraints, and the need to protect critical workloads.

The business also required near real-time data for underwriting, claims, risk insights, and data science in a highly regulated environment, where governance, security, lineage, and access control were mandatory.

The challenge was to evolve the platform into a modern, real-time, cloud-ready data platform while maintaining operational and regulatory compliance.

Our Solution

A global data mesh architecture providing governed real-time and analytical data products to the business.

Domain driven Architecture

Designed and implemented a production data mesh structured across five core domains: Underwriting, Rating, Broker, Cyber, and Central. Each domain owns its data products while relying on shared platform services for governance, security, and operational consistency.

Real-Time Data Products

Built high-volume streaming data products using Confluent Kafka, enabling near real-time processing of operational events across submission, policy, claim, premium, and cyber workloads. Designed for scalability, resilience, and low-latency delivery.

Analytical Data Products

Delivered governed batch data products in Snowflake and ADLS, enabling scalable analytics, regulatory reporting, and complex financial and actuarial processing workloads.

Multi-Regional Azure Platform

Engineered a multi-region Azure platform to support distributed teams, regulatory boundaries, disaster recovery, and high availability—ensuring production resilience across markets.

Core Insurance Data Products

Implemented domain-level data products including Submission, Policy, Claim, Premium, Workbench, and PMDR, supporting underwriting, regulatory reporting, and operational analytics in production.

Data Catalog (Data.world)

Integrated Data.world as the enterprise data catalog to manage domains, data products, lineage, and ownership. Extended the catalog to integrate directly with Kafka topics, bridging streaming metadata with governed domain models.

Real-time insurance data mesh architecture

The Value

The Modern Data Platform transformed how business teams access, trust, and act on data, enabling faster decisions, stronger governance, and scalable growth.

Data domains onboarded

Core domains onboarded with the platform launch: Underwriting, Claim, Broker, Finance, Central

Engineers working on the Platform

50+

Multiple teams working in parallel for the different domains following identical delivery practices and standards

Real Time Data Products

10+

Core Data Products (Policy, PolicySection, Claim, etc.) and Derived DPs (Premium, PMDR) generated in Real Time.

Faster, Better Underwriting & Risk Decisions

Real-time streaming data products make submissions, policies, claims, and cyber events available almost immediately. This gives underwriting and risk teams faster access to data, improving responsiveness and enabling more timely, informed decisions across core insurance processes.

Trusted, Governed and Compliant Data

An integrated Data Catalog provides visibility of domains and data products, improving discovery, transparency, and controlled access. In a regulated insurance environment, this ensures compliance and audit readiness—while giving business teams confidence in the data they use.

A Platform That Scales With the Business

A modular, domain-driven architecture enables federated delivery of data products, allowing teams to work in parallel under shared standards. This increases productivity and supports growth across new domains, regulatory requirements, and future data and AI initiatives.

View More Success Stories