Perspective

Data or AI first? Why it’s the wrong question

As someone who has worked in data strategy for over half a decade, I can tell you: it’s a great time to be working in this field. After years of funding great web and user experiences, budget holders are turning their attention to data strategy to support their AI aspirations. They’re recognising that a business run on myriad Excel spreadsheets is no longer sustainable, and it’s time to do something about it. AI ambitions have turned data modernisation from an ideal future state into an urgent requirement.

Why is it so important to fix your data foundations? Our recent survey found that every senior leader we spoke to has moved an AI pilot into production, and 23% describe AI as transforming their business. Yet a staggering 86% say data quality or accessibility is limiting their AI ambitions, and 48% name lineage and auditability as their biggest data challenge. With regulators placing ever stronger demands to explain why a decision was made, there’s a clear business case for better data management.

So, should you focus on data or AI first? Well, it’s neither: they work hand in hand. Let your AI use cases decide which data to fix first and let AI do the heavy lifting of fixing it.

But where do you begin?

Two modernisation traps to avoid

Let’s say a business has reached the point where it agrees its data estate needs modernising, and there’s a strong appetite for AI-driven use cases. Two things can go wrong:

1. The big bang transformation

Let’s just pull the plaster off and tackle the problem once and for all. Move all your existing data processes into a shiny new data platform, put best-practice governance in place along the way, and generally make your tech teams very happy.

The problem is that this doesn’t deliver immediate, tangible business value. You might not be using agents to drive your business processes for another year or two while you’re stuck in the transformation. Stakeholders lose faith as they watch their budget disappear.

2. AI bolted onto legacy

The second failure mode is to keep the legacy estate as untouched as possible. You use a façade model or apply surgical fixes to give your agents access to critical data and generate new AI-driven products. Your full modernisation plan moves further and further into the future, as you keep building new features on brittle, risky foundations.

While this might deliver an AI use case sooner, it will likely not be a revolutionary one – you might find yourself with yet another chatbot that only needs access to surface-level data (e.g. documents). To make a real impact and deliver a differentiating use case, you’ll likely need to bite the bullet and tackle some of your modernisation issues now.

As our report shows, most insurers face a systemic data issue. Despite valiant efforts to get their data in order, the sheer complexity means there’s fragility across the board.

So there’s no simple solution: no single new pipeline or database migration will get you where you need to be without more fundamental changes. The good news is that this is where AI can come to the rescue.

AI as the great enabler

AI may have been the catalyst for your data transformation in the first place. It can also help you move faster.

A lot of the work in moving away from a legacy data estate is understanding existing transformations and data models. It often means moving from older patterns, such as stored procedures, to code that’s easy to read, version control and run in integrated environments for validation and testing, such as Python notebooks.

LLMs can turn what used to be a daunting migration task into a streamlined process: by having agents read existing transformations, developers can learn and validate business rules, identify edge cases, and guide the agent to rewrite thousands of lines of complex code into more modular formats.

For example, we built configuration files to codify the relationship between different layers of our medallion architecture for a current insurance client – an LLM can then apply this pattern to any new data we are bringing into the platform, (near-enough) instantly generating configs for even complex relationships. The human data engineer can then validate correctness in a format that is easy to understand, rather than sifting through many lines of complex SQL manually.

Of course, using AI to write code always carries risks, and trust in the outputs is paramount. But a definition of correct is actually much clearer on a data project than when building a user journey with many different edge cases: by identifying key, critical business measures, we can easily compare the output of old and new transformation pipelines and be confident that our new code is operating correctly.

Start small, keep the big picture

With AI helping through the most daunting “cranking-the-handle" parts of a data transformation (the actual rewrite of business logic in new technology), organisations can get back to what they actually want to achieve: real business value from AI use cases.

To avoid the "big bang transformation” trap, it’s still wise to build in stages – even if the fact that an LLM can make the rewrite a lot faster can make it tempting to do it all at once. Focusing on a subset of data that can form the basis of a first data product or AI use case lets you validate your technical approach as early as possible while keeping the business stakeholders who’ll benefit from the investment on board through regular value-add.

For our insurance client, this meant first focusing on the data underwriters need to support their yearly renewals process (risks, premiums, organisations). This allowed us to go live with the platform much sooner than by trying to get all the data (including incidents and claims) fully modelled.

Developing in tranches naturally means having to run two platforms in parallel for longer – the legacy one and the replacement. This can feel like both a monetary and resource overhead, but it’s an investment worth making to ensure that you can focus on building the right use cases first.

Find your path

As ever with technology transformation, especially data, there’s no obvious path. A lot hinges on where you are now, where you want to get to and what your priorities are. You’ve probably got ideas, and there’s rarely a shortage of those in this industry. Picking the right one and making it happen is the hard part.

So if “data or AI first?” is the wrong question, what’s the right one? It’s probably something along the lines of “which use case is the one that most needs your data to be migrated to a modern platform?”

Close the gap between AI ambition and data foundations

A new Softwire survey, based on 77 interviews with senior leaders, explores the gap between where firms think they are and the reality. Grab your report to see how your peers are tackling the AI transition and the four priorities leaders say would most help them realise AI value.

Laura Hughes

Technical Principal

09 November 2026

to our monthly newsletter for our latest expert content.

About the AuthorLaura Hughes

Laura is Head of Data at Softwire with a particular focus on Data Strategy, Governance & Data Engineering. Having completed a degree in theoretical physics and a PhD in theoretical cosmology, Laura has a passion for deriving structure from chaos and applies her problem-solving skills to help clients achieve simple, elegant solutions to complex problems. Laura recently worked with Manchester Metropolitan University on their data strategy and has a wealth of experience in both data engineering and web development in sectors ranging from financial services, insurance, education, and media. She has also recently taken on more public speaking, writing and editorial opportunities to share technical insights in simple terms with wide audiences. In her spare time, Laura is a founding member of a somewhat critically acclaimed post-garage-punk band and spends a good chunk of her annual leave on tour.