The Idea in 60 Seconds

  • Embedded AI is quietly being included in to all manner of enterprise software.
  • Yet the big consulting firms I’d usually go to for insight, Deloitte, E&Y, and the rest, aren’t talking about it.
  • By 2026, Gartner predicts that 80% of enterprise software will have embedded AI.
  • This shift is reshaping the economics of software and creating new risks for organisations.
  • Trends are putting enterprises into a metaphorical vice.
  • On one side, legislative changes around AI are due to be implemented by the Australian government by the end of 2025.
  • The new rules will tighten the screws with increased penalties for AI misuse.
  • On the other side of the vice, software vendors are embedding AI into their products, often in ways that expose customers to risks while absolving themselves of responsibility.

Embedded AI – Creeping Risk

Enterprises are caught in a vice, getting squeezed between :

  • Software vendors fitting AI in to their products and
  • Likely imminent changes to legislation.

The First End Of The Vice

Generative AI and the more glamorous aspects of AI have dominated headlines since ChatGPT’s release. The real, not very glamorous at all revolution is happening quietly in the background. Software vendors are embedding AI into their products hell for leather.

It’s not hard to see why. Imagine being a software vendor in 2025 that isn’t adding AI to your product? How would your shareholders and customers react? AI in the product is now a customer a customer an board expectation.

A line graph showing the percentage of enterprise software apps with embedded AI, rising sharply from 2023 to 2026, based on Gartner data.

Gartner say the proportion of enterprise software with embedded AI is expected to reach 80% by 2026.

The result is a reshaping the global software market, which is projected to grow significantly in the coming years. I’m personally pleased to see this. As I’ve written elsewhere in my blog, I invested in the NASDAQ in anticipation of this.

A bar chart showing the global software market value in billions, with steady growth from 2023 to 2026.

The global AI software market is expanding rapidly, driven in part by the integration of AI.

A bar chart showing the rising proportion of generative AI costs relative to total software costs from 2023 to 2026.

Generative AI is accounting for an increasing proportion of overall software costs.

For vendors, embedding AI is a way to justify higher prices, lock in customers, and differentiate their products. For enterprises, it’s a double-edged sword that involves a lot of creeping risk, many of which are quietly shifted onto the Enterprise.

  • 92% claim rights to customer data.
  • 17% commit to regulatory compliance.
  • 33% indemnify for third-party IP claims.

Source : Termscout shown on Stanford’s website.

The Other Side Of The Vice – Legal Considerations

legislative changes are coming to Australia around ISO 42001 (industry scuttlebutt suggests they may arrive by the end of 2025.) The new rules will likely increase penalties for organisations that “get it wrong” with AI.

New laws are likely to require organisations to ensure their AI systems are  explainable, fair, and that they are acting responsibly. These are standards that many AI systems, particularly “black box” models, struggle to meet.

Managing Embedded AI: A New Approach

So, how do organisation  navigate this new reality? Preparation and governance.

  1. Understand the Risks
    Embedded AI is sneaking in to organisations. We need to scrutinise vendor contracts, particularly around data rights, compliance, and indemnity. Write to software providers and say ‘don’t add AI to this product.’
  2. Invest in Explainability
    Black box models are a compliance nightmare. Enterprises should push vendors for transparency or invest in tools that provide explainability for AI decisions. This is a big deal. Noone knows exactly how a particular answer was generated
  3. Strengthen Governance
    AI governance isn’t just for bespoke solutions. Embedded AI requires the same level of oversight, including risk assessments, ethical safeguards, and accountability frameworks.
  4. Prepare for Legislative Change
    With new regulations on the horizon, organisations need to ensure their AI systems—embedded or otherwise—meet standards for fairness, transparency, and accountability.

Address The Vice

Embedded AI is transforming enterprise software, whether we’re ready or not. While the big consulting firms may not be talking about it, the shift is happening all around us.

The vice is tightening. On one side, legislation is raising the stakes for AI compliance. On the other, software vendors are embedding AI in ways that shift risks onto their customers.

It’s an important and underrepresented issue.