Dispatch 002 · Enterprise adoption

How To Make AI Pilots Succeed

Why South African organisations need to rethink how they move from AI experimentation to measurable business value.

South African businesses are not short of interest in artificial intelligence. Across banking, insurance, retail, mining and the public sector, organisations are experimenting with AI-powered assistants, document processing, workflow automation and increasingly, autonomous AI agents.

The problem is no longer getting organisations to experiment with AI. It is getting those experiments to deliver measurable business results.

According to PwC's May 2026 AI performance research, 82% of surveyed African organisations are running AI pilots, yet relatively few have achieved enterprise-wide adoption.

This reveals an important distinction: successfully demonstrating AI is not the same as successfully implementing it.

A pilot might produce impressive outputs, attract executive attention and receive positive feedback from employees. But six months later, the organisation may still be operating exactly as it did before.

The technology worked. The business process never changed.

So how can South African organisations approach AI pilots differently?

1. Understand the process before introducing AI

Consider a common business process: supplier onboarding.

In a South African organisation, onboarding a new supplier might involve collecting company registration documents, tax compliance information, B-BBEE documentation and banking details, followed by internal verification and approval.

In the public sector, supplier information may also need to be verified through National Treasury's Central Supplier Database (CSD).

On paper, this appears straightforward.

In practice, an application might sit with procurement for several days, return to the supplier because documents are missing, wait for finance to verify banking information, and then move through additional approval stages.

Now imagine an AI pilot designed to extract supplier information from submitted documents.

The AI might complete that particular task in seconds rather than minutes.

But if the supplier still spends two weeks moving between departments, has the organisation meaningfully improved its onboarding process?

Not necessarily.

This is why process mapping should come before AI development.

Organisations need to understand how work actually moves through their departments, not simply how their standard operating procedures say it should move.

That means interviewing employees, reviewing workflow logs, identifying delays and examining the exceptions that routinely interrupt operations.

It also means asking whether some steps can be removed entirely before deciding which ones should be automated.

2. Automate outcomes, not isolated tasks

One of the easiest mistakes to make with AI is selecting a task because it produces an impressive demonstration.

Extracting information from an invoice, summarising a document or drafting an email can all demonstrate useful capabilities.

But the commercial value of these activities depends on the broader process.

Suppose a finance department processes 5,000 invoices per month. An AI system saves two minutes of manual data entry per invoice.

That represents approximately 167 hours of work each month.

That is a meaningful productivity improvement, but it does not automatically translate into financial savings.

If invoices still require multiple approvals, spend days waiting in queues and regularly return for correction, the organisation has only accelerated one part of the workflow.

A more ambitious approach would examine the entire invoice-processing lifecycle.

Could AI classify incoming invoices, match them against purchase orders, identify discrepancies, route exceptions and prepare approved transactions for payment?

The objective should not simply be to make individual employees faster.

It should be to improve the performance of the process itself.

That is where AI can begin generating measurable operational value.

3. Establish a baseline before starting the pilot

Many organisations only begin asking about return on investment after their AI pilot has been running for several months.

By then, it can be difficult to demonstrate what has improved.

Employees might report feeling more productive. The application might have hundreds of users. Executives might be satisfied with the technology.

But none of these measures necessarily establish financial value.

Before implementation, organisations should identify the metrics that matter.

These could include:

For example, an insurance company deploying AI for claims administration should understand its current claims-processing costs and turnaround times before introducing automation.

The pilot should then have a clearly defined objective, such as reducing average processing time while maintaining accuracy and appropriate human oversight.

Crucially, the organisation must also account for implementation costs, AI usage, integrations, maintenance and ongoing supervision.

A pilot is successful when its benefits can be demonstrated against an agreed baseline, not merely when the technology performs as expected.

4. Build AI into existing operations

Another common limitation of AI pilots is that they operate outside the organisation's established systems.

Employees are given another application to open, another dashboard to monitor and another workflow to remember.

For organisations already operating multiple enterprise systems, this creates additional complexity.

Consider a South African financial services company using Microsoft 365, an enterprise resource planning system, internal databases and specialised compliance tools.

An AI solution that requires employees to repeatedly copy information between these environments may improve certain tasks while introducing new inefficiencies.

Where appropriate, AI should operate within the systems employees already use.

Rather than requiring a procurement officer to open an AI chatbot and ask it to review a supplier application, an integrated AI agent could automatically identify incoming applications, check whether required documents are present and flag missing information.

The procurement officer would then review exceptions instead of manually initiating every step.

This distinction between an AI assistant and a background agent is important.

Assistants can make individuals more productive. Well-designed background agents can improve how entire workflows operate.

However, autonomy should be introduced according to the risks involved. Financial transactions, sensitive information and consequential decisions may still require explicit human approval.

5. Treat governance as part of implementation

South African organisations also need to account for their regulatory and operational environment.

AI systems handling customer, employee or supplier information must be designed with the Protection of Personal Information Act (POPIA) in mind.

Organisations should understand what information the system processes, which providers have access to it, where information is stored or transferred, and what security controls are in place.

For regulated industries, additional sector-specific requirements may apply.

This should not become an exercise in creating governance documents that never influence the actual system.

Governance needs to be operational.

An AI agent should have clearly defined permissions, activity logs, escalation procedures and limits on which actions it can perform independently.

Its outputs should also be evaluated against representative scenarios before deployment.

For example, an AI system reviewing customer documentation should be tested not only against well-formatted submissions, but also incomplete applications, inconsistent information and unusual cases.

A reliable AI system is not one that never makes mistakes. It is one whose errors can be identified, contained and corrected.

For higher-risk processes, organisations should begin with supervised or shadow-mode testing, comparing the AI system's decisions with existing human workflows before allowing it to act independently.

6. Give the AI system a business owner

AI pilots frequently begin as technology initiatives.

An innovation team identifies an opportunity, IT develops a solution, and the business department is invited to use it.

But what happens when the pilot becomes a permanent operational system?

Who is responsible when its performance deteriorates?

Who monitors exceptions, approves changes or ensures that employees continue using it correctly?

These questions should be answered before deployment.

An AI agent that supports procurement should have an accountable owner within procurement. A system supporting claims administration should have ownership within the relevant insurance function.

Technical teams remain responsible for engineering, infrastructure and support, but the business function must own the operational outcomes.

This creates accountability for both system performance and business value.

It also helps prevent AI projects from becoming isolated technology experiments that lose momentum once their original project teams move on.

7. Plan for what happens after the pilot

The most important question may be one that receives the least attention:

What happens when the pilot ends?

A six-month proof of concept might be funded through an innovation budget, but a production system requires ongoing investment.

There are infrastructure costs, model usage fees, monitoring requirements, software integrations and continued evaluation as business processes evolve.

Models can change. Providers can adjust pricing. Internal systems can be upgraded. Employees responsible for the workflow can leave.

Organisations therefore need a long-term operating model before committing to deployment.

This should address ownership, recurring costs, support responsibilities, vendor dependencies, data control and the ability to replace underlying AI models where feasible.

For South African organisations working within constrained budgets, this is particularly important.

An AI pilot should not depend indefinitely on temporary funding or external consultants to remain operational.

The transition from experimentation to production must be planned from the beginning.

A better approach to enterprise AI adoption

The path to successful AI adoption does not necessarily begin with a more capable model or a more sophisticated agent architecture.

It begins with understanding the business.

Organisations should identify a process with a meaningful operational problem, measure its existing performance and investigate why it operates inefficiently.

They should then redesign the workflow, introduce AI where it can deliver measurable value and evaluate the solution under realistic operating conditions.

Only after the system demonstrates reliability and value should the organisation expand its autonomy and scale deployment.

For South African companies, the opportunity is significant.

AI can help businesses increase operational capacity, reduce administrative costs, improve service delivery and strengthen decision-making.

But these outcomes will not come from running more pilots for the sake of experimentation.

They will come from treating AI as part of business transformation rather than as a technology project.

The question executives should be asking is no longer, "What can we build with AI?"

It is, "Which business process can we meaningfully improve, and how will we prove it?"

That is how AI pilots become sustainable business capabilities.


Sources and further reading

David Rammutla is an AI Solutions Architect at Sulta Tech and a contributor to State of AI SA. Read his contributor profile.