AI Integration in Business Processes: The Reality of Training, Judgment and Practical Use

As businesses increasingly look into AI integration for various processes, a common concern is the reliability of these systems. While AI integration promises efficiency and innovation, it also brings challenges, particularly around accuracy and trust.

A recurring conversation with clients highlights these concerns:

Client: “We are thinking about deploying AI in a business process X. Will it be 100% reliable?”

Me: “No single ‘classic’ Machine Learning (ML) nor Generative AI (GenAI) model has 100% accuracy. Is your current staff handling process X flawlessly?”

Client: “Um… No!  🙁”

This exchange underscores a critical point: every process, automated or not, has a margin of error. The focus should not solely be on the potential mistakes of AI but on the overall impact and efficiency it brings to the business.

Introducing AI, especially GenAI, into business workflows necessitates a paradigm shift in how AI-generated outputs are reviewed and utilized. Users must be trained to critically assess AI-generated content, cross-check citations and apply their judgment to ensure accuracy. This is a fundamental aspect of change management in AI deployment.

LLMs (Large Language Models) like GPT-4, used in tools like ChatGPT and Microsoft Copilot, are designed to generate human-like text by predicting the next word in a sequence. Despite advancements, these models can and do produce incorrect information, a phenomenon often referred to as “hallucinations.” This issue is intrinsic to the way they operate and is unlikely to be fully eliminated.

Consider a real-world scenario from the BCGPT (before ChatGPT) olden days where a contract reading AI solution was implemented, only to be abandoned shortly after. The AI could identify key clauses and summarize documents, but an expensive lawyer had to conduct a 100% review pass over the results to catch any missed key clauses. This negated the benefits of automation, demonstrating that in high-stakes applications, the cost of errors can outweigh the efficiencies gained, a matter that needs to be considered upfront. 

Despite these limitations, many professionals, myself included, continue to use LLM-powered services daily. Take Perplexity.ai, for example. Even though some of its generated summaries contain factual inconsistencies, it significantly reduces the time I spend sifting through vast amounts of information. Providing a concise overview and bulleted lists of main themes allows me to quickly identify relevant pages for further detailed review, thus streamlining my initial research process. 

LLMs will continue to produce errors and draw incorrect inferences. In their current state, they may hallucinate, draw incorrect inferences from supplied text and bungle basic logic easily accessible to us humans.  This reality does not render them useless. Instead, like with any tool, it highlights the importance of proper user and handling. Effective use of AI involves a combination of automated processing and human oversight.

AI, particularly LLMs, holds immense potential for enhancing business processes, provided they are implemented with a clear understanding of their limitations. While AI will never achieve 100% accuracy, its value lies in augmenting human capabilities and improving overall efficiency. Proper training, critical review and a balanced approach to automation and oversight are essential for maximizing the benefits of AI in business.

By embracing these principles, businesses can navigate the challenges of AI integration and harness its full potential, transforming processes and driving innovation in the digital age.