AI in the Real World

Recently, I was lucky enough to be part of a networking group that got to see a real world demo of Chat GPT in a highly practical use case. The use case was focused on internal search within an organization and utilized the strengths of Chat GPT as a Large Language Model (LLM).

The organization demonstrating the use case explained they used a conversational use interface (UI) that focused on user experience and making it easier for the user to find the information they were looking for without having specific domain knowledge or expertise in "prompt engineering." 

The demo sparked great conversations about what various organizations are doing with pre-trained Large Language Model tools like those provided by Open AI and Microsoft.

Bad Input Begets Bad Output 

The group's facilitator pointed out that I can be a bit of a glass half full pessimist when it comes to artificial intelligence (AI). I would suggest I am a realist who tries to cut through the hype.

Alongside the demo mentioned above, we also had a demo of Copilot in Git Hub by another member. The speaker explained they loved Copilot because it is a very tactically useful tool, operated within a set of rules and constraints they have created as internal governance. When used appropriately it enhances productivity, while remaining contextually safe, secure, and reliable.

I often echo the sentiment of my fellow Yorkshireman Alan Pelz-Sharpe that he expressed during his AI panel at the AIIM Conference 2023: AI is not yet so intelligent that it does not suffer from "garbage in equals garbage out."

This concept of bad input begets bad output applies to the now decades-old Machine Learning capabilities. With Machine Learning, you need to train your model with a high quality data set. With pre-trained LLM's, you need both good data for the massive corpus of content used to build the LLM, and also to the prompt's you provide it. There is research now demonstrating that bad data in training models and bad data from prompts can diminish the quality of generative AI engines over time. 

Narrow AI versus Generative AI

What's the difference between 'old school' AI or Narrow AI (e.g., Machine Learning, Natural Language Processing etc.) and generative AI (e.g., ChatGPT, LLM's, etc.)?

I asked ChatGPT to "Explain the difference in non-technical language between a custom Machine Learning model trained on a sample dataset, and a pre-trained Large Language Model based system." Here is what it responded with:

• A custom machine learning model learns to do specific tasks from examples you provide.
• A pre-trained large language model is already really good with language and can perform a wide range of language-related tasks right out of the box."

So, there we have it, straight from the horses mouth. LLM's are really good with language, but not necessarily good with facts, concepts, complex math or a variety of other knowledge types. LLM's won't necessarily be any better at processing your invoices than good old ML model trained on 200 of your own invoices, but they certainly can provide a great natural language based conversational UI for searching across your invoices to find the data you need from them.

This article was originally published on LinkedIn. It has been edited and republished with the permission of the author.