It seems like everyone out there is trying to do something with AI nowadays, almost to the extent where seemingly any problem your business has can be solved by some sort of AI-enabled solution. This begs the question: which of these things should you be solving with AI, and which is it particularly well suited to?
How do LLMs work
To begin to answer this question, we should have a baseline understanding of how LLMs (large language models) work, so we can reason what they might be good at.
Most of the time when someone is talking about AI nowadays they are talking about LLMs. When you're talking to ChatGPT, you're talking to an LLM. There are other “types” of AI such as neural networks, computer vision, reinforcement learning, and more. For the purposes of this article we'll be talking about LLMs, as they're the technology that's most applicable to solving business problems in the general case.
So how does an LLM actually work? Well, to really simplify things, an LLM is a neural network that predicts the next word (or to be more accurate, the next token) in a sequence of text. LLMs are trained on vast amounts of material (articles on the internet, books, magazines, etc.) to “learn” how to make these predictions. As part of the process of training the model, the model's parameters are adjusted based on these huge corpuses of data to generate a set of matrices that encode language patterns and can be used to predict what might come after given text.
What's the significance of how LLMs work
The core thing to remember is that an LLM is a prediction model for text. LLMs don't reason, they don't “learn”, and they don't “think”.
We can evaluate tasks that LLMs can complete effectively based on this context. If the core work behind a task is producing text that is derived from pre-existing information or facts, it's likely something that an LLM will be good at accomplishing. Writing an essay based on text-based research? That's something LLMs are good at. Feeding a business proposal into an LLM for evaluation? It'll probably do okay, but you should keep in mind that it's not reasoning about the proposal, it's generating a likely response to it, probably based on similar proposals that are part of its training and language patterns therein.
Think of an LLM as an extremely advanced autocomplete: great for generating or transforming text, but not a substitute for human judgment.
Can an LLM “learn” my business
Often when you see businesses trying to incorporate AI, the framing is that the business operator wants the AI to “learn” their business, or “learn” about their products. In the support context, someone might want the LLM to “learn” how to answer customer questions based on an existing dataset of support responses.
While many of these business processes could be made more efficient with AI, it's not because the LLM is “learning” them.
The support problem is an interesting one to examine: having a lot of prior support history can be helpful when setting up an LLM for support, but not just to dump into the LLM. One example of how this could be useful is for evaluation and fine-tuning. If your support dataset is labeled (i.e. you know which responses are good and which are bad), you could fine-tune a model to perform better for those sorts of asks. This doesn't instantly make the model able to answer those questions, but it does make it more likely to produce "good" response similar to those in your dataset.
Could you train a model entirely on your support dataset? Maybe. But most likely you don't have enough data, it would be extremely expensive, and not as performant as off-the-shelf models.
So how does it work if it doesn't “learn”
Let's take the support case again. While the model can't “learn” to provide support for your business, you can equip it with tools that make it more capable at doing so. A big part of this is providing the model with the context it needs in order to answer user questions, in the same way that a human providing support would need these resources to know about your products and services.
The most common way of doing this is with RAG, or retrieval augmented generation. Basically, this involves setting up a dataset, or knowledge base, that can be searched semantically and injected as context to the LLM to help it generate an accurate response. Say a user asks a question about your return policy, with RAG you could search for a document that could be helpful for the LLM (the return policy) and inject it into its context, helping it answer the user's question.
RAG is incredibly effective at improving the accuracy rate of LLM responses and reducing hallucinations (when the LLM makes something up that isn't in the context provided). Given this, the most important thing your business can have to make AI implementations more effective is a good knowledge base of resources (that might be the same you use for human employees).
What else makes LLMs effective at business processes
Another piece of the puzzle that impacts your success is equipping your AI with the right tools for the task. As we all know, most business processes aren't just chatting with someone and getting the right response. In the same way that humans need access to be able to perform actions, so do LLMs. The LLM can't help your business if it can't look up a customer's order, generate a document, or add a note to your CRM. In the same way that human actions that touch these business processes need guardrails, so do those that are used by LLMs.
Probably the most important thing that will determine the success of your AI implementation is how it's structured: what tasks are you trying to optimize with AI, are they things that AI is actually good at, and how effective is your implementation at achieving those things.
Having the right partner help map these things out is essential. Someone with an appropriate understanding of how LLMs work can tell you which tasks can be accomplished with non-AI automations, which ones likely require a human in the loop, and which ones are appropriate for AI.
What about “reasoning” models
Some providers have introduced new versions of their models that they market as “reasoning” models with chain-of-thought capabilities. These models don't actually reason in the same way that humans do, they just use LLM capabilities in a pattern that can make them more effective at problem solving. Basically, the LLM has a “thinking” step (which just involves the generation of text) where it can expand on options for how to respond, and that informs the ultimate response. This can be effective for certain more complex tasks (such as math word problems, or planning), but is not well-suited to all situations.
The takeaway
LLMs are incredible technology that can be effective for a lot of different things, but when it comes to using them for your business there are several important things to keep in mind: what sorts of tasks are LLMs good at, how do you put the appropriate systems in place, and how do you measure their effectiveness.
This article is not exhaustive, and certainly glosses over a lot, but hopefully it provides a starting point for thinking about how you can implement AI effectively in your business.
Want to leverage our expertise to set up AI effectively as part of your support operation? Get in touch.