AI fundamentals

Large Language Model (LLM)

A large language model is an AI system trained on vast amounts of text data to understand and generate human language. LLMs like Claude, GPT-4, and Gemini are the foundation of most modern AI tools. They predict the most likely next word in a sequence — which, at scale and with enough data, produces remarkably capable language understanding and generation.

The "large" refers to the number of parameters (internal settings) the model learned during training, often in the billions or trillions.

Foundation Model

A foundation model is a large AI model trained on broad, general data that can be adapted for many different tasks. Rather than training a separate model for each use case, a foundation model serves as a starting point that can be fine-tuned or prompted for specific applications. Claude, GPT-4, and Gemini are all foundation models.

The term reflects the idea that these models form the foundation on which more specialized applications are built.

Generative AI

Generative AI refers to AI systems that create new content — text, images, audio, video, code — rather than just classifying or analyzing existing content. The current wave of AI tools (Claude, ChatGPT, Midjourney, Suno) are all generative AI. What makes generative AI practically significant for businesses is that content creation — writing, drafting, summarizing, formatting — is a large proportion of knowledge worker time, and generative AI compresses that time dramatically.

Machine Learning

Machine learning is the branch of AI in which systems learn patterns from data rather than following explicitly programmed rules. Instead of a programmer writing "if this, then that," a machine learning model is trained on thousands or millions of examples and learns to recognize patterns itself. Most modern AI tools — including LLMs — are built on machine learning.

The practical implication: AI capabilities improve as more data becomes available and training methods improve, without requiring manual reprogramming.

Natural Language Processing (NLP)

Natural language processing is the field of AI focused on enabling machines to understand, interpret, and generate human language. NLP is what allows AI tools to read your emails, understand your questions, and respond in coherent sentences. Modern LLMs have dramatically advanced NLP capabilities — where earlier NLP systems struggled with context, nuance, and ambiguity, today's models handle them with remarkable accuracy.

AI Hallucination

AI hallucination is when an AI model generates information that sounds confident and plausible but is factually incorrect or entirely fabricated. The term reflects that the model isn't lying — it genuinely has no awareness that the output is wrong. Hallucinations happen because LLMs generate text based on pattern matching, not factual lookup.

They are most common when models are asked about specific facts, recent events, or niche topics where training data is sparse. The practical implication: always verify AI-generated factual claims before using them in important contexts.

Context Window

The context window is the maximum amount of text an AI model can process and "remember" in a single conversation. Everything outside the context window — earlier parts of a long conversation, documents uploaded previously — is invisible to the model. Context windows are measured in tokens (roughly 0.75 words per token).

Modern models have context windows ranging from tens of thousands to over a million tokens. For business users, context window size determines how long your documents can be, how much conversation history the AI retains, and how complex the tasks it can handle in one session.

Token

A token is the basic unit of text that AI models process. Roughly equivalent to three-quarters of a word in English — "understanding" might be two tokens, "AI" one token, "the" one token. Tokens matter for two practical reasons: they determine the cost of using AI APIs (most are priced per token), and they determine how much content fits in a context window.

For most business users, tokens are an invisible implementation detail — but understanding the concept helps explain why very long documents sometimes need to be broken into sections.

Training Data

Training data is the collection of text, images, or other content that an AI model learned from during its development. The capabilities, biases, and knowledge of an AI model are largely determined by its training data. An LLM trained primarily on internet text will reflect the patterns, perspectives, and information present on the internet — including its gaps and inaccuracies.

Training data has a cutoff date, which is why AI models don't know about events that happened after their training was completed.

Fine-Tuning

Fine-tuning is the process of taking a pre-trained foundation model and further training it on a smaller, more specific dataset to improve its performance for a particular task or domain. A general-purpose LLM fine-tuned on medical literature becomes better at clinical language. One fine-tuned on a company's internal documents becomes better at answering questions about that company's products and processes.

Fine-tuning is how organizations customize AI behavior without training a model from scratch — which would require enormous compute resources and data.

RAG (Retrieval-Augmented Generation)

Retrieval-Augmented Generation is a technique that combines an AI model's language capabilities with the ability to look up specific information from a defined knowledge base. Instead of relying solely on what the model learned during training, RAG retrieves relevant documents or data at the moment of a query and uses them to inform the response. For businesses, RAG is what makes it possible to build AI tools that answer questions accurately about your specific products, policies, and internal knowledge — without the hallucination risk that comes from asking a general model to remember specific facts.

Inference

Inference is the process of using a trained AI model to generate outputs — in other words, actually running the model to get a response. Training is what happens when the model learns from data (expensive, done once or periodically). Inference is what happens every time you send a message to an AI tool (cheaper, happens constantly).

The speed and cost of inference determine how quickly AI tools respond and how much it costs to run them at scale. Advances in inference efficiency are a major driver of falling AI costs.

AI Model

An AI model is a mathematical system trained to perform a specific task — recognizing images, translating language, generating text, predicting outcomes. In everyday business usage, "AI model" usually refers to a large language model like Claude or GPT-4. Different models have different strengths, sizes, costs, and capabilities.

Choosing the right model for a task is a practical decision: a smaller, faster model is appropriate for high-volume, simpler tasks; a larger, more capable model for complex reasoning and generation.

Multimodal AI

Multimodal AI refers to models that can process and generate multiple types of content — text, images, audio, video — rather than just one. Most early AI models were text-only. Modern multimodal models can analyze an image and describe it, read a chart and interpret the data, transcribe audio and summarize the content, or generate images from text descriptions.

For businesses, multimodal AI opens up use cases that weren't possible with text-only tools: analyzing visual reports, processing scanned documents, reviewing design mockups, or working with video content.

Vector Database

A vector database is a type of database designed to store and search data based on semantic similarity rather than exact matches. It's a key component of RAG systems: documents are converted into numerical representations (vectors) that capture their meaning, stored in the vector database, and retrieved based on how semantically similar they are to a query. Vector databases are what allow AI tools to answer "what does our policy say about X?" accurately — they find the most semantically relevant document sections even when the query doesn't exactly match the document wording.

Embedding

An embedding is a numerical representation of text (or other data) that captures its meaning in a form a computer can process and compare. When you convert a sentence into an embedding, similar sentences produce similar numbers — which allows AI systems to find conceptually related content, not just exact keyword matches. Embeddings are the technology behind semantic search, recommendation systems, and RAG.

For business users, embeddings are invisible — they're the mechanism that makes "search by meaning" possible in AI-powered knowledge management systems.

Reinforcement Learning from Human Feedback (RLHF)

RLHF is a training technique used to align AI models with human preferences. Human raters compare AI outputs and indicate which is better; these preferences are used to train a reward model that the AI then tries to maximize. RLHF is how models like Claude and GPT are trained to be helpful, harmless, and honest rather than just statistically likely — it takes a capable model and shapes its behavior to match what humans actually want.

Most modern LLMs use some form of RLHF or its variants as a key part of their training process.

Model Collapse

Model collapse is a theoretical failure mode in which AI models trained on AI-generated data gradually lose the diversity and quality of their outputs — because they're learning from synthetic data rather than the rich variety of human-generated content. As AI generates more of the text on the internet, future training data increasingly contains AI output, potentially degrading model quality over time. Model collapse is more of a concern for AI developers than for business users, but it's relevant context for understanding why data provenance and quality matter in AI development.

Overfitting

Overfitting is when an AI model performs well on the data it was trained on but poorly on new, unseen data — because it learned the specific patterns of the training set rather than generalizable principles. An overfitted model is essentially memorizing rather than learning. For business users, overfitting becomes relevant when building or fine-tuning custom AI models: a model fine-tuned on a small dataset might appear to work perfectly on examples from that dataset but fail on new inputs.

The practical fix is ensuring training data is diverse and representative.