Large Language Models (LLMs) Explained
WHAT ARE LARGE LANGUAGE MODELS (LLMs)?
A large language model (LLM) is a type of artificial intelligence system that uses deep learning techniques, particularly transformer-based neural networks, to understand, generate, and predict text. LLMs are trained on massive datasets, often containing billions or trillions of words from various sources like books, websites, and conversations. These models are capable of generating human-like text, answering questions, and performing a wide range of language-related tasks.
Notable Examples:
- OpenAI's GPT-4: A highly advanced LLM that builds on its predecessor (GPT-3) with enhanced capabilities in natural language understanding and generation.
- Gemini: A multimodal AI that integrates both text and images, expanding beyond just language processing to handle visual data.
- Claude: An AI model focused on safe and human-aligned applications, useful for academic research, content creation, and problem-solving.
HOW DO LLMS WORK WITH ALGORITHMS?
Think of it like this: If you type, “peanut butter and…,” into ChatGPT, the model (or algorithm) has seen enough examples to guess “jelly” is the word to most likely follow. The software program used an algorithm to make that prediction, based on the patterns it learned during training.
But what makes the LLM powerful—and sometimes risky—is that the predication process is mostly invisible. The person doesn't see the decisions the algorithm is making or the biases it might have learned from its training data. That’s why learning how LLMs and algorithms work matters: so students can begin to notice what’s usually hidden and ask better questions about what is being shown—and why.
Now think of a more complicated example to illustrate said risk like "teens and...." or "schools and...." or even "3-D printing and...". Given the wrong algorithm in the right hands, "gun" make be a likely predictor for all three.
Ultimately, this highlights why critical awareness is essential when using tools like LLMs. These software systems don’t understand context or consequences, they simply automate what is statistically most likely to come next based on its training data. The output, if left unchecked, doesn’t just shape what students see--it subtly influences how a person may think. If you don’t question the output, one may take it as neutral or factual, when it’s anything but.
Look Under the Hood | Deep Dive LLM Explanation
"And the remarkable thing is that when ChatGPT does something like write an essay what it’s essentially doing is just asking over and over again “given the text so far, what should the next word be?”—and each time adding a word. (More precisely, as I’ll explain, it’s adding a “token”, which could be just a part of a word, which is why it can sometimes “make up new words”.)"
" You can think of the attention mechanism as a matchmaking service for words. Each word makes a checklist (called a query vector) describing the characteristics of words it is looking for. Each word also makes a checklist (called a key vector) describing its own characteristics. The network compares each key vector to each query vector (by computing a dot product) to find the words that are the best match. Once it finds a match, it transfers information from the word that produced the key vector to the word that produced the query vector.
For example, in the previous section we showed a hypothetical transformer figuring out that in the partial sentence “John wants his bank to cash the,” his refers to John. Here’s what that might look like under the hood. The query vector for his might effectively say “I’m seeking: a noun describing a male person.” The key vector for John might effectively say “I am: a noun describing a male person.” The network would detect that these two vectors match and move information about the vector for John into the vector for his."
Discussion Questions
After choosing one of our 'deep dive' readings, answer these questions with your group and share your findings with the class:
- What did you learn about algorithms and LLMs?
- What do you still have questions on?
- How does learning about algorithms inform your day-to-day social media/smart phone (X) usage?
- How does learning about algorithms inform your future writing and research? (REVISE)
- Do you trust algorithms? Why or why not?
- What is an example of an algorithm you rely on the most?
