How Large Language Models Work: Explained Simply

Introduction Recently, about a week ago, I finally figured out how the statistical system behind LLM models actually works. Most people only see the surface, but underneath there are transformers, attention mechanisms, training methods, alignment, and self-evaluation. In this post I would like to give only a minimal understanding of everything that is hidden under machine learning and LLM or RAG systems. Also, there was no depth in such topics as self-attention, cross-attention, generation of pictures and more specific topics. Here we will only go through the top to understand how the model answers our questions. Let’s break it down step by step so it’s clear even for someone who is hearing about neural networks for the first time. How rregular LLM works A Regular Model Is Simply a Next-Token Predictor A regular large language model is essentially a generator of the next most probable token. At the core of LLMs is the transformer architecture, which uses attention mechanisms to determi