Linear Algebra for LLMs: A Geometric & Code-First Approach
A focused curriculum that replaces manual matrix calculation with TypeScript implementation, specifically designed to build the mathematical intuition required to understand Transformer architectures and vector databases.
Course Syllabus
Mapping Meaning to Vector Spaces
How word meanings are turned into spatial coordinates and compared using vector arithmetic.
- 01Representing text as TypeScript arrays
Moving from strings to numerical coordinates and why 'direction' represents semantic meaning.
- 02Measuring similarity with the Dot Product
Coding the dot product from scratch and visualizing how it measures alignment between two embedding vectors.
- 03Normalizing vectors for Cosine Similarity
How to implement L2 normalization to ensure length doesn't distort semantic similarity in LLMs.
Linear Transformations as Data Processing
How matrices act as the 'engines' that transform input embeddings into useful hidden states.
- 01Building a Matrix from scratch
Representing weight matrices in TypeScript and understanding them as foundations for neural layers.
- 02Rotating and Stretching data spaces
Coding a transformation function to see how a matrix 'moves' a vector from one position to another.
- 03Detecting features via Basis Vectors
Understanding how a matrix transformation looks for specific patterns or 'features' in the input data.
Dimensionality and Projection
The mechanics of moving data between different levels of abstraction (latent spaces).
- 01Squashing data with Projections
Implementing the projection formula to see how one vector can be 'mapped' onto another, mimicking the core of attention scores.
- 02Handling the curse of High Dimensions
Why 1536-dimensional vectors (like OpenAI embeddings) behave differently than 2D/3D visualizations.
- 03Identifying linear independence in Data
Writing a check to see if your data vectors provide new information or are just 'echoes' of existing dimensions.