Umar Jamil

Attention is all you need (Transformer) - Model explanation (including math), Inference and Training

Attention is all you need (Transformer) - Model explanation (including math), Inference and Training Umar Jamil 151K 502,261 1 год назад

Coding a Multimodal (Vision) Language Model from scratch in PyTorch with full explanation

Coding a Multimodal (Vision) Language Model from scratch in PyTorch with full explanation Umar Jamil 23K 76,318 8 месяцев назад

Flash Attention derived and coded from first principles with Triton (Python)

Flash Attention derived and coded from first principles with Triton (Python) Umar Jamil 12K 40,191 4 месяца назад

Coding a Transformer from scratch on PyTorch, with full explanation, training and inference.

Coding a Transformer from scratch on PyTorch, with full explanation, training and inference. Umar Jamil 79K 264,247 1 год назад

LLaMA explained: KV-Cache, Rotary Positional Embedding, RMS Norm, Grouped Query Attention, SwiGLU

LLaMA explained: KV-Cache, Rotary Positional Embedding, RMS Norm, Grouped Query Attention, SwiGLU Umar Jamil 27K 89,084 1 год назад

Reinforcement Learning from Human Feedback explained with math derivations and the PyTorch code.

Reinforcement Learning from Human Feedback explained with math derivations and the PyTorch code. Umar Jamil 13K 44,130 1 год назад

Paper: DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning

Paper: DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning Umar Jamil 15K 50,474 2 месяца назад

Retrieval Augmented Generation (RAG) Explained: Embedding, Sentence BERT, Vector Database (HNSW)

Retrieval Augmented Generation (RAG) Explained: Embedding, Sentence BERT, Vector Database (HNSW) Umar Jamil 21K 70,070 1 год назад

LoRA: Low-Rank Adaptation of Large Language Models - Explained visually + PyTorch code from scratch

LoRA: Low-Rank Adaptation of Large Language Models - Explained visually + PyTorch code from scratch Umar Jamil 11K 35,914 1 год назад

Decoding machine learning complexity with Writer’s Umar Jamil

Decoding machine learning complexity with Writer’s Umar Jamil Writer 268 892 5 месяцев назад

How diffusion models work - explanation and code!

How diffusion models work - explanation and code! Umar Jamil 7K 22,523 1 год назад

Coding Stable Diffusion from scratch in PyTorch

Coding Stable Diffusion from scratch in PyTorch Umar Jamil 51K 170,780 1 год назад

Mistral / Mixtral Explained: Sliding Window Attention, Sparse Mixture of Experts, Rolling Buffer

Mistral / Mixtral Explained: Sliding Window Attention, Sparse Mixture of Experts, Rolling Buffer Umar Jamil 10K 33,190 1 год назад

BERT explained: Training, Inference, BERT vs GPT/LLamA, Fine tuning, [CLS] token

BERT explained: Training, Inference, BERT vs GPT/LLamA, Fine tuning, [CLS] token Umar Jamil 17K 58,077 1 год назад

Variational Autoencoder - Model, ELBO, loss function and maths explained easily!

Variational Autoencoder - Model, ELBO, loss function and maths explained easily! Umar Jamil 14K 45,902 1 год назад

Coding LLaMA 2 from scratch in PyTorch - KV Cache, Grouped Query Attention, Rotary PE, RMSNorm

Coding LLaMA 2 from scratch in PyTorch - KV Cache, Grouped Query Attention, Rotary PE, RMSNorm Umar Jamil 15K 51,593 1 год назад

Quantization explained with PyTorch - Post-Training Quantization, Quantization-Aware Training

Quantization explained with PyTorch - Post-Training Quantization, Quantization-Aware Training Umar Jamil 10K 32,889 1 год назад

Titans: Learning to Memorize at Test Time

Titans: Learning to Memorize at Test Time Umar Jamil 3K 8,717 2 месяца назад

Distributed Training with PyTorch: complete tutorial with cloud infrastructure and code

Distributed Training with PyTorch: complete tutorial with cloud infrastructure and code Umar Jamil 7K 24,190 1 год назад

Direct Preference Optimization (DPO) explained: Bradley-Terry model, log probabilities, math

Direct Preference Optimization (DPO) explained: Bradley-Terry model, log probabilities, math Umar Jamil 7K 22,899 11 месяцев назад

Umar Jamil. Смотреть видео: Attention Is All You Need Transformer Model Explanation Including Math Inference And Training, Coding A Multimodal Vision Language Model From Scratch In PyTorch With Full Explanation, Flash Attention Derived And Coded From First Principles With Triton Python, Coding A Transformer From Scratch On PyTorch With Full Explanation Training And Inference.