Notes on LLMs, deep learning, and building things that work.
Paper Deep-Dives 25 posts
Architecture & Attention 3
-
RoPE: Rotary Position Embedding 07/03
-
FlashAttention: IO-Aware Exact Attention 04/12
-
The Transformer: Attention Is All You Need 03/01
Training & Scaling 7
-
Chinchilla: Training Compute-Optimal Large Language Models 06/30
-
Scaling Laws for Neural Language Models 06/28
-
Mixtral of Experts: Sparse Mixture of Experts 04/26
-
LLaMA: The Open-Weight Catalyst 04/19
-
LoRA: Low-Rank Adaptation of Large Models 04/05
-
GPT-3 and Few-Shot Learning at Scale 03/15
-
BERT: Bidirectional Pretraining from Transformers 03/08
Alignment & RLHF 4
-
Constitutional AI: Harmlessness from AI Feedback 07/09
-
DPO: Direct Preference Optimization 07/07
-
RLHF: Deep Reinforcement Learning from Human Preferences 07/05
-
InstructGPT: Aligning LLMs with Human Preference (RLHF) 03/22
Reasoning 4
-
Self-Consistency: Improving Chain-of-Thought Reasoning 07/10
-
Tree of Thoughts: Deliberate Problem Solving 06/21
-
DeepSeek-R1: Reasoning via Reinforcement Learning 05/03
-
Chain-of-Thought Prompting: Let the Model Think Step by Step 03/29
Agents & Tools 6
-
AutoGen: Multi-Agent Conversation as Computation 06/14
-
Generative Agents: Believable Simulacra of Human Behavior 06/07
-
Reflexion: Verbal Reinforcement Learning for Language Agents 05/31
-
Toolformer: Language Models That Teach Themselves to Use Tools 05/24
-
ReAct: Reasoning + Acting in Language Models 05/17
-
RAG: Retrieval-Augmented Generation 05/10
Multimodal 1
AI & Machine Learning 4 posts
-
Computer Vision 01/15
-
Machine Learning Explainability 12/28
-
Game AI and Reinforcement Learning 12/20
-
Deep Learning 10/22
Data Science 3 posts
Projects 3 posts
-
Deploy OpenClaw AI on Raspberry Pi: Our Open Source Guide 02/04
-
Homeware Sense: Open Source Smart Home Integration for OpenClaw AI 02/02
-
wechat-dev-mcp: Bridging AI and WeChat Mini-Program Development 01/31