Blog

42 notes on LLMs, deep learning, and building things — written to be understood.

All (42)aidataengineeringllmprojects

Deep Learning

Deep learning is the subfield of machine learning built around neural networks with many layers — "deep" because of the stack of transformations between input and…

ai
Wed Oct 22 2025 08:00:00 GMT+0800 (Singapore Standard Time)1 tag

Chinchilla: Training Compute-Optimal Large Language Models

Chinchilla (Hoffmann et al., 2022) is the paper that corrected a mistake the entire field was making about how to train large language models. It asked a deceptively…

llm
Tue Jun 30 2026 08:00:00 GMT+0800 (Singapore Standard Time)1 tag

DPO: Direct Preference Optimization

DPO (Rafailov et al., 2023) is the method I now reach for by default when aligning a model. It delivers what RLHF gives you — a model that prefers responses humans like…

llm
Tue Jul 07 2026 08:00:00 GMT+0800 (Singapore Standard Time)1 tag

Connection-Oriented Transport - TCP

These are my notes on the TCP chapters from Computer Networking: A TopDown Approach (Kurose & Ross). TCP is the protocol that makes the unreliable IP layer feel…

engineering
Thu May 18 2017 08:00:00 GMT+0800 (Singapore Standard Time)1 tag

Advanced JavaScript - Exercises and Gotchas

A while back I worked through a set of JavaScript exercises designed to expose the parts of the language people gloss over: function binding, named vs. anonymous…

engineering
Thu Mar 16 2017 08:00:00 GMT+0800 (Singapore Standard Time)1 tag

Constitutional AI: Harmlessness from AI Feedback

Constitutional AI (Bai et al., 2022, arXiv:2212.08073) asked a pragmatic question: can we make a model harmless without relying on armies of human raters labeling toxic…

llm
Thu Jul 09 2026 08:00:00 GMT+0800 (Singapore Standard Time)1 tag

Computer Vision

Computer vision (CV) is the field of getting computers to understand images — classify them, find objects, segment regions, describe scenes. For a long time it was…

ai
Thu Jan 15 2026 08:00:00 GMT+0800 (Singapore Standard Time)1 tag

IE Quirk: Clicks on <a> Inside <button> Don't Fire

A friend in the US asked me to help with a parentchild education website. During our conversation I learned there are still a fair number of IE users over there, so he…

engineering
Thu Apr 26 2018 08:00:00 GMT+0800 (Singapore Standard Time)1 tag

Reflexion: Verbal Reinforcement Learning for Language Agents

Reflexion (Shinn et al., 2023, arXiv:2303.11366) solves a frustrating problem with LLM agents: they make the same mistake twice. A standard ReAct agent that fails a task…

llm
Sun May 31 2026 08:00:00 GMT+0800 (Singapore Standard Time)1 tag

ReAct: Reasoning + Acting in Language Models

ReAct (Yao et al., 2022) is the paper that gave us the recognizable shape of an LLM agent: think, act, observe, repeat. Before ReAct, reasoning (ChainofThought) and…

llm
Sun May 17 2026 08:00:00 GMT+0800 (Singapore Standard Time)1 tag

RAG: Retrieval-Augmented Generation

RetrievalAugmented Generation (RAG, Lewis et al., 2020, arXiv:2005.11401) is, hands down, the most widely deployed LLM architecture in production. If you've used a…

llm
Sun May 10 2026 08:00:00 GMT+0800 (Singapore Standard Time)1 tag

DeepSeek-R1: Reasoning via Reinforcement Learning

DeepSeekR1 (DeepSeek, 2025, arXiv:2501.12948) is the release that showed the world you can get o1level reasoning from an openweight model — and, more importantly, that…

llm
Sun May 03 2026 08:00:00 GMT+0800 (Singapore Standard Time)1 tag

Chain-of-Thought Prompting: Let the Model Think Step by Step

ChainofThought (CoT) prompting (Wei et al., 2022) is probably the highestROI prompting technique ever published. It costs you almost nothing — a sentence in the prompt —…

llm
Sun Mar 29 2026 08:00:00 GMT+0800 (Singapore Standard Time)1 tag

InstructGPT: Aligning LLMs with Human Preference (RLHF)

InstructGPT (Ouyang et al., 2022) is the paper that made LLMs useful as assistants. Before it, GPT3 was a brilliant nexttoken predictor that rarely did what you asked.…

llm
Sun Mar 22 2026 08:00:00 GMT+0800 (Singapore Standard Time)1 tag

GPT-3 and Few-Shot Learning at Scale

GPT3 (Brown et al., 2020) is the moment the world stopped thinking of language models as "text completers" and started thinking of them as programmable via prompts. The…

llm
Sun Mar 15 2026 08:00:00 GMT+0800 (Singapore Standard Time)1 tag

BERT: Bidirectional Pretraining from Transformers

BERT (Devlin et al., 2018) is the model that made "pretrain on a huge corpus, then finetune on your task" the default playbook for NLP — and for a few years, almost…

llm
Sun Mar 08 2026 08:00:00 GMT+0800 (Singapore Standard Time)1 tag

The Transformer: Attention Is All You Need

Every large language model you use today — GPT, Claude, LLaMA, Gemini — is built on a single 2017 paper that, frankly, I did not fully appreciate when it came out.…

llm
Sun Mar 01 2026 08:00:00 GMT+0800 (Singapore Standard Time)1 tag

Scaling Laws for Neural Language Models

Scaling Laws (Kaplan et al., 2020) is the paper that turned "make the model bigger" from folklore into engineering guidance. Before it, nobody could say how much bigger,…

llm
Sun Jun 28 2026 08:00:00 GMT+0800 (Singapore Standard Time)1 tag

Tree of Thoughts: Deliberate Problem Solving

Tree of Thoughts (ToT, Yao et al., 2023, arXiv:2305.10601) takes ChainofThought and makes it deliberate. Instead of generating one linear reasoning path and hoping it's…

llm
Sun Jun 21 2026 08:00:00 GMT+0800 (Singapore Standard Time)1 tag

AutoGen: Multi-Agent Conversation as Computation

AutoGen (Microsoft, 2023) is a framework that treats multiagent conversation as the unit of computation. Instead of one prompt solving a task, you spin up several agents…

llm
Sun Jun 14 2026 08:00:00 GMT+0800 (Singapore Standard Time)1 tag

Generative Agents: Believable Simulacra of Human Behavior

Generative Agents (Park et al., 2023) is the famous "AI town" paper — 25 LLMpowered characters who go to work, gossip, throw parties, and form relationships in a…

llm
Sun Jun 07 2026 08:00:00 GMT+0800 (Singapore Standard Time)1 tag

RLHF: Deep Reinforcement Learning from Human Preferences

RLHF — Reinforcement Learning from Human Feedback — is the technique family behind every helpful modern chatbot. While InstructGPT (covered separately) popularized the…

llm
Sun Jul 05 2026 08:00:00 GMT+0800 (Singapore Standard Time)1 tag

Machine Learning Explainability

Machine learning explainability is the set of methods for answering "why did the model predict that?" As models moved from simple linear regressions to opaque neural…

ai
Sun Dec 28 2025 08:00:00 GMT+0800 (Singapore Standard Time)1 tag

Mixtral of Experts: Sparse Mixture of Experts

Mixtral (Mistral AI, 2023) is the model that made mixtureofexperts (MoE) practical and popular in the open ecosystem. The idea behind MoE is old, but Mixtral shipped it…

llm
Sun Apr 26 2026 08:00:00 GMT+0800 (Singapore Standard Time)1 tag

LLaMA: The Open-Weight Catalyst

LLaMA (Meta, 2023) is the release that kicked off the openweight LLM ecosystem we now take for granted. Before it, if you wanted a capable model you used a closed API…

llm
Sun Apr 19 2026 08:00:00 GMT+0800 (Singapore Standard Time)1 tag

FlashAttention: IO-Aware Exact Attention

FlashAttention (Dao et al., 2022, arXiv:2205.14135) is one of those papers that looks like a systems optimization but quietly reshaped what models are even possible. The…

llm
Sun Apr 12 2026 08:00:00 GMT+0800 (Singapore Standard Time)1 tag

LoRA: Low-Rank Adaptation of Large Models

LoRA (Hu et al., 2021) is the technique that made it practical for ordinary people — not just labs with clusters — to specialize a large model. If you've ever downloaded…

llm
Sun Apr 05 2026 08:00:00 GMT+0800 (Singapore Standard Time)1 tag

Getting Started with iptables

is the userspace tool for configuring the Linux kernel firewall (netfilter). It looks arcane at first, but it's really just a set of tables, each containing chains of…

engineering
Sat Nov 25 2017 08:00:00 GMT+0800 (Singapore Standard Time)1 tag

Softmax: Turning Logits into Probabilities

Softmax is the function that turns a model's raw scores into a probability distribution. It is tiny, it is everywhere, and it has a numerical trap that bites almost…

llm
Sat Jul 18 2026 08:00:00 GMT+0800 (Singapore Standard Time)1 tag

Advanced SQL

SQL is easy to start and surprisingly deep to master. The "advanced" parts — window functions, CTEs, query planning, indexing — are what separate "I can fetch rows" from…

data
Sat Jan 10 2026 08:00:00 GMT+0800 (Singapore Standard Time)1 tag

Game AI and Reinforcement Learning

Reinforcement learning (RL) is the branch of ML where an agent learns by trial and error, receiving rewards for good actions and penalties for bad ones — rather than…

ai
Sat Dec 20 2025 08:00:00 GMT+0800 (Singapore Standard Time)1 tag

Principles of Economics (Mankiw)

Notes from Mankiw's Principles of Economics. The book's framing is deceptively simple — ten principles that supposedly explain most of what economics studies. What I…

engineering
Mon Jun 26 2017 08:00:00 GMT+0800 (Singapore Standard Time)1 tag

Self-Consistency: Improving Chain-of-Thought Reasoning

SelfConsistency (Wang et al., 2022, arXiv:2203.11171) is, in my experience, the best accuracyperdollar trick in LLM reasoning. It builds directly on ChainofThought:…

llm
Fri Jul 10 2026 08:00:00 GMT+0800 (Singapore Standard Time)1 tag

RoPE: Rotary Position Embedding

Rotary Position Embedding (RoPE, Su et al., 2021) is the positional encoding used in LLaMA, Mistral, Qwen, and most openweight models you'll actually deploy. If you've…

llm
Fri Jul 03 2026 08:00:00 GMT+0800 (Singapore Standard Time)1 tag

Geospatial Analysis

Geospatial analysis is working with data that has a location — coordinates, regions, routes. It shows up everywhere once you notice it: delivery zones, disease spread,…

data
Fri Jan 02 2026 08:00:00 GMT+0800 (Singapore Standard Time)1 tag