CS229 机器学习：数学直觉与算法 · 开篇与课程概览
资料来源：斯坦福大学 CS229 “Machine Learning” 官方网站（https://cs229.stanford.edu/） 笔记定位：基于公开 Syllabus 与公开课程讲义整理的系统学习笔记，重点解释数学公式的直觉、给出核心算法的伪代码与逻辑解说。 生成日期：2026-09-01
0. 数据获取与可访问性记录（Phase 1 交付）
0.1 访问过程摘要
| 资源 | URL | 可访问性 | 说明 |
|---|---|---|---|
| 课程主页 | https://cs229.stanford.edu/ | ✅ 公开 | 课程描述、讲师信息、课程信息、日程表（Summer 2026 版，含 20 讲） |
| Course Logistics & FAQ | Google Docs（主页 Quick Links） | ✅ 公开 | CS229 Summer 2026 完整 FAQ：先修、Honor Code、评分、补交、AI 工具政策等 |
| Syllabus（Fall 2021） | https://cs229.stanford.edu/syllabus-fall2021.html | ✅ 公开 | 含完整讲座日程表与讲义链接 |
| Syllabus（Fall 2020 / Spring 2021） | https://cs229.stanford.edu/syllabus-fall2020.html 等 | ✅ 公开 | 含完整日程表 |
| 讲义（Lecture Notes） | https://cs229.stanford.edu/notes2021fall/*.pdf | ✅ 公开（本季度主页声明“仅斯坦福成员”后实际仍公开于旧目录） | cs229-notes1~12、deep_learning_notes、ML-advice 等 |
| Lecture Notes（2026 当季） | 主页 “Syllabus and Course Materials” | 🔒 受限 | 需 Stanford 邮箱登录 Google Drive 查看 |
| Problem Sets / 作业 | Ed / Gradescope | 🔒 受限 | FAQ 明确说明作业仅通过 Ed 发布，网站与 Canvas 均不公开 |
| Canvas | https://canvas.stanford.edu/courses/228316 | 🔒 受限 | 需 Stanford 账号；录播、办公时间日历、成绩 |
| Ed 论坛 | https://edstem.org/us/courses/99528 | 🔒 受限 | 课程唯一官方沟通渠道 |
| 期末考试 | 线下（Summer 2026: 2026-08-15 19:00–22:00） | 🔒 受限 | 需在校参加；SCPD/CGOE 学生远程监考 |
0.2 未公开资源记录
按主页说明，以下资源仅对斯坦福大学附属人员开放，本笔记无法获取其内容，仅记录其存在：
- Problem Set 0–4（作业仅通过 Ed 发布，不公开）
- Lecture 录播视频（仅 Canvas）
- Ed 论坛讨论与官方答疑
- 当季（2026）讲义 PDF（Google Drive 分享链接需 Stanford 登录）
- Midterm / Final Exam 试卷与答案
- 最终项目报告与 Poster
⚠️ 提示：主页 Quick Links 明确写道 “All links will require you to be logged into your Stanford email to access. Course documents are only shared with Stanford University affiliates.” 本笔记的所有内容均来自公开渠道（主页、公开 Syllabus、公开讲义 PDF、公开 FAQ）。
1. 课程概览（Course Overview）
1.1 课程定位
CS229 是斯坦福大学计算机科学系的研究生/高年级本科课程，提供对机器学习与统计模式识别的广泛而深入的理论介绍。它不只是一门“调库”课：课程从概率与线性代数的第一性原理出发，推导每个算法背后的数学，并训练学生用 Python/NumPy 从零实现核心算法。
1.2 课程范围（官方 Course Description）
This course provides a broad introduction to machine learning and statistical pattern recognition. Topics include: supervised learning (generative learning, parametric/non-parametric learning, neural networks); unsupervised learning (clustering, dimensionality reduction); learning theory (bias/variance tradeoffs, practical advice); reinforcement learning and adaptive control. The course will also discuss recent applications of machine learning, such as to robotic control, data mining, autonomous navigation, bioinformatics, speech recognition, and text and web data processing.
1.3 四大主题模块
| 模块 | 覆盖讲座 | 核心问题 | 代表算法 |
|---|---|---|---|
| 监督学习 Supervised Learning | L2–L7, L9–L12 | 给定带标签数据 $(x,y)$，学习映射 $x \to y$ | 线性回归、逻辑回归、GLM、GDA、朴素贝叶斯、SVM、决策树、Boosting、神经网络 |
| 无监督学习 Unsupervised Learning | L8, L16 | 给定无标签数据，发现隐藏结构 | K-Means、GMM、EM、PCA、ICA |
| 学习理论 Learning Theory | L5, L13 | 模型为何泛化？偏差/方差如何权衡？如何调试 ML 系统？ | 正则化、模型选择、交叉验证、误差分析、Ablation |
| 强化学习 Reinforcement Learning | L14–L15 | 智能体如何通过试错（奖励信号）学习序贯决策？ | MDP、值迭代、策略迭代、Q-learning、REINFORCE |
| 现代主题 Modern Topics | L17–L20 | 大规模语言模型、公平性、可解释性、隐私 | Transformer、RAG、微调、prompt 优化 |
1.4 教学目标
- 理论深度：能读懂并推导 $J(\theta) = \frac{1}{2m}\sum_{i=1}^m (h_\theta(x^{(i)}) - y^{(i)})^2$ 这类公式，理解其来源（概率解释、MLE）。
- 算法直觉：知道每个算法在做什么、为什么这样做、什么时候适用。
- 工程能力：用 Python/NumPy 实现算法、调试学习算法、做误差分析与消融实验。
2. 课程信息与政策速览（来自主页 + 公开 FAQ）
2.1 课程基本信息（Summer 2026）
- 讲师：Jehangir Amjad、Anand Avati（当季）；经典版本讲师为 Andrew Ng、Tengyu Ma 等。
- 时间地点：周二/周四 16:30–18:15，NVIDIA Auditorium；周五 CA Lecture 13:30–15:00（Skilling Auditorium）。
- 先修要求：
- 计算机基础与 Python/NumPy 编程能力（CS106A/CS106B 等效）；
- 概率论（CS109 / MATH151 / STATS116 等效）；
- 多元微积分与线性代数（MATH51 / CS205L 等效）。
- 无指定教材：讲义（Notes）是主要学习材料，公开于 Syllabus 页面。
2.2 评分与政策（来自公开 FAQ）
- 评分构成（Summer 2026）：3 个作业共 50% + 期末考试 50%（期末必须及格才能通过课程）。
- Late Policy：每人共 3 个免费迟到日；之后每天扣 20%；超过 3 天不接受。
- Honor Code：允许讨论，但必须独立撰写答案；禁止参考往年答案；禁止公开张贴作业解。
- AI 工具政策：可将生成式 AI 视为“人类合作者”——不得直接索取答案或复制解法，使用需注明。
- Ed 是唯一官方沟通渠道；禁止直接私信讲师。
- 3 单位 vs 4 单位：工作量相同，可自由选择。
3. 课程日程表（Summer 2026，来自主页）
注：主页日程表日期标注存在模板残留（Lecture 1 为 2026-01-05，其后为 2025 年日期），本笔记以讲座主题顺序为准。
| 讲次 | 主题 | 说明/关联作业 |
|---|---|---|
| L1 | Introduction | PS0 发布 |
| L2 | Supervised learning setup. LMS | PS1 发布 |
| L3 | Weighted Least Squares. Logistic regression. Newton’s Method | |
| L4 | Dataset split; Exponential family. Generalized Linear Models | PS0 截止（不计分） |
| L5 | Bias-variance tradeoff, regularization | Final Project Proposal 截止 |
| L6 | Gaussian discriminant analysis. Naive Bayes, Laplace Smoothing | PS2 发布；PS1 截止 |
| L7 | Kernels. SVM | |
| L8 | K-Means. GMM. Expectation Maximization | |
| L9 | Decision trees | |
| L10 | Boosting | PS3 发布；PS2 截止 |
| L11 | Neural Networks 1 | |
| L12 | Neural Networks 2 (backprop) | |
| — | MIDTERM（第 6 周） | 3 小时笔试 |
| L13 | ML Advice | |
| L14 | Basic concepts in RL, value iteration, policy iteration | PS4 发布；PS3 截止 |
| L15 | Model-based RL, value function approximator | |
| L16 | PCA | Final Project Milestone 截止 |
| L17 | LLMs — learning tasks, language modeling, embeddings, transformers | |
| L18 | LLMs — RAG, fine-tuning, prompt optimization, safety | PS4 截止 |
| L19 | Fairness, algorithmic bias, explainability, privacy | |
| L20 | Fairness, algorithmic bias, explainability, privacy | |
| — | Final Project Report / Poster Session |
4. 按讲次的数据记录（Phase 1 结构化产物）
[
{"lecture_number": 1, "topic": "Introduction", "key_concepts_raw": ["machine learning landscape", "supervised vs unsupervised vs reinforcement learning", "course logistics"], "available_public_info": "主页课程描述、FAQ 中的评分与政策公开；当季讲义与作业受限。"},
{"lecture_number": 2, "topic": "Supervised learning setup. LMS (Linear Regression)", "key_concepts_raw": ["supervised learning setup", "hypothesis function", "cost function", "gradient descent", "stochastic gradient descent", "normal equations", "probabilistic interpretation", "MLE"], "available_public_info": "公开讲义 cs229-notes1.pdf (Sections 1-3)；课程主页提及监督学习主题。"},
{"lecture_number": 3, "topic": "Weighted Least Squares. Logistic regression. Newton's Method", "key_concepts_raw": ["locally weighted linear regression", "logistic regression", "sigmoid function", "cross-entropy loss", "gradient ascent", "Newton-Raphson method", "Hessian"], "available_public_info": "公开讲义 cs229-notes1.pdf (Sections 4, 5, 7)。"},
{"lecture_number": 4, "topic": "Dataset split; Exponential family. Generalized Linear Models", "key_concepts_raw": ["train/dev/test split", "exponential family", "canonical link function", "GLM design choices", "softmax regression"], "available_public_info": "公开讲义 cs229-notes1.pdf (Sections 6, 8, 9)；主页提及生成学习与参数/非参数学习。"},
{"lecture_number": 5, "topic": "Bias-variance tradeoff, regularization", "key_concepts_raw": ["bias-variance decomposition", "regularization", "ridge regression", "lasso", "feature selection", "model selection", "cross-validation"], "available_public_info": "公开讲义 cs229-notes5.pdf、lecture10-bias-variance.pdf。"},
{"lecture_number": 6, "topic": "Gaussian discriminant analysis. Naive Bayes, Laplace Smoothing", "key_concepts_raw": ["generative learning", "GDA", "multivariate Gaussian", "Naive Bayes", "Laplace smoothing", "text classification"], "available_public_info": "公开讲义 cs229-notes2.pdf。"},
{"lecture_number": 7, "topic": "Kernels. SVM", "key_concepts_raw": ["feature mapping", "kernel trick", "Mercer's theorem", "SVM", "margin", "dual problem", "KKT", "SMO"], "available_public_info": "公开讲义 cs229-notes3.pdf。"},
{"lecture_number": 8, "topic": "K-Means. GMM. Expectation Maximization", "key_concepts_raw": ["clustering", "k-means", "mixture of Gaussians", "EM algorithm", "Jensen's inequality", "latent variables"], "available_public_info": "公开讲义 cs229-notes7a.pdf、cs229-notes7b.pdf、cs229-notes8.pdf。"},
{"lecture_number": 9, "topic": "Decision trees", "key_concepts_raw": ["decision trees", "entropy", "information gain", "Gini impurity", "overfitting", "pruning", "random forests"], "available_public_info": "公开讲义 lecture11-decision-trees.pdf。"},
{"lecture_number": 10, "topic": "Boosting", "key_concepts_raw": ["ensemble learning", "AdaBoost", "weak learners", "weighted error", "boosting margin"], "available_public_info": "公开讲义 lecture11-boosting.pdf。"},
{"lecture_number": 11, "topic": "Neural Networks 1", "key_concepts_raw": ["neural networks", "activation functions", "forward propagation", "vectorization", "logistic regression as neuron"], "available_public_info": "公开讲义 deep_learning_notes.pdf。"},
{"lecture_number": 12, "topic": "Neural Networks 2 (backprop)", "key_concepts_raw": ["backpropagation", "auto-differentiation", "chain rule", "gradient computation", "training dynamics"], "available_public_info": "公开讲义 deep_learning_notes.pdf (Sec 3)。"},
{"lecture_number": 13, "topic": "ML Advice", "key_concepts_raw": ["debugging learning algorithms", "bias vs variance diagnosis", "error analysis", "ablations", "hyperparameter tuning"], "available_public_info": "公开讲义 ML-advice.pdf。"},
{"lecture_number": 14, "topic": "Basic concepts in RL, value iteration, policy iteration", "key_concepts_raw": ["MDP", "reward function", "discount factor", "value function", "Bellman equation", "value iteration", "policy iteration"], "available_public_info": "公开讲义 cs229-notes12.pdf (Sections 1-2)。"},
{"lecture_number": 15, "topic": "Model-based RL, value function approximator", "key_concepts_raw": ["learning MDP model", "continuous states", "value function approximation", "fitted value iteration", "Q-learning", "policy search", "REINFORCE"], "available_public_info": "公开讲义 cs229-notes12.pdf (Sections 3-4)。"},
{"lecture_number": 16, "topic": "PCA", "key_concepts_raw": ["dimensionality reduction", "principal components", "covariance matrix", "eigenvectors", "data preprocessing", "ICA"], "available_public_info": "公开讲义 cs229-notes10.pdf (PCA)、cs229-notes11.pdf (ICA)。"},
{"lecture_number": 17, "topic": "Large language models — learning tasks, language modeling, embeddings, transformers", "key_concepts_raw": ["language modeling", "next-token prediction", "embeddings", "attention", "transformers", "self-supervised learning"], "available_public_info": "公开讲义中无专门 PDF（现代主题）；基于课程公开材料与通识知识整理。"},
{"lecture_number": 18, "topic": "Large language models — RAG, fine-tuning, prompt optimization, safety", "key_concepts_raw": ["retrieval-augmented generation", "fine-tuning", "instruction tuning", "RLHF", "prompt optimization", "safety"], "available_public_info": "同上；当季讲义受限。"},
{"lecture_number": 19, "topic": "Fairness, algorithmic bias, explainability, privacy", "key_concepts_raw": ["algorithmic bias", "fairness metrics", "group fairness", "explainability", "privacy", "differential privacy"], "available_public_info": "公开 Syllabus（Fall 2021）确认该主题存在；当季材料受限。"},
{"lecture_number": 20, "topic": "Fairness, algorithmic bias, explainability, privacy", "key_concepts_raw": ["explainability methods", "SHAP", "privacy-preserving ML", "federated learning"], "available_public_info": "同上。"}
]