Chapter 3: Binary Classification Systems
The Next Step: Logistic Regression and SVM
With supply predictions working, Ethan and the team tackle no-shows; Hazel guides them through logistic regression and SVM to turn yes/no decisions into better scheduling.
This chapter introduces binary classification: turning signals into a yes/no decision, evaluating performance correctly, and controlling overfitting with regularization.
Use the interactive sections below to experiment with decision boundaries, thresholds, validation, and regularization:
3.1 - SVM Classification Game: Learn about Support Vector Machines, one of the most powerful classification algorithms. Find the optimal decision boundary that separates two classes by maximizing the margin.
3.1 - The Probability Translator: Discover logistic regression, the engine behind the team’s new model. See how it transforms input signals into actionable probabilities.
3.2 - The Classifier: Learn to find the optimal classification boundary by tuning the model’s different parameters. Experiment with the decision threshold and observe how it affects predictions.
3.3 - The Risk Manager: Step into the shoes of a manager and adjust the model’s decision threshold. Experience the critical balance between the cost of false positives and false negatives.
3.3 - The Complexity Tamer: Battle overfitting. Adjust complexity and regularization to create a model that learns real patterns instead of memorizing noise, ensuring it performs well on future cases.
3.3 - Regularization Comparison: Discover the differences between L1 (Lasso), L2 (Ridge), and Elastic Net. See in real-time how each regularization type affects model weights and feature selection, helping you choose the best strategy for your problem.
3.5 - The Honest Validator: Understand why a single test isn’t enough. Compare simple validation with cross-validation (K-Fold) — the method the team uses to obtain a stable and reliable error estimate.
3.5 - ROC Curves and AUC: Explore the trade-offs between true positive and false positive rates. Learn how to interpret ROC curves and calculate AUC to evaluate model performance across different thresholds.
Algorithm Pseudocode
- 📝 Logistic Regression Pseudocode: Complete pseudocode for binary and multiclass logistic regression with gradient descent training.
Bibliography and Additional Resources
- 📚 Logistic Regression & Probability Translators: Theoretical foundations, tooling, and interpretability for The Probability Translator.
- 📚 Classification Thresholds, Validation & Regularization: Metrics, cross-validation strategies, and complexity control supporting the Risk Manager, Honest Validator, and Complexity Tamer demos.