{"id":3087,"date":"2026-01-13T14:57:58","date_gmt":"2026-01-13T06:57:58","guid":{"rendered":"https:\/\/teen.aiproinstitute.com\/?p=3087"},"modified":"2026-01-13T15:35:48","modified_gmt":"2026-01-13T07:35:48","slug":"machine-learning-algorithms-quick-reference","status":"publish","type":"post","link":"https:\/\/teen.aiproinstitute.com\/zh\/machine-learning-algorithms-quick-reference\/","title":{"rendered":"Machine Learning Algorithms Quick Reference"},"content":{"rendered":"
\n\t\t\t\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t\n\n\n \n \n Machine Learning Algorithms Quick Reference | AiPro Institute\u2122<\/title>\n <style>\n * {\n margin: 0;\n padding: 0;\n box-sizing: border-box;\n }\n\n body {\n font-family: 'Segoe UI', Tahoma, Geneva, Verdana, sans-serif;\n background: #ffffff;\n color: #333;\n line-height: 1.6;\n padding: 20px;\n }\n\n .container {\n max-width: 1400px;\n margin: 0 auto;\n background: white;\n padding: 40px;\n box-shadow: 0 0 20px rgba(0,0,0,0.1);\n }\n\n .header {\n background: linear-gradient(135deg, #7B3FF2 0%, #00A8E8 100%);\n color: white;\n padding: 40px;\n border-radius: 10px;\n margin-bottom: 30px;\n text-align: center;\n }\n\n .header h1 {\n font-size: 2.5em;\n margin-bottom: 10px;\n font-weight: 700;\n }\n\n .header .subtitle {\n font-size: 1.2em;\n opacity: 0.95;\n font-weight: 300;\n }\n\n .member-badge {\n display: inline-block;\n background: rgba(255,255,255,0.2);\n padding: 8px 20px;\n border-radius: 20px;\n margin-top: 15px;\n font-size: 0.9em;\n font-weight: 500;\n }\n\n .section {\n margin-bottom: 40px;\n }\n\n .section-header {\n background: linear-gradient(135deg, #7B3FF2 0%, #00A8E8 100%);\n color: white;\n padding: 20px 30px;\n border-radius: 8px;\n margin-bottom: 20px;\n font-size: 1.8em;\n font-weight: 600;\n }\n\n .subsection {\n margin-bottom: 30px;\n background: #f8f9fa;\n padding: 25px;\n border-radius: 8px;\n border-left: 5px solid #7B3FF2;\n }\n\n .subsection h3 {\n color: #7B3FF2;\n font-size: 1.5em;\n margin-bottom: 15px;\n font-weight: 600;\n }\n\n .algorithm-card {\n background: white;\n padding: 20px;\n margin-bottom: 20px;\n border-radius: 8px;\n border: 2px solid #e0e0e0;\n transition: all 0.3s ease;\n }\n\n .algorithm-card:hover {\n border-color: #7B3FF2;\n box-shadow: 0 4px 12px rgba(123, 63, 242, 0.1);\n }\n\n .algorithm-card h4 {\n color: #00A8E8;\n font-size: 1.3em;\n margin-bottom: 12px;\n font-weight: 600;\n }\n\n .algo-detail {\n margin-bottom: 15px;\n }\n\n .algo-label {\n font-weight: 600;\n color: #7B3FF2;\n display: inline-block;\n min-width: 150px;\n }\n\n .comparison-table {\n width: 100%;\n border-collapse: collapse;\n margin: 20px 0;\n background: white;\n box-shadow: 0 2px 8px rgba(0,0,0,0.1);\n }\n\n .comparison-table th {\n background: linear-gradient(135deg, #7B3FF2 0%, #00A8E8 100%);\n color: white;\n padding: 15px;\n text-align: left;\n font-weight: 600;\n }\n\n .comparison-table td {\n padding: 15px;\n border-bottom: 1px solid #e0e0e0;\n }\n\n .comparison-table tr:hover {\n background: #f8f9fa;\n }\n\n .metric-grid {\n display: grid;\n grid-template-columns: repeat(auto-fit, minmax(300px, 1fr));\n gap: 20px;\n margin: 20px 0;\n }\n\n .metric-card {\n background: white;\n padding: 20px;\n border-radius: 8px;\n border: 2px solid #e0e0e0;\n }\n\n .metric-card h4 {\n color: #7B3FF2;\n margin-bottom: 10px;\n font-size: 1.2em;\n }\n\n .formula-box {\n background: #f0f0f0;\n padding: 15px;\n border-radius: 5px;\n font-family: 'Courier New', monospace;\n margin: 10px 0;\n border-left: 4px solid #00A8E8;\n }\n\n .pro-tip {\n background: #e8f5e9;\n border-left: 5px solid #4caf50;\n padding: 15px;\n margin: 15px 0;\n border-radius: 5px;\n }\n\n .pro-tip strong {\n color: #2e7d32;\n }\n\n .warning-box {\n background: #fff3e0;\n border-left: 5px solid #ff9800;\n padding: 15px;\n margin: 15px 0;\n border-radius: 5px;\n }\n\n .warning-box strong {\n color: #e65100;\n }\n\n .flowchart {\n background: white;\n padding: 30px;\n border-radius: 8px;\n border: 2px solid #7B3FF2;\n margin: 20px 0;\n }\n\n .flowchart-step {\n background: linear-gradient(135deg, #7B3FF2 0%, #00A8E8 100%);\n color: white;\n padding: 15px;\n margin: 10px 0;\n border-radius: 8px;\n text-align: center;\n font-weight: 500;\n }\n\n .flowchart-arrow {\n text-align: center;\n color: #7B3FF2;\n font-size: 2em;\n margin: 5px 0;\n }\n\n .code-snippet {\n background: #1e1e1e;\n color: #d4d4d4;\n padding: 20px;\n border-radius: 8px;\n overflow-x: auto;\n font-family: 'Courier New', monospace;\n margin: 15px 0;\n }\n\n .tag {\n display: inline-block;\n background: #7B3FF2;\n color: white;\n padding: 5px 12px;\n border-radius: 15px;\n font-size: 0.85em;\n margin: 5px 5px 5px 0;\n font-weight: 500;\n }\n\n .tag.supervised { background: #4caf50; }\n .tag.unsupervised { background: #ff9800; }\n .tag.reinforcement { background: #e91e63; }\n .tag.ensemble { background: #9c27b0; }\n\n ul {\n margin-left: 20px;\n margin-top: 10px;\n }\n\n ul li {\n margin-bottom: 8px;\n }\n\n .hyperparameter-grid {\n display: grid;\n grid-template-columns: repeat(auto-fit, minmax(250px, 1fr));\n gap: 15px;\n margin: 20px 0;\n }\n\n .hyperparam-item {\n background: white;\n padding: 15px;\n border-radius: 5px;\n border: 1px solid #e0e0e0;\n }\n\n .hyperparam-item strong {\n color: #00A8E8;\n }\n\n footer {\n margin-top: 50px;\n padding-top: 30px;\n border-top: 3px solid #7B3FF2;\n text-align: center;\n color: #666;\n }\n\n .footer-logo {\n font-size: 1.5em;\n font-weight: 700;\n background: linear-gradient(135deg, #7B3FF2 0%, #00A8E8 100%);\n -webkit-background-clip: text;\n -webkit-text-fill-color: transparent;\n background-clip: text;\n margin-bottom: 10px;\n }\n\n @media (max-width: 768px) {\n .container {\n padding: 20px;\n }\n\n .header h1 {\n font-size: 1.8em;\n }\n\n .section-header {\n font-size: 1.4em;\n padding: 15px 20px;\n }\n\n .comparison-table {\n font-size: 0.9em;\n }\n\n .metric-grid {\n grid-template-columns: 1fr;\n }\n }\n <\/style>\n<\/head>\n<body>\n <div class=\"container\">\n <div class=\"header\">\n <h1>\ud83e\udd16 Machine Learning Algorithms Quick Reference<\/h1>\n <div class=\"subtitle\">Comprehensive Guide to ML Algorithm Selection & Implementation<\/div>\n <div class=\"member-badge\">AiPro Institute\u2122 Members Only<\/div>\n <\/div>\n\n <!-- Algorithm Selection Flowchart -->\n <div class=\"section\">\n <div class=\"section-header\">\ud83c\udfaf Algorithm Selection Flowchart<\/div>\n <div class=\"flowchart\">\n <div class=\"flowchart-step\">START: What is your learning task?<\/div>\n <div class=\"flowchart-arrow\">\u2193<\/div>\n <div class=\"flowchart-step\">Do you have labeled data?<\/div>\n <div class=\"flowchart-arrow\">\u2193<\/div>\n <div style=\"display: grid; grid-template-columns: 1fr 1fr; gap: 20px; margin: 20px 0;\">\n <div>\n <div class=\"flowchart-step\" style=\"background: #4caf50;\">YES \u2192 Supervised Learning<\/div>\n <div style=\"padding: 15px; background: #f8f9fa; border-radius: 8px; margin-top: 10px;\">\n <strong>Classification:<\/strong> Logistic Regression, SVM, Random Forest, Neural Networks<br>\n <strong>Regression:<\/strong> Linear Regression, Ridge, Lasso, XGBoost\n <\/div>\n <\/div>\n <div>\n <div class=\"flowchart-step\" style=\"background: #ff9800;\">NO \u2192 Unsupervised Learning<\/div>\n <div style=\"padding: 15px; background: #f8f9fa; border-radius: 8px; margin-top: 10px;\">\n <strong>Clustering:<\/strong> K-Means, DBSCAN, Hierarchical<br>\n <strong>Dimensionality Reduction:<\/strong> PCA, t-SNE, Autoencoders\n <\/div>\n <\/div>\n <\/div>\n <div class=\"flowchart-arrow\">\u2193<\/div>\n <div class=\"flowchart-step\">Consider: Data size, interpretability needs, computational resources<\/div>\n <\/div>\n <\/div>\n\n <!-- Supervised Learning Algorithms -->\n <div class=\"section\">\n <div class=\"section-header\">\ud83d\udcca Supervised Learning Algorithms<\/div>\n\n <div class=\"subsection\">\n <h3>Regression Algorithms<\/h3>\n\n <div class=\"algorithm-card\">\n <h4>Linear Regression<\/h4>\n <span class=\"tag supervised\">Supervised<\/span>\n <span class=\"tag\">Regression<\/span>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Use Case:<\/span> Predicting continuous values with linear relationships\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Formula:<\/span>\n <div class=\"formula-box\">y = \u03b2\u2080 + \u03b2\u2081x\u2081 + \u03b2\u2082x\u2082 + ... + \u03b2\u2099x\u2099 + \u03b5<\/div>\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Assumptions:<\/span> Linearity, independence, homoscedasticity, normality\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Pros:<\/span> Simple, interpretable, fast training, probabilistic predictions\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Cons:<\/span> Cannot model non-linear relationships, sensitive to outliers\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Best For:<\/span> Feature analysis, baseline models, interpretable predictions\n <\/div>\n <div class=\"pro-tip\">\n <strong>\ud83d\udca1 Pro Tip:<\/strong> Check multicollinearity using VIF (Variance Inflation Factor). Values > 10 indicate problematic correlation.\n <\/div>\n <\/div>\n\n <div class=\"algorithm-card\">\n <h4>Ridge Regression (L2 Regularization)<\/h4>\n <span class=\"tag supervised\">Supervised<\/span>\n <span class=\"tag\">Regression<\/span>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Use Case:<\/span> Linear regression with multicollinearity\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Cost Function:<\/span>\n <div class=\"formula-box\">J(\u03b2) = RSS + \u03b1 \u03a3\u03b2\u00b2<\/div>\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Hyperparameter \u03b1:<\/span> Controls regularization strength (0 to \u221e)\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Effect:<\/span> Shrinks coefficients toward zero, reduces overfitting\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Best For:<\/span> High-dimensional data, correlated features\n <\/div>\n <\/div>\n\n <div class=\"algorithm-card\">\n <h4>Lasso Regression (L1 Regularization)<\/h4>\n <span class=\"tag supervised\">Supervised<\/span>\n <span class=\"tag\">Regression<\/span>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Use Case:<\/span> Feature selection with regression\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Cost Function:<\/span>\n <div class=\"formula-box\">J(\u03b2) = RSS + \u03b1 \u03a3|\u03b2|<\/div>\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Effect:<\/span> Can reduce coefficients to exactly zero (feature elimination)\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Best For:<\/span> Feature selection, sparse models, interpretability\n <\/div>\n <div class=\"pro-tip\">\n <strong>\ud83d\udca1 Pro Tip:<\/strong> Use ElasticNet (combines L1 + L2) when you have many correlated features and need feature selection.\n <\/div>\n <\/div>\n\n <div class=\"algorithm-card\">\n <h4>Polynomial Regression<\/h4>\n <span class=\"tag supervised\">Supervised<\/span>\n <span class=\"tag\">Regression<\/span>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Use Case:<\/span> Non-linear relationships\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Formula:<\/span>\n <div class=\"formula-box\">y = \u03b2\u2080 + \u03b2\u2081x + \u03b2\u2082x\u00b2 + \u03b2\u2083x\u00b3 + ... + \u03b2\u2099x\u207f<\/div>\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Degree Selection:<\/span> Start with 2-3, use cross-validation to optimize\n <\/div>\n <div class=\"warning-box\">\n <strong>\u26a0\ufe0f Warning:<\/strong> High degree polynomials can cause severe overfitting. Always use regularization and validate carefully.\n <\/div>\n <\/div>\n <\/div>\n\n <div class=\"subsection\">\n <h3>Classification Algorithms<\/h3>\n\n <div class=\"algorithm-card\">\n <h4>Logistic Regression<\/h4>\n <span class=\"tag supervised\">Supervised<\/span>\n <span class=\"tag\">Classification<\/span>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Use Case:<\/span> Binary and multiclass classification\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Activation Function:<\/span>\n <div class=\"formula-box\">\u03c3(z) = 1 \/ (1 + e\u207b\u1dbb)<\/div>\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Output:<\/span> Probability scores (0 to 1)\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Pros:<\/span> Probabilistic output, interpretable, fast, works well with linearly separable data\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Cons:<\/span> Assumes linear decision boundary, sensitive to outliers\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Best For:<\/span> Binary classification, baseline model, probability calibration needed\n <\/div>\n <\/div>\n\n <div class=\"algorithm-card\">\n <h4>Decision Trees<\/h4>\n <span class=\"tag supervised\">Supervised<\/span>\n <span class=\"tag\">Classification<\/span>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Use Case:<\/span> Non-linear classification\/regression\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Splitting Criteria:<\/span>\n <ul>\n <li><strong>Gini Impurity:<\/strong> 1 - \u03a3p\u1d62\u00b2 (classification)<\/li>\n <li><strong>Entropy:<\/strong> -\u03a3p\u1d62log\u2082(p\u1d62) (information gain)<\/li>\n <li><strong>MSE:<\/strong> Mean squared error (regression)<\/li>\n <\/ul>\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Key Hyperparameters:<\/span>\n <div class=\"hyperparameter-grid\">\n <div class=\"hyperparam-item\">\n <strong>max_depth:<\/strong> Maximum tree depth (3-10 typical)\n <\/div>\n <div class=\"hyperparam-item\">\n <strong>min_samples_split:<\/strong> Min samples to split node (2-20)\n <\/div>\n <div class=\"hyperparam-item\">\n <strong>min_samples_leaf:<\/strong> Min samples in leaf (1-10)\n <\/div>\n <div class=\"hyperparam-item\">\n <strong>max_features:<\/strong> Features to consider for split\n <\/div>\n <\/div>\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Pros:<\/span> Highly interpretable, handles non-linear data, no feature scaling needed\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Cons:<\/span> Prone to overfitting, unstable, biased toward dominant classes\n <\/div>\n <div class=\"warning-box\">\n <strong>\u26a0\ufe0f Warning:<\/strong> Single decision trees overfit easily. Use ensemble methods (Random Forest, XGBoost) for production.\n <\/div>\n <\/div>\n\n <div class=\"algorithm-card\">\n <h4>Random Forest<\/h4>\n <span class=\"tag ensemble\">Ensemble<\/span>\n <span class=\"tag\">Classification<\/span>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Use Case:<\/span> Robust classification\/regression\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Mechanism:<\/span> Ensemble of decision trees with bagging + feature randomness\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Key Hyperparameters:<\/span>\n <div class=\"hyperparameter-grid\">\n <div class=\"hyperparam-item\">\n <strong>n_estimators:<\/strong> Number of trees (100-500)\n <\/div>\n <div class=\"hyperparam-item\">\n <strong>max_depth:<\/strong> Tree depth (10-30)\n <\/div>\n <div class=\"hyperparam-item\">\n <strong>max_features:<\/strong> sqrt(n) for classification, n\/3 for regression\n <\/div>\n <div class=\"hyperparam-item\">\n <strong>min_samples_split:<\/strong> Minimum samples to split (2-10)\n <\/div>\n <\/div>\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Pros:<\/span> Reduces overfitting, handles high-dimensional data, provides feature importance\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Cons:<\/span> Less interpretable, slower prediction, larger memory footprint\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Best For:<\/span> Tabular data, feature importance analysis, robust predictions\n <\/div>\n <div class=\"pro-tip\">\n <strong>\ud83d\udca1 Pro Tip:<\/strong> Use Out-of-Bag (OOB) score to estimate generalization without separate validation set.\n <\/div>\n <\/div>\n\n <div class=\"algorithm-card\">\n <h4>Support Vector Machine (SVM)<\/h4>\n <span class=\"tag supervised\">Supervised<\/span>\n <span class=\"tag\">Classification<\/span>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Use Case:<\/span> High-dimensional classification\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Objective:<\/span> Find hyperplane that maximizes margin between classes\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Kernel Functions:<\/span>\n <ul>\n <li><strong>Linear:<\/strong> K(x, y) = x\u00b7y (linearly separable data)<\/li>\n <li><strong>Polynomial:<\/strong> K(x, y) = (x\u00b7y + c)\u1d48 (non-linear)<\/li>\n <li><strong>RBF (Gaussian):<\/strong> K(x, y) = exp(-\u03b3||x-y||\u00b2) (most common)<\/li>\n <li><strong>Sigmoid:<\/strong> K(x, y) = tanh(\u03b1x\u00b7y + c)<\/li>\n <\/ul>\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Key Hyperparameters:<\/span>\n <div class=\"hyperparameter-grid\">\n <div class=\"hyperparam-item\">\n <strong>C:<\/strong> Regularization (0.1-100, higher = less regularization)\n <\/div>\n <div class=\"hyperparam-item\">\n <strong>\u03b3 (gamma):<\/strong> RBF kernel width (0.001-10)\n <\/div>\n <div class=\"hyperparam-item\">\n <strong>kernel:<\/strong> Type of kernel function\n <\/div>\n <\/div>\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Pros:<\/span> Effective in high dimensions, memory efficient (uses support vectors)\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Cons:<\/span> Slow on large datasets, requires feature scaling, difficult to interpret\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Best For:<\/span> Text classification, image recognition, small-to-medium datasets\n <\/div>\n <\/div>\n\n <div class=\"algorithm-card\">\n <h4>Gradient Boosting (XGBoost, LightGBM, CatBoost)<\/h4>\n <span class=\"tag ensemble\">Ensemble<\/span>\n <span class=\"tag\">Classification<\/span>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Use Case:<\/span> State-of-the-art tabular data performance\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Mechanism:<\/span> Sequential ensemble where each tree corrects previous errors\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Key Hyperparameters:<\/span>\n <div class=\"hyperparameter-grid\">\n <div class=\"hyperparam-item\">\n <strong>n_estimators:<\/strong> Number of boosting rounds (100-1000)\n <\/div>\n <div class=\"hyperparam-item\">\n <strong>learning_rate:<\/strong> Shrinkage (0.01-0.3)\n <\/div>\n <div class=\"hyperparam-item\">\n <strong>max_depth:<\/strong> Tree depth (3-10)\n <\/div>\n <div class=\"hyperparam-item\">\n <strong>subsample:<\/strong> Row sampling (0.5-1.0)\n <\/div>\n <div class=\"hyperparam-item\">\n <strong>colsample_bytree:<\/strong> Column sampling (0.5-1.0)\n <\/div>\n <div class=\"hyperparam-item\">\n <strong>min_child_weight:<\/strong> Minimum sum of weights (1-10)\n <\/div>\n <\/div>\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Comparison:<\/span>\n <ul>\n <li><strong>XGBoost:<\/strong> Most popular, handles missing values, L1\/L2 regularization<\/li>\n <li><strong>LightGBM:<\/strong> Fastest, leaf-wise growth, best for large datasets<\/li>\n <li><strong>CatBoost:<\/strong> Best for categorical features, robust to overfitting<\/li>\n <\/ul>\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Best For:<\/span> Kaggle competitions, structured data, feature engineering\n <\/div>\n <div class=\"pro-tip\">\n <strong>\ud83d\udca1 Pro Tip:<\/strong> Use early stopping with validation set to prevent overfitting. Monitor train\/val loss divergence.\n <\/div>\n <\/div>\n\n <div class=\"algorithm-card\">\n <h4>K-Nearest Neighbors (KNN)<\/h4>\n <span class=\"tag supervised\">Supervised<\/span>\n <span class=\"tag\">Classification<\/span>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Use Case:<\/span> Instance-based learning, pattern recognition\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Mechanism:<\/span> Classify based on k nearest neighbors (majority vote)\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Distance Metrics:<\/span>\n <ul>\n <li><strong>Euclidean:<\/strong> \u221a\u03a3(x\u1d62 - y\u1d62)\u00b2 (most common)<\/li>\n <li><strong>Manhattan:<\/strong> \u03a3|x\u1d62 - y\u1d62|<\/li>\n <li><strong>Minkowski:<\/strong> (\u03a3|x\u1d62 - y\u1d62|\u1d56)^(1\/p)<\/li>\n <li><strong>Cosine:<\/strong> 1 - (x\u00b7y)\/(||x|| ||y||)<\/li>\n <\/ul>\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Choosing K:<\/span> Odd numbers for binary classification, use cross-validation (typical: 3-11)\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Pros:<\/span> Simple, no training phase, naturally handles multi-class\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Cons:<\/span> Slow prediction on large datasets, curse of dimensionality, requires feature scaling\n <\/div>\n <div class=\"warning-box\">\n <strong>\u26a0\ufe0f Warning:<\/strong> Always scale features! KNN is distance-based and sensitive to feature magnitudes.\n <\/div>\n <\/div>\n\n <div class=\"algorithm-card\">\n <h4>Naive Bayes<\/h4>\n <span class=\"tag supervised\">Supervised<\/span>\n <span class=\"tag\">Classification<\/span>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Use Case:<\/span> Text classification, spam detection\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Formula (Bayes' Theorem):<\/span>\n <div class=\"formula-box\">P(y|X) = P(X|y) \u00d7 P(y) \/ P(X)<\/div>\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Variants:<\/span>\n <ul>\n <li><strong>Gaussian NB:<\/strong> Continuous features (assumes normal distribution)<\/li>\n <li><strong>Multinomial NB:<\/strong> Discrete counts (text, word frequencies)<\/li>\n <li><strong>Bernoulli NB:<\/strong> Binary features (presence\/absence)<\/li>\n <\/ul>\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Assumption:<\/span> Features are conditionally independent (often violated but works well)\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Pros:<\/span> Fast, works with high dimensions, requires little training data\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Best For:<\/span> Text classification, spam filtering, sentiment analysis\n <\/div>\n <\/div>\n <\/div>\n\n <div class=\"subsection\">\n <h3>Neural Networks<\/h3>\n\n <div class=\"algorithm-card\">\n <h4>Feedforward Neural Network (MLP)<\/h4>\n <span class=\"tag supervised\">Supervised<\/span>\n <span class=\"tag\">Deep Learning<\/span>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Use Case:<\/span> Complex non-linear patterns\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Architecture:<\/span> Input layer \u2192 Hidden layers \u2192 Output layer\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Activation Functions:<\/span>\n <ul>\n <li><strong>ReLU:<\/strong> max(0, x) - Most common for hidden layers<\/li>\n <li><strong>Sigmoid:<\/strong> 1\/(1+e\u207b\u02e3) - Binary classification output<\/li>\n <li><strong>Softmax:<\/strong> e\u02e3\u2071\/\u03a3e\u02e3\u02b2 - Multi-class classification output<\/li>\n <li><strong>Tanh:<\/strong> (e\u02e3-e\u207b\u02e3)\/(e\u02e3+e\u207b\u02e3) - Alternative to sigmoid<\/li>\n <li><strong>Leaky ReLU:<\/strong> max(0.01x, x) - Fixes dying ReLU problem<\/li>\n <\/ul>\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Key Hyperparameters:<\/span>\n <div class=\"hyperparameter-grid\">\n <div class=\"hyperparam-item\">\n <strong>hidden_layers:<\/strong> Number and size [64, 32, 16]\n <\/div>\n <div class=\"hyperparam-item\">\n <strong>learning_rate:<\/strong> 0.001-0.1 (Adam: 0.001)\n <\/div>\n <div class=\"hyperparam-item\">\n <strong>batch_size:<\/strong> 32, 64, 128, 256\n <\/div>\n <div class=\"hyperparam-item\">\n <strong>dropout:<\/strong> 0.2-0.5 (prevent overfitting)\n <\/div>\n <div class=\"hyperparam-item\">\n <strong>optimizer:<\/strong> Adam, SGD, RMSprop\n <\/div>\n <div class=\"hyperparam-item\">\n <strong>epochs:<\/strong> 50-500 with early stopping\n <\/div>\n <\/div>\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Regularization Techniques:<\/span>\n <ul>\n <li><strong>Dropout:<\/strong> Randomly disable neurons during training<\/li>\n <li><strong>L1\/L2:<\/strong> Weight penalty in loss function<\/li>\n <li><strong>Batch Normalization:<\/strong> Normalize layer inputs<\/li>\n <li><strong>Early Stopping:<\/strong> Stop when validation loss stops improving<\/li>\n <\/ul>\n <\/div>\n <div class=\"pro-tip\">\n <strong>\ud83d\udca1 Pro Tip:<\/strong> Start with 2-3 hidden layers. Add more only if underfitting. Use He initialization for ReLU, Xavier for tanh\/sigmoid.\n <\/div>\n <\/div>\n\n <div class=\"algorithm-card\">\n <h4>Convolutional Neural Network (CNN)<\/h4>\n <span class=\"tag supervised\">Supervised<\/span>\n <span class=\"tag\">Deep Learning<\/span>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Use Case:<\/span> Image recognition, computer vision\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Key Layers:<\/span>\n <ul>\n <li><strong>Convolutional:<\/strong> Feature extraction with filters<\/li>\n <li><strong>Pooling:<\/strong> Downsampling (Max\/Average pooling)<\/li>\n <li><strong>Fully Connected:<\/strong> Classification layer<\/li>\n <\/ul>\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Common Architectures:<\/span> VGG, ResNet, Inception, EfficientNet\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Best For:<\/span> Image classification, object detection, facial recognition\n <\/div>\n <\/div>\n\n <div class=\"algorithm-card\">\n <h4>Recurrent Neural Network (RNN\/LSTM\/GRU)<\/h4>\n <span class=\"tag supervised\">Supervised<\/span>\n <span class=\"tag\">Deep Learning<\/span>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Use Case:<\/span> Sequential data, time series\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Variants:<\/span>\n <ul>\n <li><strong>Simple RNN:<\/strong> Vanishing gradient problem<\/li>\n <li><strong>LSTM:<\/strong> Long Short-Term Memory (solves vanishing gradient)<\/li>\n <li><strong>GRU:<\/strong> Gated Recurrent Unit (faster than LSTM)<\/li>\n <\/ul>\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Best For:<\/span> NLP, time series forecasting, speech recognition\n <\/div>\n <\/div>\n <\/div>\n <\/div>\n\n <!-- Unsupervised Learning -->\n <div class=\"section\">\n <div class=\"section-header\">\ud83d\udd0d Unsupervised Learning Algorithms<\/div>\n\n <div class=\"subsection\">\n <h3>Clustering Algorithms<\/h3>\n\n <div class=\"algorithm-card\">\n <h4>K-Means Clustering<\/h4>\n <span class=\"tag unsupervised\">Unsupervised<\/span>\n <span class=\"tag\">Clustering<\/span>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Use Case:<\/span> Customer segmentation, pattern grouping\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Algorithm:<\/span>\n <ol>\n <li>Initialize k random centroids<\/li>\n <li>Assign each point to nearest centroid<\/li>\n <li>Update centroids to mean of assigned points<\/li>\n <li>Repeat until convergence<\/li>\n <\/ol>\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Choosing K (Elbow Method):<\/span>\n <div class=\"formula-box\">Plot Within-Cluster Sum of Squares (WCSS) vs K<br>WCSS = \u03a3\u03a3||x - \u03bc\u2096||\u00b2<\/div>\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Other K Selection Methods:<\/span>\n <ul>\n <li><strong>Silhouette Score:<\/strong> (-1 to 1, higher is better)<\/li>\n <li><strong>Gap Statistic:<\/strong> Compare WCSS to random data<\/li>\n <li><strong>Davies-Bouldin Index:<\/strong> Lower is better<\/li>\n <\/ul>\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Pros:<\/span> Fast, simple, scalable to large datasets\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Cons:<\/span> Must specify K, sensitive to outliers, assumes spherical clusters\n <\/div>\n <div class=\"warning-box\">\n <strong>\u26a0\ufe0f Warning:<\/strong> K-Means is sensitive to initialization. Use k-means++ initialization or run multiple times with different seeds.\n <\/div>\n <\/div>\n\n <div class=\"algorithm-card\">\n <h4>Hierarchical Clustering<\/h4>\n <span class=\"tag unsupervised\">Unsupervised<\/span>\n <span class=\"tag\">Clustering<\/span>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Use Case:<\/span> Taxonomy creation, hierarchical relationships\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Types:<\/span>\n <ul>\n <li><strong>Agglomerative (bottom-up):<\/strong> Start with individual points, merge clusters<\/li>\n <li><strong>Divisive (top-down):<\/strong> Start with one cluster, split recursively<\/li>\n <\/ul>\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Linkage Methods:<\/span>\n <ul>\n <li><strong>Single:<\/strong> Minimum distance between clusters<\/li>\n <li><strong>Complete:<\/strong> Maximum distance between clusters<\/li>\n <li><strong>Average:<\/strong> Average distance between all pairs<\/li>\n <li><strong>Ward:<\/strong> Minimize variance (most common)<\/li>\n <\/ul>\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Output:<\/span> Dendrogram (tree diagram showing cluster hierarchy)\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Pros:<\/span> Don't need to specify K, produces dendrogram\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Cons:<\/span> Slow (O(n\u00b3)), sensitive to noise and outliers\n <\/div>\n <\/div>\n\n <div class=\"algorithm-card\">\n <h4>DBSCAN (Density-Based Spatial Clustering)<\/h4>\n <span class=\"tag unsupervised\">Unsupervised<\/span>\n <span class=\"tag\">Clustering<\/span>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Use Case:<\/span> Arbitrary-shaped clusters, outlier detection\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Parameters:<\/span>\n <ul>\n <li><strong>\u03b5 (epsilon):<\/strong> Maximum distance for neighborhood<\/li>\n <li><strong>MinPts:<\/strong> Minimum points to form dense region<\/li>\n <\/ul>\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Point Types:<\/span>\n <ul>\n <li><strong>Core:<\/strong> Has \u2265 MinPts within \u03b5<\/li>\n <li><strong>Border:<\/strong> Within \u03b5 of core point but has < MinPts<\/li>\n <li><strong>Noise:<\/strong> Neither core nor border (outlier)<\/li>\n <\/ul>\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Pros:<\/span> Finds arbitrary shapes, robust to outliers, automatically determines number of clusters\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Cons:<\/span> Struggles with varying densities, sensitive to \u03b5 and MinPts\n <\/div>\n <div class=\"pro-tip\">\n <strong>\ud83d\udca1 Pro Tip:<\/strong> Use k-distance graph to choose \u03b5. Plot sorted k-distances and find the \"elbow\" point.\n <\/div>\n <\/div>\n\n <div class=\"algorithm-card\">\n <h4>Gaussian Mixture Model (GMM)<\/h4>\n <span class=\"tag unsupervised\">Unsupervised<\/span>\n <span class=\"tag\">Clustering<\/span>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Use Case:<\/span> Soft clustering, probability-based grouping\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Mechanism:<\/span> Assumes data generated from mixture of Gaussian distributions\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Algorithm:<\/span> Expectation-Maximization (EM)\n <ul>\n <li><strong>E-step:<\/strong> Calculate probability of each point belonging to each cluster<\/li>\n <li><strong>M-step:<\/strong> Update Gaussian parameters (mean, covariance)<\/li>\n <\/ul>\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Output:<\/span> Soft assignments (probability distribution over clusters)\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Pros:<\/span> Soft clustering, can model elliptical clusters\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Best For:<\/span> When cluster membership is uncertain, probabilistic assignments needed\n <\/div>\n <\/div>\n <\/div>\n\n <div class=\"subsection\">\n <h3>Dimensionality Reduction<\/h3>\n\n <div class=\"algorithm-card\">\n <h4>Principal Component Analysis (PCA)<\/h4>\n <span class=\"tag unsupervised\">Unsupervised<\/span>\n <span class=\"tag\">Dim Reduction<\/span>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Use Case:<\/span> Feature reduction, data visualization\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Mechanism:<\/span> Linear transformation to maximize variance\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Steps:<\/span>\n <ol>\n <li>Standardize data (mean=0, std=1)<\/li>\n <li>Compute covariance matrix<\/li>\n <li>Calculate eigenvectors and eigenvalues<\/li>\n <li>Select top K eigenvectors (principal components)<\/li>\n <li>Transform data to new feature space<\/li>\n <\/ol>\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Choosing K Components:<\/span>\n <ul>\n <li>Retain components explaining 95% variance<\/li>\n <li>Use scree plot (elbow method)<\/li>\n <li>Cross-validation with downstream task<\/li>\n <\/ul>\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Pros:<\/span> Removes correlation, reduces overfitting, fast\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Cons:<\/span> Linear only, loses interpretability\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Best For:<\/span> Visualization (reduce to 2-3D), preprocessing for ML\n <\/div>\n <\/div>\n\n <div class=\"algorithm-card\">\n <h4>t-SNE (t-Distributed Stochastic Neighbor Embedding)<\/h4>\n <span class=\"tag unsupervised\">Unsupervised<\/span>\n <span class=\"tag\">Visualization<\/span>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Use Case:<\/span> High-dimensional data visualization\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Mechanism:<\/span> Non-linear dimensionality reduction preserving local structure\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Key Hyperparameter:<\/span>\n <ul>\n <li><strong>Perplexity:<\/strong> 5-50 (typical), balance between local\/global structure<\/li>\n <li><strong>Learning rate:<\/strong> 10-1000<\/li>\n <li><strong>Iterations:<\/strong> 1000-5000<\/li>\n <\/ul>\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Pros:<\/span> Excellent for visualization, reveals clusters\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Cons:<\/span> Slow, non-deterministic, cannot transform new data\n <\/div>\n <div class=\"warning-box\">\n <strong>\u26a0\ufe0f Warning:<\/strong> t-SNE is for visualization ONLY. Don't use output features for modeling (distances are meaningless).\n <\/div>\n <\/div>\n\n <div class=\"algorithm-card\">\n <h4>UMAP (Uniform Manifold Approximation and Projection)<\/h4>\n <span class=\"tag unsupervised\">Unsupervised<\/span>\n <span class=\"tag\">Dim Reduction<\/span>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Use Case:<\/span> Faster alternative to t-SNE\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Advantages over t-SNE:<\/span>\n <ul>\n <li>Faster (10-100x)<\/li>\n <li>Preserves global structure better<\/li>\n <li>Can transform new data<\/li>\n <li>Scales to larger datasets<\/li>\n <\/ul>\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Best For:<\/span> Large-scale visualization, embedding generation\n <\/div>\n <\/div>\n\n <div class=\"algorithm-card\">\n <h4>Autoencoders<\/h4>\n <span class=\"tag unsupervised\">Unsupervised<\/span>\n <span class=\"tag\">Deep Learning<\/span>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Use Case:<\/span> Non-linear dimensionality reduction, anomaly detection\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Architecture:<\/span>\n <ul>\n <li><strong>Encoder:<\/strong> Compress input to latent representation<\/li>\n <li><strong>Bottleneck:<\/strong> Low-dimensional latent space<\/li>\n <li><strong>Decoder:<\/strong> Reconstruct original input<\/li>\n <\/ul>\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Variants:<\/span>\n <ul>\n <li><strong>Variational (VAE):<\/strong> Generative model, produces probability distributions<\/li>\n <li><strong>Denoising:<\/strong> Trained to reconstruct from corrupted input<\/li>\n <li><strong>Sparse:<\/strong> Encourages sparse activations<\/li>\n <\/ul>\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Best For:<\/span> Feature learning, anomaly detection (high reconstruction error = anomaly)\n <\/div>\n <\/div>\n <\/div>\n <\/div>\n\n <!-- Reinforcement Learning -->\n <div class=\"section\">\n <div class=\"section-header\">\ud83c\udfae Reinforcement Learning Algorithms<\/div>\n\n <div class=\"subsection\">\n <h3>Core RL Concepts<\/h3>\n <div class=\"algorithm-card\">\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Components:<\/span>\n <ul>\n <li><strong>Agent:<\/strong> Decision maker<\/li>\n <li><strong>Environment:<\/strong> World agent interacts with<\/li>\n <li><strong>State (s):<\/strong> Current situation<\/li>\n <li><strong>Action (a):<\/strong> Possible moves<\/li>\n <li><strong>Reward (r):<\/strong> Feedback signal<\/li>\n <li><strong>Policy (\u03c0):<\/strong> Strategy for selecting actions<\/li>\n <\/ul>\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Goal:<\/span> Maximize cumulative reward over time\n <\/div>\n <\/div>\n\n <div class=\"algorithm-card\">\n <h4>Q-Learning<\/h4>\n <span class=\"tag reinforcement\">Reinforcement<\/span>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Use Case:<\/span> Discrete action spaces, model-free learning\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Q-Function Update:<\/span>\n <div class=\"formula-box\">Q(s,a) \u2190 Q(s,a) + \u03b1[r + \u03b3 max Q(s',a') - Q(s,a)]<\/div>\n <ul>\n <li><strong>\u03b1:<\/strong> Learning rate<\/li>\n <li><strong>\u03b3:<\/strong> Discount factor (future reward importance)<\/li>\n <li><strong>r:<\/strong> Immediate reward<\/li>\n <\/ul>\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Exploration vs Exploitation:<\/span> \u03b5-greedy strategy (explore with probability \u03b5)\n <\/div>\n <\/div>\n\n <div class=\"algorithm-card\">\n <h4>Deep Q-Network (DQN)<\/h4>\n <span class=\"tag reinforcement\">Reinforcement<\/span>\n <span class=\"tag\">Deep Learning<\/span>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Use Case:<\/span> High-dimensional state spaces (images, complex games)\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Innovation:<\/span> Neural network approximates Q-function\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Key Techniques:<\/span>\n <ul>\n <li><strong>Experience Replay:<\/strong> Store and sample past experiences<\/li>\n <li><strong>Target Network:<\/strong> Separate network for stable targets<\/li>\n <li><strong>Frame Stacking:<\/strong> Use multiple frames to capture motion<\/li>\n <\/ul>\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Best For:<\/span> Atari games, robotic control\n <\/div>\n <\/div>\n\n <div class=\"algorithm-card\">\n <h4>Policy Gradient Methods (REINFORCE, A3C, PPO)<\/h4>\n <span class=\"tag reinforcement\">Reinforcement<\/span>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Use Case:<\/span> Continuous action spaces, stochastic policies\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Approach:<\/span> Directly optimize policy \u03c0(a|s) instead of Q-values\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Popular Algorithms:<\/span>\n <ul>\n <li><strong>PPO (Proximal Policy Optimization):<\/strong> Most popular, stable training<\/li>\n <li><strong>A3C (Asynchronous Advantage Actor-Critic):<\/strong> Parallel training<\/li>\n <li><strong>SAC (Soft Actor-Critic):<\/strong> Maximum entropy RL<\/li>\n <\/ul>\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Best For:<\/span> Robotics, continuous control, game AI\n <\/div>\n <\/div>\n <\/div>\n <\/div>\n\n <!-- Performance Metrics -->\n <div class=\"section\">\n <div class=\"section-header\">\ud83d\udcca Performance Metrics by Task Type<\/div>\n\n <div class=\"subsection\">\n <h3>Classification Metrics<\/h3>\n \n <div class=\"metric-grid\">\n <div class=\"metric-card\">\n <h4>Accuracy<\/h4>\n <div class=\"formula-box\">Accuracy = (TP + TN) \/ (TP + TN + FP + FN)<\/div>\n <p><strong>Use When:<\/strong> Balanced classes<\/p>\n <p><strong>Avoid When:<\/strong> Imbalanced datasets<\/p>\n <\/div>\n\n <div class=\"metric-card\">\n <h4>Precision<\/h4>\n <div class=\"formula-box\">Precision = TP \/ (TP + FP)<\/div>\n <p><strong>Use When:<\/strong> False positives are costly (spam detection)<\/p>\n <p><strong>Interpretation:<\/strong> \"Of predicted positives, how many are correct?\"<\/p>\n <\/div>\n\n <div class=\"metric-card\">\n <h4>Recall (Sensitivity)<\/h4>\n <div class=\"formula-box\">Recall = TP \/ (TP + FN)<\/div>\n <p><strong>Use When:<\/strong> False negatives are costly (cancer detection)<\/p>\n <p><strong>Interpretation:<\/strong> \"Of actual positives, how many did we find?\"<\/p>\n <\/div>\n\n <div class=\"metric-card\">\n <h4>F1-Score<\/h4>\n <div class=\"formula-box\">F1 = 2 \u00d7 (Precision \u00d7 Recall) \/ (Precision + Recall)<\/div>\n <p><strong>Use When:<\/strong> Need balance between precision and recall<\/p>\n <p><strong>Best For:<\/strong> Imbalanced datasets<\/p>\n <\/div>\n\n <div class=\"metric-card\">\n <h4>ROC-AUC<\/h4>\n <div class=\"formula-box\">Area Under ROC Curve<br>(TPR vs FPR)<\/div>\n <p><strong>Use When:<\/strong> Evaluating ranking quality<\/p>\n <p><strong>Range:<\/strong> 0.5 (random) to 1.0 (perfect)<\/p>\n <\/div>\n\n <div class=\"metric-card\">\n <h4>Log Loss<\/h4>\n <div class=\"formula-box\">-1\/N \u03a3[y log(p) + (1-y) log(1-p)]<\/div>\n <p><strong>Use When:<\/strong> Probability calibration matters<\/p>\n <p><strong>Best For:<\/strong> Multi-class problems<\/p>\n <\/div>\n <\/div>\n\n <div class=\"pro-tip\">\n <strong>\ud83d\udca1 Pro Tip:<\/strong> For imbalanced data, use F1-Score, Precision-Recall AUC, or Matthews Correlation Coefficient (MCC) instead of accuracy.\n <\/div>\n <\/div>\n\n <div class=\"subsection\">\n <h3>Regression Metrics<\/h3>\n \n <div class=\"metric-grid\">\n <div class=\"metric-card\">\n <h4>Mean Absolute Error (MAE)<\/h4>\n <div class=\"formula-box\">MAE = 1\/n \u03a3|y\u1d62 - \u0177\u1d62|<\/div>\n <p><strong>Use When:<\/strong> Outliers present, want interpretable error<\/p>\n <p><strong>Units:<\/strong> Same as target variable<\/p>\n <\/div>\n\n <div class=\"metric-card\">\n <h4>Mean Squared Error (MSE)<\/h4>\n <div class=\"formula-box\">MSE = 1\/n \u03a3(y\u1d62 - \u0177\u1d62)\u00b2<\/div>\n <p><strong>Use When:<\/strong> Want to penalize large errors heavily<\/p>\n <p><strong>Note:<\/strong> Sensitive to outliers<\/p>\n <\/div>\n\n <div class=\"metric-card\">\n <h4>Root Mean Squared Error (RMSE)<\/h4>\n <div class=\"formula-box\">RMSE = \u221aMSE<\/div>\n <p><strong>Use When:<\/strong> Need interpretable units like MAE<\/p>\n <p><strong>Advantage:<\/strong> Same units as target<\/p>\n <\/div>\n\n <div class=\"metric-card\">\n <h4>R\u00b2 Score (Coefficient of Determination)<\/h4>\n <div class=\"formula-box\">R\u00b2 = 1 - (SS_res \/ SS_tot)<\/div>\n <p><strong>Range:<\/strong> -\u221e to 1 (1 = perfect fit)<\/p>\n <p><strong>Use When:<\/strong> Comparing models, understanding variance explained<\/p>\n <\/div>\n\n <div class=\"metric-card\">\n <h4>Mean Absolute Percentage Error (MAPE)<\/h4>\n <div class=\"formula-box\">MAPE = 100\/n \u03a3|y\u1d62 - \u0177\u1d62|\/|y\u1d62|<\/div>\n <p><strong>Use When:<\/strong> Need scale-independent metric<\/p>\n <p><strong>Warning:<\/strong> Undefined when y\u1d62 = 0<\/p>\n <\/div>\n\n <div class=\"metric-card\">\n <h4>Adjusted R\u00b2<\/h4>\n <div class=\"formula-box\">R\u00b2_adj = 1 - [(1-R\u00b2)(n-1)\/(n-p-1)]<\/div>\n <p><strong>Use When:<\/strong> Comparing models with different feature counts<\/p>\n <p><strong>Advantage:<\/strong> Penalizes unnecessary features<\/p>\n <\/div>\n <\/div>\n <\/div>\n\n <div class=\"subsection\">\n <h3>Clustering Metrics<\/h3>\n \n <div class=\"metric-grid\">\n <div class=\"metric-card\">\n <h4>Silhouette Score<\/h4>\n <div class=\"formula-box\">s = (b - a) \/ max(a, b)<\/div>\n <p><strong>Range:<\/strong> -1 to 1 (higher is better)<\/p>\n <p><strong>Use When:<\/strong> Evaluating cluster quality<\/p>\n <\/div>\n\n <div class=\"metric-card\">\n <h4>Davies-Bouldin Index<\/h4>\n <div class=\"formula-box\">DB = 1\/k \u03a3 max[(\u03c3\u1d62 + \u03c3\u2c7c)\/d(c\u1d62,c\u2c7c)]<\/div>\n <p><strong>Range:<\/strong> 0 to \u221e (lower is better)<\/p>\n <p><strong>Use When:<\/strong> Comparing different K values<\/p>\n <\/div>\n\n <div class=\"metric-card\">\n <h4>Calinski-Harabasz Index<\/h4>\n <div class=\"formula-box\">CH = (SSB\/SSW) \u00d7 [(n-k)\/(k-1)]<\/div>\n <p><strong>Range:<\/strong> 0 to \u221e (higher is better)<\/p>\n <p><strong>Best For:<\/strong> Dense, well-separated clusters<\/p>\n <\/div>\n <\/div>\n <\/div>\n <\/div>\n\n <!-- Common Pitfalls -->\n <div class=\"section\">\n <div class=\"section-header\">\u26a0\ufe0f Common Pitfalls & Solutions<\/div>\n\n <table class=\"comparison-table\">\n <thead>\n <tr>\n <th>Problem<\/th>\n <th>Symptoms<\/th>\n <th>Solutions<\/th>\n <\/tr>\n <\/thead>\n <tbody>\n <tr>\n <td><strong>Overfitting<\/strong><\/td>\n <td>High training accuracy, low test accuracy; large gap between train\/val loss<\/td>\n <td>\n \u2022 Increase training data<br>\n \u2022 Reduce model complexity<br>\n \u2022 Apply regularization (L1\/L2, dropout)<br>\n \u2022 Early stopping<br>\n \u2022 Cross-validation\n <\/td>\n <\/tr>\n <tr>\n <td><strong>Underfitting<\/strong><\/td>\n <td>Low training and test accuracy; high bias<\/td>\n <td>\n \u2022 Increase model complexity<br>\n \u2022 Add more features<br>\n \u2022 Reduce regularization<br>\n \u2022 Train longer<br>\n \u2022 Try non-linear models\n <\/td>\n <\/tr>\n <tr>\n <td><strong>Class Imbalance<\/strong><\/td>\n <td>High accuracy but poor minority class performance<\/td>\n <td>\n \u2022 SMOTE or oversampling<br>\n \u2022 Class weights<br>\n \u2022 Stratified sampling<br>\n \u2022 Use F1-score instead of accuracy<br>\n \u2022 Ensemble methods\n <\/td>\n <\/tr>\n <tr>\n <td><strong>Data Leakage<\/strong><\/td>\n <td>Unrealistically high performance; test accuracy > train accuracy<\/td>\n <td>\n \u2022 Separate test set before any processing<br>\n \u2022 Fit preprocessing on train only<br>\n \u2022 Check for future information in features<br>\n \u2022 Time-based splits for time series\n <\/td>\n <\/tr>\n <tr>\n <td><strong>Vanishing Gradients<\/strong><\/td>\n <td>Deep networks stop learning; weights don't update<\/td>\n <td>\n \u2022 Use ReLU instead of sigmoid\/tanh<br>\n \u2022 Batch normalization<br>\n \u2022 Residual connections (ResNet)<br>\n \u2022 Gradient clipping<br>\n \u2022 Better initialization\n <\/td>\n <\/tr>\n <tr>\n <td><strong>Exploding Gradients<\/strong><\/td>\n <td>NaN or Inf losses; unstable training<\/td>\n <td>\n \u2022 Gradient clipping<br>\n \u2022 Lower learning rate<br>\n \u2022 Batch normalization<br>\n \u2022 Weight regularization\n <\/td>\n <\/tr>\n <tr>\n <td><strong>Feature Scaling Issues<\/strong><\/td>\n <td>Slow convergence; poor performance with distance-based algorithms<\/td>\n <td>\n \u2022 StandardScaler (mean=0, std=1)<br>\n \u2022 MinMaxScaler (0-1 range)<br>\n \u2022 RobustScaler (use median, resistant to outliers)<br>\n \u2022 Required for: SVM, KNN, Neural Networks, PCA\n <\/td>\n <\/tr>\n <tr>\n <td><strong>Curse of Dimensionality<\/strong><\/td>\n <td>Too many features; poor generalization; distance metrics break down<\/td>\n <td>\n \u2022 Feature selection (Lasso, feature importance)<br>\n \u2022 Dimensionality reduction (PCA, UMAP)<br>\n \u2022 Regularization<br>\n \u2022 More training data\n <\/td>\n <\/tr>\n <\/tbody>\n <\/table>\n <\/div>\n\n <!-- Algorithm Comparison -->\n <div class=\"section\">\n <div class=\"section-header\">\u2696\ufe0f Algorithm Comparison Table<\/div>\n\n <table class=\"comparison-table\">\n <thead>\n <tr>\n <th>Algorithm<\/th>\n <th>Training Speed<\/th>\n <th>Prediction Speed<\/th>\n <th>Interpretability<\/th>\n <th>Handles Non-linearity<\/th>\n <th>Scales to Big Data<\/th>\n <\/tr>\n <\/thead>\n <tbody>\n <tr>\n <td><strong>Linear Regression<\/strong><\/td>\n <td>\u2b50\u2b50\u2b50\u2b50\u2b50<\/td>\n <td>\u2b50\u2b50\u2b50\u2b50\u2b50<\/td>\n <td>\u2b50\u2b50\u2b50\u2b50\u2b50<\/td>\n <td>\u274c<\/td>\n <td>\u2b50\u2b50\u2b50\u2b50\u2b50<\/td>\n <\/tr>\n <tr>\n <td><strong>Logistic Regression<\/strong><\/td>\n <td>\u2b50\u2b50\u2b50\u2b50\u2b50<\/td>\n <td>\u2b50\u2b50\u2b50\u2b50\u2b50<\/td>\n <td>\u2b50\u2b50\u2b50\u2b50<\/td>\n <td>\u274c<\/td>\n <td>\u2b50\u2b50\u2b50\u2b50\u2b50<\/td>\n <\/tr>\n <tr>\n <td><strong>Decision Trees<\/strong><\/td>\n <td>\u2b50\u2b50\u2b50\u2b50<\/td>\n <td>\u2b50\u2b50\u2b50\u2b50\u2b50<\/td>\n <td>\u2b50\u2b50\u2b50\u2b50\u2b50<\/td>\n <td>\u2705<\/td>\n <td>\u2b50\u2b50\u2b50<\/td>\n <\/tr>\n <tr>\n <td><strong>Random Forest<\/strong><\/td>\n <td>\u2b50\u2b50\u2b50<\/td>\n <td>\u2b50\u2b50\u2b50\u2b50<\/td>\n <td>\u2b50\u2b50\u2b50<\/td>\n <td>\u2705<\/td>\n <td>\u2b50\u2b50\u2b50\u2b50<\/td>\n <\/tr>\n <tr>\n <td><strong>XGBoost<\/strong><\/td>\n <td>\u2b50\u2b50\u2b50<\/td>\n <td>\u2b50\u2b50\u2b50\u2b50<\/td>\n <td>\u2b50\u2b50<\/td>\n <td>\u2705<\/td>\n <td>\u2b50\u2b50\u2b50\u2b50\u2b50<\/td>\n <\/tr>\n <tr>\n <td><strong>SVM<\/strong><\/td>\n <td>\u2b50\u2b50<\/td>\n <td>\u2b50\u2b50\u2b50<\/td>\n <td>\u2b50\u2b50<\/td>\n <td>\u2705 (with kernels)<\/td>\n <td>\u2b50\u2b50<\/td>\n <\/tr>\n <tr>\n <td><strong>KNN<\/strong><\/td>\n <td>\u2b50\u2b50\u2b50\u2b50\u2b50 (no training)<\/td>\n <td>\u2b50\u2b50<\/td>\n <td>\u2b50\u2b50\u2b50\u2b50<\/td>\n <td>\u2705<\/td>\n <td>\u2b50<\/td>\n <\/tr>\n <tr>\n <td><strong>Neural Networks<\/strong><\/td>\n <td>\u2b50<\/td>\n <td>\u2b50\u2b50\u2b50\u2b50<\/td>\n <td>\u2b50<\/td>\n <td>\u2705<\/td>\n <td>\u2b50\u2b50\u2b50\u2b50\u2b50<\/td>\n <\/tr>\n <tr>\n <td><strong>K-Means<\/strong><\/td>\n <td>\u2b50\u2b50\u2b50\u2b50<\/td>\n <td>\u2b50\u2b50\u2b50\u2b50\u2b50<\/td>\n <td>\u2b50\u2b50\u2b50\u2b50<\/td>\n <td>\u274c<\/td>\n <td>\u2b50\u2b50\u2b50\u2b50<\/td>\n <\/tr>\n <tr>\n <td><strong>DBSCAN<\/strong><\/td>\n <td>\u2b50\u2b50\u2b50<\/td>\n <td>N\/A<\/td>\n <td>\u2b50\u2b50\u2b50<\/td>\n <td>\u2705<\/td>\n <td>\u2b50\u2b50<\/td>\n <\/tr>\n <\/tbody>\n <\/table>\n <\/div>\n\n <!-- Hyperparameter Tuning Guide -->\n <div class=\"section\">\n <div class=\"section-header\">\ud83c\udf9b\ufe0f Hyperparameter Tuning Guide<\/div>\n\n <div class=\"subsection\">\n <h3>Tuning Strategies<\/h3>\n\n <div class=\"algorithm-card\">\n <h4>Grid Search<\/h4>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Approach:<\/span> Exhaustive search over specified parameter values\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Pros:<\/span> Guaranteed to find best combination in grid\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Cons:<\/span> Exponentially slow with more parameters\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Best For:<\/span> Small parameter space, computational resources available\n <\/div>\n <\/div>\n\n <div class=\"algorithm-card\">\n <h4>Random Search<\/h4>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Approach:<\/span> Random combinations from parameter distributions\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Pros:<\/span> More efficient than grid search, explores broader space\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Cons:<\/span> May miss optimal combination\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Best For:<\/span> Large parameter space, limited time\n <\/div>\n <div class=\"pro-tip\">\n <strong>\ud83d\udca1 Pro Tip:<\/strong> Random search often outperforms grid search with the same computation budget.\n <\/div>\n <\/div>\n\n <div class=\"algorithm-card\">\n <h4>Bayesian Optimization<\/h4>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Approach:<\/span> Use previous results to inform next parameter choices\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Tools:<\/span> Optuna, Hyperopt, Scikit-Optimize\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Pros:<\/span> Most efficient, learns from past trials\n <\/div>\n <div class=\"algo-detail\">\n <span class=\"algo-label\">Best For:<\/span> Expensive models (neural networks), complex parameter spaces\n <\/div>\n <\/div>\n\n <div class=\"warning-box\">\n <strong>\u26a0\ufe0f Warning:<\/strong> Always use cross-validation during hyperparameter tuning to avoid overfitting to validation set.\n <\/div>\n <\/div>\n <\/div>\n\n <!-- Quick Reference Cheat Codes -->\n <div class=\"section\">\n <div class=\"section-header\">\ud83d\udcbb Quick Reference Code Snippets<\/div>\n\n <div class=\"subsection\">\n <h3>Scikit-learn Template<\/h3>\n <div class=\"code-snippet\">\n# Complete ML Pipeline Template\nfrom sklearn.model_selection import train_test_split, cross_val_score\nfrom sklearn.preprocessing import StandardScaler\nfrom sklearn.ensemble import RandomForestClassifier\nfrom sklearn.metrics import classification_report, confusion_matrix\n\n# 1. Split data\nX_train, X_test, y_train, y_test = train_test_split(\n X, y, test_size=0.2, random_state=42, stratify=y\n)\n\n# 2. Scale features (fit on train only!)\nscaler = StandardScaler()\nX_train_scaled = scaler.fit_transform(X_train)\nX_test_scaled = scaler.transform(X_test)\n\n# 3. Train model\nmodel = RandomForestClassifier(n_estimators=100, max_depth=10, random_state=42)\nmodel.fit(X_train_scaled, y_train)\n\n# 4. Cross-validation\ncv_scores = cross_val_score(model, X_train_scaled, y_train, cv=5)\nprint(f\"CV Score: {cv_scores.mean():.3f} (+\/- {cv_scores.std():.3f})\")\n\n# 5. Evaluate\ny_pred = model.predict(X_test_scaled)\nprint(classification_report(y_test, y_pred))\nprint(confusion_matrix(y_test, y_pred))\n <\/div>\n <\/div>\n\n <div class=\"subsection\">\n <h3>Model Selection Template<\/h3>\n <div class=\"code-snippet\">\nfrom sklearn.model_selection import GridSearchCV\n\n# Define parameter grid\nparam_grid = {\n 'n_estimators': [100, 200, 300],\n 'max_depth': [5, 10, 15, None],\n 'min_samples_split': [2, 5, 10],\n 'min_samples_leaf': [1, 2, 4]\n}\n\n# Grid search with cross-validation\ngrid_search = GridSearchCV(\n RandomForestClassifier(random_state=42),\n param_grid,\n cv=5,\n scoring='f1_macro',\n n_jobs=-1,\n verbose=1\n)\n\ngrid_search.fit(X_train_scaled, y_train)\n\nprint(f\"Best parameters: {grid_search.best_params_}\")\nprint(f\"Best CV score: {grid_search.best_score_:.3f}\")\n\n# Use best model\nbest_model = grid_search.best_estimator_\n <\/div>\n <\/div>\n <\/div>\n\n <footer>\n <div class=\"footer-logo\">AiPro Institute\u2122<\/div>\n <p>Machine Learning Algorithms Quick Reference | Members Only Resource<\/p>\n <p style=\"margin-top: 10px; font-size: 0.9em;\">\n \u00a9 2024 AiPro Institute. For educational purposes. \n Keep this guide handy for algorithm selection and implementation.\n <\/p>\n <\/footer>\n <\/div>\n<\/body>\n<\/html>\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<\/div>","protected":false},"excerpt":{"rendered":"<p>Machine Learning Algorithms Quick Reference | AiPro Institute\u2122 \ud83e\udd16 Machine Learning Algorithms Quick Reference Comprehensive Guide to ML Algorithm Selection & Implementation AiPro Institute\u2122 Members Only \ud83c\udfaf Algorithm Selection Flowchart START: What is your learning task? \u2193 Do you have labeled data? \u2193 YES \u2192 Supervised Learning Classification: Logistic Regression, SVM, Random Forest, Neural Networks…<\/p>","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[83],"tags":[],"class_list":["post-3087","post","type-post","status-publish","format-standard","hentry","category-ai-machine-learning"],"acf":[],"_links":{"self":[{"href":"https:\/\/teen.aiproinstitute.com\/zh\/wp-json\/wp\/v2\/posts\/3087","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/teen.aiproinstitute.com\/zh\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/teen.aiproinstitute.com\/zh\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/teen.aiproinstitute.com\/zh\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/teen.aiproinstitute.com\/zh\/wp-json\/wp\/v2\/comments?post=3087"}],"version-history":[{"count":4,"href":"https:\/\/teen.aiproinstitute.com\/zh\/wp-json\/wp\/v2\/posts\/3087\/revisions"}],"predecessor-version":[{"id":3164,"href":"https:\/\/teen.aiproinstitute.com\/zh\/wp-json\/wp\/v2\/posts\/3087\/revisions\/3164"}],"wp:attachment":[{"href":"https:\/\/teen.aiproinstitute.com\/zh\/wp-json\/wp\/v2\/media?parent=3087"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/teen.aiproinstitute.com\/zh\/wp-json\/wp\/v2\/categories?post=3087"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/teen.aiproinstitute.com\/zh\/wp-json\/wp\/v2\/tags?post=3087"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}