{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Library Tutorial\n",
    "\n",
    "## The Journey to Building the Best Classifier: A Regular Story"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It was an overcast Wednesday morning when Alex decided to tackle a classification challenge. Armed with the `sklearn` datasets and a toolbox of custom Python functions from the `classifierpromax` package, Alex set out to build the best possible model to classify data efficiently and effectively."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "from classifierpromax.ClassifierTrainer import ClassifierTrainer\n",
    "from classifierpromax.ClassifierOptimizer import ClassifierOptimizer\n",
    "from classifierpromax.FeatureSelector import FeatureSelector\n",
    "from classifierpromax.ResultHandler import ResultHandler\n",
    "from sklearn.pipeline import make_pipeline\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.metrics import classification_report\n",
    "from sklearn import datasets"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Loading the Dataset\n",
    "Alex started by loading a familiar dataset from `sklearn`. For this project, the Iris dataset served as the training grounds—a well-known dataset for classifying different types of iris flowers."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>sepal length (cm)</th>\n",
       "      <th>sepal width (cm)</th>\n",
       "      <th>petal length (cm)</th>\n",
       "      <th>petal width (cm)</th>\n",
       "      <th>0</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>90</th>\n",
       "      <td>5.5</td>\n",
       "      <td>2.6</td>\n",
       "      <td>4.4</td>\n",
       "      <td>1.2</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>4.7</td>\n",
       "      <td>3.2</td>\n",
       "      <td>1.6</td>\n",
       "      <td>0.2</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>65</th>\n",
       "      <td>6.7</td>\n",
       "      <td>3.1</td>\n",
       "      <td>4.4</td>\n",
       "      <td>1.4</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>82</th>\n",
       "      <td>5.8</td>\n",
       "      <td>2.7</td>\n",
       "      <td>3.9</td>\n",
       "      <td>1.2</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>135</th>\n",
       "      <td>7.7</td>\n",
       "      <td>3.0</td>\n",
       "      <td>6.1</td>\n",
       "      <td>2.3</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>30</th>\n",
       "      <td>4.8</td>\n",
       "      <td>3.1</td>\n",
       "      <td>1.6</td>\n",
       "      <td>0.2</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>79</th>\n",
       "      <td>5.7</td>\n",
       "      <td>2.6</td>\n",
       "      <td>3.5</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>5.1</td>\n",
       "      <td>3.8</td>\n",
       "      <td>1.5</td>\n",
       "      <td>0.3</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>53</th>\n",
       "      <td>5.5</td>\n",
       "      <td>2.3</td>\n",
       "      <td>4.0</td>\n",
       "      <td>1.3</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>87</th>\n",
       "      <td>6.3</td>\n",
       "      <td>2.3</td>\n",
       "      <td>4.4</td>\n",
       "      <td>1.3</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>75 rows × 5 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     sepal length (cm)  sepal width (cm)  petal length (cm)  petal width (cm)  \\\n",
       "90                 5.5               2.6                4.4               1.2   \n",
       "29                 4.7               3.2                1.6               0.2   \n",
       "65                 6.7               3.1                4.4               1.4   \n",
       "82                 5.8               2.7                3.9               1.2   \n",
       "135                7.7               3.0                6.1               2.3   \n",
       "..                 ...               ...                ...               ...   \n",
       "30                 4.8               3.1                1.6               0.2   \n",
       "79                 5.7               2.6                3.5               1.0   \n",
       "19                 5.1               3.8                1.5               0.3   \n",
       "53                 5.5               2.3                4.0               1.3   \n",
       "87                 6.3               2.3                4.4               1.3   \n",
       "\n",
       "     0  \n",
       "90   1  \n",
       "29   0  \n",
       "65   1  \n",
       "82   1  \n",
       "135  2  \n",
       "..  ..  \n",
       "30   0  \n",
       "79   1  \n",
       "19   0  \n",
       "53   1  \n",
       "87   1  \n",
       "\n",
       "[75 rows x 5 columns]"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Load the dataset\n",
    "iris = datasets.load_iris()\n",
    "X, y = pd.DataFrame(iris.data, columns=iris.feature_names), pd.Series(iris.target)\n",
    "\n",
    "# Split the data\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.5, random_state=421)\n",
    "pd.concat([X_train, y_train], axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "With the data ready, Alex built a preprocessing pipeline to ensure the input features were standardized."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Preprocessing pipeline\n",
    "preprocessor = make_pipeline(StandardScaler())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Training the Initial Models\n",
    "The first step in Alex’s workflow was training baseline models using the `ClassifierTrainer` function. This function automatically trained several models, including Logistic Regression, SVC, and Random Forest, and evaluated them using cross-validation metrics."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead tr th {\n",
       "        text-align: left;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <th colspan=\"2\" halign=\"left\">dummy</th>\n",
       "      <th colspan=\"2\" halign=\"left\">logreg</th>\n",
       "      <th colspan=\"2\" halign=\"left\">svc</th>\n",
       "      <th colspan=\"2\" halign=\"left\">random_forest</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <th>mean</th>\n",
       "      <th>std</th>\n",
       "      <th>mean</th>\n",
       "      <th>std</th>\n",
       "      <th>mean</th>\n",
       "      <th>std</th>\n",
       "      <th>mean</th>\n",
       "      <th>std</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>fit_time</th>\n",
       "      <td>0.002110</td>\n",
       "      <td>1.164137e-03</td>\n",
       "      <td>0.001835</td>\n",
       "      <td>0.000562</td>\n",
       "      <td>0.001043</td>\n",
       "      <td>0.000073</td>\n",
       "      <td>0.037302</td>\n",
       "      <td>0.000230</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>score_time</th>\n",
       "      <td>0.005287</td>\n",
       "      <td>2.824569e-03</td>\n",
       "      <td>0.002606</td>\n",
       "      <td>0.000303</td>\n",
       "      <td>0.002321</td>\n",
       "      <td>0.000079</td>\n",
       "      <td>0.004144</td>\n",
       "      <td>0.000036</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>test_accuracy</th>\n",
       "      <td>0.333333</td>\n",
       "      <td>0.000000e+00</td>\n",
       "      <td>0.933333</td>\n",
       "      <td>0.081650</td>\n",
       "      <td>0.946667</td>\n",
       "      <td>0.055777</td>\n",
       "      <td>0.933333</td>\n",
       "      <td>0.047140</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>train_accuracy</th>\n",
       "      <td>0.346667</td>\n",
       "      <td>7.453560e-03</td>\n",
       "      <td>0.953333</td>\n",
       "      <td>0.013944</td>\n",
       "      <td>0.966667</td>\n",
       "      <td>0.020412</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>test_precision</th>\n",
       "      <td>0.111111</td>\n",
       "      <td>1.551584e-17</td>\n",
       "      <td>0.939333</td>\n",
       "      <td>0.079588</td>\n",
       "      <td>0.951111</td>\n",
       "      <td>0.054772</td>\n",
       "      <td>0.948063</td>\n",
       "      <td>0.033914</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>train_precision</th>\n",
       "      <td>0.120222</td>\n",
       "      <td>5.093266e-03</td>\n",
       "      <td>0.953569</td>\n",
       "      <td>0.013877</td>\n",
       "      <td>0.967508</td>\n",
       "      <td>0.019637</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>test_recall</th>\n",
       "      <td>0.333333</td>\n",
       "      <td>0.000000e+00</td>\n",
       "      <td>0.933333</td>\n",
       "      <td>0.081650</td>\n",
       "      <td>0.946667</td>\n",
       "      <td>0.055777</td>\n",
       "      <td>0.933333</td>\n",
       "      <td>0.047140</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>train_recall</th>\n",
       "      <td>0.346667</td>\n",
       "      <td>7.453560e-03</td>\n",
       "      <td>0.953333</td>\n",
       "      <td>0.013944</td>\n",
       "      <td>0.966667</td>\n",
       "      <td>0.020412</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>test_f1</th>\n",
       "      <td>0.166667</td>\n",
       "      <td>0.000000e+00</td>\n",
       "      <td>0.932929</td>\n",
       "      <td>0.082477</td>\n",
       "      <td>0.946397</td>\n",
       "      <td>0.055859</td>\n",
       "      <td>0.932088</td>\n",
       "      <td>0.049128</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>train_f1</th>\n",
       "      <td>0.178519</td>\n",
       "      <td>6.625387e-03</td>\n",
       "      <td>0.953296</td>\n",
       "      <td>0.013956</td>\n",
       "      <td>0.966621</td>\n",
       "      <td>0.020468</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                    dummy                  logreg                 svc  \\\n",
       "                     mean           std      mean       std      mean   \n",
       "fit_time         0.002110  1.164137e-03  0.001835  0.000562  0.001043   \n",
       "score_time       0.005287  2.824569e-03  0.002606  0.000303  0.002321   \n",
       "test_accuracy    0.333333  0.000000e+00  0.933333  0.081650  0.946667   \n",
       "train_accuracy   0.346667  7.453560e-03  0.953333  0.013944  0.966667   \n",
       "test_precision   0.111111  1.551584e-17  0.939333  0.079588  0.951111   \n",
       "train_precision  0.120222  5.093266e-03  0.953569  0.013877  0.967508   \n",
       "test_recall      0.333333  0.000000e+00  0.933333  0.081650  0.946667   \n",
       "train_recall     0.346667  7.453560e-03  0.953333  0.013944  0.966667   \n",
       "test_f1          0.166667  0.000000e+00  0.932929  0.082477  0.946397   \n",
       "train_f1         0.178519  6.625387e-03  0.953296  0.013956  0.966621   \n",
       "\n",
       "                          random_forest            \n",
       "                      std          mean       std  \n",
       "fit_time         0.000073      0.037302  0.000230  \n",
       "score_time       0.000079      0.004144  0.000036  \n",
       "test_accuracy    0.055777      0.933333  0.047140  \n",
       "train_accuracy   0.020412      1.000000  0.000000  \n",
       "test_precision   0.054772      0.948063  0.033914  \n",
       "train_precision  0.019637      1.000000  0.000000  \n",
       "test_recall      0.055777      0.933333  0.047140  \n",
       "train_recall     0.020412      1.000000  0.000000  \n",
       "test_f1          0.055859      0.932088  0.049128  \n",
       "train_f1         0.020468      1.000000  0.000000  "
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "trained_models, initial_scores = ClassifierTrainer(preprocessor, X_train, y_train, seed=421)\n",
    "ResultHandler(initial_scores)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As Alex examined the results, they noticed that Random Forest showed promise, but its performance could likely improve with feature selection and hyperparameter tuning."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Selecting Features for Simplicity\n",
    "With a sense that the dataset contained redundant features, Alex decided to use the `FeatureSelector` function to refine the models further. They opted for **Recursive Feature Elimination (RFE)** to identify and select the most informative features."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "feature_selected_models = FeatureSelector(\n",
    "    preprocessor, trained_models, X_train, y_train, method='RFE', n_features_to_select=2\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now equipped with models that only used the top two features, Alex could already see improvements in simplicity and interpretability. But there was one final step to maximize the models’ potential."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Optimizing Model Hyperparameters\n",
    "The models were performing well, but Alex wanted them to be great. Enter the `ClassifierOptimizer` function. This function searched for the best hyperparameters using `RandomizedSearchCV`, tuning each model to its optimal configuration."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Training logreg...\n",
      "\n",
      "Training svc...\n",
      "\n",
      "Training random_forest...\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead tr th {\n",
       "        text-align: left;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <th colspan=\"2\" halign=\"left\">logreg</th>\n",
       "      <th colspan=\"2\" halign=\"left\">svc</th>\n",
       "      <th colspan=\"2\" halign=\"left\">random_forest</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <th>mean</th>\n",
       "      <th>std</th>\n",
       "      <th>mean</th>\n",
       "      <th>std</th>\n",
       "      <th>mean</th>\n",
       "      <th>std</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>fit_time</th>\n",
       "      <td>0.004457</td>\n",
       "      <td>0.000820</td>\n",
       "      <td>0.002409</td>\n",
       "      <td>0.000061</td>\n",
       "      <td>0.129678</td>\n",
       "      <td>0.000447</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>score_time</th>\n",
       "      <td>0.002687</td>\n",
       "      <td>0.000149</td>\n",
       "      <td>0.002588</td>\n",
       "      <td>0.000071</td>\n",
       "      <td>0.003258</td>\n",
       "      <td>0.000093</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>test_accuracy</th>\n",
       "      <td>0.946667</td>\n",
       "      <td>0.055777</td>\n",
       "      <td>0.946667</td>\n",
       "      <td>0.055777</td>\n",
       "      <td>0.933333</td>\n",
       "      <td>0.047140</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>train_accuracy</th>\n",
       "      <td>0.950000</td>\n",
       "      <td>0.011785</td>\n",
       "      <td>0.953333</td>\n",
       "      <td>0.007454</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>test_precision</th>\n",
       "      <td>0.959175</td>\n",
       "      <td>0.040828</td>\n",
       "      <td>0.951556</td>\n",
       "      <td>0.054714</td>\n",
       "      <td>0.948063</td>\n",
       "      <td>0.033914</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>train_precision</th>\n",
       "      <td>0.950475</td>\n",
       "      <td>0.011793</td>\n",
       "      <td>0.953927</td>\n",
       "      <td>0.007123</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>test_recall</th>\n",
       "      <td>0.946667</td>\n",
       "      <td>0.055777</td>\n",
       "      <td>0.946667</td>\n",
       "      <td>0.055777</td>\n",
       "      <td>0.933333</td>\n",
       "      <td>0.047140</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>train_recall</th>\n",
       "      <td>0.950000</td>\n",
       "      <td>0.011785</td>\n",
       "      <td>0.953333</td>\n",
       "      <td>0.007454</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>test_f1</th>\n",
       "      <td>0.945556</td>\n",
       "      <td>0.057790</td>\n",
       "      <td>0.946667</td>\n",
       "      <td>0.055779</td>\n",
       "      <td>0.932088</td>\n",
       "      <td>0.049128</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>train_f1</th>\n",
       "      <td>0.950006</td>\n",
       "      <td>0.011785</td>\n",
       "      <td>0.953321</td>\n",
       "      <td>0.007461</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                   logreg                 svc           random_forest  \\\n",
       "                     mean       std      mean       std          mean   \n",
       "fit_time         0.004457  0.000820  0.002409  0.000061      0.129678   \n",
       "score_time       0.002687  0.000149  0.002588  0.000071      0.003258   \n",
       "test_accuracy    0.946667  0.055777  0.946667  0.055777      0.933333   \n",
       "train_accuracy   0.950000  0.011785  0.953333  0.007454      1.000000   \n",
       "test_precision   0.959175  0.040828  0.951556  0.054714      0.948063   \n",
       "train_precision  0.950475  0.011793  0.953927  0.007123      1.000000   \n",
       "test_recall      0.946667  0.055777  0.946667  0.055777      0.933333   \n",
       "train_recall     0.950000  0.011785  0.953333  0.007454      1.000000   \n",
       "test_f1          0.945556  0.057790  0.946667  0.055779      0.932088   \n",
       "train_f1         0.950006  0.011785  0.953321  0.007461      1.000000   \n",
       "\n",
       "                           \n",
       "                      std  \n",
       "fit_time         0.000447  \n",
       "score_time       0.000093  \n",
       "test_accuracy    0.047140  \n",
       "train_accuracy   0.000000  \n",
       "test_precision   0.033914  \n",
       "train_precision  0.000000  \n",
       "test_recall      0.047140  \n",
       "train_recall     0.000000  \n",
       "test_f1          0.049128  \n",
       "train_f1         0.000000  "
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "optimized_models, optimized_scores = ClassifierOptimizer(\n",
    "    feature_selected_models, X_train, y_train, scoring='f1', n_iter=10, random_state=421\n",
    ")\n",
    "ResultHandler(optimized_scores)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After a few minutes of computation, the function returned the optimized models and their performance metrics. Alex was thrilled to see that the Random Forest classifier now had the best F1 score among all models.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Summarizing the Results\n",
    "To present the results clearly, Alex used the `ResultHandler` function to compile the performance metrics and hyperparameters into a tidy DataFrame."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>dummy_baseline</th>\n",
       "      <th>logreg_baseline</th>\n",
       "      <th>logreg_optimized</th>\n",
       "      <th>random_forest_baseline</th>\n",
       "      <th>random_forest_optimized</th>\n",
       "      <th>svc_baseline</th>\n",
       "      <th>svc_optimized</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>fit_time</th>\n",
       "      <td>0.001169</td>\n",
       "      <td>0.001690</td>\n",
       "      <td>0.004457</td>\n",
       "      <td>0.038615</td>\n",
       "      <td>0.129678</td>\n",
       "      <td>0.001124</td>\n",
       "      <td>0.002409</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>score_time</th>\n",
       "      <td>0.002642</td>\n",
       "      <td>0.002409</td>\n",
       "      <td>0.002687</td>\n",
       "      <td>0.004279</td>\n",
       "      <td>0.003258</td>\n",
       "      <td>0.002501</td>\n",
       "      <td>0.002588</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>test_accuracy</th>\n",
       "      <td>0.333333</td>\n",
       "      <td>0.933333</td>\n",
       "      <td>0.946667</td>\n",
       "      <td>0.933333</td>\n",
       "      <td>0.933333</td>\n",
       "      <td>0.946667</td>\n",
       "      <td>0.946667</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>train_accuracy</th>\n",
       "      <td>0.346667</td>\n",
       "      <td>0.953333</td>\n",
       "      <td>0.950000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.966667</td>\n",
       "      <td>0.953333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>test_precision</th>\n",
       "      <td>0.111111</td>\n",
       "      <td>0.939333</td>\n",
       "      <td>0.959175</td>\n",
       "      <td>0.948063</td>\n",
       "      <td>0.948063</td>\n",
       "      <td>0.951111</td>\n",
       "      <td>0.951556</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>train_precision</th>\n",
       "      <td>0.120222</td>\n",
       "      <td>0.953569</td>\n",
       "      <td>0.950475</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.967508</td>\n",
       "      <td>0.953927</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>test_recall</th>\n",
       "      <td>0.333333</td>\n",
       "      <td>0.933333</td>\n",
       "      <td>0.946667</td>\n",
       "      <td>0.933333</td>\n",
       "      <td>0.933333</td>\n",
       "      <td>0.946667</td>\n",
       "      <td>0.946667</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>train_recall</th>\n",
       "      <td>0.346667</td>\n",
       "      <td>0.953333</td>\n",
       "      <td>0.950000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.966667</td>\n",
       "      <td>0.953333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>test_f1</th>\n",
       "      <td>0.166667</td>\n",
       "      <td>0.932929</td>\n",
       "      <td>0.945556</td>\n",
       "      <td>0.932088</td>\n",
       "      <td>0.932088</td>\n",
       "      <td>0.946397</td>\n",
       "      <td>0.946667</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>train_f1</th>\n",
       "      <td>0.178519</td>\n",
       "      <td>0.953296</td>\n",
       "      <td>0.950006</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.966621</td>\n",
       "      <td>0.953321</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                 dummy_baseline  logreg_baseline  logreg_optimized  \\\n",
       "fit_time               0.001169         0.001690          0.004457   \n",
       "score_time             0.002642         0.002409          0.002687   \n",
       "test_accuracy          0.333333         0.933333          0.946667   \n",
       "train_accuracy         0.346667         0.953333          0.950000   \n",
       "test_precision         0.111111         0.939333          0.959175   \n",
       "train_precision        0.120222         0.953569          0.950475   \n",
       "test_recall            0.333333         0.933333          0.946667   \n",
       "train_recall           0.346667         0.953333          0.950000   \n",
       "test_f1                0.166667         0.932929          0.945556   \n",
       "train_f1               0.178519         0.953296          0.950006   \n",
       "\n",
       "                 random_forest_baseline  random_forest_optimized  \\\n",
       "fit_time                       0.038615                 0.129678   \n",
       "score_time                     0.004279                 0.003258   \n",
       "test_accuracy                  0.933333                 0.933333   \n",
       "train_accuracy                 1.000000                 1.000000   \n",
       "test_precision                 0.948063                 0.948063   \n",
       "train_precision                1.000000                 1.000000   \n",
       "test_recall                    0.933333                 0.933333   \n",
       "train_recall                   1.000000                 1.000000   \n",
       "test_f1                        0.932088                 0.932088   \n",
       "train_f1                       1.000000                 1.000000   \n",
       "\n",
       "                 svc_baseline  svc_optimized  \n",
       "fit_time             0.001124       0.002409  \n",
       "score_time           0.002501       0.002588  \n",
       "test_accuracy        0.946667       0.946667  \n",
       "train_accuracy       0.966667       0.953333  \n",
       "test_precision       0.951111       0.951556  \n",
       "train_precision      0.967508       0.953927  \n",
       "test_recall          0.946667       0.946667  \n",
       "train_recall         0.966667       0.953333  \n",
       "test_f1              0.946397       0.946667  \n",
       "train_f1             0.966621       0.953321  "
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "results_df = ResultHandler(initial_scores, optimized_scores)\n",
    "results_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Testing the Final Model\n",
    "The optimized Random Forest model was deployed for testing on the test dataset. Alex measured its performance and confirmed that it generalized well beyond the training data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       1.00      1.00      1.00        25\n",
      "           1       0.96      0.96      0.96        26\n",
      "           2       0.96      0.96      0.96        24\n",
      "\n",
      "    accuracy                           0.97        75\n",
      "   macro avg       0.97      0.97      0.97        75\n",
      "weighted avg       0.97      0.97      0.97        75\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# Testing the final Random Forest model\n",
    "final_model = optimized_models['random_forest']\n",
    "y_pred = final_model.predict(X_test)\n",
    "\n",
    "print(classification_report(y_test, y_pred))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Reflection\n",
    "By combining tools like `ClassifierTrainer`, `FeatureSelector`, `ClassifierOptimizer`, and `ResultHandler`, Alex had created an efficient, repeatable, and transparent machine learning pipeline. It was a productive day of problem-solving, leaving Alex eager to tackle their next project."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}