Discussion 3

Introduction

In this week’s discussion section, we will create some plots to better undertsand how much class imbalances can effect our classification model. Rather than creating a widget that updates the parameters of the model ( like we have done in the past couple weeks), this week we will create a widget that updates our data - specifically updating the class imbalance within our data. To do so, we will use synthesized data that is made with the intention of better understanding how relationships within data for logistic regression work. It is important to note that your results with real data may look very different - unlike this notebook, the real world data you will be working with was not made to better understand logistic regression.

Data

While our data is synthetic, we will still have it hold an environmnetal value. Our data is going to represent the prescence/absence of the invasive European green crab that is often found in California coastal waters. These crabs prefer warmer water temperatures between 64° F and 79° F and salinity levels between 26 and 39 ppt. The features for our data will be water temperature and salinity, and our target variable will be the presence (1) or absence (0) of green crabs at our different sampling sites. Import the libraries below and copy the function to generate our data below to get started.

# Import necessary libraries
import numpy as np
import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import roc_curve, auc, confusion_matrix
import matplotlib.pyplot as plt
from ipywidgets import interact, FloatSlider
import seaborn as sns

Time for some FUN(ctions)!

We will create six different functions for the different parts of our interactive output: one to generate the data, a second to create a barplot to represent the class imbalance, a third to create a confusion matrix, another to create an ROC curve, a function to wrap everything together, and a final function to add interactivity. Let’s get to it!

Function 1

Create a function that generates the species data. The parameters should be the sample size and the ratio of present green crabs.

def generate_species_data(n_samples=1000, presence_ratio=0.3):
    # Calculate number of samples for each class
    n_present = int(n_samples * presence_ratio)
    n_absent = n_samples - n_present
    
    # Generate features for presence sites 
    # Green crabs prefer warmer temps (between 64 and 79 degrees Fahrenheit) and  salinity between 26 and 39 ppt
    temp_present = np.random.normal(loc=71, scale= 4, size=n_present)
    salinity_present = np.random.normal(loc=32, scale=3, size=n_present)
    X_present = np.column_stack([temp_present, salinity_present])
    y_present = np.ones(n_present)
    
    # Generate features for absence sites
    # Sites with warmer temps or lower salinity
    temp_absent = np.random.normal(loc=26, scale=3, size=n_absent)
    salinity_absent = np.random.normal(loc=28, scale=2, size=n_absent)
    X_absent = np.column_stack([temp_absent, salinity_absent])
    y_absent = np.zeros(n_absent)
    
    # Combine and shuffle the data
    X = np.concatenate([X_present, X_absent])
    y = np.concatenate([y_present, y_absent])
    
    # Shuffle the data
    shuffle_idx = np.random.permutation(n_samples)
    X = X[shuffle_idx]
    y = y[shuffle_idx]
    
    return X, y

Function 2

Create a function that creats a bar plot of species presense distribution based on the ratio selected by the user.

def plot_class_distribution(y):
    plt.figure(figsize = (8,4))
    
    # Count the values in each category
    class_counts = pd.Series(y).value_counts()
    
    # Create the barplot of Absent and Present species
    sns.barplot(x = ['Absent', 'Present'], y = class_counts, color = '#005477')
    plt.title('Distribution of Species Presence/Absence')
    plt.ylabel('Number of Sampling sites')
    
    # Add percent over each bar
    total = len(y)
    for i,count in enumerate(class_counts):
        percentage = count/total * 100 
        plt.text(i, count, f'{percentage:.1f}%', ha = 'center', va = 'bottom')
    plt.show()

Function 3

Create a function that plots a confusion matrix of the predicted y values and true y values.

def plot_confusion_matrix(y_true, y_pred):
    
    # Create confusion matrix
    cm = confusion_matrix(y_true, y_pred)
    
    # Create confusion matrix plot
    plt.figure(figsize = (8,6))
    sns.heatmap(cm, fmt = 'd', cmap = 'GnBu',annot = True,
               xticklabels = ['Absent', 'Present'],
               yticklabels = ['Absent', 'Present'])
    plt.title('Confusion Matrix')
    plt.ylabel('True Label')
    plt.xlabel('Predicted Label')
    plt.show()
    
    
    # Calculate and display metrics
    
    TP = cm[1,1]
    TN = cm[0,0]
    FP = cm[0,1]
    FN = cm[1,0]
    
    print(f"True positives (correctly predicted presence): {TP}")
    
    
    # Calculate accuracy + various metric 
    accuracy = (TP + TN) / (TP + TN + FP + FN)
    sensitivity = TP/ (TP + FN)
    specificity = TN / (TN + FP )
    
    
    print(f"\nModel Performance Metrics:")
    print(f"Accuracy: {accuracy:.3f}")
    print(f"Sensitivity ( True positive rate): {sensitivity:.3f}")
    print(f"Specificity ( True negative rate:) {specificity:.3f}")

Function 4

Create a function that plots an ROC curve using the predicted y class probabilities and true y values.

def plot_roc_curve(y_test, y_pred_prob):
    
    fpr, tpr, _ = roc_curve(y_test, y_pred_prob)
    
    roc_auc = auc(fpr, tpr)
    
    plt.figure(figsize = (8,6))
    plt.plot(fpr, tpr, color = 'darkorange', lw =2, label = f'ROC Curve (AUC = {roc_auc:.2f})')
    plt.plot([0,1], [0,1], color = 'navy', lw = 2, linestyle = '--',
            label = 'Random Classifier (AUC = 0.5)')
    plt.xlabel('False positive rate')
    plt.ylabel('True positive rate')
    plt.title('ROC curve: Species Presence Prediction')
    plt.legend()
    plt.grid(True)
    plt.show()

Function 5

Create function that runs a logistic regression and outputs the three plots you created above.

def interactive_logistic_regression(presence_ratio = 0.3):
    
    # Generate data based on class imbalance from user
    X,y = generate_species_data(presence_ratio = presence_ratio)
    
    # Plot class distribution
    print("\nClass Distribution")
    plot_class_distribution(y)
    
    # Split data
    X_train, X_test, y_train, y_test = train_test_split(X, y , test_size = 0.3, random_state = 42)
    
    # Train model
    model = LogisticRegression()
    model.fit(X_train, y_train)
    
    # Make predict
    y_pred = model.predict(X_test)
    y_pred_prob = model.predict_proba(X_test)[:,1]
    
    # Plot confusion matrix
    print("\nConfusion matrix:")
    plot_confusion_matrix(y_test, y_pred)
    
    # Plot ROC curve
    print("\nROC Curve:")
    plot_roc_curve(y_test, y_pred_prob)

Function 6

Create a function that adds interactivity to function 5.

# Create interactive widget

def generate_log_regression():
    interact(interactive_logistic_regression, 
            presence_ratio = FloatSlider(min = .1, max = .9, step= .1, value = 0.3,
                                        description = "% Present"))
generate_log_regression()