Binclass Tools

Abstract

1. Introduction

A set of Python wrappers and interactive plots that facilitate the analysis of binary classification problems.

Finding the Best Classification Threshold for Imbalanced Classifications with the Interactive Confusion Matrix and Line Charts

2. Objective

The binclass-tools package contains a set of Python wrappers and interactive plots that facilitate the analysis of binary classification problems.

This project demonstrates how Jupyter Notebook, Machine Learning, Python can be applied to a real-world Data Science problem.

4. Technology Stack

• Powerful interactive charts that simplify the analysis of a binary classifier's performance, including calibration and any amounts and costs associated with individual observations.
• A set of functions that return the values of metrics useful for measuring the performance of a binary classifier, for each threshold value if dependent on it.
• A set of generic wrappers that help the analyst in daily operations dealing with binary classifications.
• scikit-learn (>=0.22.1)
• matplotlib
• nbformat (>= 4.2.0)

5. System Requirements

General requirements for this technology stack — check the README for exact versions.

• Python 3.8 or later with Jupyter Notebook / JupyterLab (or Google Colab)
• pip for dependencies
• Python 3.8 or later
• pip / virtualenv for dependencies
• VS Code, PyCharm or Jupyter Notebook
• Git (to clone the repository)

6. Installation & Setup

git clone https://github.com/lucazav/binclass-tools.git
cd binclass-tools

1. Python 3.6+
2. A set of the most popular packages used for working with data
3. Plotly for interactive plots
4. The size of the step separating one threshold value from the other (always considering the extremes 0 and 1 inclusive).
5. The list of individual amounts associated with each of the observables in the test dataset (since the dataset is generated by random values, the absolute value of column 13 is considered as the amount column).
6. Which currency symbol to use.

pip install binclass-tools

pip install git+https://github.com/lucazav/binclass-tools

import numpy as np
import pandas as pd
import bctools as bc

from sklearn.ensemble import RandomForestClassifier
from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split

Full setup instructions are in the project README.

7. Future Enhancements

Suggested extensions you can add to make this your own project.

• Turn the analysis into an interactive dashboard
• Automate data refresh with a scheduled job
• Add a predictive model on top of the analysis

8. Viva / Review Questions

Common questions examiners ask for projects in this domain.

1. What is the source of the dataset and how was missing data handled?
2. Which exploratory analysis steps revealed the most useful insight?
3. Why were these particular charts chosen to present the data?
4. Which statistical or ML technique supports the conclusions?
5. How could the analysis be automated or refreshed with new data?

9. Source Code & License

This project is developed by lucazav and published on GitHub under the BSD 3-Clause "New" or "Revised" License. Please follow the license terms and credit the original author when you use or modify this code.