Py POTS

Abstract

1. Introduction

⦿ Motivation: Due to all kinds of reasons like failure of collection sensors, communication error, and unexpected malfunction, missing values are common to see in time series from the real-world environment. This makes partially-observed time series (POTS) a pervasive problem in open-world modeling and prevents advanced data analysis. Although this problem is important, the area of machine learning on POTS still lacks a dedicated toolkit. PyPOTS is created to fill in this blank.

⦿ Mission: PyPOTS (pronounced "Pie Pots") is born to become a handy toolbox that is going to make machine learning on POTS easy rather than tedious, to help engineers and researchers focus more on the core problems in their hands rather than on how to deal with the missing parts in their data. PyPOTS will keep integrating classical and the latest state-of-the-art machine learning algorithms for partially-observed multivariate time series. For sure, besides various algorithms, PyPOTS is going to have unified APIs together with detailed documentation and interactive examples across algorithms as tutorials.

The rest of this readme file is organized as follows: Available Algorithms, PyPOTS Ecosystem, Installation, Usage, Citing PyPOTS, Contribution, Community.

2. Objective

A Python toolkit/library for reality-centric machine/deep learning & data mining on partially-observed time series, with 50+ SOTA neural network models for scientific analysis tasks (imputation, classification, clustering, forecasting, anomaly detection, cleaning) on incomplete industrial irregularly-sampled multivariate TS with NaN missing values

This project demonstrates how Python, Deep Learning can be applied to a real-world AI & Machine Learning problem.

4. Technology Stack

5. System Requirements

General requirements for this technology stack — check the README for exact versions.

• Python 3.8 or later
• pip / virtualenv for dependencies
• VS Code, PyCharm or Jupyter Notebook
• Git (to clone the repository)

6. Installation & Setup

git clone https://github.com/WenjieDu/PyPOTS.git
cd PyPOTS

# via pip
pip install pypots            # the first time installation
pip install pypots --upgrade  # update pypots to the latest version
# install from the latest source code with the latest features but may be not officially released yet
pip install https://github.com/WenjieDu/PyPOTS/archive/main.zip

# via conda
conda install conda-forge::pypots  # the first time installation
conda update  conda-forge::pypots  # update pypots to the latest version

# via docker
docker run -it --name pypots wenjiedu/pypots  # docker will auto pull our built image and run a instance for you
# after things settled, you can run python in the container to access the well-configured environment for running pypots
# if you'd like to detach from the container, press ctrl-P + ctrl-Q
# run `docker attach pypots` to enter the container again.

Full setup instructions are in the project README.

7. Future Enhancements

Suggested extensions you can add to make this your own project.

• Deploy the model as a web app with Streamlit, Flask or FastAPI
• Compare against an additional model and report the metric difference
• Add explainability (SHAP / Grad-CAM)

8. Viva / Review Questions

Common questions examiners ask for projects in this domain.

1. What dataset does the project use and how was it pre-processed?
2. Which algorithm / model architecture is used and why was it chosen over alternatives?
3. How are training and testing data split, and how is overfitting avoided?
4. Which evaluation metrics (accuracy, precision, recall, F1) are reported and what do they mean here?
5. How would you deploy this model for real users?

9. Source Code & License

This project is developed by WenjieDu and published on GitHub under the BSD 3-Clause "New" or "Revised" License. Please follow the license terms and credit the original author when you use or modify this code.