Time Mo E

Abstract

1. Introduction

Time-MoE comprises a family of decoder-only time series foundation models with a mixture-of-experts architecture, designed to operate in an auto-regressive manner, enabling universal forecasting with arbitrary prediction horizons and context lengths of up to 4096.

(ICLR'25 Spotlight) Time-MoE: Billion-Scale Time Series Foundation Models with Mixture of Experts

News (Feb 2025): Time-MoE has been accpeted by ICLR 2025 as a Spotlight (Top 5.1%)!

2. Objective

[ICLR 2025 Spotlight] Official implementation of "Time-MoE: Billion-Scale Time Series Foundation Models with Mixture of Experts"

This project demonstrates how Python, Deep Learning, Machine Learning can be applied to a real-world Data Science problem.

4. Technology Stack

5. System Requirements

General requirements for this technology stack — check the README for exact versions.

• Python 3.8 or later
• pip / virtualenv for dependencies
• VS Code, PyCharm or Jupyter Notebook
• Git (to clone the repository)

6. Installation & Setup

git clone https://github.com/Time-MoE/Time-MoE.git
cd Time-MoE

1. Install Python 3.10+, and then install the dependencies:
2. [Optional but recommended] Install flash-attn for faster training and inference speeds with reduced memory usage.

pip install -r requirements.txt

pip install flash-attn==2.6.3

pip install packaging
pip install ninja
# Replace "64" with the number of CPU cores available on your machine for faster compilation
MAX_JOBS=64 pip install flash-attn==2.6.3 --no-build-isolation

Full setup instructions are in the project README.

7. Future Enhancements

Suggested extensions you can add to make this your own project.

• Turn the analysis into an interactive dashboard
• Automate data refresh with a scheduled job
• Add a predictive model on top of the analysis

8. Viva / Review Questions

Common questions examiners ask for projects in this domain.

1. What is the source of the dataset and how was missing data handled?
2. Which exploratory analysis steps revealed the most useful insight?
3. Why were these particular charts chosen to present the data?
4. Which statistical or ML technique supports the conclusions?
5. How could the analysis be automated or refreshed with new data?

9. Source Code & License

This project is developed by Time-MoE and published on GitHub under the Apache License 2.0. Please follow the license terms and credit the original author when you use or modify this code.