Backend AI

Abstract

1. Introduction

Backend.AI is a streamlined, container-based computing cluster platform that hosts popular computing/ML frameworks and diverse programming languages, with pluggable heterogeneous accelerator support including CUDA GPU, ROCm GPU, Rebellions, FuriosaAI, HyperAccel, Intel Gaudi, Tenstorrent, Google TPU, Graphcore IPU and other NPUs.

It allocates and isolates the underlying computing resources for multi-tenant computation sessions on-demand or in batches with customizable job schedulers with its own orchestrator named "Sokovan".

All its functions are exposed as REST and GraphQL APIs.

2. Objective

Backend.AI is a streamlined, container-based computing cluster platform that hosts popular computing/ML frameworks and diverse programming languages, with pluggable heterogeneous accelerator support including CUDA GPU, ROCm GPU, Gaudi NPU, Google TPU, GraphCore IPU and other NPUs.

This project demonstrates how Python, Docker can be applied to a real-world Blockchain & Cloud Computing problem.

4. Technology Stack

• Python: 3.13.x (main branch requires CPython 3.13.7)
• Pantsbuild: 2.27.x
• See full version compatibility table
• Routes REST/GraphQL requests and orchestrates cluster operations
• Session scheduling via Sokovan orchestrator
• User authentication and RBAC authorization
• Plugin interfaces: backendai_scheduler_v10, backendai_agentselector_v10, backendai_hook_v20, backendai_webapp_v20, backendai_monitor_stats_v10, backendai_monitor_error_v10
• Legacy repo: https://github.com/lablup/backend.ai-manager

5. System Requirements

General requirements for this technology stack — check the README for exact versions.

• Python 3.8 or later
• pip / virtualenv for dependencies
• VS Code, PyCharm or Jupyter Notebook
• Git (to clone the repository)

6. Installation & Setup

git clone https://github.com/lablup/backend.ai.git
cd backend.ai

1. Check required dependencies (Docker, Python, etc.)
2. Set up Python virtual environment with Pantsbuild
3. Start halfstack infrastructure (PostgreSQL, Valkey, etcd, Apollo Router) — only required services by default; observability and object storage are opt-in via Compose profiles (--profile observability, --profile storage)
4. Initialize database schemas
5. Create default API keypairs and user accounts
6. System architecture diagrams and component flow
7. Port numbers and infrastructure setup
8. Component dependencies and communication protocols

git clone https://github.com/lablup/backend.ai.git
cd backend.ai
./scripts/install-dev.sh

Full setup instructions are in the project README.

7. Future Enhancements

Suggested extensions you can add to make this your own project.

• Add a CI/CD pipeline with GitHub Actions
• Deploy to a public test network or cloud free tier
• Add monitoring and cost alerts

8. Viva / Review Questions

Common questions examiners ask for projects in this domain.

1. Why does this problem need a blockchain or cloud-native design?
2. Explain the smart contract / infrastructure components and how they interact.
3. How are gas costs or cloud costs kept under control?
4. How is the system secured (keys, IAM, access control)?
5. How would the solution scale to many more users?

9. Source Code & License

This project is developed by lablup and published on GitHub under the GNU Lesser General Public License v3.0. Please follow the license terms and credit the original author when you use or modify this code.