francwa
249c5de76a
- Refactor memory system (episodic/STM/LTM with components) - Implement complete subtitle domain (scanner, matcher, placer) - Add YAML workflow infrastructure - Externalize knowledge base (patterns, release groups) - Add comprehensive testing suite - Create manual testing CLIs
434 lines
14 KiB
Markdown
434 lines
14 KiB
Markdown
# Alfred Media Organizer 🎬
|
|
|
|
An AI-powered agent for managing your local media library with natural language. Search, download, and organize movies and TV shows effortlessly through a conversational interface.
|
|
|
|
[](https://www.python.org/downloads/)
|
|
[](https://github.com/astral-sh/uv)
|
|
[](https://opensource.org/licenses/MIT)
|
|
[](https://github.com/astral-sh/ruff)
|
|
|
|
## ✨ Features
|
|
|
|
- 🤖 **Natural Language Interface** — Talk to your media library in plain language
|
|
- 🔍 **Smart Search** — Find movies and TV shows via TMDB with rich metadata
|
|
- 📥 **Torrent Integration** — Search and download via qBittorrent
|
|
- 🧠 **Contextual Memory** — Remembers your preferences and conversation history
|
|
- 📁 **Auto-Organization** — Moves and renames media files, resolves destinations, handles subtitles
|
|
- 🎞️ **Subtitle Pipeline** — Identifies, matches, and places subtitle tracks automatically
|
|
- 🔄 **Workflow Engine** — YAML-defined multi-step workflows (e.g. `organize_media`)
|
|
- 🌐 **OpenAI-Compatible API** — Works with any OpenAI-compatible client (LibreChat, OpenWebUI, etc.)
|
|
- 🔒 **Secure by Default** — Auto-generated secrets and encrypted credentials
|
|
|
|
## 🏗️ Architecture
|
|
|
|
Built with **Domain-Driven Design (DDD)** principles for clean separation of concerns:
|
|
|
|
```
|
|
alfred/
|
|
├── agent/ # AI agent orchestration
|
|
│ ├── llm/ # LLM clients (Ollama, DeepSeek)
|
|
│ ├── tools/ # Tool implementations (api, filesystem, language)
|
|
│ └── workflows/ # YAML-defined multi-step workflows
|
|
├── application/ # Use cases & DTOs
|
|
│ ├── movies/ # Movie search
|
|
│ ├── torrents/ # Torrent management
|
|
│ └── filesystem/ # File operations (move, list, subtitles, seed links)
|
|
├── domain/ # Business logic & entities
|
|
│ ├── media/ # Release parsing
|
|
│ ├── movies/ # Movie entities
|
|
│ ├── tv_shows/ # TV show entities & value objects
|
|
│ ├── subtitles/ # Subtitle scanner, services, knowledge base
|
|
│ └── shared/ # Common value objects (ImdbId, FilePath, FileSize)
|
|
└── infrastructure/ # External services & persistence
|
|
├── api/ # External API clients (TMDB, qBittorrent, Knaben)
|
|
├── filesystem/ # File manager (hard-link based, path-traversal safe)
|
|
├── persistence/ # Three-tier memory (LTM/STM/Episodic) + JSON repositories
|
|
└── subtitle/ # Subtitle infrastructure
|
|
```
|
|
|
|
### Key flows
|
|
|
|
**Agent execution:** `agent.step(user_input)` → LLM call → if tool_calls, execute each via registry → loop until no tool calls or `max_tool_iterations` → return final response.
|
|
|
|
**Media organization workflow:**
|
|
1. `resolve_destination` — Determines target folder/filename from release name
|
|
2. `move_media` — Hard-links file to library, deletes source
|
|
3. `manage_subtitles` — Scans, classifies, and places subtitle tracks
|
|
4. `create_seed_links` — Hard-links library file back to torrents/ for continued seeding
|
|
|
|
**Memory tiers:**
|
|
- **LTM** (`data/memory/ltm.json`) — Persisted config, media library, watchlist
|
|
- **STM** — Conversation history (capped at `MAX_HISTORY_MESSAGES`)
|
|
- **Episodic** — Transient search results, active downloads, recent errors
|
|
|
|
## 🚀 Quick Start
|
|
|
|
### Prerequisites
|
|
|
|
- **Python 3.14+**
|
|
- **uv** (dependency manager)
|
|
- **Docker & Docker Compose** (recommended for full stack)
|
|
- **API Keys:**
|
|
- TMDB API key ([get one here](https://www.themoviedb.org/settings/api))
|
|
- Optional: DeepSeek or other LLM provider keys
|
|
|
|
### Installation
|
|
|
|
```bash
|
|
# Clone the repository
|
|
git clone https://github.com/francwa/alfred_media_organizer.git
|
|
cd alfred_media_organizer
|
|
|
|
# Install dependencies
|
|
make install
|
|
|
|
# Install pre-commit hooks
|
|
make install-hooks
|
|
|
|
# Bootstrap environment (generates .env with secure secrets)
|
|
make bootstrap
|
|
|
|
# Validate your .env against the schema
|
|
make validate
|
|
|
|
# Edit .env with your API keys
|
|
nano .env
|
|
```
|
|
|
|
### Running with Docker (Recommended)
|
|
|
|
```bash
|
|
# Start all services (LibreChat + Alfred + MongoDB + Ollama)
|
|
make up
|
|
|
|
# Or start with specific profiles
|
|
make up p=rag,meili # Include RAG and Meilisearch
|
|
make up p=qbittorrent # Include qBittorrent
|
|
make up p=full # Everything
|
|
|
|
# View logs
|
|
make logs
|
|
|
|
# Stop all services
|
|
make down
|
|
```
|
|
|
|
The web interface will be available at **http://localhost:3080**
|
|
|
|
### Running Locally (Development)
|
|
|
|
```bash
|
|
uv run uvicorn alfred.app:app --reload --port 8000
|
|
```
|
|
|
|
## ⚙️ Configuration
|
|
|
|
### Settings system
|
|
|
|
`settings.toml` is the single source of truth. The schema flows:
|
|
|
|
```
|
|
settings.toml → settings_schema.py → settings_bootstrap.py → .env + .env.make → settings.py
|
|
```
|
|
|
|
To add a setting: define it in `settings.toml`, run `make bootstrap`, then access via `settings.my_new_setting`.
|
|
|
|
```bash
|
|
# First time setup
|
|
make bootstrap
|
|
|
|
# Validate existing .env against schema
|
|
make validate
|
|
|
|
# Re-run after settings.toml changes (existing secrets preserved)
|
|
make bootstrap
|
|
```
|
|
|
|
**Never commit `.env` or `.env.make`** — both are gitignored and auto-generated.
|
|
|
|
### Key settings (.env)
|
|
|
|
```bash
|
|
# --- CORE ---
|
|
MAX_HISTORY_MESSAGES=10
|
|
MAX_TOOL_ITERATIONS=10
|
|
|
|
# --- LLM ---
|
|
DEFAULT_LLM_PROVIDER=local # local (Ollama) | deepseek
|
|
OLLAMA_BASE_URL=http://ollama:11434
|
|
OLLAMA_MODEL=llama3.3:latest
|
|
LLM_TEMPERATURE=0.2
|
|
|
|
# --- API KEYS ---
|
|
TMDB_API_KEY=your-tmdb-key # Required for movie/show search
|
|
DEEPSEEK_API_KEY= # Optional
|
|
|
|
# --- SECURITY (auto-generated) ---
|
|
JWT_SECRET=<auto>
|
|
CREDS_KEY=<auto>
|
|
MONGO_PASSWORD=<auto>
|
|
```
|
|
|
|
## 🐳 Docker Services
|
|
|
|
### Docker Profiles
|
|
|
|
| Profile | Extra services | Use case |
|
|
|---------|---------------|----------|
|
|
| (default) | — | LibreChat + Alfred + MongoDB + Ollama |
|
|
| `meili` | Meilisearch | Fast full-text search |
|
|
| `rag` | RAG API + VectorDB (PostgreSQL) | Document retrieval |
|
|
| `qbittorrent` | qBittorrent | Torrent downloads |
|
|
| `full` | All of the above | Complete setup |
|
|
|
|
```bash
|
|
make up # Start (default profile)
|
|
make up p=full # Start with all services
|
|
make down # Stop
|
|
make restart # Restart
|
|
make logs # Follow logs
|
|
make ps # Container status
|
|
```
|
|
|
|
## 🛠️ Available Tools
|
|
|
|
| Tool | Description |
|
|
|------|-------------|
|
|
| `find_media_imdb_id` | Search for movies/TV shows on TMDB by title |
|
|
| `find_torrent` | Search for torrents across multiple indexers |
|
|
| `get_torrent_by_index` | Get detailed info about a specific result |
|
|
| `add_torrent_by_index` | Download a torrent from search results |
|
|
| `add_torrent_to_qbittorrent` | Add a torrent via magnet link directly |
|
|
| `resolve_destination` | Compute the target library path for a release |
|
|
| `move_media` | Hard-link a file to its library destination |
|
|
| `manage_subtitles` | Scan, classify, and place subtitle tracks |
|
|
| `create_seed_links` | Prepare torrent folder so qBittorrent keeps seeding |
|
|
| `learn` | Teach Alfred a new pattern (release group, naming convention) |
|
|
| `set_path_for_folder` | Configure folder paths |
|
|
| `list_folder` | List contents of a configured folder |
|
|
| `set_language` | Set preferred language for the session |
|
|
|
|
## 💬 Usage Examples
|
|
|
|
### Via Web Interface (LibreChat)
|
|
|
|
Navigate to **http://localhost:3080** and start chatting:
|
|
|
|
```
|
|
You: Find Inception in 1080p
|
|
Alfred: I found 3 torrents for Inception (2010):
|
|
1. Inception.2010.1080p.BluRay.x264 (150 seeders) - 2.1 GB
|
|
2. Inception.2010.1080p.WEB-DL.x265 (80 seeders) - 1.8 GB
|
|
3. Inception.2010.1080p.REMUX (45 seeders) - 25 GB
|
|
|
|
You: Download the first one
|
|
Alfred: ✓ Added to qBittorrent! Download started.
|
|
|
|
You: Organize the Breaking Bad S01 download
|
|
Alfred: ✓ Resolved destination: /tv_shows/Breaking.Bad/Season 01/
|
|
✓ Moved 6 episode files
|
|
✓ Placed 6 subtitle tracks (fr, en)
|
|
✓ Seed links created in /torrents/
|
|
```
|
|
|
|
### Via API
|
|
|
|
```bash
|
|
# Health check
|
|
curl http://localhost:8000/health
|
|
|
|
# Chat (OpenAI-compatible)
|
|
curl -X POST http://localhost:8000/v1/chat/completions \
|
|
-H "Content-Type: application/json" \
|
|
-d '{
|
|
"model": "alfred",
|
|
"messages": [{"role": "user", "content": "Find The Matrix 4K"}]
|
|
}'
|
|
|
|
# List models
|
|
curl http://localhost:8000/v1/models
|
|
|
|
# View memory state
|
|
curl http://localhost:8000/memory/state
|
|
```
|
|
|
|
Alfred is compatible with any OpenAI-compatible client. Point it at `http://localhost:8000/v1`, model `alfred`.
|
|
|
|
## 🧠 Memory System
|
|
|
|
Alfred uses a three-tier memory system:
|
|
|
|
| Tier | Storage | Contents | Lifetime |
|
|
|------|---------|----------|----------|
|
|
| **LTM** | JSON file (`data/memory/ltm.json`) | Config, library, watchlist, learned patterns | Permanent |
|
|
| **STM** | RAM | Conversation history (capped) | Session |
|
|
| **Episodic** | RAM | Search results, active downloads, errors | Short-lived |
|
|
|
|
## 🧪 Development
|
|
|
|
### Running Tests
|
|
|
|
```bash
|
|
# Run full suite (parallel)
|
|
make test
|
|
|
|
# Run with coverage report
|
|
make coverage
|
|
|
|
# Run a single file
|
|
uv run pytest tests/test_agent.py -v
|
|
|
|
# Run a single class
|
|
uv run pytest tests/test_agent.py::TestAgentInit -v
|
|
|
|
# Skip slow tests
|
|
uv run pytest -m "not slow"
|
|
```
|
|
|
|
### Test coverage
|
|
|
|
The suite covers:
|
|
- **Agent loop** — tool execution, history, max iterations, error handling
|
|
- **Tool registry** — OpenAI schema format, parameter extraction
|
|
- **Prompts** — system prompt building, tool inclusion
|
|
- **Memory** — LTM/STM/Episodic operations, persistence
|
|
- **Filesystem tools** — path traversal security, folder listing
|
|
- **File manager** — hard-link, move, seed links (real filesystem, no mocks)
|
|
- **Application use cases** — `resolve_destination`, `create_seed_links`, `list_folder`, `move_media`
|
|
- **Domain** — TV show/movie entities, shared value objects (`ImdbId`, `FilePath`, `FileSize`), subtitle scanner
|
|
- **Repositories** — JSON-backed movie, TV show, subtitle repos
|
|
- **Bootstrap** — secret generation, idempotency, URI construction
|
|
- **Workflows** — YAML loading, structure validation
|
|
- **Configuration** — boundary validation for all settings
|
|
|
|
### Code Quality
|
|
|
|
```bash
|
|
make lint # Ruff check --fix
|
|
make format # Ruff format + check --fix
|
|
```
|
|
|
|
### Adding a New Tool
|
|
|
|
1. Implement the function in `alfred/agent/tools/`:
|
|
|
|
```python
|
|
# alfred/agent/tools/api.py
|
|
def my_new_tool(param: str) -> dict[str, Any]:
|
|
"""Short description shown to the LLM to decide when to call this tool."""
|
|
memory = get_memory()
|
|
# ...
|
|
return {"status": "ok", "data": result}
|
|
```
|
|
|
|
2. Register it in `alfred/agent/registry.py`:
|
|
|
|
```python
|
|
tool_functions = [
|
|
# ... existing tools ...
|
|
api_tools.my_new_tool,
|
|
]
|
|
```
|
|
|
|
The registry auto-generates the JSON schema from the function signature and docstring.
|
|
|
|
### Adding a Workflow
|
|
|
|
Create a YAML file in `alfred/agent/workflows/`:
|
|
|
|
```yaml
|
|
name: my_workflow
|
|
description: What this workflow does
|
|
steps:
|
|
- tool: resolve_destination
|
|
description: Find where the file should go
|
|
- tool: move_media
|
|
description: Move the file
|
|
```
|
|
|
|
Workflows are loaded automatically at startup.
|
|
|
|
### Version Management
|
|
|
|
```bash
|
|
# Must be on main branch
|
|
make patch # 0.1.7 → 0.1.8
|
|
make minor # 0.1.7 → 0.2.0
|
|
make major # 0.1.7 → 1.0.0
|
|
```
|
|
|
|
## 📚 API Reference
|
|
|
|
### Endpoints
|
|
|
|
| Method | Path | Description |
|
|
|--------|------|-------------|
|
|
| `GET` | `/health` | Health check |
|
|
| `GET` | `/v1/models` | List models (OpenAI-compatible) |
|
|
| `POST` | `/v1/chat/completions` | Chat (OpenAI-compatible, streaming supported) |
|
|
| `GET` | `/memory/state` | Full memory dump (debug) |
|
|
| `POST` | `/memory/clear-session` | Clear STM + Episodic |
|
|
| `GET` | `/memory/episodic/search-results` | Current search results |
|
|
|
|
## 🔧 Troubleshooting
|
|
|
|
### Agent doesn't respond
|
|
|
|
1. Check API keys in `.env`
|
|
2. Verify the LLM is running:
|
|
```bash
|
|
docker logs alfred-ollama
|
|
docker exec alfred-ollama ollama list
|
|
```
|
|
3. Check Alfred logs: `docker logs alfred-core`
|
|
|
|
### qBittorrent connection failed
|
|
|
|
1. Verify qBittorrent is running: `docker ps | grep qbittorrent`
|
|
2. Check credentials in `.env` (`QBITTORRENT_URL`, `QBITTORRENT_USERNAME`, `QBITTORRENT_PASSWORD`)
|
|
|
|
### Memory not persisting
|
|
|
|
1. Check `data/` directory is writable
|
|
2. Verify volume mounts in `docker-compose.yaml`
|
|
|
|
### Bootstrap fails
|
|
|
|
```bash
|
|
make validate # Check what's wrong with .env
|
|
make bootstrap # Regenerate (preserves existing secrets)
|
|
```
|
|
|
|
### Tests failing
|
|
|
|
```bash
|
|
uv run pytest tests/test_failing.py -v --tb=long
|
|
```
|
|
|
|
## 🤝 Contributing
|
|
|
|
1. Fork the repository
|
|
2. Create a feature branch: `git checkout -b feat/my-feature`
|
|
3. Make your changes + add tests
|
|
4. Run `make test && make lint && make format`
|
|
5. Commit with [Conventional Commits](https://www.conventionalcommits.org/): `feat:`, `fix:`, `docs:`, `refactor:`, `test:`, `chore:`, `infra:`
|
|
6. Open a Pull Request
|
|
|
|
## 📄 License
|
|
|
|
MIT License — see [LICENSE](LICENSE) file for details.
|
|
|
|
## 🙏 Acknowledgments
|
|
|
|
- [LibreChat](https://github.com/danny-avila/LibreChat) — Chat interface
|
|
- [Ollama](https://ollama.ai/) — Local LLM runtime
|
|
- [DeepSeek](https://www.deepseek.com/) — LLM provider
|
|
- [TMDB](https://www.themoviedb.org/) — Movie & TV database
|
|
- [qBittorrent](https://www.qbittorrent.org/) — Torrent client
|
|
- [FastAPI](https://fastapi.tiangolo.com/) — Web framework
|
|
- [uv](https://github.com/astral-sh/uv) — Fast Python package manager
|
|
|
|
---
|
|
|
|
<p align="center">Made with ❤️ by <a href="https://github.com/francwa">Francwa</a></p>
|