StemRoller: desktop stem separator, Android stems player, and server daemons
- stemroller/: PySide6/QML desktop app and player for separating music into stems - android/: Plex music player that plays per-stem mixes, with Android Auto support, a graphic + parametric (Equalizer APO-style) EQ, and offline downloads - server/: stemserve (serves stems to the app) and stemd (separates the library on the server's GPU, with "Stem Now" priority requests) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
ClericalErrorandClaude Opus 5.5 committed
1 parent
716c5499ff
commit
1845f03c41
184 files changed
+24648
No files matched your search
@@ -0,0 +1,8 @@
|
||||
FROM python:3.12-alpine
|
||||
RUN apk add --no-cache ffmpeg
|
||||
COPY stemserve.py /app/stemserve.py
|
||||
ENV STEMS_DIR=/stems CACHE_DIR=/cache PORT=8765 PYTHONUNBUFFERED=1
|
||||
EXPOSE 8765
|
||||
# Unraid shares belong to nobody:users.
|
||||
USER 99:100
|
||||
CMD ["python", "/app/stemserve.py"]
|
||||
@@ -0,0 +1,126 @@
|
||||
# stemserve
|
||||
|
||||
Serves the StemRoller stems folder to the Android app: an index of every separated
|
||||
song, the stem files with HTTP range support (so the phone can seek and stream), per-stem
|
||||
waveforms, and optional Opus transcodes so cached songs take ~1/8 the space.
|
||||
|
||||
One Python file, standard library only. `ffmpeg` is optional; without it you get FLAC
|
||||
only and no waveforms.
|
||||
|
||||
## Run it on unraid
|
||||
|
||||
The stems live at `/mnt/user/Music/Stems` on the server (`//192.168.86.2/Music/Stems`
|
||||
from the desktop). From an unraid terminal:
|
||||
|
||||
```sh
|
||||
mkdir -p /mnt/user/appdata/stemserve/cache /mnt/user/appdata/stemd/state/queue
|
||||
chown -R 99:100 /mnt/user/appdata/stemserve /mnt/user/appdata/stemd
|
||||
cp -r /path/to/stemRoller2/server /mnt/user/appdata/stemserve/src # or copy over SMB
|
||||
docker build -t stemserve /mnt/user/appdata/stemserve/src
|
||||
docker run -d --name stemserve --restart unless-stopped -p 8765:8765 \
|
||||
-v /mnt/user/Music/Stems:/stems:ro \
|
||||
-v /mnt/user/appdata/stemserve/cache:/cache \
|
||||
-v /mnt/user/appdata/stemd/state/queue:/queue \
|
||||
stemserve
|
||||
```
|
||||
|
||||
or `docker compose up -d --build` in this folder (Compose Manager plugin).
|
||||
|
||||
Check it from any machine on the LAN:
|
||||
|
||||
```sh
|
||||
curl http://192.168.86.2:8765/api/info
|
||||
```
|
||||
|
||||
## Settings (environment)
|
||||
|
||||
| Variable | Default | Meaning |
|
||||
|-------------|----------|------------------------------------------------------------|
|
||||
| `STEMS_DIR` | `/stems` | The StemRoller library output folder |
|
||||
| `CACHE_DIR` | `/cache` | Opus transcodes and waveforms (safe to delete) |
|
||||
| `PORT` | `8765` | Listen port |
|
||||
| `TOKEN` | empty | If set, requests need `X-Stem-Token` or `?token=` |
|
||||
| `INDEX_TTL` | `30` | Seconds between rescans of the stems folder |
|
||||
| `QUEUE_DIR` | `/queue` | stemd's Stem Now folder (`stemd/state/queue` on unraid) |
|
||||
|
||||
## API
|
||||
|
||||
| Request | Returns |
|
||||
|---------------------------------------------------|-------------------------------------------------|
|
||||
| `GET /api/info` | track count, whether ffmpeg is available |
|
||||
| `GET /api/index` | every finished song (ETag; 304 when unchanged) |
|
||||
| `GET /api/peaks?key=K&n=256` | RMS per stem in `n` buckets, 0..1000 |
|
||||
| `POST /api/prepare?key=K&fmt=opus&br=256` | starts the Opus transcodes in the background |
|
||||
| `POST /api/stem` `{"paths": [plex file paths]}` | Stem Now: stemd separates these next |
|
||||
| `GET /file/K/<file>.flac[?fmt=opus&br=256]` | the stem, with `Range` support |
|
||||
|
||||
`K` is the song's folder under the stems root, e.g. `1970 - Paranoid/1-02 - Paranoid`.
|
||||
The app matches Plex tracks to it by path: the stems folder mirrors the music folder.
|
||||
|
||||
## Run it anywhere else
|
||||
|
||||
```sh
|
||||
STEMS_DIR=/var/mnt/unraid/Music/Stems CACHE_DIR=~/.cache/stemserve python3 stemserve.py
|
||||
```
|
||||
|
||||
# stemd: separate new music on the server's GPU
|
||||
|
||||
`stemd.py` keeps the stems library up to date. It walks the music folder, separates every
|
||||
track whose stems folder has no `.stemroller.json` yet (BS-RoFormer SW 6-stem by default,
|
||||
like the desktop app), and writes them in the layout above, so stemserve and the phone pick
|
||||
them up within `INDEX_TTL`. Once it has caught up it checks for new music every 30 minutes.
|
||||
Nothing runs on the desktop.
|
||||
|
||||
It writes the same layout the desktop app's old library mode did (`stemroller/library_core.py`),
|
||||
so the ~300 tracks already done on the desktop are skipped, and a track interrupted
|
||||
mid-way (`docker stop`, a reboot) is redone from scratch next time.
|
||||
|
||||
## Run it on unraid (NVIDIA)
|
||||
|
||||
Needs the **Nvidia-Driver** plugin (Apps tab). The image builds from the repo root
|
||||
because it uses the `stemroller` package. Copy the repo, or at least `stemroller/`,
|
||||
`server/` and `.dockerignore`, to `/mnt/user/appdata/stemd/src`, then from an unraid terminal:
|
||||
|
||||
```sh
|
||||
mkdir -p /mnt/user/appdata/stemd/models /mnt/user/appdata/stemd/state
|
||||
chown -R 99:100 /mnt/user/appdata/stemd
|
||||
docker build -f /mnt/user/appdata/stemd/src/server/stemd.Dockerfile -t stemd /mnt/user/appdata/stemd/src
|
||||
docker run -d --name stemd --restart unless-stopped --stop-timeout 60 \
|
||||
--runtime=nvidia -e NVIDIA_VISIBLE_DEVICES=all -e NVIDIA_DRIVER_CAPABILITIES=compute,utility \
|
||||
-v "/mnt/user/Music/Actual music/Flacs":/music:ro \
|
||||
-v /mnt/user/Music/Stems:/stems \
|
||||
-v /mnt/user/appdata/stemd/models:/models \
|
||||
-v /mnt/user/appdata/stemd/state:/state \
|
||||
stemd
|
||||
docker logs -f stemd
|
||||
```
|
||||
|
||||
The first track downloads the model (~700 MB) into `models/`. Each scan logs how many
|
||||
tracks are left, and each track logs its name, how long it took and how many times
|
||||
realtime that was.
|
||||
|
||||
Separation runs one track at a time at low CPU priority, and the model is unloaded between
|
||||
passes so the VRAM is free while idle. It can share the card with Plex transcoding; if
|
||||
VRAM runs out mid-track, that track is retried on the next pass rather than marked failed.
|
||||
Tracks that fail (corrupt files, over 40 minutes) are listed in `state/failed.json` and
|
||||
retried when the file changes; delete that file to retry them all.
|
||||
|
||||
**Stem Now** (hold a song, album, artist or playlist in the app) jumps the queue: stemserve
|
||||
writes the request into `state/queue/`, and stemd separates those tracks next, after the
|
||||
one it's on (or within 5 seconds if it's idle). Requests run oldest first, and survive a
|
||||
restart. stemd matches the Plex paths to files under `/music` by their artist/album/file
|
||||
tail, so the two containers don't need the same mount points. Tracks in `failed.json` are
|
||||
skipped here too.
|
||||
|
||||
To update after changing the code: copy the files over again, rebuild, then
|
||||
`docker rm -f stemd` and repeat the `docker run`.
|
||||
|
||||
| Variable | Default | Meaning |
|
||||
|--------------|------------------|------------------------------------------------------|
|
||||
| `MODEL` | `roformer_bs_sw` | any id from `stemroller/catalog.py` |
|
||||
| `DEVICE` | `cuda` | `cuda`, `cuda:N` or `cpu` |
|
||||
| `RESCAN_MIN` | `30` | minutes between scans once caught up |
|
||||
| `FP16` | `1` | half precision on the GPU (`0` for fp32) |
|
||||
| `OVERLAP`, `SEGMENT`, `SHIFTS` | `0.25`, `0`, `0` | as in the desktop app's settings |
|
||||
|
||||
`docker build --build-arg TORCH_INDEX=cpu ...` makes a small image without CUDA for testing.
|
||||
@@ -0,0 +1,16 @@
|
||||
services:
|
||||
stemserve:
|
||||
build: .
|
||||
image: stemserve:latest
|
||||
container_name: stemserve
|
||||
restart: unless-stopped
|
||||
ports:
|
||||
- "8765:8765"
|
||||
volumes:
|
||||
- /mnt/user/Music/Stems:/stems:ro
|
||||
- /mnt/user/appdata/stemserve/cache:/cache
|
||||
# stemd's Stem Now folder (create it owned by 99:100 first; see README).
|
||||
- /mnt/user/appdata/stemd/state/queue:/queue
|
||||
environment:
|
||||
# Optional shared secret; set the same token in the app (Settings > Stem Server).
|
||||
- TOKEN=
|
||||
@@ -0,0 +1,73 @@
|
||||
# stemd's Python deps, pinned to match uv.lock (torch and torchvision come from the CUDA index; see stemd.Dockerfile).
|
||||
# Regenerate: uv export --frozen --no-hashes --no-emit-project --no-header --no-annotate \
|
||||
# | grep -vE '^(torch|torchvision|triton-rocm|pyside6|pyside6-addons|pyside6-essentials|shiboken6|sounddevice)[=@ ]' | sed 's/ ;.*//' \
|
||||
# | sed 's#^demucs @ git+\(.*\)@\(.*\)#demucs @ \1/archive/\2.tar.gz#' (no git in the image)
|
||||
absl-py==2.5.0
|
||||
anyio==4.15.1
|
||||
audio-separator==0.47.0
|
||||
audioread==3.1.0
|
||||
beartype==0.18.5
|
||||
certifi==2026.7.22
|
||||
cffi==2.1.1
|
||||
charset-normalizer==3.5.2
|
||||
click==8.5.0
|
||||
cloudpickle==3.1.2
|
||||
cython==3.3.0
|
||||
decorator==5.3.1
|
||||
demucs @ https://github.com/adefossez/demucs/archive/2883f3db65617d6d178c6ed10d869dc14e44e59b.tar.gz
|
||||
diffq==0.2.4
|
||||
einops==0.8.2
|
||||
filelock==4.0.12
|
||||
flatbuffers==25.12.19
|
||||
fsspec==2026.9.0
|
||||
h11==0.16.0
|
||||
hf-xet==1.7.0
|
||||
httpcore2==2.13.1
|
||||
httpx2==2.13.1
|
||||
huggingface-hub==2.1.1
|
||||
idna==3.20
|
||||
jinja2==3.1.6
|
||||
joblib==1.6.0
|
||||
julius==0.2.8
|
||||
lameenc==1.8.4
|
||||
lazy-loader==0.6
|
||||
librosa==1.0.0
|
||||
llvmlite==0.50.0
|
||||
markupsafe==3.0.4
|
||||
ml-collections==1.1.0
|
||||
ml-dtypes==0.6.0
|
||||
mpmath==1.3.0
|
||||
msgpack==1.2.3
|
||||
narwhals==2.26.0
|
||||
networkx==3.7
|
||||
numba==0.68.0
|
||||
numpy==2.5.3
|
||||
onnx-weekly==1.24.0.dev20261005
|
||||
onnx2torch-py313==1.6.0
|
||||
onnxruntime==1.30.0
|
||||
packaging==26.3
|
||||
pillow==12.3.0
|
||||
platformdirs==4.12.4
|
||||
pooch==1.9.0
|
||||
protobuf==7.36.2
|
||||
pycparser==3.0
|
||||
pydub==0.25.1
|
||||
pyyaml==6.0.3
|
||||
requests==2.34.2
|
||||
resampy==0.4.3
|
||||
rotary-embedding-torch==0.6.5
|
||||
safetensors==0.8.0
|
||||
samplerate==0.1.0
|
||||
scikit-learn==1.9.1
|
||||
scipy==1.18.1
|
||||
setuptools==84.0.0
|
||||
six==1.17.0
|
||||
soundfile==0.14.0
|
||||
soxr==1.1.0
|
||||
sphn==0.2.1
|
||||
sympy==1.14.0
|
||||
threadpoolctl==3.7.0
|
||||
tqdm==4.70.1
|
||||
truststore==0.10.4
|
||||
typing-extensions==4.16.0
|
||||
urllib3==2.8.0
|
||||
@@ -0,0 +1,24 @@
|
||||
# stemd: separates new music into stems on the server's NVIDIA GPU (see stemd.py).
|
||||
# Build from the repo root, since it needs the stemroller package:
|
||||
# docker build -f server/stemd.Dockerfile -t stemd .
|
||||
FROM python:3.12-slim
|
||||
RUN apt-get update && apt-get install -y --no-install-recommends ffmpeg && rm -rf /var/lib/apt/lists/*
|
||||
# cu126 runs on NVIDIA driver 525 and up. TORCH_INDEX=cpu builds a small GPU-less image for testing.
|
||||
ARG TORCH_INDEX=cu126
|
||||
RUN pip install --no-cache-dir torch==2.14.1 torchvision==0.29.1 --index-url https://download.pytorch.org/whl/${TORCH_INDEX}
|
||||
COPY server/stemd-requirements.txt /tmp/
|
||||
# Everything is pinned, so --no-deps keeps pip from swapping torch for a non-CUDA build.
|
||||
# gcc is only for diffq, which has no wheel.
|
||||
RUN apt-get update && apt-get install -y --no-install-recommends gcc libc6-dev \
|
||||
&& pip install --no-cache-dir --no-deps -r /tmp/stemd-requirements.txt && pip check \
|
||||
&& apt-get purge -y gcc libc6-dev && apt-get autoremove -y && rm -rf /var/lib/apt/lists/*
|
||||
COPY stemroller/__init__.py stemroller/catalog.py stemroller/library_core.py /app/stemroller/
|
||||
COPY stemroller/worker/__init__.py stemroller/worker/__main__.py stemroller/worker/audio_io.py \
|
||||
stemroller/worker/backends.py /app/stemroller/worker/
|
||||
COPY server/stemd.py /app/server/
|
||||
ENV MUSIC_DIR=/music STEMS_DIR=/stems STATE_DIR=/state MODELS_DIR=/models \
|
||||
TORCH_HOME=/models/torch HF_HOME=/models/hf NUMBA_CACHE_DIR=/tmp/numba HOME=/tmp \
|
||||
PYTHONUNBUFFERED=1 PYTHONDONTWRITEBYTECODE=1 PYTHONWARNINGS=ignore::FutureWarning
|
||||
# Unraid shares belong to nobody:users.
|
||||
USER 99:100
|
||||
CMD ["python", "/app/server/stemd.py"]
|
||||
+348
@@ -0,0 +1,348 @@
|
||||
#!/usr/bin/env python3
|
||||
"""stemd: separates every track in the music library that doesn't have stems yet.
|
||||
|
||||
Runs on the server next to stemserve. Walks MUSIC_DIR, and for each track whose folder
|
||||
under STEMS_DIR has no marker it runs the separation on the GPU and writes the stems with
|
||||
the same layout and marker as the desktop app (stemroller/library_core.py), so stemserve
|
||||
and the phone pick them up as they land. Once caught up it looks again every RESCAN_MIN.
|
||||
|
||||
"Stem Now" in the app goes through stemserve, which drops a request (a JSON list of Plex
|
||||
file paths) into QUEUE_DIR. Those tracks are separated before the rest: between tracks
|
||||
during a pass, and within a few seconds while idle.
|
||||
|
||||
Settings (environment):
|
||||
|
||||
MUSIC_DIR /music the music library (read-only is fine)
|
||||
STEMS_DIR /stems where stems go
|
||||
STATE_DIR /state failed-track list
|
||||
QUEUE_DIR STATE_DIR/queue "Stem Now" requests from stemserve
|
||||
MODELS_DIR /models model weights, downloaded on first use
|
||||
MODEL roformer_bs_sw any id from stemroller/catalog.py
|
||||
DEVICE cuda cuda, cuda:N or cpu
|
||||
RESCAN_MIN 30 minutes between scans once caught up
|
||||
FP16 1 half precision on the GPU
|
||||
OVERLAP / SEGMENT / SHIFTS as in the desktop app's settings
|
||||
"""
|
||||
|
||||
import glob
|
||||
import json
|
||||
import os
|
||||
import shutil
|
||||
import signal
|
||||
import sys
|
||||
import tempfile
|
||||
import threading
|
||||
import time
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||
|
||||
from stemroller.catalog import MODELS_BY_ID # noqa: E402
|
||||
from stemroller.library_core import (MAX_SECONDS, SOURCE_RANK, duration, is_done, is_share_error, # noqa: E402
|
||||
iter_tracks, signature, write_track)
|
||||
|
||||
MUSIC_DIR = os.environ.get("MUSIC_DIR", "/music")
|
||||
STEMS_DIR = os.environ.get("STEMS_DIR", "/stems")
|
||||
STATE_DIR = os.environ.get("STATE_DIR", "/state")
|
||||
MODELS_DIR = os.environ.get("MODELS_DIR", "/models")
|
||||
MODEL = os.environ.get("MODEL", "roformer_bs_sw")
|
||||
DEVICE = os.environ.get("DEVICE", "cuda")
|
||||
RESCAN_S = float(os.environ.get("RESCAN_MIN", "30")) * 60
|
||||
FP16 = os.environ.get("FP16", "1") not in ("0", "false", "no", "")
|
||||
OVERLAP = float(os.environ.get("OVERLAP", "0.25"))
|
||||
SEGMENT = float(os.environ.get("SEGMENT", "0"))
|
||||
SHIFTS = int(os.environ.get("SHIFTS", "0"))
|
||||
QUEUE_DIR = os.environ.get("QUEUE_DIR", os.path.join(STATE_DIR, "queue"))
|
||||
FAILED_PATH = os.path.join(STATE_DIR, "failed.json")
|
||||
QUEUE_POLL_S = 5 # how often an idle stemd looks for Stem Now requests
|
||||
SHARE_RETRY_S = 5 * 60 # stems folder unwritable: try again this soon rather than at RESCAN_S
|
||||
MAX_CRASHES = 2 # GPU crashes on one track before it's given up on
|
||||
|
||||
stop = threading.Event()
|
||||
|
||||
|
||||
def log(*args):
|
||||
print(time.strftime("%Y-%m-%d %H:%M:%S"), *args, flush=True)
|
||||
|
||||
|
||||
class Failed:
|
||||
"""Tracks that didn't separate, keyed by path, retried once the source file changes."""
|
||||
|
||||
def __init__(self):
|
||||
try:
|
||||
with open(FAILED_PATH) as f:
|
||||
self.data = json.load(f)
|
||||
except (OSError, ValueError):
|
||||
self.data = {}
|
||||
|
||||
def skip(self, path, sig):
|
||||
e = self.data.get(path)
|
||||
return bool(e) and e["sig"] == sig and e.get("tries", MAX_CRASHES) >= MAX_CRASHES
|
||||
|
||||
def add(self, path, sig, msg, crash=False):
|
||||
tries = MAX_CRASHES
|
||||
if crash:
|
||||
e = self.data.get(path)
|
||||
tries = (e.get("tries", 0) if e and e["sig"] == sig else 0) + 1
|
||||
self.data[path] = {"sig": sig, "msg": msg, "tries": tries, "at": time.time()}
|
||||
os.makedirs(STATE_DIR, exist_ok=True)
|
||||
tmp = FAILED_PATH + ".tmp"
|
||||
with open(tmp, "w") as f:
|
||||
json.dump(self.data, f, indent=1)
|
||||
os.replace(tmp, FAILED_PATH)
|
||||
|
||||
|
||||
class _NoProgress:
|
||||
"""Stands in for the worker's socket; per-track progress isn't worth a log line."""
|
||||
|
||||
def send(self, *_, **__):
|
||||
pass
|
||||
|
||||
|
||||
class Separator:
|
||||
def __init__(self):
|
||||
import torch
|
||||
from stemroller.worker.backends import DemucsBackend, RoformerBackend
|
||||
|
||||
self.torch = torch
|
||||
self.model = MODELS_BY_ID[MODEL]
|
||||
self.device = torch.device(DEVICE if DEVICE == "cpu" or torch.cuda.is_available() else "cpu")
|
||||
if DEVICE != "cpu" and self.device.type == "cpu":
|
||||
log("WARNING: no CUDA device visible; separating on the CPU (slow). Is the container "
|
||||
"running with the nvidia runtime?")
|
||||
self.backend = (DemucsBackend() if self.model["family"] == "demucs" else RoformerBackend(MODELS_DIR))
|
||||
name = torch.cuda.get_device_name(self.device) if self.device.type == "cuda" else "CPU"
|
||||
log(f"model {self.model['label']} on {name}, torch {torch.__version__}")
|
||||
|
||||
def run(self, path, tmp):
|
||||
# Imported late: worker/__main__ sets ROCm env defaults, harmless here, and owns Job's
|
||||
# parameter rules (overlap, fp16, ...), which must match the desktop's.
|
||||
from stemroller.worker.__main__ import Job
|
||||
job = Job({"id": "stemd", "file": path, "model": MODEL, "out_dir": tmp, "segment": SEGMENT,
|
||||
"shifts": SHIFTS, "overlap": OVERLAP, "fp16": FP16}, _NoProgress())
|
||||
if "file" in self.model and not os.path.exists(os.path.join(MODELS_DIR, self.model["file"])):
|
||||
log(" downloading the model (first run only)")
|
||||
job.cancel_event = stop
|
||||
stems, sr = self.backend.run(job, self.device)
|
||||
# Backends return (channels, samples); the writer wants (samples, channels).
|
||||
return {name: audio.T for name, audio in stems.items()}, sr
|
||||
|
||||
def release(self):
|
||||
self.backend.release()
|
||||
|
||||
def is_fatal(self, e):
|
||||
"""CUDA errors other than OOM poison the context; only a fresh process recovers."""
|
||||
if isinstance(e, self.torch.OutOfMemoryError):
|
||||
return False
|
||||
return isinstance(e, getattr(self.torch, "AcceleratorError", ())) or "CUDA error" in str(e)
|
||||
|
||||
|
||||
def separate(sep, failed, path, dest, sig, label):
|
||||
"""Separate one track into dest. Returns True if it was written.
|
||||
|
||||
Raises on a stop request, an unwritable stems share, or a GPU error that needs a restart.
|
||||
"""
|
||||
rel = os.path.relpath(path, MUSIC_DIR)
|
||||
dur = duration(path)
|
||||
if dur is not None and dur > MAX_SECONDS:
|
||||
log(f"{label} skip {rel}: {dur / 60:.0f} min long")
|
||||
failed.add(path, sig, f"longer than {MAX_SECONDS // 60} minutes")
|
||||
return False
|
||||
log(f"{label} {rel}" + (f" ({int(dur) // 60}:{int(dur) % 60:02d})" if dur else ""))
|
||||
t1 = time.monotonic()
|
||||
tmp = tempfile.mkdtemp(prefix="stemd-")
|
||||
try:
|
||||
stems, sr = sep.run(path, tmp)
|
||||
meta = {"source": path, "size": sig[0], "mtime_ns": sig[1], "model": MODEL, "samplerate": sr,
|
||||
"duration": len(next(iter(stems.values()))) / sr}
|
||||
write_track(dest, stems, meta)
|
||||
except BaseException as e:
|
||||
if stop.is_set():
|
||||
raise # Cancelled; the track is redone next start
|
||||
if is_share_error(e):
|
||||
log(f" can't write to {STEMS_DIR}: {e}; pausing")
|
||||
raise
|
||||
if not isinstance(e, Exception):
|
||||
raise
|
||||
msg = str(e).strip().splitlines()[0] if str(e).strip() else e.__class__.__name__
|
||||
if isinstance(e, sep.torch.OutOfMemoryError):
|
||||
# Something else (Plex, a game server) may be holding VRAM; try again next pass.
|
||||
log(" out of GPU memory; will retry next pass")
|
||||
sep.release()
|
||||
return False
|
||||
if sep.is_fatal(e):
|
||||
failed.add(path, sig, msg, crash=True)
|
||||
log(f" GPU error: {msg}; restarting")
|
||||
sys.exit(75)
|
||||
failed.add(path, sig, msg)
|
||||
log(f" failed: {msg}")
|
||||
return False
|
||||
finally:
|
||||
shutil.rmtree(tmp, ignore_errors=True)
|
||||
took = time.monotonic() - t1
|
||||
log(f" done in {took:.0f}s" + (f" ({meta['duration'] / took:.1f}x realtime)" if took else ""))
|
||||
return True
|
||||
|
||||
|
||||
def resolve(plex_path):
|
||||
"""The music file a Plex path names, or None. Plex sees the library under its own mount
|
||||
point, so this takes the longest tail of the path (folder and file at least) that exists
|
||||
under MUSIC_DIR, and prefers the best format when a folder holds the song in several."""
|
||||
parts = [p for p in plex_path.replace("\\", "/").split("/") if p not in ("", ".", "..")]
|
||||
root = os.path.realpath(MUSIC_DIR)
|
||||
for i in range(len(parts) - 1):
|
||||
path = os.path.join(root, *parts[i:])
|
||||
if os.path.isfile(path) and os.path.realpath(path).startswith(root + os.sep):
|
||||
break
|
||||
else:
|
||||
return None
|
||||
folder, name = os.path.split(path)
|
||||
base = os.path.splitext(name)[0].casefold()
|
||||
try:
|
||||
same = [f for f in os.listdir(folder) if os.path.splitext(f)[0].casefold() == base
|
||||
and os.path.splitext(f)[1].lower() in SOURCE_RANK]
|
||||
except OSError:
|
||||
same = []
|
||||
if same:
|
||||
path = os.path.join(folder, min(same, key=lambda f: (SOURCE_RANK[os.path.splitext(f)[1].lower()], f != name)))
|
||||
return path if os.path.splitext(path)[1].lower() in SOURCE_RANK else None
|
||||
|
||||
|
||||
def serve_queue(sep, failed):
|
||||
"""Separate every Stem Now request, oldest first. Returns whether any track was attempted.
|
||||
|
||||
A request file is deleted once all its tracks are done, so one cut short by a restart
|
||||
picks up where it left off (finished tracks are skipped)."""
|
||||
used = False
|
||||
while not stop.is_set():
|
||||
reqs = sorted(glob.glob(os.path.join(QUEUE_DIR, "*.json")))
|
||||
if not reqs:
|
||||
break
|
||||
req = reqs[0]
|
||||
try:
|
||||
with open(req) as f:
|
||||
paths = json.load(f).get("paths", [])
|
||||
except (OSError, ValueError, AttributeError) as e:
|
||||
log(f"stem now: unreadable request {os.path.basename(req)}: {e}")
|
||||
paths = []
|
||||
todo = []
|
||||
for p in paths if isinstance(paths, list) else []:
|
||||
src = resolve(p) if isinstance(p, str) else None
|
||||
if src is None:
|
||||
log(f"stem now: not in {MUSIC_DIR}: {p}")
|
||||
continue
|
||||
dest = os.path.join(STEMS_DIR, os.path.splitext(os.path.relpath(src, MUSIC_DIR))[0])
|
||||
if is_done(dest):
|
||||
continue
|
||||
try:
|
||||
sig = signature(src)
|
||||
except OSError:
|
||||
continue
|
||||
if failed.skip(src, sig):
|
||||
log(f"stem now: skip {os.path.relpath(src, MUSIC_DIR)}: failed before ({failed.data[src]['msg']})")
|
||||
continue
|
||||
todo.append((src, dest, sig))
|
||||
if todo:
|
||||
log(f"stem now: {len(todo)} track{'' if len(todo) == 1 else 's'} ahead of the rest")
|
||||
for i, (src, dest, sig) in enumerate(todo, 1):
|
||||
if stop.is_set():
|
||||
return used
|
||||
if is_done(dest):
|
||||
continue
|
||||
used = True
|
||||
separate(sep, failed, src, dest, sig, f"[now {i}/{len(todo)}]")
|
||||
if stop.is_set():
|
||||
break
|
||||
try:
|
||||
os.remove(req)
|
||||
except FileNotFoundError:
|
||||
pass
|
||||
return used
|
||||
|
||||
|
||||
def run_pass(sep, failed):
|
||||
"""Separate everything that needs it. Returns (tracks done, whether the GPU was used)."""
|
||||
t0 = time.monotonic()
|
||||
todo, total = [], 0
|
||||
for path, dest in iter_tracks(MUSIC_DIR, STEMS_DIR):
|
||||
if stop.is_set():
|
||||
return 0, False
|
||||
total += 1
|
||||
if is_done(dest):
|
||||
continue
|
||||
try:
|
||||
sig = signature(path)
|
||||
except OSError:
|
||||
continue
|
||||
if not failed.skip(path, sig):
|
||||
todo.append((path, dest, sig))
|
||||
log(f"scan: {total:,} tracks, {total - len(todo):,} done or skipped, {len(todo):,} to do "
|
||||
f"({time.monotonic() - t0:.0f}s)")
|
||||
if not total:
|
||||
log(f"WARNING: no music found in {MUSIC_DIR}; is the share mounted and readable?")
|
||||
|
||||
done, used = 0, False
|
||||
for i, (path, dest, sig) in enumerate(todo, 1):
|
||||
used = serve_queue(sep, failed) or used
|
||||
if stop.is_set():
|
||||
break
|
||||
if is_done(dest): # a full pass takes days; the desktop or a Stem Now may have got there first
|
||||
continue
|
||||
used = True
|
||||
if separate(sep, failed, path, dest, sig, f"[{i}/{len(todo)}]"):
|
||||
done += 1
|
||||
return done, used
|
||||
|
||||
|
||||
def idle(sep, failed, seconds):
|
||||
"""Wait for the next scan, separating Stem Now requests as they arrive."""
|
||||
end = time.monotonic() + seconds
|
||||
while not stop.is_set() and time.monotonic() < end:
|
||||
if serve_queue(sep, failed):
|
||||
sep.release()
|
||||
stop.wait(min(QUEUE_POLL_S, max(0.0, end - time.monotonic())))
|
||||
|
||||
|
||||
def main():
|
||||
for name, path in (("MUSIC_DIR", MUSIC_DIR), ("STEMS_DIR", STEMS_DIR)):
|
||||
if not os.path.isdir(path):
|
||||
sys.exit(f"{name} not found: {path}")
|
||||
if MODEL not in MODELS_BY_ID:
|
||||
sys.exit(f"unknown MODEL {MODEL!r}; pick one of: {', '.join(MODELS_BY_ID)}")
|
||||
# docker stop sends SIGTERM; stop between (or inside) tracks instead of dying mid-write.
|
||||
signal.signal(signal.SIGTERM, lambda *_: stop.set())
|
||||
signal.signal(signal.SIGINT, lambda *_: stop.set())
|
||||
try:
|
||||
os.nice(10)
|
||||
except OSError:
|
||||
pass
|
||||
try:
|
||||
os.makedirs(QUEUE_DIR, exist_ok=True)
|
||||
except OSError as e:
|
||||
log(f"WARNING: can't create {QUEUE_DIR} ({e}); Stem Now won't work")
|
||||
log(f"stemd: {MUSIC_DIR} -> {STEMS_DIR}")
|
||||
sep = Separator()
|
||||
failed = Failed()
|
||||
while not stop.is_set():
|
||||
try:
|
||||
done, used = run_pass(sep, failed)
|
||||
if done:
|
||||
log(f"pass finished: {done:,} tracks separated")
|
||||
if used:
|
||||
sep.release() # hand the VRAM back while idle
|
||||
idle(sep, failed, RESCAN_S)
|
||||
except OSError as e:
|
||||
if stop.is_set():
|
||||
break
|
||||
if not is_share_error(e):
|
||||
raise
|
||||
sep.release()
|
||||
stop.wait(SHARE_RETRY_S)
|
||||
except BaseException:
|
||||
if stop.is_set():
|
||||
break
|
||||
raise
|
||||
log("stopped")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,454 @@
|
||||
#!/usr/bin/env python3
|
||||
"""stemserve: serves StemRoller library stems to the Android app over HTTP.
|
||||
|
||||
Stdlib only; ffmpeg is optional and enables Opus transcodes and waveforms.
|
||||
|
||||
The stems folder is what stemroller.library writes:
|
||||
|
||||
<STEMS_DIR>/Artist/Album/01 Song/.stemroller.json
|
||||
<STEMS_DIR>/Artist/Album/01 Song/01 Song - vocals.flac, ...
|
||||
|
||||
Endpoints (all GET unless noted; add ?token=... or X-Stem-Token when TOKEN is set):
|
||||
|
||||
/api/info server facts
|
||||
/api/index every finished track (ETag / If-None-Match)
|
||||
/api/peaks?key=<key>&n=<buckets> per-stem RMS envelopes, 0..1000
|
||||
POST /api/prepare?key=<key>&fmt=opus&br=160 start transcodes in the background
|
||||
POST /api/stem {"paths": [...]} "Stem Now": stemd separates these Plex files next
|
||||
/file/<key>/<file>[?fmt=opus&br=160] the stem itself, with Range support
|
||||
|
||||
<key> is the track's folder relative to STEMS_DIR, e.g. "Artist/Album/01 Song".
|
||||
"""
|
||||
|
||||
import email.utils
|
||||
import hashlib
|
||||
import json
|
||||
import operator
|
||||
import os
|
||||
import shutil
|
||||
import signal
|
||||
import subprocess
|
||||
import sys
|
||||
import threading
|
||||
import time
|
||||
import urllib.parse
|
||||
from array import array
|
||||
from concurrent.futures import ThreadPoolExecutor
|
||||
from http import HTTPStatus
|
||||
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
|
||||
|
||||
STEMS_DIR = os.path.realpath(os.environ.get("STEMS_DIR", "/stems"))
|
||||
CACHE_DIR = os.environ.get("CACHE_DIR", "/cache")
|
||||
PORT = int(os.environ.get("PORT", "8765"))
|
||||
TOKEN = os.environ.get("TOKEN", "")
|
||||
INDEX_TTL = float(os.environ.get("INDEX_TTL", "30"))
|
||||
QUEUE_DIR = os.environ.get("QUEUE_DIR", "/queue") # stemd's request folder, shared with it
|
||||
MAX_BODY = 8 << 20
|
||||
MARKER = ".stemroller.json"
|
||||
VERSION = 1
|
||||
FFMPEG = shutil.which("ffmpeg")
|
||||
OPUS_RATES = (64, 96, 128, 160, 192, 256)
|
||||
PEAK_RATE = 2000 # Hz the envelope is computed at; plenty for a few hundred bars
|
||||
CHUNK = 1 << 20
|
||||
|
||||
_pool = ThreadPoolExecutor(max_workers=max(2, (os.cpu_count() or 2) // 2))
|
||||
_locks: dict[str, threading.Lock] = {}
|
||||
_locks_guard = threading.Lock()
|
||||
|
||||
|
||||
def log(*args):
|
||||
print(time.strftime("%H:%M:%S"), *args, file=sys.stderr, flush=True)
|
||||
|
||||
|
||||
def _lock_for(path):
|
||||
with _locks_guard:
|
||||
return _locks.setdefault(path, threading.Lock())
|
||||
|
||||
|
||||
# ---- index ------------------------------------------------------------------
|
||||
|
||||
class Index:
|
||||
def __init__(self):
|
||||
self._lock = threading.Lock()
|
||||
self._scanning = threading.Lock()
|
||||
self._at = 0.0
|
||||
self._body = b'{"tracks":[]}'
|
||||
self._etag = '"empty"'
|
||||
self._count = 0
|
||||
self._by_key: dict[str, dict] = {}
|
||||
|
||||
def get(self):
|
||||
if time.monotonic() - self._at > INDEX_TTL:
|
||||
self.refresh()
|
||||
with self._lock:
|
||||
return self._body, self._etag
|
||||
|
||||
def entry(self, key):
|
||||
if time.monotonic() - self._at > INDEX_TTL:
|
||||
self.refresh()
|
||||
with self._lock:
|
||||
return self._by_key.get(key)
|
||||
|
||||
@property
|
||||
def count(self):
|
||||
return self._count
|
||||
|
||||
def refresh(self):
|
||||
# One scan at a time; callers that lose the race use the previous result.
|
||||
if not self._scanning.acquire(blocking=self._at == 0):
|
||||
return
|
||||
try:
|
||||
tracks = []
|
||||
for root, dirs, files in os.walk(STEMS_DIR):
|
||||
dirs[:] = sorted(d for d in dirs if not d.startswith("."))
|
||||
if MARKER not in files:
|
||||
continue
|
||||
entry = _read_marker(root, set(files))
|
||||
if entry:
|
||||
tracks.append(entry)
|
||||
tracks.sort(key=lambda t: t["key"].casefold())
|
||||
body = json.dumps({"version": VERSION, "tracks": tracks}, separators=(",", ":")).encode()
|
||||
with self._lock:
|
||||
self._body = body
|
||||
self._etag = '"%s"' % hashlib.sha1(body).hexdigest()[:20]
|
||||
self._count = len(tracks)
|
||||
self._by_key = {t["key"]: t for t in tracks}
|
||||
self._at = time.monotonic()
|
||||
finally:
|
||||
self._scanning.release()
|
||||
|
||||
|
||||
def _read_marker(folder, files):
|
||||
try:
|
||||
with open(os.path.join(folder, MARKER), encoding="utf-8") as f:
|
||||
meta = json.load(f)
|
||||
except (OSError, ValueError):
|
||||
return None
|
||||
key = os.path.relpath(folder, STEMS_DIR).replace(os.sep, "/")
|
||||
stems = []
|
||||
for s in meta.get("stems", []):
|
||||
name, fname = s.get("name"), s.get("file")
|
||||
if not name or not fname or fname not in files:
|
||||
continue
|
||||
try:
|
||||
st = os.stat(os.path.join(folder, fname))
|
||||
except OSError:
|
||||
continue
|
||||
stems.append({"name": name, "file": fname, "size": st.st_size, "mtime": int(st.st_mtime)})
|
||||
if not stems:
|
||||
return None
|
||||
# The stems folder mirrors the music folder, so the source's path relative to the
|
||||
# music root is this folder's parent plus the source's file name.
|
||||
src_name = os.path.basename(meta.get("source") or "") or os.path.basename(folder)
|
||||
parent = os.path.dirname(key)
|
||||
return {
|
||||
"key": key,
|
||||
"source": f"{parent}/{src_name}" if parent else src_name,
|
||||
"duration": meta.get("duration"),
|
||||
"samplerate": meta.get("samplerate"),
|
||||
"model": meta.get("model"),
|
||||
"finished": meta.get("finished"),
|
||||
"stems": stems,
|
||||
}
|
||||
|
||||
|
||||
INDEX = Index()
|
||||
|
||||
|
||||
# ---- transcodes and peaks ---------------------------------------------------
|
||||
|
||||
def _resolve(key, fname):
|
||||
"""Absolute path of a stem file, or None if it escapes STEMS_DIR or is missing."""
|
||||
path = os.path.realpath(os.path.join(STEMS_DIR, key, fname))
|
||||
if not path.startswith(STEMS_DIR + os.sep) or not path.lower().endswith(".flac"):
|
||||
return None
|
||||
return path if os.path.isfile(path) else None
|
||||
|
||||
|
||||
def _cache_name(src, tag, ext):
|
||||
st = os.stat(src)
|
||||
h = hashlib.sha1(f"{src}|{st.st_size}|{st.st_mtime_ns}|{tag}".encode()).hexdigest()
|
||||
return os.path.join(CACHE_DIR, tag.split(":")[0], h[:2], h + ext)
|
||||
|
||||
|
||||
def transcode(src, br):
|
||||
"""Path of an Ogg Opus copy of src at br kbps, transcoding it first if needed."""
|
||||
out = _cache_name(src, f"opus:{br}", ".opus")
|
||||
if os.path.exists(out):
|
||||
os.utime(out) # for LRU-ish cleanup by mtime
|
||||
return out
|
||||
with _lock_for(out):
|
||||
if os.path.exists(out):
|
||||
return out
|
||||
os.makedirs(os.path.dirname(out), exist_ok=True)
|
||||
tmp = out + ".part"
|
||||
t0 = time.monotonic()
|
||||
subprocess.run([FFMPEG, "-nostdin", "-v", "error", "-y", "-i", src, "-map", "0:a:0",
|
||||
"-c:a", "libopus", "-b:a", f"{br}k", "-vbr", "on", "-application", "audio",
|
||||
"-map_metadata", "-1", "-f", "ogg", tmp], check=True, timeout=600)
|
||||
os.replace(tmp, out)
|
||||
log(f"opus {br}k {os.path.basename(src)} in {time.monotonic() - t0:.1f}s")
|
||||
return out
|
||||
|
||||
|
||||
def _envelope(src, n, duration):
|
||||
"""RMS of src in n equal buckets across `duration` seconds, as 0..1 of full scale."""
|
||||
raw = subprocess.run([FFMPEG, "-nostdin", "-v", "error", "-i", src, "-map", "0:a:0", "-ac", "1",
|
||||
"-ar", str(PEAK_RATE), "-f", "s16le", "-"],
|
||||
check=True, capture_output=True, timeout=300).stdout
|
||||
pcm = array("h")
|
||||
pcm.frombytes(raw[:len(raw) // 2 * 2])
|
||||
if sys.byteorder == "big":
|
||||
pcm.byteswap()
|
||||
total = max(1, int(round((duration or len(pcm) / PEAK_RATE) * PEAK_RATE)))
|
||||
out = []
|
||||
for i in range(n):
|
||||
a, b = i * total // n, (i + 1) * total // n
|
||||
chunk = pcm[a:max(b, a + 1)]
|
||||
if not chunk:
|
||||
out.append(0)
|
||||
continue
|
||||
ms = sum(map(operator.mul, chunk, chunk)) / len(chunk)
|
||||
out.append(min(1000, int(round((ms ** 0.5) / 32768 * 1000))))
|
||||
return out
|
||||
|
||||
|
||||
def peaks(entry, n):
|
||||
folder = os.path.join(STEMS_DIR, entry["key"])
|
||||
first = _resolve(entry["key"], entry["stems"][0]["file"])
|
||||
sig = "|".join(f"{s['file']}:{s['size']}:{s['mtime']}" for s in entry["stems"])
|
||||
out = _cache_name(first, f"peaks:{n}:{hashlib.sha1(sig.encode()).hexdigest()}", ".json")
|
||||
if os.path.exists(out):
|
||||
with open(out, "rb") as f:
|
||||
return f.read()
|
||||
with _lock_for(out):
|
||||
if os.path.exists(out):
|
||||
with open(out, "rb") as f:
|
||||
return f.read()
|
||||
t0 = time.monotonic()
|
||||
jobs = {s["name"]: _pool.submit(_envelope, os.path.join(folder, s["file"]), n, entry.get("duration"))
|
||||
for s in entry["stems"]}
|
||||
body = json.dumps({"key": entry["key"], "n": n, "duration": entry.get("duration"),
|
||||
"stems": {name: job.result() for name, job in jobs.items()}},
|
||||
separators=(",", ":")).encode()
|
||||
os.makedirs(os.path.dirname(out), exist_ok=True)
|
||||
with open(out + ".part", "wb") as f:
|
||||
f.write(body)
|
||||
os.replace(out + ".part", out)
|
||||
log(f"peaks {entry['key']} in {time.monotonic() - t0:.1f}s")
|
||||
return body
|
||||
|
||||
|
||||
# ---- HTTP -------------------------------------------------------------------
|
||||
|
||||
class Handler(BaseHTTPRequestHandler):
|
||||
protocol_version = "HTTP/1.1"
|
||||
server_version = f"stemserve/{VERSION}"
|
||||
|
||||
def log_message(self, fmt, *args):
|
||||
if os.environ.get("VERBOSE"):
|
||||
log(self.address_string(), fmt % args)
|
||||
|
||||
def _authorized(self, query):
|
||||
if not TOKEN:
|
||||
return True
|
||||
return TOKEN in (self.headers.get("X-Stem-Token"), query.get("token", [None])[0])
|
||||
|
||||
def _send(self, status, body=b"", ctype="application/json", headers=None):
|
||||
self.send_response(status)
|
||||
self.send_header("Content-Type", ctype)
|
||||
self.send_header("Content-Length", str(len(body)))
|
||||
for k, v in (headers or {}).items():
|
||||
self.send_header(k, v)
|
||||
self.end_headers()
|
||||
if self.command != "HEAD":
|
||||
self.wfile.write(body)
|
||||
|
||||
def _error(self, status, msg):
|
||||
self._send(status, json.dumps({"error": msg}).encode())
|
||||
|
||||
def do_HEAD(self):
|
||||
self.do_GET()
|
||||
|
||||
def do_POST(self):
|
||||
self.do_GET()
|
||||
|
||||
def do_GET(self):
|
||||
url = urllib.parse.urlsplit(self.path)
|
||||
query = urllib.parse.parse_qs(url.query)
|
||||
path = urllib.parse.unquote(url.path)
|
||||
if not self._authorized(query):
|
||||
return self._error(HTTPStatus.UNAUTHORIZED, "bad token")
|
||||
try:
|
||||
if path == "/api/info":
|
||||
INDEX.get()
|
||||
return self._send(200, json.dumps({
|
||||
"name": "stemserve", "version": VERSION, "tracks": INDEX.count,
|
||||
"ffmpeg": bool(FFMPEG), "opus": list(OPUS_RATES) if FFMPEG else [],
|
||||
"stemNow": os.access(QUEUE_DIR, os.W_OK)}).encode())
|
||||
if path == "/api/index":
|
||||
body, etag = INDEX.get()
|
||||
if self.headers.get("If-None-Match") == etag:
|
||||
return self._send(304, headers={"ETag": etag})
|
||||
return self._send(200, body, headers={"ETag": etag, "Cache-Control": "no-cache"})
|
||||
if path == "/api/peaks":
|
||||
return self._peaks(query)
|
||||
if path == "/api/prepare":
|
||||
return self._prepare(query)
|
||||
if path == "/api/stem":
|
||||
return self._stem_now()
|
||||
if path.startswith("/file/"):
|
||||
return self._file(path[len("/file/"):], query)
|
||||
return self._error(404, "not found")
|
||||
except (BrokenPipeError, ConnectionResetError):
|
||||
pass
|
||||
except subprocess.CalledProcessError as e:
|
||||
self._error(500, f"ffmpeg failed ({e.returncode})")
|
||||
except Exception as e: # keep serving; report what broke
|
||||
log("error", self.path, repr(e))
|
||||
try:
|
||||
self._error(500, str(e) or e.__class__.__name__)
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
def _entry(self, query):
|
||||
key = (query.get("key") or [""])[0]
|
||||
entry = INDEX.entry(key)
|
||||
if entry is None:
|
||||
self._error(404, "unknown key")
|
||||
return entry
|
||||
|
||||
def _peaks(self, query):
|
||||
if not FFMPEG:
|
||||
return self._error(501, "ffmpeg not installed")
|
||||
entry = self._entry(query)
|
||||
if entry is None:
|
||||
return
|
||||
n = max(16, min(4096, int((query.get("n") or ["512"])[0])))
|
||||
self._send(200, peaks(entry, n), headers={"Cache-Control": "max-age=86400"})
|
||||
|
||||
def _prepare(self, query):
|
||||
entry = self._entry(query)
|
||||
if entry is None:
|
||||
return
|
||||
fmt, br = _format(query)
|
||||
if fmt == "opus":
|
||||
for s in entry["stems"]:
|
||||
src = _resolve(entry["key"], s["file"])
|
||||
if src:
|
||||
_pool.submit(transcode, src, br)
|
||||
self._send(202, b'{"ok":true}')
|
||||
|
||||
def _stem_now(self):
|
||||
"""Hands a list of Plex file paths to stemd, which separates them before the rest."""
|
||||
if self.command != "POST":
|
||||
return self._error(405, "POST a JSON body: {\"paths\": [...]}")
|
||||
n = int(self.headers.get("Content-Length") or 0)
|
||||
if n > MAX_BODY:
|
||||
self.close_connection = True
|
||||
return self._error(413, "too many paths")
|
||||
try:
|
||||
paths = json.loads(self.rfile.read(n) or b"{}").get("paths")
|
||||
except (ValueError, AttributeError):
|
||||
paths = None
|
||||
if not isinstance(paths, list) or not all(isinstance(p, str) for p in paths):
|
||||
return self._error(400, "expected {\"paths\": [...]}")
|
||||
if not os.access(QUEUE_DIR, os.W_OK):
|
||||
return self._error(503, f"stemd's queue folder isn't mounted at {QUEUE_DIR}")
|
||||
if paths:
|
||||
# stemd reads requests in name order; the dot keeps a half-written one out of its glob.
|
||||
name = f"{time.time_ns()}-{os.getpid()}-{threading.get_ident()}.json"
|
||||
tmp = os.path.join(QUEUE_DIR, "." + name)
|
||||
with open(tmp, "w", encoding="utf-8") as f:
|
||||
json.dump({"paths": paths, "at": time.time()}, f)
|
||||
os.replace(tmp, os.path.join(QUEUE_DIR, name))
|
||||
log(f"stem now: {len(paths)} track{'' if len(paths) == 1 else 's'} queued")
|
||||
self._send(202, json.dumps({"queued": len(paths)}).encode())
|
||||
|
||||
def _file(self, rel, query):
|
||||
key, _, fname = rel.rpartition("/")
|
||||
src = _resolve(key, fname)
|
||||
if not src:
|
||||
return self._error(404, "no such stem")
|
||||
fmt, br = _format(query)
|
||||
path, ctype = (transcode(src, br), "audio/ogg") if fmt == "opus" else (src, "audio/flac")
|
||||
self._serve_file(path, ctype)
|
||||
|
||||
def _serve_file(self, path, ctype):
|
||||
st = os.stat(path)
|
||||
size = st.st_size
|
||||
etag = '"%x-%x"' % (st.st_size, st.st_mtime_ns)
|
||||
start, end = 0, size - 1
|
||||
status = 200
|
||||
rng = self.headers.get("Range")
|
||||
if rng and rng.startswith("bytes=") and "," not in rng and self.headers.get("If-Range", etag) == etag:
|
||||
a, _, b = rng[6:].strip().partition("-")
|
||||
try:
|
||||
if a:
|
||||
start = int(a)
|
||||
end = min(int(b), size - 1) if b else size - 1
|
||||
else: # suffix range: last N bytes
|
||||
start = max(0, size - int(b))
|
||||
except ValueError:
|
||||
return self._error(400, "bad range")
|
||||
if start >= size or start > end:
|
||||
self.send_response(416)
|
||||
self.send_header("Content-Range", f"bytes */{size}")
|
||||
self.send_header("Content-Length", "0")
|
||||
self.end_headers()
|
||||
return
|
||||
status = 206
|
||||
length = end - start + 1
|
||||
self.send_response(status)
|
||||
self.send_header("Content-Type", ctype)
|
||||
self.send_header("Content-Length", str(length))
|
||||
self.send_header("Accept-Ranges", "bytes")
|
||||
self.send_header("ETag", etag)
|
||||
self.send_header("Last-Modified", email.utils.formatdate(st.st_mtime, usegmt=True))
|
||||
self.send_header("Cache-Control", "max-age=31536000, immutable")
|
||||
if status == 206:
|
||||
self.send_header("Content-Range", f"bytes {start}-{end}/{size}")
|
||||
self.end_headers()
|
||||
if self.command == "HEAD":
|
||||
return
|
||||
with open(path, "rb") as f:
|
||||
f.seek(start)
|
||||
left = length
|
||||
while left > 0:
|
||||
buf = f.read(min(CHUNK, left))
|
||||
if not buf:
|
||||
break
|
||||
self.wfile.write(buf)
|
||||
left -= len(buf)
|
||||
|
||||
|
||||
def _format(query):
|
||||
fmt = (query.get("fmt") or ["flac"])[0]
|
||||
if fmt != "opus" or not FFMPEG:
|
||||
return "flac", 0
|
||||
try:
|
||||
br = int((query.get("br") or ["160"])[0])
|
||||
except ValueError:
|
||||
br = 160
|
||||
return "opus", min(OPUS_RATES, key=lambda r: abs(r - br))
|
||||
|
||||
|
||||
def main():
|
||||
if not os.path.isdir(STEMS_DIR):
|
||||
sys.exit(f"STEMS_DIR not found: {STEMS_DIR}")
|
||||
os.makedirs(CACHE_DIR, exist_ok=True)
|
||||
INDEX.refresh()
|
||||
log(f"stemserve on :{PORT}, {INDEX.count} tracks in {STEMS_DIR}, ffmpeg={'yes' if FFMPEG else 'no'}, "
|
||||
f"token={'on' if TOKEN else 'off'}")
|
||||
server = ThreadingHTTPServer(("0.0.0.0", PORT), Handler)
|
||||
server.daemon_threads = True
|
||||
# As PID 1 in a container, SIGTERM is ignored unless handled; docker stop would hang.
|
||||
signal.signal(signal.SIGTERM, lambda *_: sys.exit(0))
|
||||
try:
|
||||
server.serve_forever()
|
||||
except KeyboardInterrupt:
|
||||
pass
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
Reference in new issue
Block a user