Rename everything to Stemify, and add a README
- Python package stemroller/ -> stemify/, Android package com.stemroller.mobile -> com.stemify.mobile, app names, QML module, icons, desktop entries, MPRIS and Plex product names - Done-markers are now .stemify.json; stemd renames the old ones once at startup so songs separated before the rename aren't separated again - A top-level README covering the server, the Android and Android Auto app, the EQ, Stem Now and the desktop apps Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
ClericalErrorandClaude Opus 5.5 committed
1 parent
1845f03c41
commit
45bc0eefe0
159 files changed
+1136
-542
No files matched your search
+12
-7
@@ -1,6 +1,6 @@
|
||||
# stemserve
|
||||
|
||||
Serves the StemRoller stems folder to the Android app: an index of every separated
|
||||
Serves the Stemify stems folder to the Android app: an index of every separated
|
||||
song, the stem files with HTTP range support (so the phone can seek and stream), per-stem
|
||||
waveforms, and optional Opus transcodes so cached songs take ~1/8 the space.
|
||||
|
||||
@@ -15,7 +15,7 @@ from the desktop). From an unraid terminal:
|
||||
```sh
|
||||
mkdir -p /mnt/user/appdata/stemserve/cache /mnt/user/appdata/stemd/state/queue
|
||||
chown -R 99:100 /mnt/user/appdata/stemserve /mnt/user/appdata/stemd
|
||||
cp -r /path/to/stemRoller2/server /mnt/user/appdata/stemserve/src # or copy over SMB
|
||||
cp -r /path/to/stemify/server /mnt/user/appdata/stemserve/src # or copy over SMB
|
||||
docker build -t stemserve /mnt/user/appdata/stemserve/src
|
||||
docker run -d --name stemserve --restart unless-stopped -p 8765:8765 \
|
||||
-v /mnt/user/Music/Stems:/stems:ro \
|
||||
@@ -36,7 +36,7 @@ curl http://192.168.86.2:8765/api/info
|
||||
|
||||
| Variable | Default | Meaning |
|
||||
|-------------|----------|------------------------------------------------------------|
|
||||
| `STEMS_DIR` | `/stems` | The StemRoller library output folder |
|
||||
| `STEMS_DIR` | `/stems` | The Stemify library output folder |
|
||||
| `CACHE_DIR` | `/cache` | Opus transcodes and waveforms (safe to delete) |
|
||||
| `PORT` | `8765` | Listen port |
|
||||
| `TOKEN` | empty | If set, requests need `X-Stem-Token` or `?token=` |
|
||||
@@ -66,19 +66,19 @@ STEMS_DIR=/var/mnt/unraid/Music/Stems CACHE_DIR=~/.cache/stemserve python3 stems
|
||||
# stemd: separate new music on the server's GPU
|
||||
|
||||
`stemd.py` keeps the stems library up to date. It walks the music folder, separates every
|
||||
track whose stems folder has no `.stemroller.json` yet (BS-RoFormer SW 6-stem by default,
|
||||
track whose stems folder has no `.stemify.json` yet (BS-RoFormer SW 6-stem by default,
|
||||
like the desktop app), and writes them in the layout above, so stemserve and the phone pick
|
||||
them up within `INDEX_TTL`. Once it has caught up it checks for new music every 30 minutes.
|
||||
Nothing runs on the desktop.
|
||||
|
||||
It writes the same layout the desktop app's old library mode did (`stemroller/library_core.py`),
|
||||
It writes the same layout the desktop app's old library mode did (`stemify/library_core.py`),
|
||||
so the ~300 tracks already done on the desktop are skipped, and a track interrupted
|
||||
mid-way (`docker stop`, a reboot) is redone from scratch next time.
|
||||
|
||||
## Run it on unraid (NVIDIA)
|
||||
|
||||
Needs the **Nvidia-Driver** plugin (Apps tab). The image builds from the repo root
|
||||
because it uses the `stemroller` package. Copy the repo, or at least `stemroller/`,
|
||||
because it uses the `stemify` package. Copy the repo, or at least `stemify/`,
|
||||
`server/` and `.dockerignore`, to `/mnt/user/appdata/stemd/src`, then from an unraid terminal:
|
||||
|
||||
```sh
|
||||
@@ -115,9 +115,14 @@ skipped here too.
|
||||
To update after changing the code: copy the files over again, rebuild, then
|
||||
`docker rm -f stemd` and repeat the `docker run`.
|
||||
|
||||
Coming from before the rename to Stemify: update stemd first. On its first start it renames
|
||||
the old done-markers in the stems folder to `.stemify.json` (it logs how many), so nothing is
|
||||
separated twice. The new stemserve only looks for the new name, so until stemd has done that
|
||||
the app would briefly see no stems.
|
||||
|
||||
| Variable | Default | Meaning |
|
||||
|--------------|------------------|------------------------------------------------------|
|
||||
| `MODEL` | `roformer_bs_sw` | any id from `stemroller/catalog.py` |
|
||||
| `MODEL` | `roformer_bs_sw` | any id from `stemify/catalog.py` |
|
||||
| `DEVICE` | `cuda` | `cuda`, `cuda:N` or `cpu` |
|
||||
| `RESCAN_MIN` | `30` | minutes between scans once caught up |
|
||||
| `FP16` | `1` | half precision on the GPU (`0` for fp32) |
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
# stemd: separates new music into stems on the server's NVIDIA GPU (see stemd.py).
|
||||
# Build from the repo root, since it needs the stemroller package:
|
||||
# Build from the repo root, since it needs the stemify package:
|
||||
# docker build -f server/stemd.Dockerfile -t stemd .
|
||||
FROM python:3.12-slim
|
||||
RUN apt-get update && apt-get install -y --no-install-recommends ffmpeg && rm -rf /var/lib/apt/lists/*
|
||||
@@ -12,9 +12,9 @@ COPY server/stemd-requirements.txt /tmp/
|
||||
RUN apt-get update && apt-get install -y --no-install-recommends gcc libc6-dev \
|
||||
&& pip install --no-cache-dir --no-deps -r /tmp/stemd-requirements.txt && pip check \
|
||||
&& apt-get purge -y gcc libc6-dev && apt-get autoremove -y && rm -rf /var/lib/apt/lists/*
|
||||
COPY stemroller/__init__.py stemroller/catalog.py stemroller/library_core.py /app/stemroller/
|
||||
COPY stemroller/worker/__init__.py stemroller/worker/__main__.py stemroller/worker/audio_io.py \
|
||||
stemroller/worker/backends.py /app/stemroller/worker/
|
||||
COPY stemify/__init__.py stemify/catalog.py stemify/library_core.py /app/stemify/
|
||||
COPY stemify/worker/__init__.py stemify/worker/__main__.py stemify/worker/audio_io.py \
|
||||
stemify/worker/backends.py /app/stemify/worker/
|
||||
COPY server/stemd.py /app/server/
|
||||
ENV MUSIC_DIR=/music STEMS_DIR=/stems STATE_DIR=/state MODELS_DIR=/models \
|
||||
TORCH_HOME=/models/torch HF_HOME=/models/hf NUMBA_CACHE_DIR=/tmp/numba HOME=/tmp \
|
||||
|
||||
+33
-6
@@ -3,7 +3,7 @@
|
||||
|
||||
Runs on the server next to stemserve. Walks MUSIC_DIR, and for each track whose folder
|
||||
under STEMS_DIR has no marker it runs the separation on the GPU and writes the stems with
|
||||
the same layout and marker as the desktop app (stemroller/library_core.py), so stemserve
|
||||
the same layout and marker as the desktop app (stemify/library_core.py), so stemserve
|
||||
and the phone pick them up as they land. Once caught up it looks again every RESCAN_MIN.
|
||||
|
||||
"Stem Now" in the app goes through stemserve, which drops a request (a JSON list of Plex
|
||||
@@ -17,7 +17,7 @@ Settings (environment):
|
||||
STATE_DIR /state failed-track list
|
||||
QUEUE_DIR STATE_DIR/queue "Stem Now" requests from stemserve
|
||||
MODELS_DIR /models model weights, downloaded on first use
|
||||
MODEL roformer_bs_sw any id from stemroller/catalog.py
|
||||
MODEL roformer_bs_sw any id from stemify/catalog.py
|
||||
DEVICE cuda cuda, cuda:N or cpu
|
||||
RESCAN_MIN 30 minutes between scans once caught up
|
||||
FP16 1 half precision on the GPU
|
||||
@@ -36,8 +36,8 @@ import time
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||
|
||||
from stemroller.catalog import MODELS_BY_ID # noqa: E402
|
||||
from stemroller.library_core import (MAX_SECONDS, SOURCE_RANK, duration, is_done, is_share_error, # noqa: E402
|
||||
from stemify.catalog import MODELS_BY_ID # noqa: E402
|
||||
from stemify.library_core import (MARKER, MAX_SECONDS, SOURCE_RANK, duration, is_done, is_share_error, # noqa: E402
|
||||
iter_tracks, signature, write_track)
|
||||
|
||||
MUSIC_DIR = os.environ.get("MUSIC_DIR", "/music")
|
||||
@@ -101,7 +101,7 @@ class _NoProgress:
|
||||
class Separator:
|
||||
def __init__(self):
|
||||
import torch
|
||||
from stemroller.worker.backends import DemucsBackend, RoformerBackend
|
||||
from stemify.worker.backends import DemucsBackend, RoformerBackend
|
||||
|
||||
self.torch = torch
|
||||
self.model = MODELS_BY_ID[MODEL]
|
||||
@@ -116,7 +116,7 @@ class Separator:
|
||||
def run(self, path, tmp):
|
||||
# Imported late: worker/__main__ sets ROCm env defaults, harmless here, and owns Job's
|
||||
# parameter rules (overlap, fp16, ...), which must match the desktop's.
|
||||
from stemroller.worker.__main__ import Job
|
||||
from stemify.worker.__main__ import Job
|
||||
job = Job({"id": "stemd", "file": path, "model": MODEL, "out_dir": tmp, "segment": SEGMENT,
|
||||
"shifts": SHIFTS, "overlap": OVERLAP, "fp16": FP16}, _NoProgress())
|
||||
if "file" in self.model and not os.path.exists(os.path.join(MODELS_DIR, self.model["file"])):
|
||||
@@ -183,6 +183,32 @@ def separate(sep, failed, path, dest, sig, label):
|
||||
return True
|
||||
|
||||
|
||||
# The project was renamed to Stemify; songs separated before that carry the old marker name.
|
||||
# migrate_markers() renames them once so they aren't separated again. Once stemd has run it
|
||||
# against your library, this constant, the function and its call in main() can be deleted.
|
||||
LEGACY_MARKER = ".stemroller.json"
|
||||
|
||||
|
||||
def migrate_markers():
|
||||
"""Rename every legacy done-marker under STEMS_DIR to MARKER (a few seconds per 10k songs)."""
|
||||
n = 0
|
||||
for root, dirs, files in os.walk(STEMS_DIR):
|
||||
dirs[:] = [d for d in dirs if not d.startswith(".")]
|
||||
if LEGACY_MARKER not in files:
|
||||
continue
|
||||
old, new = os.path.join(root, LEGACY_MARKER), os.path.join(root, MARKER)
|
||||
try:
|
||||
if MARKER in files:
|
||||
os.remove(old)
|
||||
else:
|
||||
os.replace(old, new)
|
||||
n += 1
|
||||
except OSError as e:
|
||||
log(f"can't rename {old}: {e}")
|
||||
if n:
|
||||
log(f"renamed {n:,} {LEGACY_MARKER} markers to {MARKER}")
|
||||
|
||||
|
||||
def resolve(plex_path):
|
||||
"""The music file a Plex path names, or None. Plex sees the library under its own mount
|
||||
point, so this takes the longest tail of the path (folder and file at least) that exists
|
||||
@@ -320,6 +346,7 @@ def main():
|
||||
except OSError as e:
|
||||
log(f"WARNING: can't create {QUEUE_DIR} ({e}); Stem Now won't work")
|
||||
log(f"stemd: {MUSIC_DIR} -> {STEMS_DIR}")
|
||||
migrate_markers()
|
||||
sep = Separator()
|
||||
failed = Failed()
|
||||
while not stop.is_set():
|
||||
|
||||
+4
-4
@@ -1,11 +1,11 @@
|
||||
#!/usr/bin/env python3
|
||||
"""stemserve: serves StemRoller library stems to the Android app over HTTP.
|
||||
"""stemserve: serves Stemify library stems to the Android app over HTTP.
|
||||
|
||||
Stdlib only; ffmpeg is optional and enables Opus transcodes and waveforms.
|
||||
|
||||
The stems folder is what stemroller.library writes:
|
||||
The stems folder is what stemify.library writes:
|
||||
|
||||
<STEMS_DIR>/Artist/Album/01 Song/.stemroller.json
|
||||
<STEMS_DIR>/Artist/Album/01 Song/.stemify.json
|
||||
<STEMS_DIR>/Artist/Album/01 Song/01 Song - vocals.flac, ...
|
||||
|
||||
Endpoints (all GET unless noted; add ?token=... or X-Stem-Token when TOKEN is set):
|
||||
@@ -44,7 +44,7 @@ TOKEN = os.environ.get("TOKEN", "")
|
||||
INDEX_TTL = float(os.environ.get("INDEX_TTL", "30"))
|
||||
QUEUE_DIR = os.environ.get("QUEUE_DIR", "/queue") # stemd's request folder, shared with it
|
||||
MAX_BODY = 8 << 20
|
||||
MARKER = ".stemroller.json"
|
||||
MARKER = ".stemify.json"
|
||||
VERSION = 1
|
||||
FFMPEG = shutil.which("ffmpeg")
|
||||
OPUS_RATES = (64, 96, 128, 160, 192, 256)
|
||||
|
||||
Reference in new issue
Block a user