stemify/README.md

### ⚠️ THIS PROJECT WAS ENTIRELY VIBECODED AND IS SLOP. DO NOT PROCEED IF YOU DO NOT ALIGN WITH THESE VALUES. DO NOT COMPLAIN ABOUT IT IN PULL RELEASES, ISSUES, ETC. UNLESS THERE ARE SEVERE SECURITY RISKS. THEY WILL BE REMOVED. PROCEED AT YOUR OWN RISK! ⚠️

# Stemify

Stemify splits every song in your Plex music library into stems (vocals, drums, bass, guitar, piano and other) and gives you players that mix them live. Mute the vocals and sing along, solo the bass to learn a part, or save an instrumental of a song and have it play that way every time.

It runs on iPhone (with CarPlay), Android (with Android Auto) and Linux desktop. A GPU on your server does the separating in the background.

- [How it works](#how-it-works)
- [Apps](#apps)
- [Setup](#setup)
- [iOS](#ios)
- [Android](#android)
- [Desktop](#desktop)
- [Stem Now](#stem-now)
- [Reference](#reference)
- [Building and testing](#building-and-testing)
- [Troubleshooting](#troubleshooting)
- [Credits and license](#credits)

## How it works

The stems folder mirrors the music folder. If a song lives at `Kansas/Leftoverture/01 Carry On.flac`, its stems live in `Stems/Kansas/Leftoverture/01 Carry On/`.

```
                         your server
  Music/ ──▶ stemd (GPU) ──▶ Stems/ ──▶ stemserve :8765 ──▶ phone, car, desktop
                ▲                           │
                └──── Stem Now requests ◀───┘
  Plex Media Server :32400 ───────────────────────────────▶ library, artwork, playlists
```

- **stemd** works through the library on the GPU, one song at a time, and rescans every 30 minutes for new music.
- **stemserve** serves the finished stems over HTTP with range requests, waveforms and optional Opus transcodes.
- **Plex** handles the library, metadata, artwork and playlists.
- **The apps** play stems when a song has them and fall back to the original file when it doesn't, so everything plays.

## Apps

| | iOS | Android | Desktop player |
|---|---|---|---|
| Per-stem faders, mute, solo, mix presets | yes | yes | yes |
| Waveform seek bar, live stem meters | yes | yes | |
| Saved mix per song | yes | | |
| Karaoke mode with synced lyrics | yes | | |
| Stem playlists (a mix per song) | yes | | |
| Plex playlists | yes | yes | yes |
| Downloads for offline | yes | yes | |
| 10-band and parametric EQ, AutoEQ import | yes | yes | |
| Car | CarPlay | Android Auto | |
| Voice | Siri | Google Assistant | |
| Stem Now | yes | yes | |

Mix presets are Full, Karaoke, A Cappella, No Drums, Rhythm, No Guitar and No Bass.

## Setup

Set up the server first, then the apps. The examples use Unraid paths and `192.168.86.2` for the server; use your own.

### Server

You need Docker, and an NVIDIA GPU for stemd (on Unraid, install the Nvidia-Driver plugin). A CPU build exists for testing, but it's far too slow for a whole library.

Make the folders, owned by `99:100` (Unraid's nobody:users):

```sh
mkdir -p /mnt/user/appdata/stemserve/cache \
         /mnt/user/appdata/stemd/models \
         /mnt/user/appdata/stemd/state/queue
chown -R 99:100 /mnt/user/appdata/stemserve /mnt/user/appdata/stemd
```

**stemd** builds from the repo root because it uses the `stemify` package. Copy the repo (at least `stemify/`, `server/` and `.dockerignore`) to `/mnt/user/appdata/stemd/src`, then:

```sh
docker build -f /mnt/user/appdata/stemd/src/server/stemd.Dockerfile -t stemd /mnt/user/appdata/stemd/src
docker run -d --name stemd --restart unless-stopped --stop-timeout 60 \
  --runtime=nvidia -e NVIDIA_VISIBLE_DEVICES=all -e NVIDIA_DRIVER_CAPABILITIES=compute,utility \
  -v "/mnt/user/Music/Actual music/Flacs":/music:ro \
  -v /mnt/user/Music/Stems:/stems \
  -v /mnt/user/appdata/stemd/models:/models \
  -v /mnt/user/appdata/stemd/state:/state \
  stemd
```

The first song downloads the model (about 700 MB). After that the log shows what's left, what it's on, and how fast it's going.

**stemserve** is one Python file using only the standard library. Copy `server/` to `/mnt/user/appdata/stemserve` and run `docker compose up -d --build` there, or by hand:

```sh
docker build -t stemserve /mnt/user/appdata/stemserve
docker run -d --name stemserve --restart unless-stopped -p 8765:8765 \
  -v /mnt/user/Music/Stems:/stems:ro \
  -v /mnt/user/appdata/stemserve/cache:/cache \
  -v /mnt/user/appdata/stemd/state/queue:/queue \
  stemserve
```

`curl http://192.168.86.2:8765/api/info` should return JSON with a track count.

To update, copy the new files, rebuild, `docker rm -f` the container and run it again. Everything that matters lives in the mounted folders. Update stemd before stemserve.

<details>
<summary>Server environment variables</summary>

**stemd**

| Variable | Default | What it does |
|---|---|---|
| `MUSIC_DIR` | `/music` | Music library (read-only is fine) |
| `STEMS_DIR` | `/stems` | Where stems go |
| `STATE_DIR` | `/state` | Holds the failed-track list |
| `QUEUE_DIR` | `STATE_DIR/queue` | Where Stem Now requests arrive |
| `MODELS_DIR` | `/models` | Model weights, downloaded on first use |
| `MODEL` | `roformer_bs_sw` | Any id from [the model table](#separation-models) |
| `DEVICE` | `cuda` | `cuda`, `cuda:N` or `cpu` |
| `RESCAN_MIN` | `30` | Minutes between library scans once caught up |
| `FP16` | `1` | Half precision on the GPU (`0` for fp32) |
| `OVERLAP`, `SEGMENT`, `SHIFTS` | `0.25`, `0`, `0` | Separation quality, same as the desktop app |

`docker build --build-arg TORCH_INDEX=cpu ...` makes a small CPU-only image.

**stemserve**

| Variable | Default | What it does |
|---|---|---|
| `STEMS_DIR` | `/stems` | The stems folder |
| `CACHE_DIR` | `/cache` | Opus transcodes and waveforms (safe to delete) |
| `QUEUE_DIR` | `/queue` | stemd's Stem Now folder, shared with it |
| `PORT` | `8765` | Listen port |
| `TOKEN` | empty | If set, requests need `X-Stem-Token` or `?token=` |
| `INDEX_TTL` | `30` | Seconds between rescans of the stems folder |

</details>

### Connecting an app

Every app asks for the same two things:

1. **Plex:** sign in with the 4-character code at [plex.tv/link](https://plex.tv/link) and pick your music library.
2. **Stem server:** in Settings, enter your stemserve address. It should say Online with a song count.

The apps check stemserve for newly separated songs every minute, so the library fills in as stemd works.

## iOS

A SwiftUI app for iOS 17 or newer, with CarPlay and Siri ("play Paranoid in Stemify").

### Install

Grab `stemify-<version>.ipa` from [Releases](https://git.iamchrisama.com/cdnutter/stemify/releases) and sideload it with AltStore, SideStore, Sideloadly or Feather. To get updates in AltStore, SideStore or Feather, add this source:

```
https://git.iamchrisama.com/cdnutter/stemify/raw/branch/main/ios/source.json
```

CarPlay and Siri need the `carplay-audio` and `siri` entitlements, which a free Apple ID can't sign. On a free sideload the app works on the phone but not in CarPlay or Siri. A paid developer account covers Siri; CarPlay audio also needs Apple to approve the entitlement for your team.

### Using it

- **Mixer:** on Now Playing, under the waveform. Tap **Save** to keep a mix for that song; it comes back every time the song plays.
- **Karaoke:** the mic button mutes vocals on every song and shows synced lyrics from [LRCLIB](https://lrclib.net). Tap a line to jump to it. Vocals can be On, Soft or Off, and Soft has a slider for how quiet.
- **Playlists:** the Playlists tab switches between Plex and Stem.
  - Plex playlists sync with Plex and Plexamp.
  - Stem playlists stay on the phone, and each song in them has its own mix, so one playlist can hold an instrumental of one song and an a cappella of the next.
  - Copy any Plex playlist to a stem playlist from its menu.
- **Add to Playlist:** hold any song, album, artist or playlist. One sheet covers both kinds: pick Plex or Stem, pick the mix for stem playlists, then tap a playlist or make a new one right there.
- **Sorting:** artists, albums, songs and playlists sort like Plexamp. Songs has a shuffle-everything button.
- **Hold anything** for Add to Front or End of Queue, Download, Add to Playlist and Stem Now.
- **Equalizer:** same graphic EQ, parametric filters and APO/AutoEQ import as [Android](#equalizer).

## Android

Kotlin and Jetpack Compose, built and tested on a OnePlus 15. Grab `Stemify.apk` from Releases (or [build it](#building-and-testing)) and `adb install -r Stemify.apk`.

### Using it

- **Tabs:** Library, Playlists, Search, Downloads and Settings.
- **Badges:** songs without stems yet say NO STEMS, and albums and artists show NO STEMS or a count like 7/10 STEMS. They still play from the original file.
- **Now Playing:** waveform seek bar (drag sideways to seek, up and down for volume), mixer, meters and queue. Your mix can carry over between songs or reset.
- **Stem Quality:** lossless FLAC, or Opus at a bitrate you choose (about an eighth of the size, transcoded and cached on the server).
- **Streaming** needs your home network, since Plex and stemserve are plain HTTP on the LAN. Downloads and already-streamed songs play anywhere.

### Equalizer

Settings → Equalizer has two parts.

- **Graphic EQ:** ten bands from 31 Hz to 16 kHz, with presets like Bass Booster, Late Night and Vocal Booster.
- **Parametric filters**, modeled on [Equalizer APO](https://sourceforge.net/projects/equalizerapo/): Peak, Low/High Shelf, Low/High Pass (6 to 48 dB/oct), Band Pass, Notch and All Pass. Each has frequency, gain and Q, works on both channels or one, and can be toggled on its own. Drag the handle on the curve or tap a value to type it.

To correct headphones, copy a `ParametricEQ.txt` from [AutoEQ](https://github.com/jaakkopasanen/AutoEq) and choose **••• → Paste APO Config** on the Filters page. **Copy as APO Config** goes the other way.

Profiles are tied to outputs (speaker, wired, each Bluetooth device, USB DACs) and switch automatically. Each has its own preamp, and anti-clipping adds just enough cut to keep the loudest point of the curve at 0 dBFS. A limiter on the master bus catches the rest.

### Android Auto

Built for a 2016 Mazda MX-5 with Mazda Connect: 800×480, no touch, knob and wheel buttons only.

- Tabs for Playlists, Albums, Artists and Downloads (Downloads first when you're away from home).
- Every song list starts with Shuffle all, and lists over 40 items split into A to Z folders.
- Now Playing has a Vocals on/off button and a Mix button that cycles presets. The mix shows on the artist line, like "Black Sabbath · Karaoke".
- Voice: "play Paranoid on Stemify" works for playlists, artists, albums and songs.
- Away from home it notices the server is gone and only lists songs that will play.

Android Auto hides sideloaded apps until you allow them: in the phone's Android Auto settings, tap Version ten times, then ⋮ → Developer settings → Unknown sources.

To test without a car, start the head unit server from Auto's developer settings, then:

```sh
adb forward tcp:5277 tcp:5277
~/Android/Sdk/extras/google/auto/desktop-head-unit -c ~/Android/Sdk/extras/google/auto/config/rotary.ini
```

## Desktop

Linux, Python 3.12 and [uv](https://docs.astral.sh/uv/). PyTorch defaults to ROCm (AMD); point the torch index in `pyproject.toml` elsewhere for NVIDIA or CPU.

```sh
uv sync
uv run stemify                # the drag-and-drop separator
uv run stemify-player         # the player
./scripts/install-desktop.sh  # add both to your app launcher (--uninstall removes them)
```

**Stemify** is the separator. Drop files on the window and each one is separated with your last-used model and device. You get a lane per stem with a waveform, solo, mute, gain and export (everything, one stem, or the current mix, as WAV, FLAC or MP3). Separation runs in its own process, results are cached, and it works on AMD, NVIDIA and CPU.

**Stemify Player** is your Plex library and its stems, dressed up as Microsoft Excel 2016. Sheets are your library, the zoom slider is master volume, Cell Styles are mix presets, and Undo/Redo skip songs. It reports plays to Plex, supports media keys and MPRIS, and remembers your queue and mix. More in [`stemify/player/README.md`](stemify/player/README.md).

## Stem Now

A big library takes stemd days. Hold a song, album, artist or playlist and choose **Stem Now** to move its unseparated songs to the front.

1. The app sends the songs to stemserve, which drops a request file in stemd's queue folder.
2. stemd finishes the song it's on and does yours next. If it was idle, it starts within about 5 seconds.
3. About a minute after each song finishes, the app picks up its stems.

Requests run oldest first and survive restarts. stemd matches Plex's paths to its own files by the artist/album/file tail, so different mount points don't matter. Songs that failed before are skipped; delete `state/failed.json` to retry them.

## Reference

### Separation models

| Model id | Stems | Notes |
|---|---|---|
| `htdemucs` | 4 | HT Demucs, fast |
| `htdemucs_ft` | 4 | Fine-tuned, about 4x slower |
| `htdemucs_6s` | 6 | Adds guitar and piano |
| `roformer_bs_vocals` | 2 | BS-RoFormer vocals/instrumental (Viperx 1297) |
| `roformer_mel_kim_ft` | 2 | Mel-RoFormer, cleanest vocals (Kim FT by unwa) |
| `roformer_bs_sw` | 6 | BS-RoFormer SW by jarredou, stemd's default |

### Folder layout

```
Music/Kansas/Leftoverture/01 Carry On Wayward Son.flac
Stems/Kansas/Leftoverture/01 Carry On Wayward Son/
    .stemify.json                              done marker
    01 Carry On Wayward Son - vocals.flac
    01 Carry On Wayward Son - drums.flac
    ...
```

- Stems are written as 24-bit FLAC to a hidden `.partial` folder and the marker goes in last, so an interrupted separation is just redone.
- If a folder has the same song in several formats, the best one wins (FLAC/WAV, then M4A, Opus/Ogg, MP3, WMA).
- Plex reports paths as its container sees them, so the apps match songs to stems by path suffix, check durations, and learn the prefix from the first confident match.

### stemserve API

| Request | Returns |
|---|---|
| `GET /api/info` | Track count, ffmpeg availability, whether Stem Now is set up |
| `GET /api/index` | Every finished song (with ETag) |
| `GET /api/peaks?key=K&n=256` | Per-stem loudness envelopes for waveforms, 0 to 1000 |
| `POST /api/prepare?key=K&fmt=opus&br=256` | Starts Opus transcodes in the background |
| `POST /api/stem` with `{"paths": [...]}` | Stem Now for these Plex files |
| `GET /file/K/<file>.flac[?fmt=opus&br=256]` | The stem, with range support |

`K` is the song's folder under the stems root, like `Kansas/Leftoverture/01 Carry On Wayward Son`. Outside Docker:

```sh
STEMS_DIR=/path/to/Stems CACHE_DIR=~/.cache/stemserve python3 server/stemserve.py
```

### Repo layout

```
ios/                  iOS and CarPlay app (SwiftUI, xcodegen project)
  Stemify/            App, Plex, Playback, Stems, Engine, EQ, CarPlay, UI
  scripts/            build-ipa.sh, make-source.py
android/              Android and Android Auto app (Kotlin, Compose, Media3)
server/               stemd.py, stemserve.py, Dockerfiles, compose file
stemify/              desktop separator (PySide6 + QML) and shared separation code
  worker/             Demucs and RoFormer backends, run in a separate process
  player/             Stemify Player
  library_core.py     folder layout rules shared with stemd
scripts/              install-desktop.sh
specs.txt             original design doc for the desktop separator
```

## Building and testing

**iOS** needs a Mac with Xcode and xcodegen (`brew install xcodegen`):

```sh
cd ios
scripts/build-ipa.sh        # unsigned build/stemify-<version>.ipa
xcodegen generate           # or open Stemify.xcodeproj in Xcode
```

The version lives in `ios/project.yml` (`MARKETING_VERSION` and `CURRENT_PROJECT_VERSION`). To release, tag `ios-v<version>`, attach `stemify-<version>.ipa`, then run `GITEA_TOKEN=... ios/scripts/make-source.py` and commit the new `ios/source.json`.

**Android** needs JDK 21 and the Android SDK (`android/local.properties` points at it):

```sh
cd android
JAVA_HOME=/path/to/jdk-21 ./gradlew assembleRelease     # app/build/outputs/apk/release/
JAVA_HOME=/path/to/jdk-21 ./gradlew testDebugUnitTest   # matching, EQ maths, limiter, cache, car tree
```

The cache and index tests use a stemserve on `localhost:8765` and skip themselves without one.

**Desktop** can be checked without a window:

```sh
QT_QPA_PLATFORM=offscreen uv run stemify-player --no-mpris --screenshot /tmp/p.png@5 --quit-after 6
```

## Troubleshooting

**A song says NO STEMS.** Hold it and choose Stem Now.

**Stem Now says "stemserve needs updating" or "queue folder isn't mounted".** stemserve is older than the app or is missing the `/queue` mount. The queue folder has to be owned by `99:100` before the container first starts, or Docker creates it as root.

**stemd is separating on the CPU.** The container can't see the GPU. Run it with `--runtime=nvidia` and the NVIDIA variables above.

**stemd says "out of GPU memory".** Something else is using VRAM. stemd retries that song next pass without marking it failed.

**The stem server shows Offline.** Check the address in Settings, try `curl http://<server>:8765/api/info` from another machine, and if stemserve has a `TOKEN`, set the same one in the app.

**Stemify isn't in CarPlay or Siri.** The IPA was signed without those entitlements. See [iOS install](#install).

**Stemify isn't in Android Auto.** Turn on Unknown sources in Auto's developer settings. See [Android Auto](#android-auto).

**Songs listed in `state/failed.json`.** Usually corrupt files or "songs" over 40 minutes (DJ mixes, radio rips). They retry when the file changes; delete the file to retry everything.

## Credits

- [Demucs](https://github.com/adefossez/demucs) by Alexandre Défossez
- BS-RoFormer and Mel-RoFormer models (SW 6-stem by jarredou, Viperx 1297, Kim FT by unwa) via [python-audio-separator](https://github.com/nomadkaraoke/python-audio-separator)
- [PyTorch](https://pytorch.org/), [PySide6](https://www.qt.io/qt-for-python), [Jetpack Compose](https://developer.android.com/compose), [Media3](https://developer.android.com/media/media3)
- [Equalizer APO](https://sourceforge.net/projects/equalizerapo/) and [AutoEQ](https://github.com/jaakkopasanen/AutoEq), whose formats the EQ reads and writes
- [LRCLIB](https://lrclib.net) for synced lyrics
- Inter, Selawik and Carlito fonts (licenses included alongside them)

Stemify isn't affiliated with Plex, Apple, Google, Microsoft or Mazda.

## License

[GPL-3.0](LICENSE)