diff --git a/README.md b/README.md index 1c2ed19..f5521b4 100644 --- a/README.md +++ b/README.md @@ -2,117 +2,66 @@ # Stemify -**Your whole music library, taken apart and put back together however you like.** +Stemify splits every song in your Plex music library into stems (vocals, drums, bass, guitar, piano and other) and gives you players that mix them live. Mute the vocals and sing along, solo the bass to learn a part, or save an instrumental of a song and have it play that way every time. -Ever wanted to hear "Carry On Wayward Son" without the vocals so you can belt it out yourself? Or -mute the drums on a song to figure out what the bass player is actually doing back there? Stemify -splits every song in your library into its separate parts (vocals, drums, bass, guitar, piano and -"other"), keeps them organised next to your music, and gives you players that let you mix them -live, on your phone, in your car, and on your desktop. +It runs on iPhone (with CarPlay), Android (with Android Auto) and Linux desktop. A GPU on your server does the separating in the background. -It's a home-grown, slightly over-engineered love letter to karaoke, music nerdery and the joy of -pressing the "mute vocals" button at exactly the wrong moment. - ---- - -## Contents - -- [The big picture](#the-big-picture) -- [What you get](#what-you-get) -- [Setting it up](#setting-it-up) - - [1. The server: stemd and stemserve](#1-the-server-stemd-and-stemserve) - - [2. The Android app](#2-the-android-app) - - [3. The desktop apps (optional)](#3-the-desktop-apps-optional) -- [Using the Android app](#using-the-android-app) -- [Stem Now: skipping the line](#stem-now-skipping-the-line) -- [The Equalizer](#the-equalizer) -- [Android Auto](#android-auto) -- [iOS and CarPlay](#ios-and-carplay) -- [The desktop apps](#the-desktop-apps) -- [Separation models](#separation-models) -- [How the pieces find each other](#how-the-pieces-find-each-other) -- [stemserve API](#stemserve-api) +- [How it works](#how-it-works) +- [Apps](#apps) +- [Setup](#setup) +- [iOS](#ios) +- [Android](#android) +- [Desktop](#desktop) +- [Stem Now](#stem-now) +- [Reference](#reference) - [Building and testing](#building-and-testing) -- [Repo tour](#repo-tour) -- [Troubleshooting and FAQ](#troubleshooting-and-faq) -- [Credits](#credits) -- [License](#license) +- [Troubleshooting](#troubleshooting) +- [Credits and license](#credits) ---- +## How it works -## The big picture - -There are three moving parts, and they all agree on one simple idea: **the stems folder mirrors -the music folder.** If your song lives at `Kansas/Leftoverture/01 Carry On.flac`, its stems live at -`Stems/Kansas/Leftoverture/01 Carry On/`. That's it. That's the whole trick. +The stems folder mirrors the music folder. If a song lives at `Kansas/Leftoverture/01 Carry On.flac`, its stems live in `Stems/Kansas/Leftoverture/01 Carry On/`. ``` - ┌──────────────────────────── your server (unraid, in our case) ───────────────────────────┐ - │ │ - │ Music/ ──read──▶ stemd (GPU) ──writes──▶ Stems/ ──read──▶ stemserve (:8765) ─────┼──▶ phone, car, desktop - │ ▲ ▲ │ │ - │ │ └──────── "Stem Now" requests ◀──────────────┘ │ - │ Plex Media Server (:32400) ──────────────────────────────────────────────────────────────┼──▶ library, artwork, playlists - └───────────────────────────────────────────────────────────────────────────────────────────┘ + your server + Music/ ──▶ stemd (GPU) ──▶ Stems/ ──▶ stemserve :8765 ──▶ phone, car, desktop + ▲ │ + └──── Stem Now requests ◀───┘ + Plex Media Server :32400 ───────────────────────────────▶ library, artwork, playlists ``` -- **stemd** sits on the server and chews through your library on the GPU, one song at a time, - forever. New album? It notices within half an hour and gets to work. -- **stemserve** hands the finished stems to your devices over HTTP, with seeking, waveforms and - optional Opus transcodes so your phone storage doesn't cry. -- **Plex** stays in charge of what it's good at: your library, metadata, artwork and playlists. -- **The apps** (Android, Android Auto, and two desktop apps) play everything, and quietly fall back - to the original file for any song that hasn't been separated yet. Nothing is ever unplayable. +- **stemd** works through the library on the GPU, one song at a time, and rescans every 30 minutes for new music. +- **stemserve** serves the finished stems over HTTP with range requests, waveforms and optional Opus transcodes. +- **Plex** handles the library, metadata, artwork and playlists. +- **The apps** play stems when a song has them and fall back to the original file when it doesn't, so everything plays. ---- +## Apps -## What you get +| | iOS | Android | Desktop player | +|---|---|---|---| +| Per-stem faders, mute, solo, mix presets | yes | yes | yes | +| Waveform seek bar, live stem meters | yes | yes | | +| Saved mix per song | yes | | | +| Karaoke mode with synced lyrics | yes | | | +| Stem playlists (a mix per song) | yes | | | +| Plex playlists | yes | yes | yes | +| Downloads for offline | yes | yes | | +| 10-band and parametric EQ, AutoEQ import | | yes | | +| Car | CarPlay | Android Auto | | +| Voice | Siri | Google Assistant | | +| Stem Now | yes | yes | | -**On your phone (Android)** -- A Plex music player with per-stem faders, mute and solo, and one-tap mixes: Full, Karaoke, - A Cappella, No Drums, Rhythm, No Guitar, No Bass. -- A Plexamp-style waveform seek bar that doubles as a volume control (drag sideways to seek, up and - down for volume). The waveform is scaled to each song, so you can actually see the quiet verse - and the big chorus. -- Live level meters for every stem, updated every frame and in sync with what you hear. -- A 10-band graphic EQ **plus** Equalizer APO-style parametric filters, with per-device profiles - that switch automatically when you plug in headphones or connect to Bluetooth. -- Paste in an AutoEQ correction for your headphones and you're done. -- Downloads for offline listening, stems and all. -- Gapless playback, a proper queue, playlists (create, reorder, rename), search, and "On This - Day" album birthdays on the home screen. -- "Stem Now": jump any song, album, artist or playlist to the front of the separation queue. -- An iOS-flavoured look in Black, Dark or White, at whatever refresh rate your panel can manage. +Mix presets are Full, Karaoke, A Cappella, No Drums, Rhythm, No Guitar and No Bass. -**In your car (Android Auto)** -- A browse tree built for a rotary knob and no touchscreen, with voice search, a Vocals on/off - button and a Mix button right next to play/pause. +## Setup -**On your desktop (Linux)** -- **Stemify (the separator)**: drag a song onto the window, get stems out the other side, preview them in sync, - export them. Material layout meets Windows 7 Aero glass, running at 240 Hz because why not. -- **Stemify Player**: your Plex library and its stems, with full mixing... dressed up as - Microsoft Excel 2016. Yes, really. Your boss will think you're working on the quarterly numbers. +Set up the server first, then the apps. The examples use Unraid paths and `192.168.86.2` for the server; use your own. -**On the server** -- **stemd** separates everything automatically with BS-RoFormer SW (6 stems) on an NVIDIA GPU, - politely, at low priority, sharing the card with Plex transcodes. -- **stemserve** is one Python file using only the standard library. ffmpeg is optional. +### Server ---- +You need Docker, and an NVIDIA GPU for stemd (on Unraid, install the Nvidia-Driver plugin). A CPU build exists for testing, but it's far too slow for a whole library. -## Setting it up - -The recommended order is server first, then phone, then (if you like) desktop. The examples below -use unraid paths and the addresses from the original setup (`192.168.86.2`). Swap in your own. - -### 1. The server: stemd and stemserve - -You need Docker, and for stemd an NVIDIA GPU (on unraid, install the **Nvidia-Driver** plugin from -the Apps tab). A CPU-only build exists for testing, but separating a whole library on a CPU is a -good way to celebrate several birthdays while you wait. - -**Folders first** (owned by `99:100`, unraid's nobody:users): +Make the folders, owned by `99:100` (Unraid's nobody:users): ```sh mkdir -p /mnt/user/appdata/stemserve/cache \ @@ -121,8 +70,7 @@ mkdir -p /mnt/user/appdata/stemserve/cache \ chown -R 99:100 /mnt/user/appdata/stemserve /mnt/user/appdata/stemd ``` -**stemd** builds from the repo root because it borrows the `stemify` package. Copy the repo (or -at least `stemify/`, `server/` and `.dockerignore`) to `/mnt/user/appdata/stemd/src`, then: +**stemd** builds from the repo root because it uses the `stemify` package. Copy the repo (at least `stemify/`, `server/` and `.dockerignore`) to `/mnt/user/appdata/stemd/src`, then: ```sh docker build -f /mnt/user/appdata/stemd/src/server/stemd.Dockerfile -t stemd /mnt/user/appdata/stemd/src @@ -133,21 +81,11 @@ docker run -d --name stemd --restart unless-stopped --stop-timeout 60 \ -v /mnt/user/appdata/stemd/models:/models \ -v /mnt/user/appdata/stemd/state:/state \ stemd -docker logs -f stemd ``` -The first song downloads the model (about 700 MB). After that the log reads like a very calm -sports commentary: how many tracks are left, which one it's on, and how many times faster than -realtime it went. +The first song downloads the model (about 700 MB). After that the log shows what's left, what it's on, and how fast it's going. -**stemserve** is tiny. Copy the `server/` folder to `/mnt/user/appdata/stemserve` and either use -the included compose file: - -```sh -cd /mnt/user/appdata/stemserve && docker compose up -d --build -``` - -or run it by hand: +**stemserve** is one Python file using only the standard library. Copy `server/` to `/mnt/user/appdata/stemserve` and run `docker compose up -d --build` there, or by hand: ```sh docker build -t stemserve /mnt/user/appdata/stemserve @@ -158,433 +96,263 @@ docker run -d --name stemserve --restart unless-stopped -p 8765:8765 \ stemserve ``` -Check it from any machine on your network: +`curl http://192.168.86.2:8765/api/info` should return JSON with a track count. -```sh -curl http://192.168.86.2:8765/api/info -``` - -If you see a happy little JSON blob with a track count, you're in business. +To update, copy the new files, rebuild, `docker rm -f` the container and run it again. Everything that matters lives in the mounted folders. Update stemd before stemserve.
-Server settings (environment variables) +Server environment variables **stemd** | Variable | Default | What it does | |---|---|---| -| `MUSIC_DIR` | `/music` | Your music library (read-only is fine) | +| `MUSIC_DIR` | `/music` | Music library (read-only is fine) | | `STEMS_DIR` | `/stems` | Where stems go | -| `STATE_DIR` | `/state` | The failed-track list lives here | -| `QUEUE_DIR` | `STATE_DIR/queue` | Where "Stem Now" requests arrive | +| `STATE_DIR` | `/state` | Holds the failed-track list | +| `QUEUE_DIR` | `STATE_DIR/queue` | Where Stem Now requests arrive | | `MODELS_DIR` | `/models` | Model weights, downloaded on first use | -| `MODEL` | `roformer_bs_sw` | Any model id from [the table below](#separation-models) | +| `MODEL` | `roformer_bs_sw` | Any id from [the model table](#separation-models) | | `DEVICE` | `cuda` | `cuda`, `cuda:N` or `cpu` | -| `RESCAN_MIN` | `30` | Minutes between library scans once it's caught up | -| `FP16` | `1` | Half precision on the GPU (`0` for full fp32) | -| `OVERLAP`, `SEGMENT`, `SHIFTS` | `0.25`, `0`, `0` | Separation quality knobs, same as the desktop app | +| `RESCAN_MIN` | `30` | Minutes between library scans once caught up | +| `FP16` | `1` | Half precision on the GPU (`0` for fp32) | +| `OVERLAP`, `SEGMENT`, `SHIFTS` | `0.25`, `0`, `0` | Separation quality, same as the desktop app | -`docker build --build-arg TORCH_INDEX=cpu ...` makes a small GPU-less image for testing. +`docker build --build-arg TORCH_INDEX=cpu ...` makes a small CPU-only image. **stemserve** | Variable | Default | What it does | |---|---|---| | `STEMS_DIR` | `/stems` | The stems folder | -| `CACHE_DIR` | `/cache` | Opus transcodes and waveforms (safe to delete any time) | -| `QUEUE_DIR` | `/queue` | stemd's "Stem Now" folder, shared with it | +| `CACHE_DIR` | `/cache` | Opus transcodes and waveforms (safe to delete) | +| `QUEUE_DIR` | `/queue` | stemd's Stem Now folder, shared with it | | `PORT` | `8765` | Listen port | -| `TOKEN` | empty | If set, every request needs `X-Stem-Token` or `?token=` | +| `TOKEN` | empty | If set, requests need `X-Stem-Token` or `?token=` | | `INDEX_TTL` | `30` | Seconds between rescans of the stems folder |
-**Updating later:** copy the new files over, rebuild the image, `docker rm -f` the container, and -run it again. Your stems, models and settings live in the mounted folders, so nothing is lost. -Update stemd before stemserve: if you separated songs before the project was called Stemify, -stemd renames their old done-markers on its first start so nothing gets separated twice. +### Connecting an app -### 2. The Android app +Every app asks for the same two things: -Built for (and lovingly tested on) a OnePlus 15, but any recent Android phone should be happy. +1. **Plex:** sign in with the 4-character code at [plex.tv/link](https://plex.tv/link) and pick your music library. +2. **Stem server:** in Settings, enter your stemserve address. It should say Online with a song count. -1. Grab `Stemify.apk` from the Releases page, or [build it yourself](#building-and-testing), - then sideload it: - ```sh - adb install -r Stemify.apk - ``` -2. Open it and go to **Library → Connect** (or Settings → Plex). Enter the 4-character code at - [plex.tv/link](https://plex.tv/link) and pick your music library. -3. Visit **Settings → Stem Server** and point it at your stemserve. It should say "Online" with a - song count. (The default address is `http://192.168.86.2:8765`, which is almost certainly not - yours.) +The apps check stemserve for newly separated songs every minute, so the library fills in as stemd works. -The app checks stemserve for freshly separated songs every minute, so your library slowly "lights -up" as stemd works through it. +## iOS -### 3. The desktop apps (optional) +A SwiftUI app for iOS 17 or newer, with CarPlay and Siri ("play Paranoid in Stemify"). -Linux, Python 3.12 and [uv](https://docs.astral.sh/uv/). The default setup uses PyTorch for ROCm -(AMD GPUs, developed on an RX 9070 XT). NVIDIA and CPU work too if you point the torch index -elsewhere in `pyproject.toml`. +### Install -```sh -uv sync -uv run stemify # the drag-and-drop separator -uv run stemify-player # the Excel-shaped music player -./scripts/install-desktop.sh # adds both to your app grid ("--uninstall" removes them) +Grab `stemify-.ipa` from [Releases](https://git.iamchrisama.com/cdnutter/stemify/releases) and sideload it with AltStore, SideStore, Sideloadly or Feather. To get updates in AltStore, SideStore or Feather, add this source: + +``` +https://git.iamchrisama.com/cdnutter/stemify/raw/branch/main/ios/source.json ``` ---- +CarPlay and Siri need the `carplay-audio` and `siri` entitlements, which a free Apple ID can't sign. On a free sideload the app works on the phone but not in CarPlay or Siri. A paid developer account covers Siri; CarPlay audio also needs Apple to approve the entitlement for your team. -## Using the Android app +### Using it -**Tabs:** Library, Playlists, Search, Downloads and Settings. +- **Mixer:** on Now Playing, under the waveform. Tap **Save** to keep a mix for that song; it comes back every time the song plays. +- **Karaoke:** the mic button mutes vocals on every song and shows synced lyrics from [LRCLIB](https://lrclib.net). Tap a line to jump to it. Vocals can be On, Soft or Off, and Soft has a slider for how quiet. +- **Playlists:** the Playlists tab switches between Plex and Stem. + - Plex playlists sync with Plex and Plexamp. + - Stem playlists stay on the phone, and each song in them has its own mix, so one playlist can hold an instrumental of one song and an a cappella of the next. + - Copy any Plex playlist to a stem playlist from its menu. +- **Add to Playlist:** hold any song, album, artist or playlist. One sheet covers both kinds: pick Plex or Stem, pick the mix for stem playlists, then tap a playlist or make a new one right there. +- **Sorting:** artists, albums, songs and playlists sort like Plexamp. Songs has a shuffle-everything button. +- **Hold anything** for Add to Front or End of Queue, Download, Add to Playlist and Stem Now. -**Stems are the norm, so the exceptions are what's marked.** Songs without stems yet say -**NO STEMS**, and albums and artists say **NO STEMS** or something like **7/10 STEMS** until -they're done. Those songs still play, just from the original file. +## Android -**Hold anything** (a song, album, artist or playlist) for: +Kotlin and Jetpack Compose, built and tested on a OnePlus 15. Grab `Stemify.apk` from Releases (or [build it](#building-and-testing)) and `adb install -r Stemify.apk`. -- **Add to Front of Queue** / **Add to End of Queue** -- **Download** (or **Remove Download**) -- **Add to Playlist** (playlists can even go into other playlists) -- **Stem Now**, while any of it lacks stems +### Using it -**The Now Playing screen** has the waveform seek bar, the mixer (faders, mute, solo, mix presets, -meters, master volume) and the queue. Your mix can carry over from song to song or reset each -time, your choice. +- **Tabs:** Library, Playlists, Search, Downloads and Settings. +- **Badges:** songs without stems yet say NO STEMS, and albums and artists show NO STEMS or a count like 7/10 STEMS. They still play from the original file. +- **Now Playing:** waveform seek bar (drag sideways to seek, up and down for volume), mixer, meters and queue. Your mix can carry over between songs or reset. +- **Stem Quality:** lossless FLAC, or Opus at a bitrate you choose (about an eighth of the size, transcoded and cached on the server). +- **Streaming** needs your home network, since Plex and stemserve are plain HTTP on the LAN. Downloads and already-streamed songs play anywhere. -**Downloads** are proper offline copies: every stem fully fetched (or the original file for songs -without stems). Sort by download order, artist or release date. +### Equalizer -**Stem Quality** decides how stems travel from the server: lossless FLAC, or Opus at a bitrate of -your choosing, which takes roughly an eighth of the space and is transcoded and cached on the -server. +Settings → Equalizer has two parts. -**Streaming** needs your home network (Plex and stemserve are plain HTTP on the LAN). Anything -you've downloaded or already streamed plays anywhere. +- **Graphic EQ:** ten bands from 31 Hz to 16 kHz, with presets like Bass Booster, Late Night and Vocal Booster. +- **Parametric filters**, modeled on [Equalizer APO](https://sourceforge.net/projects/equalizerapo/): Peak, Low/High Shelf, Low/High Pass (6 to 48 dB/oct), Band Pass, Notch and All Pass. Each has frequency, gain and Q, works on both channels or one, and can be toggled on its own. Drag the handle on the curve or tap a value to type it. ---- +To correct headphones, copy a `ParametricEQ.txt` from [AutoEQ](https://github.com/jaakkopasanen/AutoEq) and choose **••• → Paste APO Config** on the Filters page. **Copy as APO Config** goes the other way. -## Stem Now: skipping the line +Profiles are tied to outputs (speaker, wired, each Bluetooth device, USB DACs) and switch automatically. Each has its own preamp, and anti-clipping adds just enough cut to keep the loudest point of the curve at 0 dBFS. A limiter on the master bus catches the rest. -stemd works through your library on its own, but a big library takes days. If you're desperate to -karaoke your way through an album tonight, hold it and choose **Stem Now**. +### Android Auto -Here's what happens: +Built for a 2016 Mazda MX-5 with Mazda Connect: 800×480, no touch, knob and wheel buttons only. -1. The app sends the songs that still lack stems to stemserve. -2. stemserve drops a little request file into stemd's queue folder. -3. stemd finishes the song it's on, then does yours next, ahead of everything else. If it was - idle, it starts within about 5 seconds. -4. About a minute after each song finishes, the app notices and plays it with stems from then on. +- Tabs for Playlists, Albums, Artists and Downloads (Downloads first when you're away from home). +- Every song list starts with Shuffle all, and lists over 40 items split into A to Z folders. +- Now Playing has a Vocals on/off button and a Mix button that cycles presets. The mix shows on the artist line, like "Black Sabbath · Karaoke". +- Voice: "play Paranoid on Stemify" works for playlists, artists, albums and songs. +- Away from home it notices the server is gone and only lists songs that will play. -Requests are handled oldest first and survive restarts. stemd matches the paths Plex reports to -files in its own music folder by their artist/album/file tail, so it doesn't matter that Plex and -stemd see the library under different mount points. Songs that failed before (corrupt files, or -anything over 40 minutes) are skipped; delete `state/failed.json` to give them all another shot. +Android Auto hides sideloaded apps until you allow them: in the phone's Android Auto settings, tap Version ten times, then ⋮ → Developer settings → Unknown sources. ---- - -## The Equalizer - -Find it in **Settings → Equalizer**. It's two equalizers wearing one trench coat. - -**The graphic EQ** has ten octave bands from 31 Hz to 16 kHz, a response curve, and a pile of -presets borrowed in spirit from iOS (Bass Booster, Late Night, Vocal Booster and friends). - -**Parametric Filters** are modelled on [Equalizer APO](https://sourceforge.net/projects/equalizerapo/): - -| Type | What it's for | -|---|---| -| Peak | Boost or cut around one frequency | -| Low Shelf / High Shelf | Tilt everything below or above a frequency | -| Low Pass / High Pass | Roll off the top or bottom, at 6, 12, 24, 36 or 48 dB per octave | -| Band Pass | Keep only a band | -| Notch | Surgically remove one frequency (hello, 60 Hz hum) | -| All Pass | Shift phase without changing level, for the true connoisseur | - -Every filter has frequency, gain and Q, can apply to **both channels, or just left or right**, -and can be switched off on its own to compare. Drag a filter's handle on its curve (sideways for -frequency, up and down for gain), use the sliders, or tap any value to type it in. - -**Headphone corrections in two taps:** copy a `ParametricEQ.txt` from -[AutoEQ](https://github.com/jaakkopasanen/AutoEq) for your headphones, then on the Filters page -choose **••• → Paste APO Config**. You can also **Copy as APO Config** to take your settings to a -PC. - -**Profiles** link to output devices: the phone speaker, wired headphones, each Bluetooth headset, -USB DACs. Plug something in and its profile switches on automatically, or pick one by hand. - -**Preamp and anti-clipping:** each profile has its own manual preamp, and the anti-clipping switch -adds just enough extra cut that a full-scale signal at the curve's loudest point still peaks at -0 dBFS. A limiter on the master bus catches anything else, so cranking the bass won't make things -crackle. - ---- - -## Android Auto - -Designed for a 2016 Mazda MX-5 with the Mazda Connect retrofit: an 800×480 screen with touch -disabled, driven entirely by the commander knob and the steering wheel buttons. If it works there, -it'll work in your car. - -- **Four tabs:** Playlists, Albums, Artists, Downloads (Downloads comes first when you're away - from home). -- **Shallow and short:** every song list starts with **Shuffle all**, and anything longer than 40 - items splits into A to Z folders, so you never have to twiddle the knob through 400 songs at a - red light. -- **Now playing:** next to the transport controls you get **Vocals** on/off (instant karaoke), - **Mix** (cycles through the presets), shuffle and repeat. The current mix shows on the artist - line, e.g. "Black Sabbath · Karaoke". -- **Voice:** hold the talk button and say "play Paranoid on Stemify". Playlists, artists, - albums and songs all work. -- **Offline:** away from home, the app quickly notices the server is unreachable and lists only - songs that will actually play (downloaded or fully cached). - -**Setting up the car (once):** Android Auto hides sideloaded apps until you allow them. On the -phone, search Settings for "Android Auto", tap **Version** ten times, then in the ⋮ menu go to -**Developer settings** and tick **Unknown sources**. Stemify then shows up in Auto's app -launcher. - -**Testing without a car:** Google's Desktop Head Unit can impersonate the Miata. On the phone, -start the head unit server from Auto's developer settings, then: +To test without a car, start the head unit server from Auto's developer settings, then: ```sh adb forward tcp:5277 tcp:5277 ~/Android/Sdk/extras/google/auto/desktop-head-unit -c ~/Android/Sdk/extras/google/auto/config/rotary.ini ``` ---- +## Desktop -## iOS and CarPlay +Linux, Python 3.12 and [uv](https://docs.astral.sh/uv/). PyTorch defaults to ROCm (AMD); point the torch index in `pyproject.toml` elsewhere for NVIDIA or CPU. -`ios/` is a SwiftUI port of the Android app with the same features, plus CarPlay (browse tabs, Now Playing with Vocals and Mix buttons) and Siri ("play X in Stemify"). iOS 17 or newer. - -On top of the Android features: the Save chip in the mixer remembers a mix for that song and puts it back every time it plays, and the mic button turns on karaoke mode (vocals muted on every song, synced lyrics from [LRCLIB](https://lrclib.net) on Now Playing; tap a line to jump to it). - -Build an unsigned IPA on a Mac with Xcode and xcodegen (`brew install xcodegen`): - -``` -cd ios -scripts/build-ipa.sh # writes build/stemify-.ipa +```sh +uv sync +uv run stemify # the drag-and-drop separator +uv run stemify-player # the player +./scripts/install-desktop.sh # add both to your app launcher (--uninstall removes them) ``` -Sideload the IPA with Sideloadly or AltStore, which sign it with your Apple ID. +**Stemify** is the separator. Drop files on the window and each one is separated with your last-used model and device. You get a lane per stem with a waveform, solo, mute, gain and export (everything, one stem, or the current mix, as WAV, FLAC or MP3). Separation runs in its own process, results are cached, and it works on AMD, NVIDIA and CPU. -To get updates in AltStore, SideStore or Feather, add this source: +**Stemify Player** is your Plex library and its stems, dressed up as Microsoft Excel 2016. Sheets are your library, the zoom slider is master volume, Cell Styles are mix presets, and Undo/Redo skip songs. It reports plays to Plex, supports media keys and MPRIS, and remembers your queue and mix. More in [`stemify/player/README.md`](stemify/player/README.md). -``` -https://git.iamchrisama.com/cdnutter/stemify/raw/branch/main/ios/source.json -``` +## Stem Now -After uploading a new release (tag `ios-v`, asset `stemify-.ipa`), run `ios/scripts/make-source.py` to regenerate `ios/source.json` and commit it. To work in Xcode instead, run `xcodegen generate` and open `Stemify.xcodeproj`. +A big library takes stemd days. Hold a song, album, artist or playlist and choose **Stem Now** to move its unseparated songs to the front. -CarPlay and Siri need the `carplay-audio` and `siri` entitlements. A free Apple ID can't sign those, so on a free sideload the app works on the phone but won't show up in CarPlay or Siri. A paid developer account covers Siri; CarPlay audio also needs Apple to approve the entitlement for your team. +1. The app sends the songs to stemserve, which drops a request file in stemd's queue folder. +2. stemd finishes the song it's on and does yours next. If it was idle, it starts within about 5 seconds. +3. About a minute after each song finishes, the app picks up its stems. -## The desktop apps +Requests run oldest first and survive restarts. stemd matches Plex's paths to its own files by the artist/album/file tail, so different mount points don't matter. Songs that failed before are skipped; delete `state/failed.json` to retry them. -### Stemify (the separator) +## Reference -Drop files onto the window. Each becomes a queue item, separation starts right away with your -last-used model and device, and when it's done you get a lane per stem with a waveform, solo, mute, -gain and export. Export everything, one stem, or your current mix, as WAV, FLAC or MP3. +### Separation models -Other niceties: +| Model id | Stems | Notes | +|---|---|---| +| `htdemucs` | 4 | HT Demucs, fast | +| `htdemucs_ft` | 4 | Fine-tuned, about 4x slower | +| `htdemucs_6s` | 6 | Adds guitar and piano | +| `roformer_bs_vocals` | 2 | BS-RoFormer vocals/instrumental (Viperx 1297) | +| `roformer_mel_kim_ft` | 2 | Mel-RoFormer, cleanest vocals (Kim FT by unwa) | +| `roformer_bs_sw` | 6 | BS-RoFormer SW by jarredou, stemd's default | -- Separation runs in its own worker process, so the UI never stutters, even mid-separation. -- Works on AMD (ROCm), NVIDIA and CPU, and shows free VRAM. -- Results are cached, so flipping back to an old song is instant (with a size limit you can set). -- A dev frame-time overlay, for anyone who wants proof of the 240 fps claims. - -### Stemify Player (the spreadsheet) - -The desktop sibling of the phone app, disguised as Microsoft Excel 2016. Some highlights from the -translation guide: - -| In Excel | In Stemify Player | -|---|---| -| Sheets (Songs, Artists, Albums, Playlists, Stems, Queue) | Your library | -| "Tell me what you want to do" | Search | -| Zoom slider | Master volume | -| Cell Styles | Mix presets (Karaoke is filed under "Bad", A Cappella under "Good") | -| AutoSum | AutoShuffle | -| Undo / Redo | Previous / next song | -| Format task pane | Now playing, with the stems as a column chart you can drag | -| `#GETTING_DATA` | A playlist loading from Plex | - -It reports plays to Plex, supports media keys and desktop media controls (MPRIS), and remembers -your queue, position and mix between launches. The full tour is in -[`stemify/player/README.md`](stemify/player/README.md). - ---- - -## Separation models - -| Model id | Name | Stems | Notes | -|---|---|---|---| -| `htdemucs` | HT Demucs | 4 | Fast | -| `htdemucs_ft` | HT Demucs FT | 4 | Fine-tuned, about 4x slower | -| `htdemucs_6s` | HT Demucs 6-stem | 6 | Adds guitar and piano | -| `roformer_bs_vocals` | BS-RoFormer Vocals | 2 | Vocals / instrumental (Viperx 1297) | -| `roformer_mel_kim_ft` | Mel-RoFormer Vocals FT | 2 | Cleanest vocals (Kim FT by unwa) | -| `roformer_bs_sw` | BS-RoFormer SW 6-stem | 6 | stemd's default (by jarredou) | - -The 4-stem models give vocals, drums, bass and other. The 6-stem ones add guitar and piano. - ---- - -## How the pieces find each other - -The stems folder mirrors the music folder, with one folder per song: +### Folder layout ``` Music/Kansas/Leftoverture/01 Carry On Wayward Son.flac Stems/Kansas/Leftoverture/01 Carry On Wayward Son/ - .stemify.json ← the "this one's done" marker + .stemify.json done marker 01 Carry On Wayward Son - vocals.flac 01 Carry On Wayward Son - drums.flac - 01 Carry On Wayward Son - bass.flac ... ``` -- Stems are written as 24-bit FLAC into a hidden `.partial` folder first, and the marker goes in - last, so a separation interrupted by a reboot is simply redone next time. No half-baked songs. -- When a folder holds the same song in several formats, the best one wins (FLAC/WAV, then M4A, - Opus/Ogg, MP3, WMA). -- Plex reports file paths as *its* container sees them, so the apps match them to stems by suffix, - checking durations along the way, and learn the path prefix from the first confident match. A - song called "Intro.flac" on one album will never steal another album's intro. +- Stems are written as 24-bit FLAC to a hidden `.partial` folder and the marker goes in last, so an interrupted separation is just redone. +- If a folder has the same song in several formats, the best one wins (FLAC/WAV, then M4A, Opus/Ogg, MP3, WMA). +- Plex reports paths as its container sees them, so the apps match songs to stems by path suffix, check durations, and learn the prefix from the first confident match. ---- - -## stemserve API +### stemserve API | Request | Returns | |---|---| -| `GET /api/info` | Track count, whether ffmpeg is available, whether Stem Now is set up | -| `GET /api/index` | Every finished song (with ETag, so unchanged indexes cost one tiny 304) | +| `GET /api/info` | Track count, ffmpeg availability, whether Stem Now is set up | +| `GET /api/index` | Every finished song (with ETag) | | `GET /api/peaks?key=K&n=256` | Per-stem loudness envelopes for waveforms, 0 to 1000 | | `POST /api/prepare?key=K&fmt=opus&br=256` | Starts Opus transcodes in the background | -| `POST /api/stem` with `{"paths": [...]}` | Stem Now: stemd separates these Plex files next | -| `GET /file/K/.flac[?fmt=opus&br=256]` | The stem itself, with HTTP range support for seeking | +| `POST /api/stem` with `{"paths": [...]}` | Stem Now for these Plex files | +| `GET /file/K/.flac[?fmt=opus&br=256]` | The stem, with range support | -`K` is the song's folder under the stems root, e.g. `Kansas/Leftoverture/01 Carry On Wayward Son`. -Want to run it outside Docker? It's one file: +`K` is the song's folder under the stems root, like `Kansas/Leftoverture/01 Carry On Wayward Son`. Outside Docker: ```sh STEMS_DIR=/path/to/Stems CACHE_DIR=~/.cache/stemserve python3 server/stemserve.py ``` ---- +### Repo layout + +``` +ios/ iOS and CarPlay app (SwiftUI, xcodegen project) + Stemify/ App, Plex, Playback, Stems, CarPlay, UI + scripts/ build-ipa.sh, make-source.py +android/ Android and Android Auto app (Kotlin, Compose, Media3) +server/ stemd.py, stemserve.py, Dockerfiles, compose file +stemify/ desktop separator (PySide6 + QML) and shared separation code + worker/ Demucs and RoFormer backends, run in a separate process + player/ Stemify Player + library_core.py folder layout rules shared with stemd +scripts/ install-desktop.sh +specs.txt original design doc for the desktop separator +``` ## Building and testing +**iOS** needs a Mac with Xcode and xcodegen (`brew install xcodegen`): + +```sh +cd ios +scripts/build-ipa.sh # unsigned build/stemify-.ipa +xcodegen generate # or open Stemify.xcodeproj in Xcode +``` + +The version lives in `ios/project.yml` (`MARKETING_VERSION` and `CURRENT_PROJECT_VERSION`). To release, tag `ios-v`, attach `stemify-.ipa`, then run `GITEA_TOKEN=... ios/scripts/make-source.py` and commit the new `ios/source.json`. + **Android** needs JDK 21 and the Android SDK (`android/local.properties` points at it): ```sh cd android JAVA_HOME=/path/to/jdk-21 ./gradlew assembleRelease # app/build/outputs/apk/release/ -JAVA_HOME=/path/to/jdk-21 ./gradlew testDebugUnitTest # stem matching, EQ and filter maths, limiter, cache, car tree +JAVA_HOME=/path/to/jdk-21 ./gradlew testDebugUnitTest # matching, EQ maths, limiter, cache, car tree ``` -The cache and index tests stream real stems from a stemserve on `localhost:8765` and skip -themselves when there isn't one. Debug builds include a hook for screenshots and back-gesture -testing (`app/src/debug`); release builds don't. +The cache and index tests use a stemserve on `localhost:8765` and skip themselves without one. -**Desktop** apps can be checked without opening a window: +**Desktop** can be checked without a window: ```sh QT_QPA_PLATFORM=offscreen uv run stemify-player --no-mpris --screenshot /tmp/p.png@5 --quit-after 6 ``` ---- +## Troubleshooting -## Repo tour +**A song says NO STEMS.** Hold it and choose Stem Now. -``` -android/ the Android and Android Auto app (Kotlin, Jetpack Compose, Media3) - app/src/main/java/com/stemify/mobile/ - plex/ Plex library, search, playlists, artwork, play reporting - stems/ stemserve client, path matching, which albums have stems - cache/ sparse block cache for streaming, and downloads - engine/ decoders, the sample-locked mixer, EQ, limiter, meters - eq/ EQ profiles, device switching, APO config import/export - playback/ queue, audio focus, mix presets, the Media3 session - auto/ Android Auto browse tree, voice search, offline mode - ui/ every screen, hand-built iOS-style components -server/ - stemd.py the GPU separation daemon - stemserve.py the stem file server (standard library only) - *.Dockerfile, docker-compose.yml -stemify/ the desktop separator (PySide6 + QML) and shared separation code - worker/ separation backends (Demucs, RoFormer) in a separate process - player/ Stemify Player, the Excel-shaped one - library_core.py the folder layout rules shared with stemd -scripts/install-desktop.sh -specs.txt the original design doc for the desktop separator -``` +**Stem Now says "stemserve needs updating" or "queue folder isn't mounted".** stemserve is older than the app or is missing the `/queue` mount. The queue folder has to be owned by `99:100` before the container first starts, or Docker creates it as root. ---- +**stemd is separating on the CPU.** The container can't see the GPU. Run it with `--runtime=nvidia` and the NVIDIA variables above. -## Troubleshooting and FAQ +**stemd says "out of GPU memory".** Something else is using VRAM. stemd retries that song next pass without marking it failed. -**A song says NO STEMS and I'm impatient.** -Hold it and choose **Stem Now**. It'll be next in line. +**The stem server shows Offline.** Check the address in Settings, try `curl http://:8765/api/info` from another machine, and if stemserve has a `TOKEN`, set the same one in the app. -**Stem Now says "stemserve needs updating" or "queue folder isn't mounted".** -stemserve is older than the app, or it's missing the `/queue` mount. See -[the server setup](#1-the-server-stemd-and-stemserve). The queue folder needs to be owned by -`99:100` *before* the container first starts, or Docker creates it as root and nobody can write -to it. +**Stemify isn't in CarPlay or Siri.** The IPA was signed without those entitlements. See [iOS install](#install). -**stemd says it's separating on the CPU.** -The container can't see the GPU. Make sure it's running with `--runtime=nvidia` and the NVIDIA -environment variables shown above. +**Stemify isn't in Android Auto.** Turn on Unknown sources in Auto's developer settings. See [Android Auto](#android-auto). -**stemd keeps saying "out of GPU memory".** -Something else (Plex transcoding, a game server, your ambitions) is hogging VRAM. stemd shrugs and -retries that song on the next pass. Nothing is marked failed. - -**The Stem Server page says Offline.** -Check the address in Settings → Stem Server, check `curl http://:8765/api/info` from -another machine, and if you set a `TOKEN` on the server, set the same one in the app. - -**The app doesn't show up in Android Auto.** -Sideloaded apps need **Unknown sources** turned on in Auto's developer settings. See -[setting up the car](#android-auto). - -**Some songs are marked failed in `state/failed.json`.** -Usually corrupt files, or "songs" over 40 minutes (those are almost always DJ mixes or radio rips). -Failed tracks retry automatically when the file changes; delete the file to retry everything. - -**Why is the desktop player an Excel spreadsheet?** -We don't talk about it. - ---- +**Songs listed in `state/failed.json`.** Usually corrupt files or "songs" over 40 minutes (DJ mixes, radio rips). They retry when the file changes; delete the file to retry everything. ## Credits -Standing on the shoulders of some very clever people: - - [Demucs](https://github.com/adefossez/demucs) by Alexandre Défossez -- BS-RoFormer and Mel-RoFormer models, including the SW 6-stem model by jarredou, Viperx's 1297, - and Kim's vocal model fine-tuned by unwa, via - [python-audio-separator](https://github.com/nomadkaraoke/python-audio-separator) -- [PyTorch](https://pytorch.org/), [PySide6 / Qt](https://www.qt.io/qt-for-python), - [Jetpack Compose](https://developer.android.com/compose) and - [Media3](https://developer.android.com/media/media3) -- [Equalizer APO](https://sourceforge.net/projects/equalizerapo/) and - [AutoEQ](https://github.com/jaakkopasanen/AutoEq), whose formats the EQ speaks -- The Inter, Selawik and Carlito fonts (licences included alongside them) -- [Plex](https://www.plex.tv/), for keeping the library tidy so we didn't have to +- BS-RoFormer and Mel-RoFormer models (SW 6-stem by jarredou, Viperx 1297, Kim FT by unwa) via [python-audio-separator](https://github.com/nomadkaraoke/python-audio-separator) +- [PyTorch](https://pytorch.org/), [PySide6](https://www.qt.io/qt-for-python), [Jetpack Compose](https://developer.android.com/compose), [Media3](https://developer.android.com/media/media3) +- [Equalizer APO](https://sourceforge.net/projects/equalizerapo/) and [AutoEQ](https://github.com/jaakkopasanen/AutoEq), whose formats the EQ reads and writes +- [LRCLIB](https://lrclib.net) for synced lyrics +- Inter, Selawik and Carlito fonts (licenses included alongside them) -Stemify isn't affiliated with Plex, Google, Microsoft or Mazda. It just likes them. - ---- +Stemify isn't affiliated with Plex, Apple, Google, Microsoft or Mazda. ## License -[GPL-3.0](LICENSE). Take it apart and put it back together however you like, just like the songs. +[GPL-3.0](LICENSE)