cdnutter/playlink
1
0 Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Rename host library to master throughout

Browse Source 
The two libraries are now "master" and "slave" everywhere: UI text, API
JSON fields (master_root, rel_master, master_size, ...), Go identifiers,
database columns, quality verdicts, quarantine subfolders and docs.

Existing databases are upgraded by a new migration that renames the
columns in place and rewrites stored "host" values, so pairs, history,
cached verdicts and restorable quarantine entries carry over. Network
"host" usages (SplitHostPort, the Origin/Host check) are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
cdnutterandClaude Opus 5.5 committed 2026-09-22 16:50:55 -07:00
1 parent d06b6d546d
commit 5b088bb4a6
19 files changed
+320 -307
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

README.md
+9 -9
View File
@@ -1,7 +1,7 @@
# playlink
Deduplicate music libraries with hardlinks. playlink finds songs that exist in both a **host** library (e.g. your full `Albums`) and a
**slave** library (e.g. a curated `Singles` library) and replaces the slave copy with a hardlink to the host copy. Each song then takes
Deduplicate music libraries with hardlinks. playlink finds songs that exist in both a **master** library (e.g. your full `Albums`) and a
**slave** library (e.g. a curated `Singles` library) and replaces the slave copy with a hardlink to the master copy. Each song then takes
up disk space only once, while both libraries keep looking exactly the same to Plex, Navidrome, Jellyfin and other media servers.
It ships as a single Go binary with an embedded React web UI and SQLite state, packaged as a small Docker image, in the style of
@@ -26,15 +26,15 @@ path on both sides. The fallbacks below catch small differences, and a scan warn
**Manual review** shows both files side by side and ranks their quality: lossless beats lossy, then bit depth and sample rate, then
codec-weighted bitrate (Opus and AAC count for more per kbps than MP3). You then have three choices:
- **Link to host copy.** The default. The host is kept because it belongs to a complete album. If the host is *lower* quality, playlink
- **Link to master copy.** The default. The master is kept because it belongs to a complete album. If the master is *lower* quality, playlink
warns you and asks for explicit confirmation.
- **Use slave copy as host.** The override. The slave's better file replaces the host file, the old host file is quarantined, and both
- **Use slave copy as master.** The override. The slave's better file replaces the master file, the old master file is quarantined, and both
libraries share the better file.
- **Ignore.** The pair is hidden from review until you un-ignore it.
**Build mode** covers the "start fresh" workflow. Point the slave at an empty folder, browse the host library, tick the
**Build mode** covers the "start fresh" workflow. Point the slave at an empty folder, browse the master library, tick the
folders or songs you want, and playlink hardlinks them in at the same relative path. **Remove** deletes slave files, but only ones that are hardlinks of the
host file.
master file.
## Safety model
@@ -52,10 +52,10 @@ playlink is built so that no user data is ever lost, even if it crashes or loses
expires, or until you purge it.
- **Every operation can be undone.** An identical link is undone by writing an independent, verified copy back. A quarantined file is
undone by restoring the original.
- **Removal is limited to hardlinks.** Build/Remove only deletes a slave file that shares an inode with the host file, so the last
- **Removal is limited to hardlinks.** Build/Remove only deletes a slave file that shares an inode with the master file, so the last
copy of any data is never deleted.
- **Paths are confined.** Libraries and the quarantine must be inside `PLAYLINK_ALLOWED_ROOTS`. Symlinks are never followed or
replaced, `..` is rejected, and host and slave may not overlap.
replaced, `..` is rejected, and master and slave may not overlap.
- **Mutations are serialized.** Only one filesystem mutation runs at a time, and only one job runs per pair.
- **Auto-link is conservative.** It only acts on files with the *exact* same path, and it skips files modified within the settle time
(default 10 minutes) so it never touches a download in progress.
@@ -83,7 +83,7 @@ services:
Then open `http://<server>:7480`, create your account, and add a pair:
- Host: `/data/Music/Albums`
- Master: `/data/Music/Albums`
- Slave: `/data/Music/Instances/cdnutter/Singles`
Run a scan, check the Review tab, and turn off dry run once the planned actions in Activity look right.
docker-compose.yml
+1 -1
View File
@@ -16,7 +16,7 @@ services:
- /mnt/user/Aqua/playlink:/config
# IMPORTANT: mount ONE folder that contains BOTH libraries. Hardlinks
# cannot cross Docker bind mounts, even on the same disk.
# host (Albums) -> /data/Music/Albums
# master (Albums) -> /data/Music/Albums
# slave (Singles) -> /data/Music/Instances/cdnutter/Singles
- /mnt/user/Sapphire:/data
restart: unless-stopped
internal/api/server.go
+3 -3
View File
@@ -298,7 +298,7 @@ func (s *Server) handleGetPair(w http.ResponseWriter, r *http.Request) {
type pairInput struct {
Name string `json:"name"`
HostRoot string `json:"host_root"`
MasterRoot string `json:"master_root"`
SlaveRoot string `json:"slave_root"`
QuarantineDir string `json:"quarantine_dir"`
Extensions []string `json:"extensions"`
@@ -309,7 +309,7 @@ type pairInput struct {
}
func (in pairInput) apply(p *store.Pair) {
p.Name, p.HostRoot, p.SlaveRoot, p.QuarantineDir = in.Name, in.HostRoot, in.SlaveRoot, in.QuarantineDir
p.Name, p.MasterRoot, p.SlaveRoot, p.QuarantineDir = in.Name, in.MasterRoot, in.SlaveRoot, in.QuarantineDir
p.Extensions, p.AutoLinkIdentical = in.Extensions, in.AutoLinkIdentical
p.ScanIntervalMinutes, p.RetentionDays, p.SettleMinutes = in.ScanIntervalMinutes, in.RetentionDays, in.SettleMinutes
}
@@ -361,7 +361,7 @@ func (s *Server) handleSavePair(w http.ResponseWriter, r *http.Request) {
writeErr(w, 500, err)
return
}
if old.ID != 0 && (old.HostRoot != p.HostRoot || old.SlaveRoot != p.SlaveRoot) {
if old.ID != 0 && (old.MasterRoot != p.MasterRoot || old.SlaveRoot != p.SlaveRoot) {
_ = s.st.ClearMatches(r.Context(), p.ID) // results no longer apply
}
p, _ = s.st.Pair(r.Context(), p.ID)
internal/engine/engine.go
+22 -22
View File
@@ -26,8 +26,8 @@ const QuarantineDirName = ".playlink-trash"
var (
ErrBusy = errors.New("another job is already running for this library pair")
ErrDowngrade = errors.New("the host copy is lower quality than the slave copy; confirm the downgrade to continue")
ErrConfirm = errors.New("this action modifies the host library and must be confirmed")
ErrDowngrade = errors.New("the master copy is lower quality than the slave copy; confirm the downgrade to continue")
ErrConfirm = errors.New("this action modifies the master library and must be confirmed")
)
type Engine struct {
@@ -163,7 +163,7 @@ func (e *Engine) startJob(pairID int64, kind string, run func(ctx context.Contex
type libs struct {
pair store.Pair
host fsops.Root
master fsops.Root
slave fsops.Root
quarantine string
dev uint64
@@ -187,23 +187,23 @@ func (e *Engine) withinAllowed(p string) bool {
// called before every scan and operation so configuration drift (a remount,
// a moved folder) is caught before anything is touched.
func (e *Engine) open(p store.Pair) (*libs, error) {
host, err := fsops.NewRoot(p.HostRoot)
master, err := fsops.NewRoot(p.MasterRoot)
if err != nil {
return nil, fmt.Errorf("host library: %w", err)
return nil, fmt.Errorf("master library: %w", err)
}
slave, err := fsops.NewRoot(p.SlaveRoot)
if err != nil {
return nil, fmt.Errorf("slave library: %w", err)
}
for _, r := range []string{host.Path, slave.Path} {
for _, r := range []string{master.Path, slave.Path} {
if !e.withinAllowed(r) {
return nil, fmt.Errorf("%s is outside the allowed roots %v", r, e.allowedRoots)
}
}
if fsops.Within(host.Path, slave.Path) || fsops.Within(slave.Path, host.Path) {
return nil, errors.New("host and slave libraries must not overlap or contain each other")
if fsops.Within(master.Path, slave.Path) || fsops.Within(slave.Path, master.Path) {
return nil, errors.New("master and slave libraries must not overlap or contain each other")
}
hi, err := fsops.Lstat(host.Path)
mi, err := fsops.Lstat(master.Path)
if err != nil {
return nil, err
}
@@ -211,17 +211,17 @@ func (e *Engine) open(p store.Pair) (*libs, error) {
if err != nil {
return nil, err
}
if hi.Dev != si.Dev {
return nil, fmt.Errorf("host and slave are on different filesystems (device %d vs %d): %w. "+
"In Docker, mount a single common parent folder instead of two separate volumes", hi.Dev, si.Dev, fsops.ErrCrossDevice)
if mi.Dev != si.Dev {
return nil, fmt.Errorf("master and slave are on different filesystems (device %d vs %d): %w. "+
"In Docker, mount a single common parent folder instead of two separate volumes", mi.Dev, si.Dev, fsops.ErrCrossDevice)
}
q := filepath.Clean(p.QuarantineDir)
if !filepath.IsAbs(q) {
return nil, errors.New("quarantine directory must be absolute")
}
if fsops.Within(host.Path, q) {
return nil, errors.New("quarantine directory must not be inside the host library")
if fsops.Within(master.Path, q) {
return nil, errors.New("quarantine directory must not be inside the master library")
}
if q == slave.Path || !e.withinAllowed(q) {
return nil, errors.New("quarantine directory must be inside the allowed roots and not the slave root itself")
@@ -236,7 +236,7 @@ func (e *Engine) open(p store.Pair) (*libs, error) {
if err != nil {
return nil, err
}
if qi.Dev != hi.Dev {
if qi.Dev != mi.Dev {
return nil, errors.New("quarantine directory must be on the same filesystem as the libraries")
}
@@ -250,7 +250,7 @@ func (e *Engine) open(p store.Pair) (*libs, error) {
if len(exts) == 0 {
return nil, errors.New("no file extensions configured")
}
return &libs{pair: p, host: host, slave: slave, quarantine: q, dev: hi.Dev, exts: exts}, nil
return &libs{pair: p, master: master, slave: slave, quarantine: q, dev: mi.Dev, exts: exts}, nil
}
// NormalizePair fills defaults and validates a pair. It returns non-fatal
@@ -260,7 +260,7 @@ func (e *Engine) NormalizePair(p *store.Pair) ([]string, error) {
if p.Name == "" {
return nil, errors.New("name is required")
}
p.HostRoot = filepath.Clean(strings.TrimSpace(p.HostRoot))
p.MasterRoot = filepath.Clean(strings.TrimSpace(p.MasterRoot))
p.SlaveRoot = filepath.Clean(strings.TrimSpace(p.SlaveRoot))
if strings.TrimSpace(p.QuarantineDir) == "" {
p.QuarantineDir = filepath.Join(p.SlaveRoot, QuarantineDirName)
@@ -288,7 +288,7 @@ func (e *Engine) NormalizePair(p *store.Pair) ([]string, error) {
return nil, err
}
// Store resolved paths so later containment checks compare like with like.
p.HostRoot, p.SlaveRoot, p.QuarantineDir = l.host.Path, l.slave.Path, l.quarantine
p.MasterRoot, p.SlaveRoot, p.QuarantineDir = l.master.Path, l.slave.Path, l.quarantine
return layoutWarnings(l), nil
}
@@ -312,20 +312,20 @@ func layoutWarnings(l *libs) []string {
}
return
}
hd, _ := list(l.host.Path)
md, _ := list(l.master.Path)
sd, sf := list(l.slave.Path)
if len(sd) == 0 && sf == 0 {
warns = append(warns, "The slave library is empty. Use Build to pick songs from the host library to hardlink into it.")
warns = append(warns, "The slave library is empty. Use Build to pick songs from the master library to hardlink into it.")
return warns
}
shared := 0
for d := range sd {
if hd[d] {
if md[d] {
shared++
}
}
if len(sd) > 0 && shared*2 < len(sd) {
warns = append(warns, fmt.Sprintf("Only %d of %d top-level folders in the slave library also exist in the host library. "+
warns = append(warns, fmt.Sprintf("Only %d of %d top-level folders in the slave library also exist in the master library. "+
"Both libraries should use the same folder layout, or most files will not match.", shared, len(sd)))
}
return warns
internal/engine/engine_test.go
+31 -31
View File
@@ -21,7 +21,7 @@ import (
type env struct {
t *testing.T
root string
host string
master string
slave string
st *store.Store
e *Engine
@@ -34,8 +34,8 @@ func setup(t *testing.T, dryRun bool) *env {
if err != nil {
t.Fatal(err)
}
ev := &env{t: t, root: root, host: filepath.Join(root, "Albums"), slave: filepath.Join(root, "Singles")}
for _, d := range []string{ev.host, ev.slave, filepath.Join(root, "db")} {
ev := &env{t: t, root: root, master: filepath.Join(root, "Albums"), slave: filepath.Join(root, "Singles")}
for _, d := range []string{ev.master, ev.slave, filepath.Join(root, "db")} {
if err := os.MkdirAll(d, 0o755); err != nil {
t.Fatal(err)
}
@@ -54,7 +54,7 @@ func setup(t *testing.T, dryRun bool) *env {
}
ev.st = st
ev.e = New(st, probe.Prober{}, []string{root}, slog.New(slog.NewTextHandler(io.Discard, nil)))
ev.pair = store.Pair{Name: "test", HostRoot: ev.host, SlaveRoot: ev.slave, RetentionDays: 30}
ev.pair = store.Pair{Name: "test", MasterRoot: ev.master, SlaveRoot: ev.slave, RetentionDays: 30}
if _, err := ev.e.NormalizePair(&ev.pair); err != nil {
t.Fatal(err)
}
@@ -136,7 +136,7 @@ var ctx = context.Background()
func TestIdenticalLinkAndUndo(t *testing.T) {
ev := setup(t, false)
rel := "bbno$/Bag or Die/edamame.flac"
h := ev.write(ev.host, rel, "same-bytes")
h := ev.write(ev.master, rel, "same-bytes")
s := ev.write(ev.slave, rel, "same-bytes")
ev.scan()
@@ -152,7 +152,7 @@ func TestIdenticalLinkAndUndo(t *testing.T) {
t.Fatal("identical link must not quarantine")
}
if !sameInode(t, h, s) {
t.Fatal("slave is not a hardlink of host")
t.Fatal("slave is not a hardlink of master")
}
if ev.match(rel).Status != store.StatusLinked {
t.Fatal("match not marked linked")
@@ -169,7 +169,7 @@ func TestIdenticalLinkAndUndo(t *testing.T) {
func TestDifferentLinkQuarantinesAndUndoRestores(t *testing.T) {
ev := setup(t, false)
rel := "Artist/Album/song.flac"
h := ev.write(ev.host, rel, "host-version-longer")
h := ev.write(ev.master, rel, "master-version-longer")
s := ev.write(ev.slave, rel, "slave-version")
ev.scan()
@@ -190,7 +190,7 @@ func TestDifferentLinkQuarantinesAndUndoRestores(t *testing.T) {
if err := ev.e.Undo(ctx, op.ID); err != nil {
t.Fatal(err)
}
if ev.read(s) != "slave-version" || ev.read(h) != "host-version-longer" {
if ev.read(s) != "slave-version" || ev.read(h) != "master-version-longer" {
t.Fatal("undo did not restore the original")
}
if _, err := os.Stat(op.QuarantinePath); !errors.Is(err, os.ErrNotExist) {
@@ -203,12 +203,12 @@ func TestDifferentLinkQuarantinesAndUndoRestores(t *testing.T) {
func TestSimilarExtensionLinkRenamesAndUndo(t *testing.T) {
ev := setup(t, false)
h := ev.write(ev.host, "A/B/01 - Song.flac", "flac-data")
h := ev.write(ev.master, "A/B/01 - Song.flac", "flac-data")
s := ev.write(ev.slave, "A/B/01 - Song.mp3", "mp3")
ev.scan()
m := ev.match("A/B/01 - Song.mp3")
if m.Status != store.StatusSimilar || m.Kind != store.KindExtension || m.RelHost != "A/B/01 - Song.flac" {
if m.Status != store.StatusSimilar || m.Kind != store.KindExtension || m.RelMaster != "A/B/01 - Song.flac" {
t.Fatalf("unexpected match %+v", m)
}
op, err := ev.e.Link(ctx, m.ID, LinkOptions{})
@@ -217,7 +217,7 @@ func TestSimilarExtensionLinkRenamesAndUndo(t *testing.T) {
}
newSlave := filepath.Join(ev.slave, "A/B/01 - Song.flac")
if !sameInode(t, h, newSlave) {
t.Fatal("slave should now hold the host's .flac as a hardlink")
t.Fatal("slave should now hold the master's .flac as a hardlink")
}
if _, err := os.Stat(s); !errors.Is(err, os.ErrNotExist) {
t.Fatal("old .mp3 should be moved to quarantine")
@@ -232,16 +232,16 @@ func TestSimilarExtensionLinkRenamesAndUndo(t *testing.T) {
t.Fatal("linked .flac should be removed on undo")
}
if ev.read(h) != "flac-data" {
t.Fatal("host must be untouched")
t.Fatal("master must be untouched")
}
}
func TestFuzzyTitleMatch(t *testing.T) {
ev := setup(t, false)
ev.write(ev.host, "A/B/03. My Song!.flac", "x")
ev.write(ev.master, "A/B/03. My Song!.flac", "x")
ev.write(ev.slave, "A/B/My Song.mp3", "y")
ev.write(ev.slave, "A/B/My Song (Live).mp3", "z")
ev.write(ev.host, "A/Other Album/Lonely.flac", "x")
ev.write(ev.master, "A/Other Album/Lonely.flac", "x")
ev.write(ev.slave, "A/Single/Lonely.flac", "x")
ev.scan()
if m := ev.match("A/B/My Song.mp3"); m.Kind != store.KindFuzzy {
@@ -258,7 +258,7 @@ func TestFuzzyTitleMatch(t *testing.T) {
func TestDowngradeGuardAndPromote(t *testing.T) {
ev := setup(t, false)
rel := "A/B/song.flac"
h := ev.write(ev.host, rel, "lossy-host")
h := ev.write(ev.master, rel, "lossy-master")
s := ev.write(ev.slave, rel, "lossless-slave!")
ev.scan()
m := ev.match(rel)
@@ -280,15 +280,15 @@ func TestDowngradeGuardAndPromote(t *testing.T) {
t.Fatal(err)
}
if !sameInode(t, h, s) || ev.read(h) != "lossless-slave!" {
t.Fatal("host should now be the slave's file")
t.Fatal("master should now be the slave's file")
}
if ev.read(op.QuarantinePath) != "lossy-host" {
t.Fatal("host original not quarantined")
if ev.read(op.QuarantinePath) != "lossy-master" {
t.Fatal("master original not quarantined")
}
if err := ev.e.Undo(ctx, op.ID); err != nil {
t.Fatal(err)
}
if ev.read(h) != "lossy-host" || ev.read(s) != "lossless-slave!" || sameInode(t, h, s) {
if ev.read(h) != "lossy-master" || ev.read(s) != "lossless-slave!" || sameInode(t, h, s) {
t.Fatal("promote undo failed")
}
}
@@ -296,7 +296,7 @@ func TestDowngradeGuardAndPromote(t *testing.T) {
func TestChangedSinceScanIsRefused(t *testing.T) {
ev := setup(t, false)
rel := "A/B/c.flac"
ev.write(ev.host, rel, "aaa")
ev.write(ev.master, rel, "aaa")
s := ev.write(ev.slave, rel, "bbb")
ev.scan()
m := ev.match(rel)
@@ -315,7 +315,7 @@ func TestChangedSinceScanIsRefused(t *testing.T) {
func TestDryRunChangesNothing(t *testing.T) {
ev := setup(t, true)
rel := "A/B/d.flac"
h := ev.write(ev.host, rel, "same")
h := ev.write(ev.master, rel, "same")
s := ev.write(ev.slave, rel, "same")
ev.scan()
op, err := ev.e.Link(ctx, ev.match(rel).ID, LinkOptions{})
@@ -337,7 +337,7 @@ func TestAutoLinkRespectsSettleTime(t *testing.T) {
if err := ev.st.SavePair(ctx, &ev.pair); err != nil {
t.Fatal(err)
}
h := ev.write(ev.host, "A/B/e.flac", "same")
h := ev.write(ev.master, "A/B/e.flac", "same")
s := ev.write(ev.slave, "A/B/e.flac", "same")
ev.scan()
if sameInode(t, h, s) {
@@ -355,9 +355,9 @@ func TestAutoLinkRespectsSettleTime(t *testing.T) {
func TestBuildAndRemove(t *testing.T) {
ev := setup(t, false)
h1 := ev.write(ev.host, "bbno$/Album/one.flac", "1")
ev.write(ev.host, "bbno$/Album/two.flac", "2")
ev.write(ev.host, "bbno$/Album/cover.jpg", "img")
h1 := ev.write(ev.master, "bbno$/Album/one.flac", "1")
ev.write(ev.master, "bbno$/Album/two.flac", "2")
ev.write(ev.master, "bbno$/Album/cover.jpg", "img")
ev.write(ev.slave, "bbno$/Album/two.flac", "user's own different file")
if _, err := ev.e.StartBuild(ev.pair.ID, []string{"bbno$"}); err != nil {
@@ -389,7 +389,7 @@ func TestBuildAndRemove(t *testing.T) {
t.Fatal("linked file should be removed")
}
if ev.read(h1) != "1" {
t.Fatal("host must be untouched")
t.Fatal("master must be untouched")
}
if ev.read(filepath.Join(ev.slave, "bbno$/Album/two.flac")) != "user's own different file" {
t.Fatal("remove must never delete a non-linked file")
@@ -404,7 +404,7 @@ func TestSymlinkEscapeIgnored(t *testing.T) {
if err := os.Symlink(outside, filepath.Join(ev.slave, "Evil", "Album")); err != nil {
t.Fatal(err)
}
ev.write(ev.host, "Evil/Album/secret.flac", "x")
ev.write(ev.master, "Evil/Album/secret.flac", "x")
ev.scan()
if _, err := ev.st.MatchBySlave(ctx, ev.pair.ID, "Evil/Album/secret.flac"); err == nil {
t.Fatal("files behind a symlinked directory must not be scanned")
@@ -421,7 +421,7 @@ func TestSymlinkEscapeIgnored(t *testing.T) {
func TestPurgeAndOverlapValidation(t *testing.T) {
ev := setup(t, false)
rel := "A/B/f.flac"
ev.write(ev.host, rel, "one")
ev.write(ev.master, rel, "one")
ev.write(ev.slave, rel, "two!")
ev.scan()
op, err := ev.e.Link(ctx, ev.match(rel).ID, LinkOptions{})
@@ -438,11 +438,11 @@ func TestPurgeAndOverlapValidation(t *testing.T) {
t.Fatal("undo after purge must fail")
}
bad := store.Pair{Name: "bad", HostRoot: ev.host, SlaveRoot: filepath.Join(ev.host, "A")}
bad := store.Pair{Name: "bad", MasterRoot: ev.master, SlaveRoot: filepath.Join(ev.master, "A")}
if _, err := ev.e.NormalizePair(&bad); err == nil {
t.Fatal("nested libraries must be rejected")
}
outside := store.Pair{Name: "bad", HostRoot: ev.host, SlaveRoot: t.TempDir()}
outside := store.Pair{Name: "bad", MasterRoot: ev.master, SlaveRoot: t.TempDir()}
if _, err := ev.e.NormalizePair(&outside); err == nil {
t.Fatal("paths outside allowed roots must be rejected")
}
@@ -485,7 +485,7 @@ func TestAnyLayoutMatchesWithoutWarnings(t *testing.T) {
ev := setup(t, false)
// Flat files, deeper Genre/Artist/Album/Disc nesting: all fine as long as both sides agree.
for _, rel := range []string{"loose.flac", "Rock/Band/Album/Disc 1/01 - Song.flac", "Band/Song.flac"} {
ev.write(ev.host, rel, "same:"+rel)
ev.write(ev.master, rel, "same:"+rel)
ev.write(ev.slave, rel, "same:"+rel)
}
ev.scan()
internal/engine/ops.go
+79 -79
View File
@@ -19,15 +19,15 @@ import (
)
type LinkOptions struct {
// AllowDowngrade confirms linking when the host copy scored lower.
// AllowDowngrade confirms linking when the master copy scored lower.
AllowDowngrade bool
Trigger string
}
// snapshot holds the verified state of both files right before an operation.
type snapshot struct {
hostAbs, slaveAbs string
hi, si fsops.Info
masterAbs, slaveAbs string
mi, si fsops.Info
}
// preflight re-validates a match against the filesystem: both files must be
@@ -36,28 +36,28 @@ type snapshot struct {
func preflight(l *libs, m store.Match) (snapshot, error) {
var s snapshot
var err error
if s.hostAbs, err = l.host.Resolve(m.RelHost); err != nil {
if s.masterAbs, err = l.master.Resolve(m.RelMaster); err != nil {
return s, err
}
if s.slaveAbs, err = l.slave.Resolve(m.RelSlave); err != nil {
return s, err
}
if s.hi, err = fsops.RegularFile(s.hostAbs); err != nil {
return s, fmt.Errorf("host file: %w", err)
if s.mi, err = fsops.RegularFile(s.masterAbs); err != nil {
return s, fmt.Errorf("master file: %w", err)
}
if s.si, err = fsops.RegularFile(s.slaveAbs); err != nil {
return s, fmt.Errorf("slave file: %w", err)
}
if s.hi.Ino != m.HostIno || !s.hi.Unchanged(m.HostSize, m.HostMtime) {
return s, fmt.Errorf("host %w", fsops.ErrChanged)
if s.mi.Ino != m.MasterIno || !s.mi.Unchanged(m.MasterSize, m.MasterMtime) {
return s, fmt.Errorf("master %w", fsops.ErrChanged)
}
if s.si.Ino != m.SlaveIno || !s.si.Unchanged(m.SlaveSize, m.SlaveMtime) {
return s, fmt.Errorf("slave %w", fsops.ErrChanged)
}
if s.hi.Dev != s.si.Dev || s.hi.Dev != l.dev {
if s.mi.Dev != s.si.Dev || s.mi.Dev != l.dev {
return s, fsops.ErrCrossDevice
}
if s.hi.SameInode(s.si) {
if s.mi.SameInode(s.si) {
return s, errors.New("already linked")
}
return s, nil
@@ -128,7 +128,7 @@ func (e *Engine) finish(ctx context.Context, o *store.Operation, err error) erro
e.log.Warn("operation failed", "op", o.ID, "action", o.Action, "slave", o.RelSlave, "err", err)
} else {
o.Status = store.OpDone
e.log.Info("operation done", "op", o.ID, "action", o.Action, "slave", o.RelSlave, "host", o.RelHost)
e.log.Info("operation done", "op", o.ID, "action", o.Action, "slave", o.RelSlave, "master", o.RelMaster)
}
if ferr := e.st.FinishOp(ctx, o.ID, o.Status, o.Error); ferr != nil && err == nil {
return ferr
@@ -142,10 +142,10 @@ func verdictOf(m store.Match) probe.Verdict {
return v
}
// Link replaces the slave file with a hardlink of the host file.
// Link replaces the slave file with a hardlink of the master file.
//
// Identical files are re-compared byte for byte right before linking and no
// quarantine copy is kept (the data exists in the host). Non-identical files
// quarantine copy is kept (the data exists in the master). Non-identical files
// are quarantined first so the operation can be undone.
func (e *Engine) Link(ctx context.Context, matchID int64, opt LinkOptions) (store.Operation, error) {
e.mu.Lock()
@@ -176,7 +176,7 @@ func (e *Engine) Link(ctx context.Context, matchID int64, opt LinkOptions) (stor
return store.Operation{}, err
}
if identical {
same, err := fsops.SameContent(s.hostAbs, s.slaveAbs)
same, err := fsops.SameContent(s.masterAbs, s.slaveAbs)
if err != nil {
return store.Operation{}, err
}
@@ -186,7 +186,7 @@ func (e *Engine) Link(ctx context.Context, matchID int64, opt LinkOptions) (stor
}
}
newRel := filepath.Join(filepath.Dir(m.RelSlave), filepath.Base(m.RelHost))
newRel := filepath.Join(filepath.Dir(m.RelSlave), filepath.Base(m.RelMaster))
newAbs, err := l.slave.Resolve(newRel)
if err != nil {
return store.Operation{}, err
@@ -198,7 +198,7 @@ func (e *Engine) Link(ctx context.Context, matchID int64, opt LinkOptions) (stor
}
}
o := store.Operation{PairID: l.pair.ID, Action: store.ActLink, RelHost: m.RelHost, RelSlave: m.RelSlave,
o := store.Operation{PairID: l.pair.ID, Action: store.ActLink, RelMaster: m.RelMaster, RelSlave: m.RelSlave,
RelNew: newRel, Trigger: opt.Trigger}
if identical {
o.Bytes = s.si.Size // space freed
@@ -226,7 +226,7 @@ func (e *Engine) Link(ctx context.Context, matchID int64, opt LinkOptions) (stor
}
}
if !rename {
if err := fsops.ReplaceWithLink(s.hostAbs, s.hi, s.slaveAbs); err != nil {
if err := fsops.ReplaceWithLink(s.masterAbs, s.mi, s.slaveAbs); err != nil {
if q != "" {
if cur, e2 := fsops.Lstat(s.slaveAbs); e2 == nil && cur.SameInode(s.si) {
dropQuarantine(l, q, s.si) // original untouched
@@ -237,7 +237,7 @@ func (e *Engine) Link(ctx context.Context, matchID int64, opt LinkOptions) (stor
}
return nil
}
if err := fsops.LinkNew(s.hostAbs, s.hi, newAbs); err != nil {
if err := fsops.LinkNew(s.masterAbs, s.mi, newAbs); err != nil {
if q != "" {
dropQuarantine(l, q, s.si)
e.clearQuarantine(ctx, o.ID)
@@ -259,7 +259,7 @@ func (e *Engine) clearQuarantine(ctx context.Context, opID int64) {
}
// Promote is the override path: the slave copy is better, so it replaces the
// host file (whose original is quarantined) and both libraries share it.
// master file (whose original is quarantined) and both libraries share it.
func (e *Engine) Promote(ctx context.Context, matchID int64, confirmed bool) (store.Operation, error) {
if !confirmed {
return store.Operation{}, ErrConfirm
@@ -282,20 +282,20 @@ func (e *Engine) Promote(ctx context.Context, matchID int64, confirmed bool) (st
if err != nil {
return store.Operation{}, err
}
newHostRel := filepath.Join(filepath.Dir(m.RelHost), filepath.Base(m.RelSlave))
newHostAbs, err := l.host.Resolve(newHostRel)
newMasterRel := filepath.Join(filepath.Dir(m.RelMaster), filepath.Base(m.RelSlave))
newMasterAbs, err := l.master.Resolve(newMasterRel)
if err != nil {
return store.Operation{}, err
}
rename := newHostRel != m.RelHost
rename := newMasterRel != m.RelMaster
if rename {
if _, err := os.Lstat(newHostAbs); err == nil {
return store.Operation{}, fmt.Errorf("host already has %s: %w", newHostRel, fsops.ErrExists)
if _, err := os.Lstat(newMasterAbs); err == nil {
return store.Operation{}, fmt.Errorf("master already has %s: %w", newMasterRel, fsops.ErrExists)
}
}
o := store.Operation{PairID: l.pair.ID, Action: store.ActPromote, RelHost: m.RelHost, RelSlave: m.RelSlave,
RelNew: newHostRel, QuarantineSide: "host", Trigger: "manual"}
o := store.Operation{PairID: l.pair.ID, Action: store.ActPromote, RelMaster: m.RelMaster, RelSlave: m.RelSlave,
RelNew: newMasterRel, QuarantineSide: "master", Trigger: "manual"}
if err := e.begin(ctx, &o); err != nil {
return o, err
}
@@ -303,31 +303,31 @@ func (e *Engine) Promote(ctx context.Context, matchID int64, confirmed bool) (st
return o, nil
}
err = func() error {
q := quarantinePath(l, o.ID, "host", m.RelHost)
if err := quarantine(s.hostAbs, s.hi, q); err != nil {
q := quarantinePath(l, o.ID, "master", m.RelMaster)
if err := quarantine(s.masterAbs, s.mi, q); err != nil {
return fmt.Errorf("quarantine: %w", err)
}
o.QuarantinePath = q
if _, err := e.st.DB.ExecContext(ctx, `UPDATE operations SET quarantine_path = ? WHERE id = ?`, q, o.ID); err != nil {
dropQuarantine(l, q, s.hi)
dropQuarantine(l, q, s.mi)
return err
}
if !rename {
if err := fsops.ReplaceWithLink(s.slaveAbs, s.si, s.hostAbs); err != nil {
if cur, e2 := fsops.Lstat(s.hostAbs); e2 == nil && cur.SameInode(s.hi) {
dropQuarantine(l, q, s.hi)
if err := fsops.ReplaceWithLink(s.slaveAbs, s.si, s.masterAbs); err != nil {
if cur, e2 := fsops.Lstat(s.masterAbs); e2 == nil && cur.SameInode(s.mi) {
dropQuarantine(l, q, s.mi)
e.clearQuarantine(ctx, o.ID)
}
return err
}
return nil
}
if err := fsops.LinkNew(s.slaveAbs, s.si, newHostAbs); err != nil {
dropQuarantine(l, q, s.hi)
if err := fsops.LinkNew(s.slaveAbs, s.si, newMasterAbs); err != nil {
dropQuarantine(l, q, s.mi)
e.clearQuarantine(ctx, o.ID)
return err
}
return removeOriginal(s.hostAbs, s.hi)
return removeOriginal(s.masterAbs, s.mi)
}()
err = e.finish(ctx, &o, err)
e.refresh(ctx, l, m.RelSlave)
@@ -368,15 +368,15 @@ func expand(ctx context.Context, l *libs, root fsops.Root, rels []string) ([]str
return sortedKeys(set), nil
}
// StartBuild hardlinks the chosen host files/folders into the slave library,
// mirroring the host path. Existing slave files are never overwritten.
// StartBuild hardlinks the chosen master files/folders into the slave library,
// mirroring the master path. Existing slave files are never overwritten.
func (e *Engine) StartBuild(pairID int64, rels []string) (*Job, error) {
return e.startJob(pairID, "build", func(ctx context.Context, j *Job) error {
l, err := e.loadPair(ctx, pairID)
if err != nil {
return err
}
files, err := expand(ctx, l, l.host, rels)
files, err := expand(ctx, l, l.master, rels)
if err != nil {
return err
}
@@ -395,7 +395,7 @@ func (e *Engine) StartBuild(pairID int64, rels []string) (*Job, error) {
func (e *Engine) buildOne(ctx context.Context, j *Job, l *libs, rel string) {
e.mu.Lock()
defer e.mu.Unlock()
hostAbs, err := l.host.Resolve(rel)
masterAbs, err := l.master.Resolve(rel)
if err != nil {
j.skip(rel + ": " + err.Error())
return
@@ -405,20 +405,20 @@ func (e *Engine) buildOne(ctx context.Context, j *Job, l *libs, rel string) {
j.skip(rel + ": " + err.Error())
return
}
hi, err := fsops.RegularFile(hostAbs)
mi, err := fsops.RegularFile(masterAbs)
if err != nil {
j.skip(rel + ": " + err.Error())
return
}
if si, err := fsops.Lstat(slaveAbs); err == nil {
if si.SameInode(hi) {
if si.SameInode(mi) {
j.count("already linked")
} else {
j.skip(rel + ": a different file already exists in the slave library (scan, then use Review)")
}
return
}
o := store.Operation{PairID: l.pair.ID, Action: store.ActBuild, RelHost: rel, RelSlave: rel, RelNew: rel, Trigger: "build"}
o := store.Operation{PairID: l.pair.ID, Action: store.ActBuild, RelMaster: rel, RelSlave: rel, RelNew: rel, Trigger: "build"}
if err := e.begin(ctx, &o); err != nil {
j.skip(rel + ": " + err.Error())
return
@@ -429,7 +429,7 @@ func (e *Engine) buildOne(ctx context.Context, j *Job, l *libs, rel string) {
}
err = fsops.MkdirAllLike(filepath.Dir(slaveAbs))
if err == nil {
err = fsops.LinkNew(hostAbs, hi, slaveAbs)
err = fsops.LinkNew(masterAbs, mi, slaveAbs)
}
if err = e.finish(ctx, &o, err); err != nil {
j.skip(rel + ": " + err.Error())
@@ -439,7 +439,7 @@ func (e *Engine) buildOne(ctx context.Context, j *Job, l *libs, rel string) {
e.refresh(ctx, l, rel)
}
// StartRemove deletes slave files that are hardlinks of the host file at the
// StartRemove deletes slave files that are hardlinks of the master file at the
// same path. Anything that is not such a link is left alone.
func (e *Engine) StartRemove(pairID int64, rels []string) (*Job, error) {
return e.startJob(pairID, "remove", func(ctx context.Context, j *Job) error {
@@ -466,15 +466,15 @@ func (e *Engine) StartRemove(pairID int64, rels []string) (*Job, error) {
func (e *Engine) removeOne(ctx context.Context, j *Job, l *libs, rel string) {
e.mu.Lock()
defer e.mu.Unlock()
hostAbs, err1 := l.host.Resolve(rel)
masterAbs, err1 := l.master.Resolve(rel)
slaveAbs, err2 := l.slave.Resolve(rel)
if err1 != nil || err2 != nil {
j.skip(rel + ": invalid path")
return
}
hi, err := fsops.RegularFile(hostAbs)
mi, err := fsops.RegularFile(masterAbs)
if err != nil {
j.skip(rel + ": no host file at the same path, so it is not removed")
j.skip(rel + ": no master file at the same path, so it is not removed")
return
}
si, err := fsops.RegularFile(slaveAbs)
@@ -482,11 +482,11 @@ func (e *Engine) removeOne(ctx context.Context, j *Job, l *libs, rel string) {
j.skip(rel + ": " + err.Error())
return
}
if !si.SameInode(hi) {
j.skip(rel + ": not a hardlink of the host file, so it is not removed")
if !si.SameInode(mi) {
j.skip(rel + ": not a hardlink of the master file, so it is not removed")
return
}
o := store.Operation{PairID: l.pair.ID, Action: store.ActRemove, RelHost: rel, RelSlave: rel, Trigger: "build"}
o := store.Operation{PairID: l.pair.ID, Action: store.ActRemove, RelMaster: rel, RelSlave: rel, Trigger: "build"}
if err := e.begin(ctx, &o); err != nil {
j.skip(rel + ": " + err.Error())
return
@@ -495,7 +495,7 @@ func (e *Engine) removeOne(ctx context.Context, j *Job, l *libs, rel string) {
j.count("planned")
return
}
err = fsops.RemoveLink(slaveAbs, hi)
err = fsops.RemoveLink(slaveAbs, mi)
if err = e.finish(ctx, &o, err); err != nil {
j.skip(rel + ": " + err.Error())
return
@@ -533,7 +533,7 @@ func (e *Engine) Undo(ctx context.Context, opID int64) error {
}
return p
}
hostAbs := res(l.host, o.RelHost)
masterAbs := res(l.master, o.RelMaster)
slaveAbs := res(l.slave, o.RelSlave)
if err != nil {
return err
@@ -541,24 +541,24 @@ func (e *Engine) Undo(ctx context.Context, opID int64) error {
switch o.Action {
case store.ActBuild:
hi, err := fsops.RegularFile(hostAbs)
mi, err := fsops.RegularFile(masterAbs)
if err != nil {
return err
}
if err := fsops.RemoveLink(slaveAbs, hi); err != nil {
if err := fsops.RemoveLink(slaveAbs, mi); err != nil {
return err
}
pruneEmpty(filepath.Dir(slaveAbs), l.slave.Path)
case store.ActRemove:
hi, err := fsops.RegularFile(hostAbs)
mi, err := fsops.RegularFile(masterAbs)
if err != nil {
return err
}
if err := fsops.MkdirAllLike(filepath.Dir(slaveAbs)); err != nil {
return err
}
if err := fsops.LinkNew(hostAbs, hi, slaveAbs); err != nil {
if err := fsops.LinkNew(masterAbs, mi, slaveAbs); err != nil {
return err
}
@@ -567,7 +567,7 @@ func (e *Engine) Undo(ctx context.Context, opID int64) error {
if err != nil {
return err
}
hi, err := fsops.RegularFile(hostAbs)
mi, err := fsops.RegularFile(masterAbs)
if err != nil {
return err
}
@@ -575,8 +575,8 @@ func (e *Engine) Undo(ctx context.Context, opID int64) error {
if err != nil {
return err
}
if !cur.SameInode(hi) {
return errors.New("the slave file is no longer linked to the host file; refusing to undo")
if !cur.SameInode(mi) {
return errors.New("the slave file is no longer linked to the master file; refusing to undo")
}
renamed := o.RelNew != o.RelSlave
if renamed {
@@ -586,7 +586,7 @@ func (e *Engine) Undo(ctx context.Context, opID int64) error {
}
if o.QuarantinePath == "" {
// Identical link: give the slave its own copy of the bytes again.
if err := fsops.CopyReplace(hostAbs, slaveAbs, hi.Mode); err != nil {
if err := fsops.CopyReplace(masterAbs, slaveAbs, mi.Mode); err != nil {
return err
}
} else {
@@ -605,14 +605,14 @@ func (e *Engine) Undo(ctx context.Context, opID int64) error {
dropQuarantine(l, o.QuarantinePath, qi)
}
if renamed {
if err := fsops.RemoveLink(newAbs, hi); err != nil {
if err := fsops.RemoveLink(newAbs, mi); err != nil {
return err
}
}
e.refresh(ctx, l, o.RelNew)
case store.ActPromote:
newHostAbs := res(l.host, o.RelNew)
newMasterAbs := res(l.master, o.RelNew)
if err != nil {
return err
}
@@ -620,25 +620,25 @@ func (e *Engine) Undo(ctx context.Context, opID int64) error {
if err != nil {
return err
}
cur, err := fsops.RegularFile(newHostAbs)
cur, err := fsops.RegularFile(newMasterAbs)
if err != nil {
return err
}
if !cur.SameInode(si) {
return errors.New("the host file changed after promotion; refusing to undo")
return errors.New("the master file changed after promotion; refusing to undo")
}
qi, err := e.quarantined(l, o.QuarantinePath)
if err != nil {
return err
}
if o.RelNew == o.RelHost {
err = fsops.ReplaceWithLink(o.QuarantinePath, qi, hostAbs)
if o.RelNew == o.RelMaster {
err = fsops.ReplaceWithLink(o.QuarantinePath, qi, masterAbs)
} else {
if _, e2 := os.Lstat(hostAbs); e2 == nil {
return fmt.Errorf("%s: %w", o.RelHost, fsops.ErrExists)
if _, e2 := os.Lstat(masterAbs); e2 == nil {
return fmt.Errorf("%s: %w", o.RelMaster, fsops.ErrExists)
}
if err = fsops.LinkNew(o.QuarantinePath, qi, hostAbs); err == nil {
err = fsops.RemoveLink(newHostAbs, si)
if err = fsops.LinkNew(o.QuarantinePath, qi, masterAbs); err == nil {
err = fsops.RemoveLink(newMasterAbs, si)
}
}
if err != nil {
@@ -732,8 +732,8 @@ func (e *Engine) Recover(ctx context.Context) {
}
for _, o := range ops {
if p, err := e.st.Pair(ctx, o.PairID); err == nil {
for _, root := range []string{p.SlaveRoot, p.HostRoot} {
for _, rel := range []string{o.RelSlave, o.RelNew, o.RelHost} {
for _, root := range []string{p.SlaveRoot, p.MasterRoot} {
for _, rel := range []string{o.RelSlave, o.RelNew, o.RelMaster} {
if rel == "" {
continue
}
@@ -763,15 +763,15 @@ type TreeEntry struct {
Linked int `json:"linked,omitempty"`
}
// Tree lists a host directory and shows how much of it is in the slave.
// Tree lists a master directory and shows how much of it is in the slave.
func (e *Engine) Tree(ctx context.Context, pairID int64, rel string) ([]TreeEntry, error) {
l, err := e.loadPair(ctx, pairID)
if err != nil {
return nil, err
}
dir := l.host.Path
dir := l.master.Path
if rel != "" {
if dir, err = l.host.Resolve(rel); err != nil {
if dir, err = l.master.Resolve(rel); err != nil {
return nil, err
}
}
@@ -779,7 +779,7 @@ func (e *Engine) Tree(ctx context.Context, pairID int64, rel string) ([]TreeEntr
if err != nil {
return nil, err
}
linked, _ := e.st.LinkedHostPaths(ctx, pairID, rel)
linked, _ := e.st.LinkedMasterPaths(ctx, pairID, rel)
perChild := map[string]int{}
for _, p := range linked {
sub := p
@@ -807,16 +807,16 @@ func (e *Engine) Tree(ctx context.Context, pairID int64, rel string) ([]TreeEntr
if !l.hasExt(name) {
continue
}
hi, err := fsops.Lstat(filepath.Join(dir, name))
mi, err := fsops.Lstat(filepath.Join(dir, name))
if err != nil {
continue
}
te.Size = hi.Size
te.Size = mi.Size
te.State = "missing"
if sp, err := l.slave.Resolve(child); err == nil {
if si, err := fsops.Lstat(sp); err == nil {
te.State = "conflict"
if si.SameInode(hi) {
if si.SameInode(mi) {
te.State = "linked"
}
}
internal/engine/scan.go
+45 -45
View File
@@ -97,17 +97,17 @@ func firstComponent(rel string) string {
return ""
}
// hostIndex supports the fallback lookups for slave files without an exact
// masterIndex supports the fallback lookups for slave files without an exact
// path counterpart.
type hostIndex struct {
type masterIndex struct {
files map[string]fsops.Info
lower map[string]string // lower(rel) -> rel
byDir map[string][]string // lower(dir) -> rels
byTitle map[string][]string // lower(top folder)|normTitle -> rels
}
func buildIndex(files map[string]fsops.Info) *hostIndex {
ix := &hostIndex{files: files, lower: map[string]string{}, byDir: map[string][]string{}, byTitle: map[string][]string{}}
func buildIndex(files map[string]fsops.Info) *masterIndex {
ix := &masterIndex{files: files, lower: map[string]string{}, byDir: map[string][]string{}, byTitle: map[string][]string{}}
for _, rel := range sortedKeys(files) {
ix.lower[strings.ToLower(rel)] = rel
d := strings.ToLower(filepath.Dir(rel))
@@ -118,7 +118,7 @@ func buildIndex(files map[string]fsops.Info) *hostIndex {
return ix
}
// losslessRank orders extension candidates so a lossless host file is
// losslessRank orders extension candidates so a lossless master file is
// preferred when several share the same title.
func extRank(rel string) int {
switch strings.ToLower(filepath.Ext(rel)) {
@@ -130,8 +130,8 @@ func extRank(rel string) int {
return 2
}
// find locates the host counterpart of a slave file.
func (ix *hostIndex) find(relSlave string) (string, string) {
// find locates the master counterpart of a slave file.
func (ix *masterIndex) find(relSlave string) (string, string) {
if _, ok := ix.files[relSlave]; ok {
return relSlave, store.KindPath
}
@@ -173,40 +173,40 @@ func (ix *hostIndex) find(relSlave string) (string, string) {
// classify fills in status, probes and verdict for a match, reusing cached
// results from prev when neither file changed.
func (e *Engine) classify(ctx context.Context, l *libs, m *store.Match, hi, si fsops.Info, prev *store.Match) {
func (e *Engine) classify(ctx context.Context, l *libs, m *store.Match, mi, si fsops.Info, prev *store.Match) {
m.SlaveIno, m.SlaveSize, m.SlaveMtime = si.Ino, si.Size, si.MtimeNs
m.HostIno, m.HostSize, m.HostMtime = 0, 0, 0
m.HostProbe, m.SlaveProbe, m.Verdict, m.Note = nil, nil, nil, ""
m.MasterIno, m.MasterSize, m.MasterMtime = 0, 0, 0
m.MasterProbe, m.SlaveProbe, m.Verdict, m.Note = nil, nil, nil, ""
if m.Kind == store.KindNone {
m.Status = store.StatusUnmatched
return
}
m.HostIno, m.HostSize, m.HostMtime = hi.Ino, hi.Size, hi.MtimeNs
m.MasterIno, m.MasterSize, m.MasterMtime = mi.Ino, mi.Size, mi.MtimeNs
unchanged := prev != nil && prev.RelHost == m.RelHost &&
prev.HostIno == hi.Ino && prev.HostSize == hi.Size && prev.HostMtime == hi.MtimeNs &&
unchanged := prev != nil && prev.RelMaster == m.RelMaster &&
prev.MasterIno == mi.Ino && prev.MasterSize == mi.Size && prev.MasterMtime == mi.MtimeNs &&
prev.SlaveIno == si.Ino && prev.SlaveSize == si.Size && prev.SlaveMtime == si.MtimeNs
if prev != nil && prev.RelHost == m.RelHost {
if prev != nil && prev.RelMaster == m.RelMaster {
m.Ignored = prev.Ignored
}
switch {
case hi.Dev != si.Dev:
case mi.Dev != si.Dev:
m.Status, m.Note = store.StatusError, "files are on different filesystems"
return
case hi.SameInode(si):
case mi.SameInode(si):
m.Status = store.StatusLinked
return
}
pathKind := m.Kind == store.KindPath || m.Kind == store.KindPathCase
if pathKind && hi.Size == si.Size {
if pathKind && mi.Size == si.Size {
if unchanged && (prev.Status == store.StatusIdentical || prev.Status == store.StatusDifferent) {
m.Status = prev.Status
} else {
hostAbs := filepath.Join(l.host.Path, m.RelHost)
masterAbs := filepath.Join(l.master.Path, m.RelMaster)
slaveAbs := filepath.Join(l.slave.Path, m.RelSlave)
same, err := fsops.SameContent(hostAbs, slaveAbs)
same, err := fsops.SameContent(masterAbs, slaveAbs)
switch {
case err != nil:
m.Status, m.Note = store.StatusError, "compare failed: "+err.Error()
@@ -227,14 +227,14 @@ func (e *Engine) classify(ctx context.Context, l *libs, m *store.Match, hi, si f
}
// Manual review: gather codec details for both sides.
if unchanged && len(prev.HostProbe) > 0 && len(prev.SlaveProbe) > 0 {
m.HostProbe, m.SlaveProbe, m.Verdict = prev.HostProbe, prev.SlaveProbe, prev.Verdict
if unchanged && len(prev.MasterProbe) > 0 && len(prev.SlaveProbe) > 0 {
m.MasterProbe, m.SlaveProbe, m.Verdict = prev.MasterProbe, prev.SlaveProbe, prev.Verdict
return
}
hp := e.prober.Probe(ctx, filepath.Join(l.host.Path, m.RelHost))
mp := e.prober.Probe(ctx, filepath.Join(l.master.Path, m.RelMaster))
sp := e.prober.Probe(ctx, filepath.Join(l.slave.Path, m.RelSlave))
v := probe.Compare(hp, sp)
m.HostProbe, _ = json.Marshal(hp)
v := probe.Compare(mp, sp)
m.MasterProbe, _ = json.Marshal(mp)
m.SlaveProbe, _ = json.Marshal(sp)
m.Verdict, _ = json.Marshal(v)
}
@@ -255,9 +255,9 @@ func (e *Engine) scan(ctx context.Context, j *Job, pairID int64, trigger string)
}
warns := layoutWarnings(l)
j.phase("Indexing host library", 0)
hostFiles := map[string]fsops.Info{}
hs, err := walk(ctx, l, l.host, hostFiles)
j.phase("Indexing master library", 0)
masterFiles := map[string]fsops.Info{}
ms, err := walk(ctx, l, l.master, masterFiles)
if err != nil {
_ = e.st.FinishScan(ctx, pairID, started.Unix(), err.Error(), warns)
return err
@@ -269,7 +269,7 @@ func (e *Engine) scan(ctx context.Context, j *Job, pairID int64, trigger string)
_ = e.st.FinishScan(ctx, pairID, started.Unix(), err.Error(), warns)
return err
}
ix := buildIndex(hostFiles)
ix := buildIndex(masterFiles)
prev, err := e.st.AllMatches(ctx, pairID)
if err != nil {
return err
@@ -284,12 +284,12 @@ func (e *Engine) scan(ctx context.Context, j *Job, pairID int64, trigger string)
}
si := slaveFiles[rel]
m := store.Match{PairID: pairID, RelSlave: rel}
m.RelHost, m.Kind = ix.find(rel)
m.RelMaster, m.Kind = ix.find(rel)
var pm *store.Match
if p, ok := prev[rel]; ok {
pm = &p
}
e.classify(ctx, l, &m, hostFiles[m.RelHost], si, pm)
e.classify(ctx, l, &m, masterFiles[m.RelMaster], si, pm)
if err := e.st.UpsertMatch(ctx, &m); err != nil {
return err
}
@@ -305,12 +305,12 @@ func (e *Engine) scan(ctx context.Context, j *Job, pairID int64, trigger string)
// Layout warnings based on what was actually found.
if ss.files > 0 {
if n := kinds[store.KindElsewhere] + kinds[store.KindNone]; n*4 > ss.files {
warns = append(warns, fmt.Sprintf("%d of %d slave files have no counterpart at the same path in the host library. "+
warns = append(warns, fmt.Sprintf("%d of %d slave files have no counterpart at the same path in the master library. "+
"Check that both libraries use the same folder layout.", n, ss.files))
}
}
if hs.symlinks+ss.symlinks > 0 {
warns = append(warns, fmt.Sprintf("Ignored %d symlinks (playlink never follows or replaces symlinks).", hs.symlinks+ss.symlinks))
if ms.symlinks+ss.symlinks > 0 {
warns = append(warns, fmt.Sprintf("Ignored %d symlinks (playlink never follows or replaces symlinks).", ms.symlinks+ss.symlinks))
}
if e.prober.Bin == "" {
warns = append(warns, "ffprobe was not found, so codec and quality details are unavailable for manual review.")
@@ -319,7 +319,7 @@ func (e *Engine) scan(ctx context.Context, j *Job, pairID int64, trigger string)
return err
}
j.update(func(j *Job) {
j.Message = fmt.Sprintf("Scanned %d host and %d slave files in %s", hs.files, ss.files, time.Since(started).Round(time.Second))
j.Message = fmt.Sprintf("Scanned %d master and %d slave files in %s", ms.files, ss.files, time.Since(started).Round(time.Second))
})
if l.pair.AutoLinkIdentical {
@@ -367,7 +367,7 @@ func (e *Engine) linkIDs(ctx context.Context, j *Job, pairID int64, ids []int64,
j.tick()
continue
}
if settle > 0 && (m.SlaveMtime > cutoff || m.HostMtime > cutoff) {
if settle > 0 && (m.SlaveMtime > cutoff || m.MasterMtime > cutoff) {
j.skip(m.RelSlave + ": modified recently, waiting for it to settle")
j.tick()
continue
@@ -415,24 +415,24 @@ func (e *Engine) refresh(ctx context.Context, l *libs, relSlave string) {
if p, err := e.st.MatchBySlave(ctx, l.pair.ID, relSlave); err == nil {
pm = &p
}
var hi fsops.Info
if hp, err := l.host.Resolve(relSlave); err == nil {
if in, err := fsops.RegularFile(hp); err == nil {
hi = in
m.RelHost, m.Kind = relSlave, store.KindPath
var mi fsops.Info
if mp, err := l.master.Resolve(relSlave); err == nil {
if in, err := fsops.RegularFile(mp); err == nil {
mi = in
m.RelMaster, m.Kind = relSlave, store.KindPath
}
}
if m.Kind == "" {
if pm != nil && pm.RelHost != "" {
if in, err := fsops.RegularFile(filepath.Join(l.host.Path, pm.RelHost)); err == nil {
hi = in
m.RelHost, m.Kind = pm.RelHost, pm.Kind
if pm != nil && pm.RelMaster != "" {
if in, err := fsops.RegularFile(filepath.Join(l.master.Path, pm.RelMaster)); err == nil {
mi = in
m.RelMaster, m.Kind = pm.RelMaster, pm.Kind
}
}
}
if m.Kind == "" {
m.Kind = store.KindNone
}
e.classify(ctx, l, &m, hi, si, pm)
e.classify(ctx, l, &m, mi, si, pm)
_ = e.st.UpsertMatch(ctx, &m)
}
internal/fsops/fsops.go
+1 -1
View File
@@ -23,7 +23,7 @@ import (
)
// TmpPrefix marks staging files created by playlink. Anything carrying this
// prefix is disposable: it is either a hardlink to a host file or a copy that
// prefix is disposable: it is either a hardlink to a master file or a copy that
// has not been renamed into place yet.
const TmpPrefix = ".playlink-tmp-"
internal/probe/probe.go
+19 -19
View File
@@ -172,48 +172,48 @@ func Score(in Info) float64 {
return float64(in.Bitrate) / 1000 * codecEfficiency(in.Codec)
}
// Verdict compares host and slave quality.
// Verdict compares master and slave quality.
type Verdict struct {
// Better is "host", "slave", "equal" or "unknown".
// Better is "master", "slave", "equal" or "unknown".
Better string `json:"better"`
// DurationDelta is |host-slave| in seconds; large deltas suggest a
// DurationDelta is |master-slave| in seconds; large deltas suggest a
// different edit or version of the song rather than a re-encode.
DurationDelta float64 `json:"duration_delta"`
Notes []string `json:"notes"`
}
// Compare ranks host against slave. Within 5% is treated as equal.
func Compare(host, slave Info) Verdict {
// Compare ranks master against slave. Within 5% is treated as equal.
func Compare(master, slave Info) Verdict {
v := Verdict{Better: "unknown"}
if host.Error != "" || slave.Error != "" {
if master.Error != "" || slave.Error != "" {
v.Notes = append(v.Notes, "could not probe one of the files")
return v
}
if host.Duration > 0 && slave.Duration > 0 {
v.DurationDelta = math.Abs(host.Duration - slave.Duration)
if master.Duration > 0 && slave.Duration > 0 {
v.DurationDelta = math.Abs(master.Duration - slave.Duration)
if v.DurationDelta > 2 {
v.Notes = append(v.Notes, fmt.Sprintf("durations differ by %.1fs: possibly a different version or edit", v.DurationDelta))
}
}
hs, ss := Score(host), Score(slave)
ms, ss := Score(master), Score(slave)
switch {
case hs <= 0 || ss <= 0:
case math.Abs(hs-ss) <= 0.05*math.Max(hs, ss):
case ms <= 0 || ss <= 0:
case math.Abs(ms-ss) <= 0.05*math.Max(ms, ss):
v.Better = "equal"
case hs > ss:
v.Better = "host"
case ms > ss:
v.Better = "master"
default:
v.Better = "slave"
}
if host.Lossless != slave.Lossless {
if host.Lossless {
v.Notes = append(v.Notes, "host is lossless, slave is lossy")
if master.Lossless != slave.Lossless {
if master.Lossless {
v.Notes = append(v.Notes, "master is lossless, slave is lossy")
} else {
v.Notes = append(v.Notes, "slave is lossless but host is lossy: linking would downgrade the slave")
v.Notes = append(v.Notes, "slave is lossless but master is lossy: linking would downgrade the slave")
}
}
if host.Channels != slave.Channels && host.Channels > 0 && slave.Channels > 0 {
v.Notes = append(v.Notes, fmt.Sprintf("channel count differs (%d vs %d)", host.Channels, slave.Channels))
if master.Channels != slave.Channels && master.Channels > 0 && slave.Channels > 0 {
v.Notes = append(v.Notes, fmt.Sprintf("channel count differs (%d vs %d)", master.Channels, slave.Channels))
}
return v
}
internal/probe/probe_test.go
+1 -1
View File
@@ -52,6 +52,6 @@ func TestProbeRealFiles(t *testing.T) {
t.Fatalf("unexpected probe results %+v / %+v", fi, mi)
}
if v := Compare(mi, fi); v.Better != "slave" {
t.Fatalf("flac slave should beat mp3 host, got %+v", v)
t.Fatalf("flac slave should beat mp3 master, got %+v", v)
}
}
internal/store/models.go
+42 -42
View File
@@ -96,7 +96,7 @@ func (s *Store) PruneSessions(ctx context.Context) {
type Pair struct {
ID int64 `json:"id"`
Name string `json:"name"`
HostRoot string `json:"host_root"`
MasterRoot string `json:"master_root"`
SlaveRoot string `json:"slave_root"`
QuarantineDir string `json:"quarantine_dir"`
Extensions []string `json:"extensions"`
@@ -110,7 +110,7 @@ type Pair struct {
CreatedAt int64 `json:"created_at"`
}
const pairCols = `id, name, host_root, slave_root, quarantine_dir, extensions, auto_link_identical,
const pairCols = `id, name, master_root, slave_root, quarantine_dir, extensions, auto_link_identical,
scan_interval_minutes, retention_days, settle_minutes, last_scan_at, last_scan_error, last_scan_warnings, created_at`
type scanner interface{ Scan(...any) error }
@@ -119,7 +119,7 @@ func scanPair(r scanner) (Pair, error) {
var p Pair
var exts, warns string
var last sql.NullInt64
err := r.Scan(&p.ID, &p.Name, &p.HostRoot, &p.SlaveRoot, &p.QuarantineDir, &exts, &p.AutoLinkIdentical,
err := r.Scan(&p.ID, &p.Name, &p.MasterRoot, &p.SlaveRoot, &p.QuarantineDir, &exts, &p.AutoLinkIdentical,
&p.ScanIntervalMinutes, &p.RetentionDays, &p.SettleMinutes, &last, &p.LastScanError, &warns, &p.CreatedAt)
if err != nil {
return p, err
@@ -163,10 +163,10 @@ func (s *Store) Pair(ctx context.Context, id int64) (Pair, error) {
func (s *Store) SavePair(ctx context.Context, p *Pair) error {
exts := strings.Join(p.Extensions, ",")
if p.ID == 0 {
res, err := s.DB.ExecContext(ctx, `INSERT INTO pairs (name, host_root, slave_root, quarantine_dir, extensions,
res, err := s.DB.ExecContext(ctx, `INSERT INTO pairs (name, master_root, slave_root, quarantine_dir, extensions,
auto_link_identical, scan_interval_minutes, retention_days, settle_minutes, created_at)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`,
p.Name, p.HostRoot, p.SlaveRoot, p.QuarantineDir, exts, p.AutoLinkIdentical,
p.Name, p.MasterRoot, p.SlaveRoot, p.QuarantineDir, exts, p.AutoLinkIdentical,
p.ScanIntervalMinutes, p.RetentionDays, p.SettleMinutes, now())
if err != nil {
return err
@@ -174,10 +174,10 @@ func (s *Store) SavePair(ctx context.Context, p *Pair) error {
p.ID, _ = res.LastInsertId()
return nil
}
_, err := s.DB.ExecContext(ctx, `UPDATE pairs SET name = ?, host_root = ?, slave_root = ?, quarantine_dir = ?,
_, err := s.DB.ExecContext(ctx, `UPDATE pairs SET name = ?, master_root = ?, slave_root = ?, quarantine_dir = ?,
extensions = ?, auto_link_identical = ?, scan_interval_minutes = ?, retention_days = ?, settle_minutes = ?
WHERE id = ?`,
p.Name, p.HostRoot, p.SlaveRoot, p.QuarantineDir, exts, p.AutoLinkIdentical,
p.Name, p.MasterRoot, p.SlaveRoot, p.QuarantineDir, exts, p.AutoLinkIdentical,
p.ScanIntervalMinutes, p.RetentionDays, p.SettleMinutes, p.ID)
return err
}
@@ -214,7 +214,7 @@ const (
StatusError = "error" // cannot be linked (e.g. other filesystem)
)
// Match kinds (how the host candidate was found).
// Match kinds (how the master candidate was found).
const (
KindPath = "path" // identical relative path
KindPathCase = "path_case" // relative path differs only by case
@@ -228,16 +228,16 @@ type Match struct {
ID int64 `json:"id"`
PairID int64 `json:"pair_id"`
RelSlave string `json:"rel_slave"`
RelHost string `json:"rel_host"`
RelMaster string `json:"rel_master"`
Kind string `json:"kind"`
Status string `json:"status"`
HostIno uint64 `json:"-"`
HostSize int64 `json:"host_size"`
HostMtime int64 `json:"host_mtime"`
MasterIno uint64 `json:"-"`
MasterSize int64 `json:"master_size"`
MasterMtime int64 `json:"master_mtime"`
SlaveIno uint64 `json:"-"`
SlaveSize int64 `json:"slave_size"`
SlaveMtime int64 `json:"slave_mtime"`
HostProbe json.RawMessage `json:"host_probe,omitempty"`
MasterProbe json.RawMessage `json:"master_probe,omitempty"`
SlaveProbe json.RawMessage `json:"slave_probe,omitempty"`
Verdict json.RawMessage `json:"verdict,omitempty"`
Note string `json:"note"`
@@ -245,16 +245,16 @@ type Match struct {
UpdatedAt int64 `json:"updated_at"`
}
const matchCols = `id, pair_id, rel_slave, rel_host, kind, status, host_ino, host_size, host_mtime,
slave_ino, slave_size, slave_mtime, host_probe, slave_probe, verdict, note, ignored, updated_at`
const matchCols = `id, pair_id, rel_slave, rel_master, kind, status, master_ino, master_size, master_mtime,
slave_ino, slave_size, slave_mtime, master_probe, slave_probe, verdict, note, ignored, updated_at`
func scanMatch(r scanner) (Match, error) {
var m Match
var hp, sp, v string
err := r.Scan(&m.ID, &m.PairID, &m.RelSlave, &m.RelHost, &m.Kind, &m.Status, &m.HostIno, &m.HostSize, &m.HostMtime,
&m.SlaveIno, &m.SlaveSize, &m.SlaveMtime, &hp, &sp, &v, &m.Note, &m.Ignored, &m.UpdatedAt)
if hp != "" {
m.HostProbe = json.RawMessage(hp)
var mp, sp, v string
err := r.Scan(&m.ID, &m.PairID, &m.RelSlave, &m.RelMaster, &m.Kind, &m.Status, &m.MasterIno, &m.MasterSize, &m.MasterMtime,
&m.SlaveIno, &m.SlaveSize, &m.SlaveMtime, &mp, &sp, &v, &m.Note, &m.Ignored, &m.UpdatedAt)
if mp != "" {
m.MasterProbe = json.RawMessage(mp)
}
if sp != "" {
m.SlaveProbe = json.RawMessage(sp)
@@ -321,7 +321,7 @@ func (s *Store) Matches(ctx context.Context, pairID int64, f MatchFilter) ([]Mat
args = append(args, f.Status)
}
if f.Query != "" {
where = append(where, "(rel_slave LIKE ? ESCAPE '\\' OR rel_host LIKE ? ESCAPE '\\')")
where = append(where, "(rel_slave LIKE ? ESCAPE '\\' OR rel_master LIKE ? ESCAPE '\\')")
q := "%" + strings.NewReplacer(`\`, `\\`, `%`, `\%`, `_`, `\_`).Replace(f.Query) + "%"
args = append(args, q, q)
}
@@ -370,16 +370,16 @@ func (s *Store) IdenticalIDs(ctx context.Context, pairID int64) ([]int64, error)
func (s *Store) UpsertMatch(ctx context.Context, m *Match) error {
nullable := func(r json.RawMessage) string { return string(r) }
_, err := s.DB.ExecContext(ctx, `INSERT INTO matches (pair_id, rel_slave, rel_host, kind, status, host_ino, host_size,
host_mtime, slave_ino, slave_size, slave_mtime, host_probe, slave_probe, verdict, note, ignored, updated_at)
_, err := s.DB.ExecContext(ctx, `INSERT INTO matches (pair_id, rel_slave, rel_master, kind, status, master_ino, master_size,
master_mtime, slave_ino, slave_size, slave_mtime, master_probe, slave_probe, verdict, note, ignored, updated_at)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
ON CONFLICT(pair_id, rel_slave) DO UPDATE SET rel_host = excluded.rel_host, kind = excluded.kind,
status = excluded.status, host_ino = excluded.host_ino, host_size = excluded.host_size,
host_mtime = excluded.host_mtime, slave_ino = excluded.slave_ino, slave_size = excluded.slave_size,
slave_mtime = excluded.slave_mtime, host_probe = excluded.host_probe, slave_probe = excluded.slave_probe,
ON CONFLICT(pair_id, rel_slave) DO UPDATE SET rel_master = excluded.rel_master, kind = excluded.kind,
status = excluded.status, master_ino = excluded.master_ino, master_size = excluded.master_size,
master_mtime = excluded.master_mtime, slave_ino = excluded.slave_ino, slave_size = excluded.slave_size,
slave_mtime = excluded.slave_mtime, master_probe = excluded.master_probe, slave_probe = excluded.slave_probe,
verdict = excluded.verdict, note = excluded.note, ignored = excluded.ignored, updated_at = excluded.updated_at`,
m.PairID, m.RelSlave, m.RelHost, m.Kind, m.Status, m.HostIno, m.HostSize, m.HostMtime, m.SlaveIno, m.SlaveSize,
m.SlaveMtime, nullable(m.HostProbe), nullable(m.SlaveProbe), nullable(m.Verdict), m.Note, m.Ignored, now())
m.PairID, m.RelSlave, m.RelMaster, m.Kind, m.Status, m.MasterIno, m.MasterSize, m.MasterMtime, m.SlaveIno, m.SlaveSize,
m.SlaveMtime, nullable(m.MasterProbe), nullable(m.SlaveProbe), nullable(m.Verdict), m.Note, m.Ignored, now())
return err
}
@@ -409,12 +409,12 @@ func (s *Store) SetIgnored(ctx context.Context, id int64, ignored bool) error {
return err
}
// LinkedHostPaths returns host paths already linked into the slave under prefix.
func (s *Store) LinkedHostPaths(ctx context.Context, pairID int64, prefix string) ([]string, error) {
q := `SELECT rel_host FROM matches WHERE pair_id = ? AND status = 'linked'`
// LinkedMasterPaths returns master paths already linked into the slave under prefix.
func (s *Store) LinkedMasterPaths(ctx context.Context, pairID int64, prefix string) ([]string, error) {
q := `SELECT rel_master FROM matches WHERE pair_id = ? AND status = 'linked'`
args := []any{pairID}
if prefix != "" {
q += ` AND rel_host LIKE ? ESCAPE '\'`
q += ` AND rel_master LIKE ? ESCAPE '\'`
args = append(args, strings.NewReplacer(`\`, `\\`, `%`, `\%`, `_`, `\_`).Replace(prefix)+"/%")
}
rows, err := s.DB.QueryContext(ctx, q, args...)
@@ -479,7 +479,7 @@ type Operation struct {
ID int64 `json:"id"`
PairID int64 `json:"pair_id"`
Action string `json:"action"`
RelHost string `json:"rel_host"`
RelMaster string `json:"rel_master"`
RelSlave string `json:"rel_slave"`
RelNew string `json:"rel_new"`
QuarantinePath string `json:"quarantine_path"`
@@ -498,10 +498,10 @@ type Operation struct {
// Operation actions.
const (
ActLink = "link" // slave replaced by hardlink of host
ActPromote = "promote" // host replaced by slave's file, then linked
ActBuild = "build" // host file linked into the slave (fresh library)
ActRemove = "remove" // slave hardlink removed (host copy kept)
ActLink = "link" // slave replaced by hardlink of master
ActPromote = "promote" // master replaced by slave's file, then linked
ActBuild = "build" // master file linked into the slave (fresh library)
ActRemove = "remove" // slave hardlink removed (master copy kept)
)
// Operation statuses.
@@ -513,13 +513,13 @@ const (
OpInterrupted = "interrupted"
)
const opCols = `id, pair_id, action, rel_host, rel_slave, rel_new, quarantine_path, quarantine_side, tmp_path, bytes,
const opCols = `id, pair_id, action, rel_master, rel_slave, rel_new, quarantine_path, quarantine_side, tmp_path, bytes,
dry_run, trigger, status, error, created_at, finished_at, undone_at, purged_at`
func scanOp(r scanner) (Operation, error) {
var o Operation
var fin, und, pur sql.NullInt64
err := r.Scan(&o.ID, &o.PairID, &o.Action, &o.RelHost, &o.RelSlave, &o.RelNew, &o.QuarantinePath, &o.QuarantineSide,
err := r.Scan(&o.ID, &o.PairID, &o.Action, &o.RelMaster, &o.RelSlave, &o.RelNew, &o.QuarantinePath, &o.QuarantineSide,
&o.TmpPath, &o.Bytes, &o.DryRun, &o.Trigger, &o.Status, &o.Error, &o.CreatedAt, &fin, &und, &pur)
if fin.Valid {
o.FinishedAt = &fin.Int64
@@ -535,10 +535,10 @@ func scanOp(r scanner) (Operation, error) {
func (s *Store) CreateOp(ctx context.Context, o *Operation) error {
o.CreatedAt = now()
res, err := s.DB.ExecContext(ctx, `INSERT INTO operations (pair_id, action, rel_host, rel_slave, rel_new, quarantine_path,
res, err := s.DB.ExecContext(ctx, `INSERT INTO operations (pair_id, action, rel_master, rel_slave, rel_new, quarantine_path,
quarantine_side, tmp_path, bytes, dry_run, trigger, status, error, created_at)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`,
o.PairID, o.Action, o.RelHost, o.RelSlave, o.RelNew, o.QuarantinePath, o.QuarantineSide, o.TmpPath, o.Bytes,
o.PairID, o.Action, o.RelMaster, o.RelSlave, o.RelNew, o.QuarantinePath, o.QuarantineSide, o.TmpPath, o.Bytes,
o.DryRun, o.Trigger, o.Status, o.Error, o.CreatedAt)
if err != nil {
return err
internal/store/store.go
+13
View File
@@ -91,6 +91,19 @@ var migrations = []string{
purged_at INTEGER
);
CREATE INDEX operations_pair ON operations(pair_id, created_at DESC);`,
// 2: the "host" library was renamed to "master". Rename in place so
// existing pairs, history and quarantine entries carry over.
`ALTER TABLE pairs RENAME COLUMN host_root TO master_root;
ALTER TABLE matches RENAME COLUMN rel_host TO rel_master;
ALTER TABLE matches RENAME COLUMN host_ino TO master_ino;
ALTER TABLE matches RENAME COLUMN host_size TO master_size;
ALTER TABLE matches RENAME COLUMN host_mtime TO master_mtime;
ALTER TABLE matches RENAME COLUMN host_probe TO master_probe;
ALTER TABLE operations RENAME COLUMN rel_host TO rel_master;
UPDATE operations SET quarantine_side = 'master' WHERE quarantine_side = 'host';
UPDATE matches SET verdict = replace(verdict, 'host', 'master');
UPDATE pairs SET last_scan_warnings = replace(last_scan_warnings, 'host', 'master');`,
}
// Open opens (and migrates) the database in dataDir.
web/src/api.ts
+8 -8
View File
@@ -62,7 +62,7 @@ export type Summary = {
export type Pair = {
id: number;
name: string;
host_root: string;
master_root: string;
slave_root: string;
quarantine_dir: string;
extensions: string[];
@@ -80,7 +80,7 @@ export type Pair = {
export type PairInput = Pick<
Pair,
| "name"
| "host_root"
| "master_root"
| "slave_root"
| "quarantine_dir"
| "extensions"
@@ -106,7 +106,7 @@ export type ProbeInfo = {
error?: string;
};
export type Verdict = { better: "host" | "slave" | "equal" | "unknown"; duration_delta: number; notes: string[] | null };
export type Verdict = { better: "master" | "slave" | "equal" | "unknown"; duration_delta: number; notes: string[] | null };
export type Status = "linked" | "identical" | "different" | "similar" | "unmatched" | "error";
@@ -114,14 +114,14 @@ export type Match = {
id: number;
pair_id: number;
rel_slave: string;
rel_host: string;
rel_master: string;
kind: "path" | "path_case" | "extension" | "fuzzy" | "elsewhere" | "none";
status: Status;
host_size: number;
host_mtime: number;
master_size: number;
master_mtime: number;
slave_size: number;
slave_mtime: number;
host_probe?: ProbeInfo;
master_probe?: ProbeInfo;
slave_probe?: ProbeInfo;
verdict?: Verdict;
note: string;
@@ -132,7 +132,7 @@ export type Operation = {
id: number;
pair_id: number;
action: "link" | "promote" | "build" | "remove";
rel_host: string;
rel_master: string;
rel_slave: string;
rel_new: string;
quarantine_path: string;
web/src/pages/Activity.tsx
+5 -5
View File
@@ -32,8 +32,8 @@ export function ActivityPage() {
}
const actionLabel: Record<Operation["action"], string> = {
link: "Linked slave → host",
promote: "Promoted slave → host",
link: "Linked slave → master",
promote: "Promoted slave → master",
build: "Built into slave",
remove: "Removed from slave",
};
@@ -97,7 +97,7 @@ export function OpsTable({ pairId, quarantine }: { pairId?: number; quarantine?:
<div className="truncate mono text-[13px]" title={o.rel_slave}>
{o.rel_slave}
</div>
{o.rel_new && o.rel_new !== o.rel_slave && o.rel_new !== o.rel_host && (
{o.rel_new && o.rel_new !== o.rel_slave && o.rel_new !== o.rel_master && (
<div className="truncate text-xs text-muted mono">now {o.rel_new}</div>
)}
{o.error && <div className="text-xs text-red-300">{o.error}</div>}
@@ -172,9 +172,9 @@ function undoText(o: Operation) {
? "The original slave file is restored from quarantine and the hardlink is removed."
: "The slave gets its own independent copy of the file again, which uses disk space.";
case "promote":
return "The original host file is restored from quarantine. The slave keeps its own file.";
return "The original master file is restored from quarantine. The slave keeps its own file.";
case "build":
return "The hardlink is removed from the slave library. The host file is not affected.";
return "The hardlink is removed from the slave library. The master file is not affected.";
case "remove":
return "The hardlink is recreated in the slave library.";
}
web/src/pages/Build.tsx
+5 -5
View File
@@ -39,15 +39,15 @@ export function Build({ pair }: { pair: Pair }) {
return (
<div className="space-y-4">
<Alert tone="blue">
Build the slave library from the host. Browse the host library, pick folders or songs, and playlink hardlinks them into the slave
Build the slave library from the master. Browse the master library, pick folders or songs, and playlink hardlinks them into the slave
library at the same relative path. No extra space is used. Existing slave files are never
overwritten, and <b>Remove</b> only deletes slave files that are hardlinks of the host file, so the host copy is always kept.
overwritten, and <b>Remove</b> only deletes slave files that are hardlinks of the master file, so the master copy is always kept.
</Alert>
<div className="flex flex-wrap items-center justify-between gap-3">
<nav className="flex min-w-0 flex-wrap items-center gap-1 text-sm">
<button className="text-accent hover:underline cursor-pointer" onClick={() => setPath("")}>
host
master
</button>
{crumbs.map((c, i) => (
<span key={i} className="flex items-center gap-1">
@@ -117,8 +117,8 @@ export function Build({ pair }: { pair: Pair }) {
</>
) : (
<>
Slave files in the {sel.size} selected item(s) that are <b>hardlinks of the host file</b> will be deleted from{" "}
<code className="mono">{pair.slave_root}</code>. The host copy is kept. Any other slave file is left untouched.
Slave files in the {sel.size} selected item(s) that are <b>hardlinks of the master file</b> will be deleted from{" "}
<code className="mono">{pair.slave_root}</code>. The master copy is kept. Any other slave file is left untouched.
</>
)}
</p>
web/src/pages/Dashboard.tsx
+4 -4
View File
@@ -19,7 +19,7 @@ export function Dashboard() {
<div className="flex items-end justify-between gap-4">
<div>
<h1 className="text-2xl font-semibold tracking-tight">Libraries</h1>
<p className="text-sm text-muted">Each pair links matching songs from a slave library to the host library's copy.</p>
<p className="text-sm text-muted">Each pair links matching songs from a slave library to the master library's copy.</p>
</div>
<Button variant="primary" onClick={() => setCreating(true)}>
<Plus className="size-4" /> New pair
@@ -30,8 +30,8 @@ export function Dashboard() {
{pairs.data?.length === 0 && (
<Card>
<Empty icon={<Library className="size-10" />} title="No library pairs yet">
Create a pair with a <b>host</b> library (e.g. Albums) and a <b>slave</b> library (e.g. your curated Singles). playlink finds songs in
both and replaces the slave copy with a hardlink to the host copy, so each song is stored only once.
Create a pair with a <b>master</b> library (e.g. Albums) and a <b>slave</b> library (e.g. your curated Singles). playlink finds songs in
both and replaces the slave copy with a hardlink to the master copy, so each song is stored only once.
<div className="mt-4">
<Button variant="primary" onClick={() => setCreating(true)}>
<Plus className="size-4" /> Create your first pair
@@ -66,7 +66,7 @@ function PairCard({ pair }: { pair: Pair }) {
</a>
<div className="mt-1 space-y-0.5 text-xs">
<div className="truncate">
<span className="text-muted">host</span> <code className="mono">{pair.host_root}</code>
<span className="text-muted">master</span> <code className="mono">{pair.master_root}</code>
</div>
<div className="truncate">
<span className="text-muted">slave</span> <code className="mono">{pair.slave_root}</code>
web/src/pages/PairForm.tsx
+7 -7
View File
@@ -14,7 +14,7 @@ export function PairForm({ pair, open, onClose }: { pair?: Pair; open: boolean;
const toast = useToast();
const blank: PairInput = {
name: "Music",
host_root: "",
master_root: "",
slave_root: "",
quarantine_dir: "",
extensions: [],
@@ -25,7 +25,7 @@ export function PairForm({ pair, open, onClose }: { pair?: Pair; open: boolean;
};
const [f, setF] = useState<PairInput>(pair ?? blank);
const [exts, setExts] = useState((pair?.extensions ?? []).join(", "));
const [picker, setPicker] = useState<null | "host_root" | "slave_root" | "quarantine_dir">(null);
const [picker, setPicker] = useState<null | "master_root" | "slave_root" | "quarantine_dir">(null);
const [validation, setValidation] = useState<Validation | null>(null);
useEffect(() => {
@@ -40,7 +40,7 @@ export function PairForm({ pair, open, onClose }: { pair?: Pair; open: boolean;
// Send only editable fields: the API rejects unknown keys.
const body = (): PairInput => ({
name: f.name,
host_root: f.host_root,
master_root: f.master_root,
slave_root: f.slave_root,
quarantine_dir: f.quarantine_dir,
auto_link_identical: f.auto_link_identical,
@@ -76,11 +76,11 @@ export function PairForm({ pair, open, onClose }: { pair?: Pair; open: boolean;
</Field>
<div className="grid gap-4 md:grid-cols-2">
<PathField
label="Host library (keeps the files)"
label="Master library (keeps the files)"
hint="Example: Albums. Its files are the source of every hardlink."
value={f.host_root}
onChange={(v) => set("host_root", v)}
onBrowse={() => setPicker("host_root")}
value={f.master_root}
onChange={(v) => set("master_root", v)}
onBrowse={() => setPicker("master_root")}
/>
<PathField
label="Slave library (gets hardlinks)"
web/src/pages/PairPage.tsx
+1 -1
View File
@@ -149,7 +149,7 @@ export function PairPage({ id, tab }: { id: number; tab: string }) {
<h1 className="text-2xl font-semibold tracking-tight">{pair.name}</h1>
<div className="mt-1 flex flex-col gap-0.5 text-xs sm:flex-row sm:gap-4">
<span className="truncate">
<Badge tone="violet">host</Badge> <code className="mono">{pair.host_root}</code>
<Badge tone="violet">master</Badge> <code className="mono">{pair.master_root}</code>
</span>
<span className="truncate">
<Badge>slave</Badge> <code className="mono">{pair.slave_root}</code>
web/src/pages/Review.tsx
+24 -24
View File
@@ -94,7 +94,7 @@ export function Review({ pair }: { pair: Pair }) {
<Card className="flex flex-wrap items-center justify-between gap-3 p-4">
<div className="text-sm">
<b>{identical}</b> byte-identical files ({formatBytes(pair.summary.reclaimable_bytes)}) can be replaced with hardlinks.
<div className="text-xs text-muted">Each one is compared byte for byte again right before linking. No quarantine is needed because the data stays in the host.</div>
<div className="text-xs text-muted">Each one is compared byte for byte again right before linking. No quarantine is needed because the data stays in the master.</div>
</div>
<Button variant="primary" onClick={() => setConfirmBulk(true)} disabled={running}>
<CopyCheck className="size-4" /> Link all identical
@@ -134,8 +134,8 @@ export function Review({ pair }: { pair: Pair }) {
<Modal open={confirmBulk} onClose={() => setConfirmBulk(false)} title="Link all identical files?">
<p className="text-sm text-gray-300">
Each of the <b>{identical}</b> slave files will be compared byte for byte with its host counterpart once more, then atomically
replaced with a hardlink to the host file. Files whose paths differ only by letter case are included.
Each of the <b>{identical}</b> slave files will be compared byte for byte with its master counterpart once more, then atomically
replaced with a hardlink to the master file. Files whose paths differ only by letter case are included.
</p>
<div className="mt-5 flex justify-end gap-2">
<Button variant="ghost" onClick={() => setConfirmBulk(false)}>
@@ -151,15 +151,15 @@ export function Review({ pair }: { pair: Pair }) {
}
function PathLine({ m }: { m: Match }) {
const same = m.rel_host === m.rel_slave || !m.rel_host;
const same = m.rel_master === m.rel_slave || !m.rel_master;
return (
<div className="min-w-0 text-sm">
<div className="truncate mono text-[13px]" title={m.rel_slave}>
{m.rel_slave}
</div>
{!same && (
<div className="flex min-w-0 items-center gap-1 text-xs text-muted" title={m.rel_host}>
<ArrowRight className="size-3 shrink-0" /> <span className="truncate mono">{m.rel_host}</span>
<div className="flex min-w-0 items-center gap-1 text-xs text-muted" title={m.rel_master}>
<ArrowRight className="size-3 shrink-0" /> <span className="truncate mono">{m.rel_master}</span>
</div>
)}
</div>
@@ -215,15 +215,15 @@ function MatchRow({ m }: { m: Match }) {
</div>
{reviewable && (
<div className="hidden items-center gap-1.5 text-xs md:flex">
<span className="text-muted">{qualityLabel(m.host_probe)}</span>
<span className="text-muted">{qualityLabel(m.master_probe)}</span>
<span className="text-gray-600">vs</span>
<span className="text-muted">{qualityLabel(m.slave_probe)}</span>
{better === "slave" && (
<Badge tone="amber">
<AlertTriangle className="size-3" /> host lower
<AlertTriangle className="size-3" /> master lower
</Badge>
)}
{better === "host" && <Badge tone="green">host better</Badge>}
{better === "master" && <Badge tone="green">master better</Badge>}
{better === "equal" && <Badge>equal</Badge>}
</div>
)}
@@ -266,10 +266,10 @@ const rows: [string, (p: ProbeInfo, size: number) => string][] = [
function ComparePanel({ m, onDone }: { m: Match; onDone: () => void }) {
const toast = useToast();
const [confirm, setConfirm] = useState<null | "downgrade" | "promote">(null);
const hp = m.host_probe;
const hp = m.master_probe;
const sp = m.slave_probe;
const v = m.verdict;
const renamed = m.rel_host.split("/").pop() !== m.rel_slave.split("/").pop();
const renamed = m.rel_master.split("/").pop() !== m.rel_slave.split("/").pop();
const link = useMutation({
mutationFn: (allow: boolean) => api.post<{ dry_run: boolean }>(`/matches/${m.id}/link`, { allow_downgrade: allow }),
@@ -287,7 +287,7 @@ function ComparePanel({ m, onDone }: { m: Match; onDone: () => void }) {
mutationFn: () => api.post<{ dry_run: boolean }>(`/matches/${m.id}/promote`, { confirm: true }),
onSuccess: (op) => {
setConfirm(null);
toast("ok", op.dry_run ? "Planned (dry run): nothing changed" : "Slave file promoted to host. The old host file is in quarantine.");
toast("ok", op.dry_run ? "Planned (dry run): nothing changed" : "Slave file promoted to master. The old master file is in quarantine.");
onDone();
},
onError: (e) => toast("err", e.message),
@@ -303,7 +303,7 @@ function ComparePanel({ m, onDone }: { m: Match; onDone: () => void }) {
<tr className="text-left text-xs text-muted">
<th className="py-1 pr-4 font-normal"></th>
<th className="py-1 pr-4 font-medium">
<Badge tone="violet">host</Badge> {better(v?.better === "host")}
<Badge tone="violet">master</Badge> {better(v?.better === "master")}
</th>
<th className="py-1 font-medium">
<Badge>slave</Badge> {better(v?.better === "slave")}
@@ -312,7 +312,7 @@ function ComparePanel({ m, onDone }: { m: Match; onDone: () => void }) {
</thead>
<tbody>
{rows.map(([label, f]) => {
const a = hp.error ? "–" : f(hp, m.host_size);
const a = hp.error ? "–" : f(hp, m.master_size);
const b = sp.error ? "–" : f(sp, m.slave_size);
return (
<tr key={label} className="border-t border-line/60">
@@ -334,30 +334,30 @@ function ComparePanel({ m, onDone }: { m: Match; onDone: () => void }) {
))}
{v?.better === "slave" && (
<Alert tone="amber" icon={<Crown className="mt-0.5 size-4 shrink-0" />}>
The host copy is lower quality. The host is normally kept because it belongs to a complete album. You can still link to it (the
slave's better copy goes to quarantine), or override and make the slave's copy the host file.
The master copy is lower quality. The master is normally kept because it belongs to a complete album. You can still link to it (the
slave's better copy goes to quarantine), or override and make the slave's copy the master file.
</Alert>
)}
{renamed && (
<Alert tone="blue">
File names differ. Linking puts <code className="mono">{m.rel_host.split("/").pop()}</code> into the slave folder in place of{" "}
File names differ. Linking puts <code className="mono">{m.rel_master.split("/").pop()}</code> into the slave folder in place of{" "}
<code className="mono">{m.rel_slave.split("/").pop()}</code>, so playlists that point at the old name will need updating.
</Alert>
)}
</div>
<div className="mt-4 flex flex-wrap gap-2">
<Button variant="primary" onClick={() => (v?.better === "slave" ? setConfirm("downgrade") : link.mutate(false))} loading={link.isPending && !confirm}>
<Link2 className="size-4" /> Link to host copy
<Link2 className="size-4" /> Link to master copy
</Button>
<Button variant={v?.better === "slave" ? "warn" : "secondary"} onClick={() => setConfirm("promote")}>
<Crown className="size-4" /> Use slave copy as host
<Crown className="size-4" /> Use slave copy as master
</Button>
</div>
<p className="mt-2 text-xs text-muted">Whatever gets replaced is kept in quarantine and can be restored from History.</p>
<Modal open={confirm === "downgrade"} onClose={() => setConfirm(null)} title="Link to a lower-quality host file?">
<Modal open={confirm === "downgrade"} onClose={() => setConfirm(null)} title="Link to a lower-quality master file?">
<p className="text-sm text-gray-300">
The slave has the better copy ({qualityLabel(sp)} vs {qualityLabel(hp)}). Linking replaces it with the host's{" "}
The slave has the better copy ({qualityLabel(sp)} vs {qualityLabel(hp)}). Linking replaces it with the master's{" "}
{qualityLabel(hp)} file. The slave's original goes to quarantine and can be restored until it is purged.
</p>
<div className="mt-5 flex justify-end gap-2">
@@ -369,10 +369,10 @@ function ComparePanel({ m, onDone }: { m: Match; onDone: () => void }) {
</Button>
</div>
</Modal>
<Modal open={confirm === "promote"} onClose={() => setConfirm(null)} title="Replace the host file with the slave copy?">
<Modal open={confirm === "promote"} onClose={() => setConfirm(null)} title="Replace the master file with the slave copy?">
<p className="text-sm text-gray-300">
This <b>changes the host library</b>. The host file <code className="mono break-all">{m.rel_host}</code> goes to quarantine, and the
slave's file (<code className="mono">{qualityLabel(sp)}</code>) is hardlinked into the host album in its place
This <b>changes the master library</b>. The master file <code className="mono break-all">{m.rel_master}</code> goes to quarantine, and the
slave's file (<code className="mono">{qualityLabel(sp)}</code>) is hardlinked into the master album in its place
{renamed && (
<>
{" "}as <code className="mono">{m.rel_slave.split("/").pop()}</code>