Loosen approximate matching and link to the album version's folders
Matching: - Prefer the studio album over singles, EPs and compilations when a song exists on several releases, including when the master also has the single at the slave's exact folder. - Ignore release labels (Single Version, Remastered, Explicit, Deluxe) inside brackets or a trailing " - ..." part; "Clean" stays a version. - Treat live/acoustic/remix/... album folders as different recordings; the strict top-folder fallback now respects this too. - Add a last-resort library-wide search that only accepts candidates whose artist tag agrees with the slave file. Linking: - Cross-folder matches now mirror the master's path in the slave (e.g. a single moves into the album folder) and prune the emptied folder; a "keep in current folder" option preserves the old behaviour. Undo recreates and prunes folders accordingly. - Operations record the match kind and reason, so undo (and promotes that rename the master file) restore the correct pairing instead of leaving the file unmatched until the next scan. UI: rewrite the approximate matching description with Avenged Sevenfold examples and clear safeguards, and show where linking will place a file. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
381b804742
commit
5f895cab6a
12 files changed
+698
-109
No files matched your search
@@ -20,15 +20,32 @@ path on both sides. The fallbacks below catch small differences, and a scan warn
|
||||
| Same title, **different format** (`Song.mp3` vs `Song.flac`) | Status `similar` (other format). Manual review. |
|
||||
| Same title after ignoring track numbers/punctuation (`01 - Song!.flac` vs `Song.mp3`) | Status `similar` (similar title). Manual review. |
|
||||
| Same top-level folder and title, but a **different subfolder** | Status `similar` (same top folder, other subfolder). Manual review. |
|
||||
| *(opt-in)* **Approximately** the same title, disc folders ignored (`Cats/Disc 02/07 Memory.flac` vs `Cats/07 Memory.mp3`) | Status `similar` (approximate match), with the reason shown. Manual review. |
|
||||
| *(opt-in)* Approximately the same song in another folder, e.g. a **single vs. the album version** or a **disc folder** | Status `similar` (approximate match), with the reason shown. Manual review. |
|
||||
| Only exists in the slave library | Status `unmatched`. Never touched. |
|
||||
| Already the same inode | Status `linked`. |
|
||||
|
||||
**Approximate matching** is an opt-in setting on each pair. For slave files the strict rules could not place, it searches the same
|
||||
album with disc folders (`Disc 02`, `CD2`, …) ignored, then the whole top-level folder, and compares titles by similarity against a
|
||||
threshold you choose (85% by default; a matching track number adds a small bonus). Several guards keep it from linking the wrong song:
|
||||
version words such as *Live*, *Remix*, *Reprise* or *Instrumental* must agree on both sides, a file is left unmatched when two
|
||||
candidates are nearly tied, and approximate matches never auto-link.
|
||||
**Approximate matching** is an opt-in setting on each pair. It finds songs whose file name or folder doesn't line up exactly with the
|
||||
master library and recommends linking them to the master's album version:
|
||||
|
||||
| Case | Slave | Master |
|
||||
|---|---|---|
|
||||
| Single → album version | `Avenged Sevenfold/Hail to the King - Single/Hail to the King.mp3` | `Avenged Sevenfold/Hail to the King/01 Hail to the King.flac` |
|
||||
| Disc folders ignored | `Avenged Sevenfold/Nightmare/04 Buried Alive.mp3` | `Avenged Sevenfold/Nightmare/Disc 1/04 Buried Alive.flac` |
|
||||
| Small title differences | `Avenged Sevenfold/City of Evil/Bat Country (Remastered).mp3` | `Avenged Sevenfold/City of Evil/03 Bat Country.flac` |
|
||||
|
||||
It searches the same album first (disc folders ignored), then the artist's other albums, then, as a last resort, the whole master
|
||||
library. Titles are compared by similarity against a threshold you choose (85% by default), and release labels such as *Single
|
||||
Version*, *Remastered* or *Explicit* are ignored. When a song exists on several releases, the studio album wins over singles, EPs
|
||||
and compilations, even if the master library also has the single. Linking such a match places the file at the master's path in the
|
||||
slave library (the emptied single folder is removed), or you can keep it in the slave's current folder.
|
||||
|
||||
Safeguards:
|
||||
|
||||
- Different recordings never match. *Live*, *Remix*, *Acoustic*, *Instrumental* and similar versions are kept apart, including
|
||||
whole live or acoustic albums.
|
||||
- Matches found outside the artist's folder require both files' artist tags to agree.
|
||||
- Ambiguous candidates are skipped rather than guessed.
|
||||
- Approximate matches are never linked automatically.
|
||||
|
||||
**Manual review** shows both files side by side and ranks their quality: lossless beats lossy, then bit depth and sample rate, then
|
||||
codec-weighted bitrate (Opus and AAC count for more per kbps than MP3). You then have three choices:
|
||||
|
||||
@@ -422,6 +422,9 @@ func (s *Server) handleMatches(w http.ResponseWriter, r *http.Request) {
|
||||
writeErr(w, 500, err)
|
||||
return
|
||||
}
|
||||
for i := range ms {
|
||||
ms[i].LinkTarget = engine.LinkTarget(ms[i].RelSlave, ms[i].RelMaster, false)
|
||||
}
|
||||
writeJSON(w, 200, map[string]any{"items": ms, "total": total})
|
||||
}
|
||||
|
||||
@@ -487,11 +490,12 @@ func (s *Server) handleLink(w http.ResponseWriter, r *http.Request) {
|
||||
id, _ := idParam(r)
|
||||
var body struct {
|
||||
AllowDowngrade bool `json:"allow_downgrade"`
|
||||
KeepFolder bool `json:"keep_folder"`
|
||||
}
|
||||
if !readJSON(w, r, &body) {
|
||||
return
|
||||
}
|
||||
op, err := s.eng.Link(r.Context(), id, engine.LinkOptions{AllowDowngrade: body.AllowDowngrade})
|
||||
op, err := s.eng.Link(r.Context(), id, engine.LinkOptions{AllowDowngrade: body.AllowDowngrade, KeepFolder: body.KeepFolder})
|
||||
if err != nil {
|
||||
opErr(w, err)
|
||||
return
|
||||
|
||||
+273
-66
@@ -11,9 +11,10 @@ import (
|
||||
)
|
||||
|
||||
// Approximate matching is an opt-in, per-pair second pass for slave files the
|
||||
// strict rules could not place. It tolerates disc subfolders ("Disc 02/",
|
||||
// "CD2/") and slightly different titles, but never auto-links: every result
|
||||
// goes to manual review.
|
||||
// strict rules could not place. It tolerates disc subfolders, small title
|
||||
// differences, release labels such as "Single Version" or "Remastered", and
|
||||
// songs filed under a different release (a single vs. the album). It never
|
||||
// auto-links: every result goes to manual review.
|
||||
|
||||
// discFolder matches disc/side subfolders such as "Disc 02", "CD2", "Disk 1 - Bonus".
|
||||
var discFolder = regexp.MustCompile(`(?i)^(disc|disk|cd|dvd|side|vol|volume)[\s._-]*\d+\b.*$`)
|
||||
@@ -21,9 +22,39 @@ var discFolder = regexp.MustCompile(`(?i)^(disc|disk|cd|dvd|side|vol|volume)[\s.
|
||||
// trackNum extracts the leading track number ("07 Memory", "2-07 Memory").
|
||||
var trackNum = regexp.MustCompile(`^\s*(?:\d{1,2}[-.]\s*)?(\d{1,3})(?:\s*[-._)]\s*|\s+)`)
|
||||
|
||||
// releaseNoise are labels describing a release rather than the recording.
|
||||
// They are only removed inside brackets or a trailing " - ..." part, so
|
||||
// "Nightmare (Single Version)" and "Nightmare - Remastered 2013" compare as
|
||||
// "Nightmare", while a song titled "Single Ladies" keeps its name. "Clean"
|
||||
// is deliberately absent: a clean edit is different audio.
|
||||
var releaseNoise = regexp.MustCompile(`(?i)\b(?:(?:single|album|lp|original)\s+version|` +
|
||||
`(?:\d{4}\s+)?remaster(?:ed)?(?:\s+\d{4})?(?:\s+version)?|explicit(?:\s+version)?|` +
|
||||
`deluxe(?:\s+edition)?|bonus\s+track|single)\b`)
|
||||
|
||||
var (
|
||||
bracketPart = regexp.MustCompile(`[(\[]([^)\]]*)[)\]]`)
|
||||
dashPart = regexp.MustCompile(`\s+-\s+([^-]+)$`)
|
||||
)
|
||||
|
||||
// stripRelease removes release labels from bracketed and trailing parts.
|
||||
func stripRelease(stem string) string {
|
||||
clean := func(re *regexp.Regexp) func(string) string {
|
||||
return func(part string) string {
|
||||
inner := re.FindStringSubmatch(part)[1]
|
||||
rest := releaseNoise.ReplaceAllString(inner, " ")
|
||||
if len(words(rest)) == 0 {
|
||||
return " "
|
||||
}
|
||||
return strings.Replace(part, inner, rest, 1)
|
||||
}
|
||||
}
|
||||
stem = bracketPart.ReplaceAllStringFunc(stem, clean(bracketPart))
|
||||
return dashPart.ReplaceAllStringFunc(stem, clean(dashPart))
|
||||
}
|
||||
|
||||
// versionWords mark a different recording or arrangement. Two titles only
|
||||
// match approximately if they carry the same set of these words, so
|
||||
// "Memory (Live)" never matches "Memory".
|
||||
// "Nightmare (Live)" never matches "Nightmare".
|
||||
var versionWords = map[string]bool{
|
||||
"live": true, "remix": true, "mix": true, "acoustic": true, "demo": true, "instrumental": true,
|
||||
"edit": true, "version": true, "reprise": true, "karaoke": true, "cover": true, "extended": true,
|
||||
@@ -32,6 +63,19 @@ var versionWords = map[string]bool{
|
||||
"sped": true, "slowed": true, "nightcore": true, "reverb": true, "mashup": true, "bootleg": true,
|
||||
}
|
||||
|
||||
// folderVersions are album-folder words that mean every track in it is a
|
||||
// different recording ("Live in the LBC", "Acoustic Sessions", "Remixes").
|
||||
var folderVersions = map[string]string{
|
||||
"live": "live", "acoustic": "acoustic", "unplugged": "unplugged", "remix": "remix", "remixes": "remix",
|
||||
"instrumental": "instrumental", "instrumentals": "instrumental", "karaoke": "karaoke", "demo": "demo",
|
||||
"demos": "demo", "orchestral": "orchestral", "acapella": "acapella",
|
||||
}
|
||||
|
||||
var (
|
||||
singleFolder = regexp.MustCompile(`(?i)\b(singles?|ep)\b`)
|
||||
compilationFolder = regexp.MustCompile(`(?i)\b(greatest hits|best of|the best|essentials|anthology|collection|compilation|hits)\b`)
|
||||
)
|
||||
|
||||
// albumKey is the lower-cased directory with disc folders removed.
|
||||
func albumKey(rel string) string {
|
||||
var parts []string
|
||||
@@ -43,40 +87,67 @@ func albumKey(rel string) string {
|
||||
return strings.Join(parts, "/")
|
||||
}
|
||||
|
||||
func hasDiscFolder(rel string) bool {
|
||||
for _, p := range strings.Split(filepath.Dir(rel), string(filepath.Separator)) {
|
||||
if discFolder.MatchString(p) {
|
||||
return true
|
||||
// releaseFolders returns the folder names below the top-level folder,
|
||||
// without disc folders: normally just the album folder.
|
||||
func releaseFolders(rel string) []string {
|
||||
parts := strings.Split(filepath.Dir(rel), string(filepath.Separator))
|
||||
var out []string
|
||||
for i, p := range parts {
|
||||
if i == 0 || p == "." || discFolder.MatchString(p) {
|
||||
continue
|
||||
}
|
||||
out = append(out, p)
|
||||
}
|
||||
return false
|
||||
return out
|
||||
}
|
||||
|
||||
func words(s string) []string {
|
||||
return strings.FieldsFunc(strings.ToLower(s), func(r rune) bool { return !unicode.IsLetter(r) && !unicode.IsDigit(r) })
|
||||
}
|
||||
|
||||
type titleInfo struct {
|
||||
norm string // normTitle: letters+digits only
|
||||
norm string // letters+digits of the cleaned title
|
||||
words []string // non-version words
|
||||
versions map[string]bool // version words present
|
||||
versions map[string]bool // version words from the title and album folder
|
||||
track int // 0 = unknown
|
||||
|
||||
single bool // filed under a single/EP folder
|
||||
compilation bool // filed under a compilation folder
|
||||
}
|
||||
|
||||
func parseTitle(name string) titleInfo {
|
||||
// parseTitle analyses a relative path (or bare file name).
|
||||
func parseTitle(rel string) titleInfo {
|
||||
name := filepath.Base(rel)
|
||||
stem := strings.TrimSuffix(name, filepath.Ext(name))
|
||||
ti := titleInfo{norm: normTitle(name), versions: map[string]bool{}}
|
||||
ti := titleInfo{versions: map[string]bool{}}
|
||||
if m := trackNum.FindStringSubmatch(stem); m != nil {
|
||||
ti.track, _ = strconv.Atoi(m[1])
|
||||
if s := trackNum.ReplaceAllString(stem, ""); strings.TrimSpace(s) != "" {
|
||||
stem = s
|
||||
}
|
||||
}
|
||||
for _, w := range strings.FieldsFunc(strings.ToLower(stem), func(r rune) bool {
|
||||
return !unicode.IsLetter(r) && !unicode.IsDigit(r)
|
||||
}) {
|
||||
if s := stripRelease(stem); len(words(s)) > 0 {
|
||||
stem = s
|
||||
}
|
||||
var norm strings.Builder
|
||||
for _, w := range words(stem) {
|
||||
norm.WriteString(w)
|
||||
if versionWords[w] {
|
||||
ti.versions[w] = true
|
||||
} else {
|
||||
ti.words = append(ti.words, w)
|
||||
}
|
||||
}
|
||||
ti.norm = norm.String()
|
||||
for _, f := range releaseFolders(rel) {
|
||||
for _, w := range words(f) {
|
||||
if v, ok := folderVersions[w]; ok {
|
||||
ti.versions[v] = true
|
||||
}
|
||||
}
|
||||
ti.single = ti.single || singleFolder.MatchString(f)
|
||||
ti.compilation = ti.compilation || compilationFolder.MatchString(f)
|
||||
}
|
||||
return ti
|
||||
}
|
||||
|
||||
@@ -118,8 +189,8 @@ func levenshteinRatio(a, b string) float64 {
|
||||
}
|
||||
|
||||
// wordContainment scores titles where one adds words to the other
|
||||
// ("Memory" vs "Memory from Cats"). It needs at least two shared words so a
|
||||
// one-word title like "Love" cannot swallow "Love Story".
|
||||
// ("So Far Away" vs "So Far Away feat. Someone"). It needs at least two
|
||||
// shared words so a one-word title cannot swallow a longer one.
|
||||
func wordContainment(a, b []string) float64 {
|
||||
if len(a) > len(b) {
|
||||
a, b = b, a
|
||||
@@ -131,15 +202,11 @@ func wordContainment(a, b []string) float64 {
|
||||
for _, w := range b {
|
||||
set[w] = true
|
||||
}
|
||||
hit := 0
|
||||
for _, w := range a {
|
||||
if set[w] {
|
||||
hit++
|
||||
}
|
||||
}
|
||||
if hit != len(a) {
|
||||
if !set[w] {
|
||||
return 0
|
||||
}
|
||||
}
|
||||
return 0.9 // a full subset is strong but never a perfect match
|
||||
}
|
||||
|
||||
@@ -160,22 +227,40 @@ func titleScore(a, b titleInfo) float64 {
|
||||
return min(s, 1)
|
||||
}
|
||||
|
||||
// stopWords are too common to narrow down library-wide candidates.
|
||||
var stopWords = map[string]bool{"the": true, "and": true, "for": true, "you": true, "feat": true, "with": true}
|
||||
|
||||
// maxPosting skips words that appear in too many titles to be useful.
|
||||
const maxPosting = 2000
|
||||
|
||||
type approxIndex struct {
|
||||
byAlbum map[string][]string // albumKey -> master rels
|
||||
byTop map[string][]string // lower(top folder) -> master rels
|
||||
byWord map[string][]string // title word -> master rels (library-wide search)
|
||||
titles map[string]titleInfo
|
||||
albumTracks map[string]int // albumKey -> number of tracks
|
||||
}
|
||||
|
||||
func buildApproxIndex(files []string) *approxIndex {
|
||||
ax := &approxIndex{byAlbum: map[string][]string{}, byTop: map[string][]string{}, titles: map[string]titleInfo{}}
|
||||
ax := &approxIndex{byAlbum: map[string][]string{}, byTop: map[string][]string{}, byWord: map[string][]string{},
|
||||
titles: map[string]titleInfo{}, albumTracks: map[string]int{}}
|
||||
for _, rel := range files {
|
||||
ax.titles[rel] = parseTitle(filepath.Base(rel))
|
||||
ti := parseTitle(rel)
|
||||
ax.titles[rel] = ti
|
||||
k := albumKey(rel)
|
||||
ax.byAlbum[k] = append(ax.byAlbum[k], rel)
|
||||
ax.albumTracks[k]++
|
||||
if top := firstComponent(rel); top != "" {
|
||||
t := strings.ToLower(top)
|
||||
ax.byTop[t] = append(ax.byTop[t], rel)
|
||||
}
|
||||
seen := map[string]bool{}
|
||||
for _, w := range ti.words {
|
||||
if len(w) >= 3 && !stopWords[w] && !seen[w] {
|
||||
seen[w] = true
|
||||
ax.byWord[w] = append(ax.byWord[w], rel)
|
||||
}
|
||||
}
|
||||
}
|
||||
return ax
|
||||
}
|
||||
@@ -185,61 +270,133 @@ type scored struct {
|
||||
score float64
|
||||
}
|
||||
|
||||
// approxMargin is how far the best candidate must beat the runner-up.
|
||||
// approxMargin: candidates within this score of the best are considered
|
||||
// equally good titles, and release preference decides between them.
|
||||
const approxMargin = 0.05
|
||||
|
||||
// find returns the best approximate master match for a slave file, plus a
|
||||
// human-readable reason. The album (disc folders ignored) is searched first,
|
||||
// then the whole top-level folder.
|
||||
func (ax *approxIndex) find(relSlave string, threshold float64) (string, string) {
|
||||
st := parseTitle(filepath.Base(relSlave))
|
||||
scopes := []struct {
|
||||
name string
|
||||
cands []string
|
||||
}{
|
||||
{"same album", ax.byAlbum[albumKey(relSlave)]},
|
||||
// VerifyFunc confirms a library-wide candidate (by comparing artist tags).
|
||||
type VerifyFunc func(relSlave, relMaster string) bool
|
||||
|
||||
// better reports whether candidate a is a preferable release to b: an album
|
||||
// over a single/EP, a regular album over a compilation, then the fuller
|
||||
// album, then the better format.
|
||||
func (ax *approxIndex) better(a, b string) (bool, bool) {
|
||||
ta, tb := ax.titles[a], ax.titles[b]
|
||||
if ta.single != tb.single {
|
||||
return !ta.single, true
|
||||
}
|
||||
if top := firstComponent(relSlave); top != "" {
|
||||
scopes = append(scopes, struct {
|
||||
name string
|
||||
cands []string
|
||||
}{"same top-level folder, different album folder", ax.byTop[strings.ToLower(top)]})
|
||||
if ta.compilation != tb.compilation {
|
||||
return !ta.compilation, true
|
||||
}
|
||||
for _, sc := range scopes {
|
||||
var best []scored
|
||||
for _, rel := range sc.cands {
|
||||
if s := titleScore(st, ax.titles[rel]); s >= threshold {
|
||||
best = append(best, scored{rel, s})
|
||||
na, nb := ax.albumTracks[albumKey(a)], ax.albumTracks[albumKey(b)]
|
||||
if na != nb {
|
||||
return na > nb, true
|
||||
}
|
||||
if ra, rb := extRank(a), extRank(b); ra != rb {
|
||||
return ra < rb, false // format alone does not tell two releases apart
|
||||
}
|
||||
return false, false
|
||||
}
|
||||
|
||||
func sameSong(a, b string) bool {
|
||||
return filepath.Dir(a) == filepath.Dir(b) &&
|
||||
strings.TrimSuffix(filepath.Base(a), filepath.Ext(a)) == strings.TrimSuffix(filepath.Base(b), filepath.Ext(b))
|
||||
}
|
||||
|
||||
// pick chooses the best candidate. It returns "" when two different songs
|
||||
// are equally good, because guessing could link the wrong file.
|
||||
func (ax *approxIndex) pick(cands []scored) (scored, bool) {
|
||||
sort.Slice(cands, func(i, j int) bool { return cands[i].score > cands[j].score })
|
||||
var tier []scored
|
||||
for _, c := range cands {
|
||||
if cands[0].score-c.score <= approxMargin {
|
||||
tier = append(tier, c)
|
||||
}
|
||||
}
|
||||
if len(best) == 0 {
|
||||
best := tier[0]
|
||||
for _, c := range tier[1:] {
|
||||
if b, _ := ax.better(c.rel, best.rel); b {
|
||||
best = c
|
||||
}
|
||||
}
|
||||
for _, c := range tier {
|
||||
if c.rel == best.rel || sameSong(c.rel, best.rel) {
|
||||
continue
|
||||
}
|
||||
sort.Slice(best, func(i, j int) bool {
|
||||
if best[i].score != best[j].score {
|
||||
return best[i].score > best[j].score
|
||||
}
|
||||
return extRank(best[i].rel) < extRank(best[j].rel)
|
||||
})
|
||||
// Several formats of the same title in one folder are one song.
|
||||
top := best[0]
|
||||
ambiguous := false
|
||||
for _, o := range best[1:] {
|
||||
sameSong := filepath.Dir(o.rel) == filepath.Dir(top.rel) &&
|
||||
strings.TrimSuffix(filepath.Base(o.rel), filepath.Ext(o.rel)) == strings.TrimSuffix(filepath.Base(top.rel), filepath.Ext(top.rel))
|
||||
if !sameSong && top.score-o.score < approxMargin {
|
||||
ambiguous = true
|
||||
break
|
||||
if _, decisive := ax.better(best.rel, c.rel); !decisive {
|
||||
return scored{}, false // two different songs, no reason to prefer either
|
||||
}
|
||||
}
|
||||
if ambiguous {
|
||||
return "", ""
|
||||
return best, true
|
||||
}
|
||||
return top.rel, approxReason(relSlave, top.rel, st, ax.titles[top.rel], top.score, sc.name)
|
||||
|
||||
// find returns the best approximate master match for a slave file and a
|
||||
// human-readable reason. Scopes are tried in order: the same album (disc
|
||||
// folders ignored), the same top-level folder, then the whole library, where
|
||||
// verify must confirm the match.
|
||||
func (ax *approxIndex) find(relSlave string, threshold float64, verify VerifyFunc) (string, string) {
|
||||
st := parseTitle(relSlave)
|
||||
type scope struct {
|
||||
name string
|
||||
cands []string
|
||||
verify bool
|
||||
}
|
||||
scopes := []scope{{"same album", ax.byAlbum[albumKey(relSlave)], false}}
|
||||
if top := firstComponent(relSlave); top != "" {
|
||||
scopes = append(scopes, scope{"same top-level folder, different album folder", ax.byTop[strings.ToLower(top)], false})
|
||||
}
|
||||
if verify != nil {
|
||||
scopes = append(scopes, scope{"elsewhere in the master library, artist tag matches", ax.wordCandidates(st), true})
|
||||
}
|
||||
tried := map[string]bool{}
|
||||
for _, sc := range scopes {
|
||||
var cands []scored
|
||||
for _, rel := range sc.cands {
|
||||
if tried[rel] {
|
||||
continue
|
||||
}
|
||||
tried[rel] = true
|
||||
if s := titleScore(st, ax.titles[rel]); s >= threshold {
|
||||
// Library-wide, the artist tag must agree before a title counts.
|
||||
if !sc.verify || verify(relSlave, rel) {
|
||||
cands = append(cands, scored{rel, s})
|
||||
}
|
||||
}
|
||||
}
|
||||
if len(cands) == 0 {
|
||||
continue
|
||||
}
|
||||
best, ok := ax.pick(cands)
|
||||
if !ok {
|
||||
return "", "" // ambiguous: never fall through to a wider scope
|
||||
}
|
||||
return best.rel, approxReason(relSlave, best.rel, st, ax.titles[best.rel], best.score, sc.name)
|
||||
}
|
||||
return "", ""
|
||||
}
|
||||
|
||||
// wordCandidates lists master files sharing a meaningful title word.
|
||||
func (ax *approxIndex) wordCandidates(st titleInfo) []string {
|
||||
seen := map[string]bool{}
|
||||
var out []string
|
||||
for _, w := range st.words {
|
||||
if len(w) < 3 || stopWords[w] {
|
||||
continue
|
||||
}
|
||||
post := ax.byWord[w]
|
||||
if len(post) > maxPosting {
|
||||
continue
|
||||
}
|
||||
for _, rel := range post {
|
||||
if !seen[rel] {
|
||||
seen[rel] = true
|
||||
out = append(out, rel)
|
||||
}
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func approxReason(relSlave, relMaster string, st, mt titleInfo, score float64, scope string) string {
|
||||
parts := []string{fmt.Sprintf("title %d%% similar", int(score*100+0.5))}
|
||||
if st.track > 0 && mt.track > 0 {
|
||||
@@ -252,6 +409,56 @@ func approxReason(relSlave, relMaster string, st, mt titleInfo, score float64, s
|
||||
if filepath.Dir(relSlave) != filepath.Dir(relMaster) && albumKey(relSlave) == albumKey(relMaster) {
|
||||
parts = append(parts, "disc folder ignored")
|
||||
}
|
||||
switch {
|
||||
case strings.HasPrefix(scope, "album version preferred"):
|
||||
// the scope already says it
|
||||
case st.single && !mt.single:
|
||||
parts = append(parts, "single → album version")
|
||||
case st.compilation && !mt.compilation:
|
||||
parts = append(parts, "compilation → album version")
|
||||
}
|
||||
parts = append(parts, scope)
|
||||
return strings.Join(parts, " · ")
|
||||
}
|
||||
|
||||
// albumAlternative looks for the studio-album version of a match that landed
|
||||
// on a single, EP or compilation release in the master library.
|
||||
func (ax *approxIndex) albumAlternative(relSlave, relMaster string, threshold float64) (string, string) {
|
||||
mt, ok := ax.titles[relMaster]
|
||||
if !ok || !(mt.single || mt.compilation) {
|
||||
return "", ""
|
||||
}
|
||||
st := parseTitle(relSlave)
|
||||
top := firstComponent(relMaster)
|
||||
if top == "" {
|
||||
return "", ""
|
||||
}
|
||||
var cands []scored
|
||||
for _, rel := range ax.byTop[strings.ToLower(top)] {
|
||||
t := ax.titles[rel]
|
||||
if t.single || t.compilation {
|
||||
continue
|
||||
}
|
||||
// Must be the same recording as the release that matched.
|
||||
if !sameVersions(mt.versions, t.versions) {
|
||||
continue
|
||||
}
|
||||
if s := titleScore(st, t); s >= threshold {
|
||||
cands = append(cands, scored{rel, s})
|
||||
}
|
||||
}
|
||||
if len(cands) == 0 {
|
||||
return "", ""
|
||||
}
|
||||
best, ok := ax.pick(cands)
|
||||
if !ok {
|
||||
return "", ""
|
||||
}
|
||||
kind := "single"
|
||||
if mt.compilation && !mt.single {
|
||||
kind = "compilation"
|
||||
}
|
||||
folder := filepath.Base(filepath.Dir(relMaster))
|
||||
return best.rel, approxReason(relSlave, best.rel, st, ax.titles[best.rel], best.score,
|
||||
fmt.Sprintf("album version preferred over the %s in %q", kind, folder))
|
||||
}
|
||||
@@ -1,10 +1,14 @@
|
||||
package engine
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"os"
|
||||
"os/exec"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/cdnutter/playlink/internal/probe"
|
||||
"github.com/cdnutter/playlink/internal/store"
|
||||
)
|
||||
|
||||
@@ -41,11 +45,12 @@ func TestApproxDiscFolderUpgrade(t *testing.T) {
|
||||
t.Fatalf("reason should explain the match, got %q", m.Reason)
|
||||
}
|
||||
|
||||
// Linking upgrades the slave: it gets the master's FLAC in its own folder.
|
||||
// Linking upgrades the slave to the master's FLAC, mirroring the master's
|
||||
// folder structure (including the disc folder).
|
||||
if _, err := ev.e.Link(ctx, m.ID, LinkOptions{}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
upgraded := filepath.Join(ev.slave, "Andrew Lloyd Webber/Cats/07 Memory.flac")
|
||||
upgraded := filepath.Join(ev.slave, "Andrew Lloyd Webber/Cats/Disc 02/07 Memory.flac")
|
||||
if !sameInode(t, master, upgraded) {
|
||||
t.Fatal("slave should now hold a hardlink of the master FLAC")
|
||||
}
|
||||
@@ -124,3 +129,174 @@ func TestTitleScore(t *testing.T) {
|
||||
t.Errorf("albumKey = %q", k)
|
||||
}
|
||||
}
|
||||
|
||||
// A single matched to the album version is linked into the album's folders.
|
||||
func TestApproxSingleToAlbumVersion(t *testing.T) {
|
||||
ev := setup(t, false)
|
||||
enableApprox(t, ev)
|
||||
a7x := "Avenged Sevenfold"
|
||||
album := a7x + "/Hail to the King/01 Hail to the King.flac"
|
||||
for i, title := range []string{"Hail to the King", "Shepherd of Fire", "Doing Time", "This Means War", "Requiem"} {
|
||||
rel := a7x + "/Hail to the King/" + []string{"01", "02", "03", "04", "05"}[i] + " " + title + ".flac"
|
||||
ev.write(ev.master, rel, "album:"+title)
|
||||
}
|
||||
ev.write(ev.master, a7x+"/Hail to the King - Single/01 Hail to the King.flac", "single release")
|
||||
ev.write(ev.master, a7x+"/Live in the LBC/Hail to the King.flac", "live recording")
|
||||
slaveRel := a7x + "/Hail to the King (Single)/Hail to the King (Single Version).mp3"
|
||||
ev.write(ev.slave, slaveRel, "slave mp3")
|
||||
ev.scan()
|
||||
|
||||
m := ev.match(slaveRel)
|
||||
if m.Kind != store.KindApprox || m.RelMaster != album {
|
||||
t.Fatalf("want the album version, got %s -> %q (%s)", m.Kind, m.RelMaster, m.Reason)
|
||||
}
|
||||
if !strings.Contains(m.Reason, "single → album version") {
|
||||
t.Fatalf("reason should mention the album preference, got %q", m.Reason)
|
||||
}
|
||||
if got := LinkTarget(m.RelSlave, m.RelMaster, false); got != album {
|
||||
t.Fatalf("link target should mirror the master's folders, got %q", got)
|
||||
}
|
||||
|
||||
op, err := ev.e.Link(ctx, m.ID, LinkOptions{})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !sameInode(t, filepath.Join(ev.master, album), filepath.Join(ev.slave, album)) {
|
||||
t.Fatal("slave should now have the album version at the album path")
|
||||
}
|
||||
if _, err := os.Stat(filepath.Join(ev.slave, a7x, "Hail to the King (Single)")); !errors.Is(err, os.ErrNotExist) {
|
||||
t.Fatal("the single's emptied folder should be removed")
|
||||
}
|
||||
|
||||
if err := ev.e.Undo(ctx, op.ID); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if ev.read(filepath.Join(ev.slave, slaveRel)) != "slave mp3" {
|
||||
t.Fatal("undo should restore the single in its original folder")
|
||||
}
|
||||
if _, err := os.Stat(filepath.Join(ev.slave, a7x, "Hail to the King")); !errors.Is(err, os.ErrNotExist) {
|
||||
t.Fatal("undo should remove the album folder it no longer needs")
|
||||
}
|
||||
if m := ev.match(slaveRel); m.Kind != store.KindApprox || m.RelMaster != album || m.Reason == "" {
|
||||
t.Fatalf("undo should restore the approximate match, got %s/%s -> %q", m.Status, m.Kind, m.RelMaster)
|
||||
}
|
||||
}
|
||||
|
||||
func TestApproxKeepFolderOption(t *testing.T) {
|
||||
ev := setup(t, false)
|
||||
enableApprox(t, ev)
|
||||
ev.write(ev.master, "Avenged Sevenfold/Nightmare/01 Nightmare.flac", "album")
|
||||
ev.write(ev.slave, "Avenged Sevenfold/Favorites/Nightmare.mp3", "mp3")
|
||||
ev.scan()
|
||||
m := ev.match("Avenged Sevenfold/Favorites/Nightmare.mp3")
|
||||
if m.Kind != store.KindApprox {
|
||||
t.Fatalf("want approximate match, got %s", m.Kind)
|
||||
}
|
||||
if _, err := ev.e.Link(ctx, m.ID, LinkOptions{KeepFolder: true}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !sameInode(t, filepath.Join(ev.master, "Avenged Sevenfold/Nightmare/01 Nightmare.flac"),
|
||||
filepath.Join(ev.slave, "Avenged Sevenfold/Favorites/01 Nightmare.flac")) {
|
||||
t.Fatal("KeepFolder should link into the slave's own folder")
|
||||
}
|
||||
}
|
||||
|
||||
func TestApproxLiveAlbumIsDifferentRecording(t *testing.T) {
|
||||
ev := setup(t, false)
|
||||
enableApprox(t, ev)
|
||||
ev.write(ev.master, "Avenged Sevenfold/Live in the LBC/04 Nightmare.flac", "live")
|
||||
ev.write(ev.slave, "Avenged Sevenfold/Nightmare/04 Nightmare.mp3", "studio")
|
||||
ev.scan()
|
||||
if m := ev.match("Avenged Sevenfold/Nightmare/04 Nightmare.mp3"); m.Status != store.StatusUnmatched {
|
||||
t.Fatalf("studio track must not match a live album, got %s -> %s", m.Kind, m.RelMaster)
|
||||
}
|
||||
}
|
||||
|
||||
func TestApproxCompilationLosesToAlbum(t *testing.T) {
|
||||
ev := setup(t, false)
|
||||
enableApprox(t, ev)
|
||||
ev.write(ev.master, "Avenged Sevenfold/City of Evil/03 Bat Country.flac", "album")
|
||||
ev.write(ev.master, "Avenged Sevenfold/City of Evil/04 Trashed and Scattered.flac", "album2")
|
||||
ev.write(ev.master, "Avenged Sevenfold/The Best of 2005-2013/02 Bat Country.flac", "comp")
|
||||
ev.write(ev.slave, "Avenged Sevenfold/Misc/Bat Country (Remastered).mp3", "mp3")
|
||||
ev.scan()
|
||||
m := ev.match("Avenged Sevenfold/Misc/Bat Country (Remastered).mp3")
|
||||
if m.RelMaster != "Avenged Sevenfold/City of Evil/03 Bat Country.flac" {
|
||||
t.Fatalf("album should win over compilation, got %q", m.RelMaster)
|
||||
}
|
||||
}
|
||||
|
||||
func TestApproxLibraryWideNeedsArtistTag(t *testing.T) {
|
||||
ffmpeg, err1 := exec.LookPath("ffmpeg")
|
||||
ffprobe, err2 := exec.LookPath("ffprobe")
|
||||
if err1 != nil || err2 != nil {
|
||||
t.Skip("ffmpeg/ffprobe not installed")
|
||||
}
|
||||
ev := setup(t, false)
|
||||
ev.e.prober = probe.Prober{Bin: ffprobe}
|
||||
enableApprox(t, ev)
|
||||
mk := func(base, rel, artist string) {
|
||||
p := filepath.Join(base, rel)
|
||||
os.MkdirAll(filepath.Dir(p), 0o755)
|
||||
if out, err := exec.Command(ffmpeg, "-v", "error", "-y", "-f", "lavfi", "-i", "sine=d=1",
|
||||
"-metadata", "artist="+artist, p).CombinedOutput(); err != nil {
|
||||
t.Fatalf("ffmpeg: %v %s", err, out)
|
||||
}
|
||||
}
|
||||
mk(ev.master, "Avenged Sevenfold/Nightmare/01 Nightmare.flac", "Avenged Sevenfold")
|
||||
mk(ev.master, "Other Band/Dreams/05 Nightmare.flac", "Other Band")
|
||||
mk(ev.slave, "Chris Picks/Nightmare.mp3", "Avenged Sevenfold")
|
||||
mk(ev.slave, "Chris Picks/Nightmare (untagged).mp3", "")
|
||||
ev.scan()
|
||||
m := ev.match("Chris Picks/Nightmare.mp3")
|
||||
if m.Kind != store.KindApprox || m.RelMaster != "Avenged Sevenfold/Nightmare/01 Nightmare.flac" {
|
||||
t.Fatalf("artist tag should pick the A7X track, got %s -> %q", m.Kind, m.RelMaster)
|
||||
}
|
||||
if !strings.Contains(m.Reason, "artist tag matches") {
|
||||
t.Fatalf("reason = %q", m.Reason)
|
||||
}
|
||||
}
|
||||
|
||||
func TestReleaseLabels(t *testing.T) {
|
||||
cases := map[string]string{
|
||||
"Nightmare (Single Version).flac": "nightmare",
|
||||
"Nightmare - Remastered 2013.flac": "nightmare",
|
||||
"Nightmare [Explicit].flac": "nightmare",
|
||||
"Single Ladies.flac": "singleladies",
|
||||
"Nightmare (Clean).flac": "nightmareclean",
|
||||
"04 Nightmare (Deluxe Edition).flac": "nightmare",
|
||||
"Afterlife (Live at Wembley).flac": "afterlifeliveatwembley",
|
||||
}
|
||||
for in, want := range cases {
|
||||
if got := parseTitle(in).norm; got != want {
|
||||
t.Errorf("parseTitle(%q).norm = %q, want %q", in, got, want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// The master has both the single and the album; the slave's single sits at
|
||||
// the same path as the master's single. The album version is still preferred.
|
||||
func TestApproxPrefersAlbumOverMatchingSingle(t *testing.T) {
|
||||
ev := setup(t, false)
|
||||
a7x := "Avenged Sevenfold"
|
||||
for i, title := range []string{"Shepherd of Fire", "Hail to the King", "Doing Time"} {
|
||||
ev.write(ev.master, a7x+"/Hail to the King/0"+string(rune('1'+i))+" "+title+".flac", "album:"+title)
|
||||
}
|
||||
ev.write(ev.master, a7x+"/Hail to the King - Single/01 Hail to the King.flac", "single")
|
||||
ev.write(ev.slave, a7x+"/Hail to the King - Single/Hail to the King.mp3", "mp3")
|
||||
|
||||
ev.scan() // approximate matching off: the strict same-folder match stands
|
||||
if m := ev.match(a7x + "/Hail to the King - Single/Hail to the King.mp3"); m.RelMaster != a7x+"/Hail to the King - Single/01 Hail to the King.flac" {
|
||||
t.Fatalf("without approximate matching the single should match, got %q", m.RelMaster)
|
||||
}
|
||||
|
||||
enableApprox(t, ev)
|
||||
ev.scan()
|
||||
m := ev.match(a7x + "/Hail to the King - Single/Hail to the King.mp3")
|
||||
if m.Kind != store.KindApprox || m.RelMaster != a7x+"/Hail to the King/02 Hail to the King.flac" {
|
||||
t.Fatalf("want the album version, got %s -> %q", m.Kind, m.RelMaster)
|
||||
}
|
||||
if !strings.Contains(m.Reason, `album version preferred over the single in "Hail to the King - Single"`) {
|
||||
t.Fatalf("reason = %q", m.Reason)
|
||||
}
|
||||
}
|
||||
@@ -228,6 +228,9 @@ func TestSimilarExtensionLinkRenamesAndUndo(t *testing.T) {
|
||||
if ev.read(s) != "mp3" {
|
||||
t.Fatal(".mp3 not restored")
|
||||
}
|
||||
if m := ev.match("A/B/01 - Song.mp3"); m.Kind != store.KindExtension || m.Status != store.StatusSimilar {
|
||||
t.Fatalf("undo should restore the match for review, got %s/%s", m.Status, m.Kind)
|
||||
}
|
||||
if _, err := os.Stat(newSlave); !errors.Is(err, os.ErrNotExist) {
|
||||
t.Fatal("linked .flac should be removed on undo")
|
||||
}
|
||||
@@ -504,3 +507,24 @@ func TestAnyLayoutMatchesWithoutWarnings(t *testing.T) {
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestPromoteRenamedMasterStaysLinked(t *testing.T) {
|
||||
ev := setup(t, false)
|
||||
ev.write(ev.master, "A/B/Song.mp3", "lossy")
|
||||
ev.write(ev.slave, "A/B/Song.flac", "lossless")
|
||||
ev.scan()
|
||||
m := ev.match("A/B/Song.flac")
|
||||
op, err := ev.e.Promote(ctx, m.ID, true)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got := ev.match("A/B/Song.flac"); got.Status != store.StatusLinked || got.RelMaster != "A/B/Song.flac" {
|
||||
t.Fatalf("after promote the pair should be linked to the renamed master, got %s -> %q", got.Status, got.RelMaster)
|
||||
}
|
||||
if err := ev.e.Undo(ctx, op.ID); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got := ev.match("A/B/Song.flac"); got.Status != store.StatusSimilar || got.RelMaster != "A/B/Song.mp3" {
|
||||
t.Fatalf("undo should restore the original match, got %s -> %q", got.Status, got.RelMaster)
|
||||
}
|
||||
}
|
||||
+48
-9
@@ -21,9 +21,28 @@ import (
|
||||
type LinkOptions struct {
|
||||
// AllowDowngrade confirms linking when the master copy scored lower.
|
||||
AllowDowngrade bool
|
||||
// KeepFolder keeps a cross-folder match in the slave's current folder
|
||||
// instead of mirroring the master's folder structure.
|
||||
KeepFolder bool
|
||||
Trigger string
|
||||
}
|
||||
|
||||
// LinkTarget is where the slave file ends up after linking. A match in the
|
||||
// same folder keeps the slave's folder (only the file name follows the
|
||||
// master). A match found in another folder, such as a single matched to its
|
||||
// album version, mirrors the master's path so the slave gets the album's
|
||||
// folder structure, unless keepFolder is set.
|
||||
func LinkTarget(relSlave, relMaster string, keepFolder bool) string {
|
||||
if relMaster == "" {
|
||||
return relSlave
|
||||
}
|
||||
sd := filepath.Dir(relSlave)
|
||||
if keepFolder || strings.EqualFold(sd, filepath.Dir(relMaster)) {
|
||||
return filepath.Join(sd, filepath.Base(relMaster))
|
||||
}
|
||||
return relMaster
|
||||
}
|
||||
|
||||
// snapshot holds the verified state of both files right before an operation.
|
||||
type snapshot struct {
|
||||
masterAbs, slaveAbs string
|
||||
@@ -186,7 +205,7 @@ func (e *Engine) Link(ctx context.Context, matchID int64, opt LinkOptions) (stor
|
||||
}
|
||||
}
|
||||
|
||||
newRel := filepath.Join(filepath.Dir(m.RelSlave), filepath.Base(m.RelMaster))
|
||||
newRel := LinkTarget(m.RelSlave, m.RelMaster, opt.KeepFolder)
|
||||
newAbs, err := l.slave.Resolve(newRel)
|
||||
if err != nil {
|
||||
return store.Operation{}, err
|
||||
@@ -199,7 +218,7 @@ func (e *Engine) Link(ctx context.Context, matchID int64, opt LinkOptions) (stor
|
||||
}
|
||||
|
||||
o := store.Operation{PairID: l.pair.ID, Action: store.ActLink, RelMaster: m.RelMaster, RelSlave: m.RelSlave,
|
||||
RelNew: newRel, Trigger: opt.Trigger}
|
||||
RelNew: newRel, Trigger: opt.Trigger, MatchKind: m.Kind, MatchReason: m.Reason}
|
||||
if identical {
|
||||
o.Bytes = s.si.Size // space freed
|
||||
} else {
|
||||
@@ -237,19 +256,28 @@ func (e *Engine) Link(ctx context.Context, matchID int64, opt LinkOptions) (stor
|
||||
}
|
||||
return nil
|
||||
}
|
||||
if err := fsops.LinkNew(s.masterAbs, s.mi, newAbs); err != nil {
|
||||
err := fsops.MkdirAllLike(filepath.Dir(newAbs))
|
||||
if err == nil {
|
||||
err = fsops.LinkNew(s.masterAbs, s.mi, newAbs)
|
||||
}
|
||||
if err != nil {
|
||||
pruneEmpty(filepath.Dir(newAbs), l.slave.Path)
|
||||
if q != "" {
|
||||
dropQuarantine(l, q, s.si)
|
||||
e.clearQuarantine(ctx, o.ID)
|
||||
}
|
||||
return err
|
||||
}
|
||||
return removeOriginal(s.slaveAbs, s.si)
|
||||
if err := removeOriginal(s.slaveAbs, s.si); err != nil {
|
||||
return err
|
||||
}
|
||||
pruneEmpty(filepath.Dir(s.slaveAbs), l.slave.Path) // e.g. the single's now-empty folder
|
||||
return nil
|
||||
}()
|
||||
err = e.finish(ctx, &o, err)
|
||||
e.refresh(ctx, l, m.RelSlave)
|
||||
e.refresh(ctx, l, m.RelSlave, m)
|
||||
if rename {
|
||||
e.refresh(ctx, l, newRel)
|
||||
e.refresh(ctx, l, newRel, m)
|
||||
}
|
||||
return o, err
|
||||
}
|
||||
@@ -295,7 +323,7 @@ func (e *Engine) Promote(ctx context.Context, matchID int64, confirmed bool) (st
|
||||
}
|
||||
|
||||
o := store.Operation{PairID: l.pair.ID, Action: store.ActPromote, RelMaster: m.RelMaster, RelSlave: m.RelSlave,
|
||||
RelNew: newMasterRel, QuarantineSide: "master", Trigger: "manual"}
|
||||
RelNew: newMasterRel, QuarantineSide: "master", Trigger: "manual", MatchKind: m.Kind, MatchReason: m.Reason}
|
||||
if err := e.begin(ctx, &o); err != nil {
|
||||
return o, err
|
||||
}
|
||||
@@ -330,7 +358,11 @@ func (e *Engine) Promote(ctx context.Context, matchID int64, confirmed bool) (st
|
||||
return removeOriginal(s.masterAbs, s.mi)
|
||||
}()
|
||||
err = e.finish(ctx, &o, err)
|
||||
e.refresh(ctx, l, m.RelSlave)
|
||||
promoted := m
|
||||
if err == nil {
|
||||
promoted.RelMaster = newMasterRel // the master file may have been renamed
|
||||
}
|
||||
e.refresh(ctx, l, m.RelSlave, promoted)
|
||||
return o, err
|
||||
}
|
||||
|
||||
@@ -583,6 +615,11 @@ func (e *Engine) Undo(ctx context.Context, opID int64) error {
|
||||
if _, err := os.Lstat(slaveAbs); err == nil {
|
||||
return fmt.Errorf("%s: %w", o.RelSlave, fsops.ErrExists)
|
||||
}
|
||||
// The original folder may have been pruned when the link moved
|
||||
// the file into the master's folder structure.
|
||||
if err := fsops.MkdirAllLike(filepath.Dir(slaveAbs)); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
if o.QuarantinePath == "" {
|
||||
// Identical link: give the slave its own copy of the bytes again.
|
||||
@@ -608,6 +645,7 @@ func (e *Engine) Undo(ctx context.Context, opID int64) error {
|
||||
if err := fsops.RemoveLink(newAbs, mi); err != nil {
|
||||
return err
|
||||
}
|
||||
pruneEmpty(filepath.Dir(newAbs), l.slave.Path)
|
||||
}
|
||||
e.refresh(ctx, l, o.RelNew)
|
||||
|
||||
@@ -653,7 +691,8 @@ func (e *Engine) Undo(ctx context.Context, opID int64) error {
|
||||
return err
|
||||
}
|
||||
e.log.Info("operation undone", "op", o.ID, "action", o.Action)
|
||||
e.refresh(ctx, l, o.RelSlave)
|
||||
// Restore the original pairing (the master path before any promote rename).
|
||||
e.refresh(ctx, l, o.RelSlave, store.Match{RelMaster: o.RelMaster, Kind: o.MatchKind, Reason: o.MatchReason})
|
||||
return nil
|
||||
}
|
||||
|
||||
|
||||
+67
-9
@@ -107,6 +107,7 @@ type masterIndex struct {
|
||||
|
||||
approx *approxIndex // nil unless the pair enables approximate matching
|
||||
threshold float64
|
||||
verify VerifyFunc // confirms library-wide approximate matches; nil disables that scope
|
||||
}
|
||||
|
||||
func buildIndex(files map[string]fsops.Info, p store.Pair) *masterIndex {
|
||||
@@ -139,8 +140,21 @@ func extRank(rel string) int {
|
||||
}
|
||||
|
||||
// find locates the master counterpart of a slave file. The reason is only
|
||||
// set for approximate matches.
|
||||
// set for approximate matches. With approximate matching on, a match that
|
||||
// lands on a single, EP or compilation is replaced by the same recording on
|
||||
// a studio album when one exists, so the slave is linked to the album
|
||||
// version and takes on the album's folder structure.
|
||||
func (ix *masterIndex) find(relSlave string) (rel, kind, reason string) {
|
||||
rel, kind, reason = ix.strictFind(relSlave)
|
||||
if ix.approx != nil && rel != "" {
|
||||
if alt, why := ix.approx.albumAlternative(relSlave, rel, ix.threshold); alt != "" {
|
||||
return alt, store.KindApprox, why
|
||||
}
|
||||
}
|
||||
return rel, kind, reason
|
||||
}
|
||||
|
||||
func (ix *masterIndex) strictFind(relSlave string) (rel, kind, reason string) {
|
||||
if _, ok := ix.files[relSlave]; ok {
|
||||
return relSlave, store.KindPath, ""
|
||||
}
|
||||
@@ -176,12 +190,15 @@ func (ix *masterIndex) find(relSlave string) (rel, kind, reason string) {
|
||||
// searches the album (disc folders ignored) first, which resolves titles
|
||||
// that appear on several albums.
|
||||
if ix.approx != nil {
|
||||
if r, why := ix.approx.find(relSlave, ix.threshold); r != "" {
|
||||
if r, why := ix.approx.find(relSlave, ix.threshold, ix.verify); r != "" {
|
||||
return r, store.KindApprox, why
|
||||
}
|
||||
}
|
||||
if top := firstComponent(relSlave); top != "" && norm != "" {
|
||||
if hits := ix.byTitle[strings.ToLower(top)+"|"+norm]; len(hits) == 1 {
|
||||
// The hit is in another album folder, so it must be the same kind of
|
||||
// recording: a studio track never matches one on a live album.
|
||||
if hits := ix.byTitle[strings.ToLower(top)+"|"+norm]; len(hits) == 1 &&
|
||||
sameVersions(parseTitle(relSlave).versions, parseTitle(hits[0]).versions) {
|
||||
return hits[0], store.KindElsewhere, ""
|
||||
}
|
||||
}
|
||||
@@ -287,6 +304,9 @@ func (e *Engine) scan(ctx context.Context, j *Job, pairID int64, trigger string)
|
||||
return err
|
||||
}
|
||||
ix := buildIndex(masterFiles, l.pair)
|
||||
if ix.approx != nil {
|
||||
ix.verify = e.artistVerifier(ctx, l)
|
||||
}
|
||||
prev, err := e.st.AllMatches(ctx, pairID)
|
||||
if err != nil {
|
||||
return err
|
||||
@@ -418,8 +438,11 @@ func (e *Engine) StartLinkIdentical(pairID int64) (*Job, error) {
|
||||
})
|
||||
}
|
||||
|
||||
// refresh re-evaluates a single slave file after an operation.
|
||||
func (e *Engine) refresh(ctx context.Context, l *libs, relSlave string) {
|
||||
// refresh re-evaluates a single slave file after an operation. The optional
|
||||
// hint names the master file the operation paired it with; it takes
|
||||
// precedence over the stored match, which may describe the pre-operation
|
||||
// state (or be gone, after a move).
|
||||
func (e *Engine) refresh(ctx context.Context, l *libs, relSlave string, hint ...store.Match) {
|
||||
si, err := fsops.RegularFile(filepath.Join(l.slave.Path, relSlave))
|
||||
if err != nil {
|
||||
if errors.Is(err, fs.ErrNotExist) {
|
||||
@@ -439,11 +462,18 @@ func (e *Engine) refresh(ctx context.Context, l *libs, relSlave string) {
|
||||
m.RelMaster, m.Kind = relSlave, store.KindPath
|
||||
}
|
||||
}
|
||||
if m.Kind == "" {
|
||||
if pm != nil && pm.RelMaster != "" {
|
||||
if in, err := fsops.RegularFile(filepath.Join(l.master.Path, pm.RelMaster)); err == nil {
|
||||
candidates := hint
|
||||
if pm != nil {
|
||||
candidates = append(candidates, *pm)
|
||||
}
|
||||
for _, c := range candidates {
|
||||
if m.Kind != "" || c.RelMaster == "" || c.Kind == "" || c.Kind == store.KindNone {
|
||||
continue
|
||||
}
|
||||
if mp, err := l.master.Resolve(c.RelMaster); err == nil {
|
||||
if in, err := fsops.RegularFile(mp); err == nil {
|
||||
mi = in
|
||||
m.RelMaster, m.Kind, m.Reason = pm.RelMaster, pm.Kind, pm.Reason
|
||||
m.RelMaster, m.Kind, m.Reason = c.RelMaster, c.Kind, c.Reason
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -453,3 +483,31 @@ func (e *Engine) refresh(ctx context.Context, l *libs, relSlave string) {
|
||||
e.classify(ctx, l, &m, mi, si, pm)
|
||||
_ = e.st.UpsertMatch(ctx, &m)
|
||||
}
|
||||
|
||||
// artistVerifier confirms a library-wide approximate match by comparing the
|
||||
// artist tags of both files. Without ffprobe, or when either file has no
|
||||
// artist tag, the match is rejected: a title alone is not enough evidence
|
||||
// once the folder structure no longer agrees.
|
||||
func (e *Engine) artistVerifier(ctx context.Context, l *libs) VerifyFunc {
|
||||
if e.prober.Bin == "" {
|
||||
return nil
|
||||
}
|
||||
cache := map[string]string{}
|
||||
artist := func(abs string) string {
|
||||
if a, ok := cache[abs]; ok {
|
||||
return a
|
||||
}
|
||||
a := strings.Join(words(e.prober.Probe(ctx, abs).Artist), " ")
|
||||
cache[abs] = a
|
||||
return a
|
||||
}
|
||||
return func(relSlave, relMaster string) bool {
|
||||
sa := artist(filepath.Join(l.slave.Path, relSlave))
|
||||
ma := artist(filepath.Join(l.master.Path, relMaster))
|
||||
if sa == "" || ma == "" {
|
||||
return false
|
||||
}
|
||||
// "Avenged Sevenfold" matches "Avenged Sevenfold feat Someone".
|
||||
return sa == ma || strings.HasPrefix(ma, sa+" ") || strings.HasPrefix(sa, ma+" ")
|
||||
}
|
||||
}
|
||||
@@ -247,6 +247,7 @@ type Match struct {
|
||||
Verdict json.RawMessage `json:"verdict,omitempty"`
|
||||
Note string `json:"note"`
|
||||
Reason string `json:"reason"` // why an approximate match was chosen
|
||||
LinkTarget string `json:"link_target"` // where linking puts the slave file (filled by the API)
|
||||
Ignored bool `json:"ignored"`
|
||||
UpdatedAt int64 `json:"updated_at"`
|
||||
}
|
||||
@@ -500,6 +501,8 @@ type Operation struct {
|
||||
RelMaster string `json:"rel_master"`
|
||||
RelSlave string `json:"rel_slave"`
|
||||
RelNew string `json:"rel_new"`
|
||||
MatchKind string `json:"match_kind"`
|
||||
MatchReason string `json:"match_reason"`
|
||||
QuarantinePath string `json:"quarantine_path"`
|
||||
QuarantineSide string `json:"quarantine_side"`
|
||||
TmpPath string `json:"-"`
|
||||
@@ -532,13 +535,13 @@ const (
|
||||
)
|
||||
|
||||
const opCols = `id, pair_id, action, rel_master, rel_slave, rel_new, quarantine_path, quarantine_side, tmp_path, bytes,
|
||||
dry_run, trigger, status, error, created_at, finished_at, undone_at, purged_at`
|
||||
dry_run, trigger, status, error, created_at, finished_at, undone_at, purged_at, match_kind, match_reason`
|
||||
|
||||
func scanOp(r scanner) (Operation, error) {
|
||||
var o Operation
|
||||
var fin, und, pur sql.NullInt64
|
||||
err := r.Scan(&o.ID, &o.PairID, &o.Action, &o.RelMaster, &o.RelSlave, &o.RelNew, &o.QuarantinePath, &o.QuarantineSide,
|
||||
&o.TmpPath, &o.Bytes, &o.DryRun, &o.Trigger, &o.Status, &o.Error, &o.CreatedAt, &fin, &und, &pur)
|
||||
&o.TmpPath, &o.Bytes, &o.DryRun, &o.Trigger, &o.Status, &o.Error, &o.CreatedAt, &fin, &und, &pur, &o.MatchKind, &o.MatchReason)
|
||||
if fin.Valid {
|
||||
o.FinishedAt = &fin.Int64
|
||||
}
|
||||
@@ -554,10 +557,10 @@ func scanOp(r scanner) (Operation, error) {
|
||||
func (s *Store) CreateOp(ctx context.Context, o *Operation) error {
|
||||
o.CreatedAt = now()
|
||||
res, err := s.DB.ExecContext(ctx, `INSERT INTO operations (pair_id, action, rel_master, rel_slave, rel_new, quarantine_path,
|
||||
quarantine_side, tmp_path, bytes, dry_run, trigger, status, error, created_at)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`,
|
||||
quarantine_side, tmp_path, bytes, dry_run, trigger, status, error, created_at, match_kind, match_reason)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`,
|
||||
o.PairID, o.Action, o.RelMaster, o.RelSlave, o.RelNew, o.QuarantinePath, o.QuarantineSide, o.TmpPath, o.Bytes,
|
||||
o.DryRun, o.Trigger, o.Status, o.Error, o.CreatedAt)
|
||||
o.DryRun, o.Trigger, o.Status, o.Error, o.CreatedAt, o.MatchKind, o.MatchReason)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
@@ -109,6 +109,11 @@ var migrations = []string{
|
||||
`ALTER TABLE pairs ADD COLUMN approx_matching INTEGER NOT NULL DEFAULT 0;
|
||||
ALTER TABLE pairs ADD COLUMN approx_threshold INTEGER NOT NULL DEFAULT 85;
|
||||
ALTER TABLE matches ADD COLUMN reason TEXT NOT NULL DEFAULT '';`,
|
||||
|
||||
// 4: remember how an operation's match was found, so undo can restore
|
||||
// matches that are not at the same path (other format, approximate, ...).
|
||||
`ALTER TABLE operations ADD COLUMN match_kind TEXT NOT NULL DEFAULT '';
|
||||
ALTER TABLE operations ADD COLUMN match_reason TEXT NOT NULL DEFAULT '';`,
|
||||
}
|
||||
|
||||
// Open opens (and migrates) the database in dataDir.
|
||||
|
||||
@@ -131,6 +131,7 @@ export type Match = {
|
||||
verdict?: Verdict;
|
||||
note: string;
|
||||
reason: string;
|
||||
link_target: string;
|
||||
ignored: boolean;
|
||||
};
|
||||
|
||||
|
||||
@@ -138,14 +138,35 @@ export function PairForm({ pair, open, onClose }: { pair?: Pair; open: boolean;
|
||||
onChange={(v) => set("approx_matching", v)}
|
||||
label={<span className="font-medium">Approximate matching</span>}
|
||||
/>
|
||||
<p className="mt-1.5 pl-11.5 text-xs text-muted">
|
||||
Also match songs whose file name is only approximately the same, and ignore disc folders. For example{" "}
|
||||
<code className="mono">Cats/Disc 02/07 Memory.flac</code> matches <code className="mono">Cats/07 Memory.mp3</code>. Version words
|
||||
must agree ("Live", "Remix", "Reprise"…), ambiguous candidates are skipped, and approximate matches always go to manual review.
|
||||
<div className="mt-1.5 space-y-3 pl-11.5 text-xs text-muted">
|
||||
<p>
|
||||
Finds songs whose file name or folder doesn't line up exactly with the master library, such as a single that also appears on an
|
||||
album, and recommends linking them to the master's album version.
|
||||
</p>
|
||||
<div className="space-y-2 rounded-md border border-line bg-bg/60 p-3">
|
||||
{approxExamples.map((ex) => (
|
||||
<div key={ex.label}>
|
||||
<div className="font-medium text-gray-300">{ex.label}</div>
|
||||
<div className="mt-0.5 grid grid-cols-[auto_1fr] gap-x-2 mono text-[11px] leading-relaxed">
|
||||
<span className="text-gray-500">slave</span>
|
||||
<span className="break-all">{ex.slave}</span>
|
||||
<span className="text-gray-500">master</span>
|
||||
<span className="break-all text-gray-200">{ex.master}</span>
|
||||
</div>
|
||||
</div>
|
||||
))}
|
||||
</div>
|
||||
<ul className="list-disc space-y-0.5 pl-4">
|
||||
<li>Different recordings never match: live, remix, acoustic, instrumental and similar versions are kept apart.</li>
|
||||
<li>When a song appears on several releases, the studio album is preferred over singles, EPs and compilations.</li>
|
||||
<li>Matches outside the artist's folder need matching artist tags. Anything ambiguous is skipped.</li>
|
||||
<li>Approximate matches are never linked automatically. Each one waits in Review.</li>
|
||||
</ul>
|
||||
</div>
|
||||
{f.approx_matching && (
|
||||
<div className="mt-3 flex flex-wrap items-center gap-x-3 gap-y-1 pl-11.5">
|
||||
<label className="whitespace-nowrap text-xs text-muted" htmlFor="approx-threshold">
|
||||
<div className="mt-4 pl-11.5">
|
||||
<div className="flex items-center gap-3">
|
||||
<label className="whitespace-nowrap text-sm font-medium" htmlFor="approx-threshold">
|
||||
Minimum title similarity
|
||||
</label>
|
||||
<input
|
||||
@@ -159,7 +180,8 @@ export function PairForm({ pair, open, onClose }: { pair?: Pair; open: boolean;
|
||||
className="w-40 accent-violet-500"
|
||||
/>
|
||||
<span className="w-10 text-sm tabular-nums">{f.approx_threshold}%</span>
|
||||
<span className="basis-full text-xs text-muted">Lower finds more matches, but also more false positives. 85% is a good default.</span>
|
||||
</div>
|
||||
<p className="mt-1 text-xs text-muted">Lower finds more matches but risks more false ones. 85% is a good default.</p>
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
@@ -199,6 +221,24 @@ export function PairForm({ pair, open, onClose }: { pair?: Pair; open: boolean;
|
||||
);
|
||||
}
|
||||
|
||||
const approxExamples = [
|
||||
{
|
||||
label: "Single → album version",
|
||||
slave: "Avenged Sevenfold/Hail to the King - Single/Hail to the King.mp3",
|
||||
master: "Avenged Sevenfold/Hail to the King/01 Hail to the King.flac",
|
||||
},
|
||||
{
|
||||
label: "Disc folders ignored",
|
||||
slave: "Avenged Sevenfold/Nightmare/04 Buried Alive.mp3",
|
||||
master: "Avenged Sevenfold/Nightmare/Disc 1/04 Buried Alive.flac",
|
||||
},
|
||||
{
|
||||
label: "Small title differences",
|
||||
slave: "Avenged Sevenfold/City of Evil/Bat Country (Remastered).mp3",
|
||||
master: "Avenged Sevenfold/City of Evil/03 Bat Country.flac",
|
||||
},
|
||||
];
|
||||
|
||||
function PathField(props: { label: string; hint: string; value: string; placeholder?: string; onChange: (v: string) => void; onBrowse: () => void }) {
|
||||
return (
|
||||
<Field label={props.label} hint={props.hint}>
|
||||
|
||||
@@ -277,9 +277,15 @@ function ComparePanel({ m, onDone }: { m: Match; onDone: () => void }) {
|
||||
const sp = m.slave_probe;
|
||||
const v = m.verdict;
|
||||
const renamed = m.rel_master.split("/").pop() !== m.rel_slave.split("/").pop();
|
||||
const dir = (p: string) => p.split("/").slice(0, -1).join("/");
|
||||
const base = (p: string) => p.split("/").pop() ?? p;
|
||||
// Cross-folder matches mirror the master's folders unless the user keeps the slave's folder.
|
||||
const [keepFolder, setKeepFolder] = useState(false);
|
||||
const movesFolder = dir(m.link_target) !== dir(m.rel_slave);
|
||||
const target = keepFolder && movesFolder ? [dir(m.rel_slave), base(m.rel_master)].filter(Boolean).join("/") : m.link_target;
|
||||
|
||||
const link = useMutation({
|
||||
mutationFn: (allow: boolean) => api.post<{ dry_run: boolean }>(`/matches/${m.id}/link`, { allow_downgrade: allow }),
|
||||
mutationFn: (allow: boolean) => api.post<{ dry_run: boolean }>(`/matches/${m.id}/link`, { allow_downgrade: allow, keep_folder: keepFolder }),
|
||||
onSuccess: (op) => {
|
||||
setConfirm(null);
|
||||
toast("ok", op.dry_run ? "Planned (dry run): nothing changed" : "Linked. The original slave file is in quarantine.");
|
||||
@@ -357,10 +363,19 @@ function ComparePanel({ m, onDone }: { m: Match; onDone: () => void }) {
|
||||
slave's better copy goes to quarantine), or override and make the slave's copy the master file.
|
||||
</Alert>
|
||||
)}
|
||||
{renamed && (
|
||||
{target !== m.rel_slave && (
|
||||
<Alert tone="blue">
|
||||
File names differ. Linking puts <code className="mono">{m.rel_master.split("/").pop()}</code> into the slave folder in place of{" "}
|
||||
<code className="mono">{m.rel_slave.split("/").pop()}</code>, so playlists that point at the old name will need updating.
|
||||
<div>
|
||||
Linking places the file at <code className="mono break-all text-white">{target}</code> in the slave library
|
||||
{movesFolder && !keepFolder ? ", matching the master's folders. The old folder is removed if it ends up empty." : "."} Playlists
|
||||
that point at the old path will need updating.
|
||||
</div>
|
||||
{movesFolder && (
|
||||
<label className="mt-2 flex items-center gap-2 text-xs cursor-pointer">
|
||||
<input type="checkbox" className="accent-violet-500" checked={keepFolder} onChange={(e) => setKeepFolder(e.target.checked)} />
|
||||
Keep it in the slave's current folder instead: <code className="mono">{dir(m.rel_slave) || "library root"}</code>
|
||||
</label>
|
||||
)}
|
||||
</Alert>
|
||||
)}
|
||||
</div>
|
||||
|
||||
Reference in new issue
Block a user