[Bug]: New Scanner paths issue (causes duplicated albums) #3788
Description
I confirm that:
- I have searched the existing open AND closed issues to see if an issue already exists for the bug I've encountered
- I'm using the latest version (your issue may have been fixed already)
Version
0.55.0-SNAPSHOT (source_archive)
Current Behavior
Looking forwards to this! Album disambiguation affordances is something I've been looking forwards to, along with the promise of not losing track favorites because of tags updates or file moving.
Thank you for your continued efforts!
Something has gone wrong with path construction with the latest "BFR" version
Before deploying widely, I run on a subset of my music library with just one artist
Using the default new PID behavior [ie not setting PID.Album] I saw every album duplicated, with one version which would not play.
Setting
PID.Album= "album_legacy"
only one version of each album was created, but they were all broken.
Digging in a little, comparing the logs where I could see bogus filenames attempted to be accessed, and using "show full path" in the web interface, I can now see that it appears one character is being deleted from the path, as if the path splicing between MusicFolder and paths found by the scanner is broken.
If
MusicFolder = "/tank/whorfin/Audio/Musiq/yelworC/"
then the broken album version contains track paths such as:
/tank/whorfin/Audio/Musiq/yelworC//990 - A.I.W.A.S.S (Cassette, C60)/01-great_volunteers.mp3
while the working version of each album has paths such as:
/tank/whorfin/Audio/Musiq/yelworC//1990 - A.I.W.A.S.S (Cassette, C60)/01-great_volunteers.mp3
As an aside it seem MusicFolder paths aren't normalized... if I remove the trailing slash, things are broken in exactly the same way, but the double slash makes it rather clear what seems to be happening - the first character is eaten, breaking the media path.
Expected Behavior
I would expect media paths not to be broken.
The broken paths seem to co-exist with unbroken paths with non-legacy PID.Album behavior, resulting in duplicated albums, where half the duplicates are broken. Using PID.Album="album_legacy", only one version is created, but it is broken.
Steps To Reproduce
No response
Environment
- OS: OmniOS r151046co [LTS]
- Browser: broken in all browsers and clients, as it is a database/paths issue
- Client: doesn't matterHow Navidrome is installed?
Built from sources
Configuration
# simple config for testing
Port = 14533 # instead of 4533
MusicFolder = "/tank/whorfin/Audio/Musiq/yelworC"
DataFolder = "/home/whorfin/navidrome/navidrome-0.55.0/0runhere"
ImageCacheSize = 0
TranscodingCacheSize = 0
RecentlyAddedByModTime = true
PasswordEncryptionKey = "redacted lol"
EnableInsightsCollector = false
DefaultReportRealPath = true
Scanner.Schedule = "@weekly"
Subsonic.DefaultReportRealPath = true
Scanner.WatcherWait = "5m"Relevant log output
Here's a quick attempt to just grab the relevant bits of the log mentioning one affected track; i restored the database then ran with log level "trace"
$ grep great_volunteer rawlog.txt
time="2025-03-09T23:04:08-07:00" level=trace msg="Scanner: Found directory" path="1990 - A.I.W.A.S.S (Cassette, C60)" audioFiles="[`12-rage_hard.mp3`,`13-earth fire.mp3`,`02-active_life.mp3`,`03-spy_vs_spy.mp3`,`04-face_to_face.mp3`,`05-bloodthirst.mp3`,`06-mind_section_ii.mp3`,`07-nausea.mp3`,`08-data_control_ii.mp3`,`11-body_shift.mp3`,`01-great_volunteers.mp3`,`09-inquest.mp3`,`10-get_vomit.mp3`]" images="[`R-1228378-1202168700.jpg`,`cover.jpg`]" imagesUpdatedAt="2021-11-27 17:45:09.3446346 -0800 PST" modTime="2021-11-27 17:45:09.3446346 -0800 PST" numChildren=0 playlists=0 updTime="2025-03-09 23:04:06 +0000 UTC"
time="2025-03-09T23:04:08-07:00" level=trace msg="extractor: reading tags" filename="/tank/whorfin/Audio/Musiq/yelworC/1990 - A.I.W.A.S.S (Cassette, C60)/01-great_volunteers.mp3" map_id=112
time="2025-03-09T23:04:08-07:00" level=trace msg="extractor: read tags" filename="/tank/whorfin/Audio/Musiq/yelworC/1990 - A.I.W.A.S.S (Cassette, C60)/01-great_volunteers.mp3" id=112 tags="{\"_bitrate\":[\"192\"],\"_channels\":[\"2\"],\"_lengthinmilliseconds\":[\"259030\"],\"_samplerate\":[\"44100\"],\"album\":[\"A.I.W.A.S.S Tape\"],\"albumartist\":[\"yelworC\"],\"artist\":[\"yelworC\"],\"comment\":[\"\"],\"date\":[\"1990\"],\"genre\":[\"Electronic\"],\"has_picture\":[\"true\"],\"title\":[\"Great Volunteers\"],\"tracknumber\":[\"1\"]}"
time="2025-03-09T23:04:09-07:00" level=trace msg="SQL: `UPDATE album SET album_artist = {:p0}, album_artist_id = {:p1}, catalog_num = {:p2}, comment = {:p3}, compilation = {:p4}, date = {:p5}, description = {:p6}, discs = {:p7}, duration = {:p8}, embed_art_path = {:p9}, explicit_status = {:p10}, external_info_updated_at = {:p11}, external_url = {:p12}, folder_ids = {:p13}, full_text = {:p14}, genre = {:p15}, id = {:p16}, imported_at = {:p17}, large_image_url = {:p18}, library_id = {:p19}, max_original_year = {:p20}, max_year = {:p21}, mbz_album_artist_id = {:p22}, mbz_album_comment = {:p23}, mbz_album_id = {:p24}, mbz_album_type = {:p25}, mbz_release_group_id = {:p26}, medium_image_url = {:p27}, min_original_year = {:p28}, min_year = {:p29}, missing = {:p30}, name = {:p31}, order_album_artist_name = {:p32}, order_album_name = {:p33}, original_date = {:p34}, participants = {:p35}, release_date = {:p36}, size = {:p37}, small_image_url = {:p38}, song_count = {:p39}, sort_album_artist_name = {:p40}, sort_album_name = {:p41}, tags = {:p42}, updated_at = {:p43} WHERE id = {:p44}`" args="map[p0:yelworC p1:29TfAbE1p0dZWRemh5Veyb p10: p11:<nil> p12: p13:[\"5GZoVUFKYSbD6GiRcuGcqu\"] p14: a.i.w.a.s.s tape yelworc p15: p16:44a55T6CHiasWdG8fnvBoG p17:2025-03-09 23:04:09.062312414 -0700 PDT m=+0.859597093 p18: p19:1 p2: p20:0 p21:1990 p22: p23: p24: p25: p26: p27: p28:0 p29:1990 p3: p30:false p31:A.I.W.A.S.S Tape p32:yelworc p33:a.i.w.a.s.s tape p34: p35:{\"albumartist\":[{\"id\":\"29TfAbE1p0dZWRemh5Veyb\",\"name\":\"yelworC\"}],\"artist\":[{\"id\":\"29TfAbE1p0dZWRemh5Veyb\",\"name\":\"yelworC\"}]} p36: p37:84796080 p38: p39:13 p4:false p40: p41: p42:{\"genre\":[{\"id\":\"7bLYq0Np81m1Wgy5N31nuG\",\"value\":\"Electronic\"}]} p43:2021-08-21 10:49:41.342255 -0700 PDT p44:44a55T6CHiasWdG8fnvBoG p5:1990 p6: p7:{\"1\":\"\"} p8:3510.94 p9:1990 - A.I.W.A.S.S (Cassette, C60)/01-great_volunteers.mp3]" elapsedTime="578.634µs" rowsAffected=0
time="2025-03-09T23:04:09-07:00" level=trace msg="SQL: `INSERT INTO album (album_artist,album_artist_id,catalog_num,comment,compilation,created_at,date,description,discs,duration,embed_art_path,explicit_status,external_info_updated_at,external_url,folder_ids,full_text,genre,id,imported_at,large_image_url,library_id,max_original_year,max_year,mbz_album_artist_id,mbz_album_comment,mbz_album_id,mbz_album_type,mbz_release_group_id,medium_image_url,min_original_year,min_year,missing,name,order_album_artist_name,order_album_name,original_date,participants,release_date,size,small_image_url,song_count,sort_album_artist_name,sort_album_name,tags,updated_at) VALUES ({:p0},{:p1},{:p2},{:p3},{:p4},{:p5},{:p6},{:p7},{:p8},{:p9},{:p10},{:p11},{:p12},{:p13},{:p14},{:p15},{:p16},{:p17},{:p18},{:p19},{:p20},{:p21},{:p22},{:p23},{:p24},{:p25},{:p26},{:p27},{:p28},{:p29},{:p30},{:p31},{:p32},{:p33},{:p34},{:p35},{:p36},{:p37},{:p38},{:p39},{:p40},{:p41},{:p42},{:p43},{:p44})`" args="map[p0:yelworC p1:29TfAbE1p0dZWRemh5Veyb p10:1990 - A.I.W.A.S.S (Cassette, C60)/01-great_volunteers.mp3 p11: p12:<nil> p13: p14:[\"5GZoVUFKYSbD6GiRcuGcqu\"] p15: a.i.w.a.s.s tape yelworc p16: p17:44a55T6CHiasWdG8fnvBoG p18:2025-03-09 23:04:09.062312414 -0700 PDT m=+0.859597093 p19: p2: p20:1 p21:0 p22:1990 p23: p24: p25: p26: p27: p28: p29:0 p3: p30:1990 p31:false p32:A.I.W.A.S.S Tape p33:yelworc p34:a.i.w.a.s.s tape p35: p36:{\"albumartist\":[{\"id\":\"29TfAbE1p0dZWRemh5Veyb\",\"name\":\"yelworC\"}],\"artist\":[{\"id\":\"29TfAbE1p0dZWRemh5Veyb\",\"name\":\"yelworC\"}]} p37: p38:84796080 p39: p4:false p40:13 p41: p42: p43:{\"genre\":[{\"id\":\"7bLYq0Np81m1Wgy5N31nuG\",\"value\":\"Electronic\"}]} p44:2021-08-21 10:49:41.342255 -0700 PDT p5:2025-03-09 23:04:08.721743913 -0700 PDT m=+0.519020700 p6:1990 p7: p8:{\"1\":\"\"} p9:3510.94]" elapsedTime="495.043µs" rowsAffected=1
time="2025-03-09T23:04:09-07:00" level=trace msg="SQL: `SELECT id FROM media_file WHERE library_id = {:p0} AND path = {:p1}`" args="map[p0:1 p1:1990 - A.I.W.A.S.S (Cassette, C60)/01-great_volunteers.mp3]" elapsedTime="164.561µs" rowsAffected=0
time="2025-03-09T23:04:09-07:00" level=trace msg="SQL: `INSERT INTO media_file (album,album_artist,album_artist_id,album_id,artist,artist_id,birth_time,bit_depth,bit_rate,bpm,catalog_num,channels,comment,compilation,created_at,date,disc_number,disc_subtitle,duration,explicit_status,folder_id,full_text,genre,has_cover_art,id,library_id,lyrics,mbz_album_artist_id,mbz_album_comment,mbz_album_id,mbz_album_type,mbz_artist_id,mbz_recording_id,mbz_release_group_id,mbz_release_track_id,missing,order_album_artist_name,order_album_name,order_artist_name,order_title,original_date,original_year,participants,path,pid,release_date,release_year,rg_album_gain,rg_album_peak,rg_track_gain,rg_track_peak,sample_rate,size,sort_album_artist_name,sort_album_name,sort_artist_name,sort_title,suffix,tags,title,track_number,updated_at,year) VALUES ({:p0},{:p1},{:p2},{:p3},{:p4},{:p5},{:p6},{:p7},{:p8},{:p9},{:p10},{:p11},{:p12},{:p13},{:p14},{:p15},{:p16},{:p17},{:p18},{:p19},{:p20},{:p21},{:p22},{:p23},{:p24},{:p25},{:p26},{:p27},{:p28},{:p29},{:p30},{:p31},{:p32},{:p33},{:p34},{:p35},{:p36},{:p37},{:p38},{:p39},{:p40},{:p41},{:p42},{:p43},{:p44},{:p45},{:p46},{:p47},{:p48},{:p49},{:p50},{:p51},{:p52},{:p53},{:p54},{:p55},{:p56},{:p57},{:p58},{:p59},{:p60},{:p61},{:p62})`" args="map[p0:A.I.W.A.S.S Tape p1:yelworC p10: p11:2 p12: p13:false p14:2025-03-09 23:04:09.106000695 -0700 PDT m=+0.903286392 p15:1990 p16:0 p17: p18:259.03 p19: p2:29TfAbE1p0dZWRemh5Veyb p20:5GZoVUFKYSbD6GiRcuGcqu p21: a.i.w.a.s.s great tape volunteers yelworc p22: p23:true p24:bZVxV8Zuv5R80OIAu8TVMP p25:1 p26:[] p27: p28: p29: p3:44a55T6CHiasWdG8fnvBoG p30: p31: p32: p33: p34: p35:false p36:yelworc p37:a.i.w.a.s.s tape p38:yelworc p39:great volunteers p4:yelworC p40: p41:0 p42:{\"albumartist\":[{\"id\":\"29TfAbE1p0dZWRemh5Veyb\",\"name\":\"yelworC\"}],\"artist\":[{\"id\":\"29TfAbE1p0dZWRemh5Veyb\",\"name\":\"yelworC\"}]} p43:1990 - A.I.W.A.S.S (Cassette, C60)/01-great_volunteers.mp3 p44:6ws7GPQHnRZwSwPEZo0yOE p45: p46:0 p47:0 p48:1 p49:0 p5:29TfAbE1p0dZWRemh5Veyb p50:1 p51:44100 p52:6257700 p53: p54: p55: p56: p57:mp3 p58:{\"genre\":[{\"id\":\"7bLYq0Np81m1Wgy5N31nuG\",\"value\":\"Electronic\"}]} p59:Great Volunteers p6:2025-03-09 23:04:08.724328652 -0700 PDT m=+0.521606355 p60:1 p61:2021-08-21 10:49:10.8249688 -0700 PDT p62:1990 p7:0 p8:192 p9:0]" elapsedTime="754.09µs" rowsAffected=1
time="2025-03-09T23:04:09-07:00" level=trace msg="SQL: `UPDATE album SET album_artist = {:p0}, album_artist_id = {:p1}, catalog_num = {:p2}, comment = {:p3}, compilation = {:p4}, date = {:p5}, description = {:p6}, discs = {:p7}, duration = {:p8}, embed_art_path = {:p9}, explicit_status = {:p10}, external_info_updated_at = {:p11}, external_url = {:p12}, folder_ids = {:p13}, full_text = {:p14}, genre = {:p15}, id = {:p16}, imported_at = {:p17}, large_image_url = {:p18}, library_id = {:p19}, max_original_year = {:p20}, max_year = {:p21}, mbz_album_artist_id = {:p22}, mbz_album_comment = {:p23}, mbz_album_id = {:p24}, mbz_album_type = {:p25}, mbz_release_group_id = {:p26}, medium_image_url = {:p27}, min_original_year = {:p28}, min_year = {:p29}, missing = {:p30}, name = {:p31}, order_album_artist_name = {:p32}, order_album_name = {:p33}, original_date = {:p34}, participants = {:p35}, release_date = {:p36}, size = {:p37}, small_image_url = {:p38}, song_count = {:p39}, sort_album_artist_name = {:p40}, sort_album_name = {:p41}, tags = {:p42}, updated_at = {:p43} WHERE id = {:p44}`" args="map[p0:yelworC p1:29TfAbE1p0dZWRemh5Veyb p10: p11:<nil> p12: p13:[\"5GZoVUFKYSbD6GiRcuGcqu\"] p14: a.i.w.a.s.s tape yelworc p15: p16:44a55T6CHiasWdG8fnvBoG p17:2025-03-09 23:04:09.513609341 -0700 PDT m=+1.310904024 p18: p19:1 p2: p20:0 p21:1990 p22: p23: p24: p25: p26: p27: p28:0 p29:1990 p3: p30:false p31:A.I.W.A.S.S Tape p32:yelworc p33:a.i.w.a.s.s tape p34: p35:{\"albumartist\":[{\"id\":\"29TfAbE1p0dZWRemh5Veyb\",\"name\":\"yelworC\"}],\"artist\":[{\"id\":\"29TfAbE1p0dZWRemh5Veyb\",\"name\":\"yelworC\"}]} p36: p37:84796080 p38: p39:13 p4:false p40: p41: p42:{\"genre\":[{\"id\":\"7bLYq0Np81m1Wgy5N31nuG\",\"value\":\"Electronic\"}]} p43:2021-08-21 10:49:41.342255 -0700 -0700 p44:44a55T6CHiasWdG8fnvBoG p5:1990 p6: p7:{\"1\":\"\"} p8:3510.94 p9:1990 - A.I.W.A.S.S (Cassette, C60)/01-great_volunteers.mp3]" elapsedTime=1.3ms rowsAffected=1
Anything else?
I should add that if I use the default PID.Album behavior, the busted albums seem to often not have an Artist, and in such a situation they don't appear when listing all Albums by a given artist. But in the "All" view for music, they are clearly visible, at least in my small test library.
Code of Conduct
- I agree to follow Navidrome's Code of Conduct