[Bug]: Navidrome Splitting Albums (not the FAQ issue) about navidrome HOT 28 CLOSED

After further investigation, this seems to be related to
#2679
and
#2674

And is hopefully fixed by
#2680

I'll check once #2680 makes it into the the latest Docker build.

from navidrome.

I can confirm that version 0.51.0 (fd61b29) fixes this on my setup. Thanks for everyone's help.

from navidrome.

Update on this. I was also pointed to the ReleaseDate tag in song files as that can pose issues. While I can't find any differences in that tag among the songs that are getting split out (in fact, I don't see that tag being set in any of the affected files), I was able to set the Docker environment variable ND_SCANNER_GROUPALBUMRELEASES=true, and that seems to have fixed the issue. I still can't figure out why the app is picking up my albums as being multiple releases, though.

from navidrome.

are there multiple file formats? like a mixture of mp3, ogg, Flac etc?

from navidrome.

Maybe the release dates are different? By default, Navidrome splits albums by release dates, to avoid merging different releases into one. See #2668

from navidrome.

are there multiple file formats? like a mixture of mp3, ogg, Flac etc?

Nope. All are m4a.

Maybe the release dates are different? By default, Navidrome splits albums by release dates, to avoid merging different releases into one. See #2668

I'm not seeing a "release date" tag (I've looked in mp3tag, MusicBrainz Picard, and iTunes). If you have a rec on software that will show me all tags, I will check. Picard did show an "original release date", which is the same on all songs in the album.

from navidrome.

Check the all date tags (date, year, originaldate, releasedate, etc...). They must all match. As discussed in #2668, you can also set Scanner.GroupAlbumReleases to true if you don't have multiple releases of the same album in your collection.

from navidrome.

(date, year, originaldate, releasedate, etc...). They must all match.

I mean, that can't possibly be right, can it? I get why people would want to split by releasedate, but "date" is different from song to song all the time. What about Greatest Hits albums where the songs come from across decades of standard albums? I will use eyed3 when I have a minute and report back on what tags are different among the songs.

from navidrome.

Yes, you are right, Date and Year will not split albums.

A quick way to validate if the dates are the issue is just setting Scanner.GroupAlbumReleases=true, restartind Navidrome and checking if the split still happens. If it does, then releasedate/releaseyear or originaldate/originalyear maybe be not be matching.

from navidrome.

I have already fixed my immediate issue using Scanner.GroupAlbumReleases=true (or, more technically, ND_SCANNER_GROUPALBUMRELEASES=true since I am using Docker). This DOES fix the issue. And I don't personally care to have different "releases" split into different albums. I have all my albums named uniquely to the extent I want them separated (e.g., "Greatest Hits [Artist Name]").

I'm mostly just trying to bug squash at this point, so here's what I've found.

The issue DOES seem to be related to the Year tag. When I use mp3tag to set the Year to be the same for all of the songs in one of the affected albums, they combine into one album within Navidrome.

The most baffling thing about this is that I then went back to a Greatest Hits album that has the Year tagged differently for each song, and it did NOT split that album. So it's only sometimes splitting an album based on Year???

I'm going to do a bit more searching to see if I can figure out any differences between those albums that are affected and those that are not, but it does have at least something to do with the Year tag. My personal recommendation would be that the software should never consider the Year tag when deciding whether to group songs in an album because there are a host of reasons you would want the year to be different within an album. Release Year is a different story, for reasons that have been discussed in numerous other threads.

from navidrome.

I'm at a loss. I found an album that is itself illustrating the problem, and it's just baffling.

The songs as I have them have three Date tags according to MusicBrainz Picard. This seems to be the same as the Year tag in mp3tag:

1. "1973"
2. "2000-09-26"
3. "1974"

When I load these songs into Navidrome, 1 and 3 combine into a single album. 2 goes into it's own album.

If I update #2 to be "1976" (all tagging done in mp3tag), they all group into the same album. I seem to be able to set it to anything from blank to 1 to 1999 and it will group it with the other two. But if I set it to 2000 or greater, it will split it out.

Here's where it gets really weird. If I set #2 to be, say, 2005, it STILL shows up in Navidrome as being "Year" 2000. If I set all three to 2005, all three songs are grouped in the same album, but all three show up as the "Year" 2000. If I set two of them to 2005 and the last to 2010, they split into two albums, but they all say "Year" 2000 within Navidrome.

I'm just spitballing here, but there seems to be a second date field that Navidrome is pulling from conditionally. Perhaps it is pulling the earlier of the two dates? And somehow it has some complicated ruleset for determining whether to split the songs? If it pulls all three from the "Year"/"Date" field, it will allow grouping even if the the date is different. But the second it starts pulling from this mystery second field, it gets very particular???

For what it's worth, "Original Release Date" and "Release Year" are both set for all three songs. The former is set for all three songs to be "2000-09-26", and the latter to "2000". So it seems like Navidrome is sometimes pulling this information to use as the "Year" of the song (e.g., if the "Year"/"Date" field is set to blank or to a later date), and when it does this it wants to be particular about the date when combining songs into an album.

So I think there may be a bug in how Navidrome selects which date tag to use and display in the "Year" field. I would suggest this should be the "Year" (or "Date" per MusicBrainz Picard terminology) IF SET, and only if not set to fall back to another field. (Though this doesn't fully explain the bug because, as I say above, I can set song #2 to have a blank "Year", it will display the 2000 year in Navidrome, but it will still group all three songs together.)

The thing is, the date chosen for display in the web interface should not matter for those who want their albums split by releases. I could have an album where all the songs have the same "Date" and display the same "Date", but the release dates are different, and I would want them split out.

from navidrome.

More possibly useful info:

The original album I was having issues with (Lover, screenshot above) was an issue with how Navidrome handles the Year field as well.

All songs in the album have an Original Release Date of "2019-08-23" and an Original Year of "2019". But they have different Date (MusicBrainz Picard field naming) with the following values:

1. 2019-08-23T07:00:00Z (tracks 1-2, 4, 6-13, 15, 17-18)
2. 2019-08-16T07:00:00Z (track 3)
3. 2019-07-23T07:00:00Z (track 5)
4. 2019-06-14T07:00:00Z (tracks 14)
5. 2019-04-26T07:00:00Z (tracks 16)

Know that I did not enter these values, so I believe these were coded by whoever sold me the songs (probably Apple since these are m4a files).

Perhaps unsurprisingly given my findings above, it is these 5 groupings that show up as 5 different albums in Navidrome DESPITE the fact that all the songs have the same release date.

Even if I remove the "T07...." time information, the album is still split based on the Year/Date tag.

Once again, if I set the Date to anything before 2019 (specifically, 1 Jan 2019), it will group the songs together. This is true whether I set just the year, the year and date, or the year and date and time. So "2018" works, but "2019" does not. "2018-12-31" but not "2019-01-01". "2018-12-31T11:59:59Z" but not "2019-01-01:00:00:00Z".

So it does seem to be related to whether it considers the Date or Original Release Date (or possibly Original Year) to be earlier. And then does some weird things where it thinks the Date is after the "original" date.

I suspect this is not intended behavior.

from navidrome.

So this is the logic used to map dates from tags:

And then, releaseDate is used to disambiguate albums, here:

I think @certuna can explain the decisions in these mappings a bit more in depth. I know most of it is related to a bug in Picard he was investigating. Maybe that's the part that is getting you?

from navidrome.

I have a hunch I've found the problem. I'm not exactly sure what each line of code means, but it seems to be making the Picard correction when the year (aka date) is greater than OR EQUAL TO the originalyear. This seems to be causing problems for me. Because I think it sees my songs that have different year but same originalyear that are all within the same year, and decides to overwrite the originalyear with the year and then decide whether to break up the album. So I think changing this line (year >= originalYear) to (year > originalYear) will fix the problem I'm seeing because it will leave my dates intact if they are sensible (i.e., the song is not tagged to have been made after the album was released). Now, I don't know if this will cause some other problem that this code was designed to fix in the first place. But I think this is the issue I'm experiencing.

Stated another way, when song has year 2019-08-23 and originalyear 2019, this code says to group by year instead of originalyear which breaks being able to code an entire album by originalyear and expect it to be considered a single album.

I know it's weird that my year is actually a date, but that's the way these files came from the factory.

I figured out why there isn't an uproar about this issue that I seem to be the only one experiencing. I have tagged my music over the decades, and have just recently started re-tagging with Picard. I very quickly realized that Picard was doing bad things to my year and/or date tags, so I told it never to update that tag.

So normal Picard usage would update that tag and put in what Navidrome is expecting as releaseyear, and this bit of code takes that and puts it back into releaseyear. But in my setup, it THINKS Picard has put the release year in the year field, and is updating accordingly, but really Picard only put in the originalyear.

I still think changing >= to > will solve 99% of the problem. But maybe I'm missing a use case where the = is needed.

I'm sort of stream-of-consciousnessing here, so forgive me. But I just realized that changing >= to > might swallow the rule and often not make the Picard correction (which I think should be officially called the Picard Maneuver). The Picard Maneuver does seem like an over-correction, though. The problem with Picard is that it apparently puts releaseyear into year, but the fix is to fiddle with metadata most any time there is an originalyear but no releaseyear. There are seemingly many instances when one would have an originalyear but not a releaseyear, so it seems like a cannon for a fly. What are people using to tag their music? I thought Picard was practically the consensus recommendation.

Possible solutions:

• Group albums by releaseyear instead of releasedate. Does it really come up that there are multiple releases within the same year?
• Go with my original thought of removing the "=". If originalyear is the same year as releaseyear (as improperly put into the year field by Picard), do we really need to move it back, or can we just fall back on originalyear? This may not swallow the rule as much as I suggest in the previous paragraph as there are undoubtedly situations when the releaseyear (as improperly put into the year field by Picard) will indeed be after the originalyear. And if it's not, do we really need releaseyear?
• Remove the Picard Maneuver entirely. How many people are tagging their music with the expectation that it's going to group albums by releasedate without ACTUALLY adding a releasedate tag??

from navidrome.

OK apologies I didn't see this earlier, here goes:

• the reason that the TaggedLikePicard logic is in there, is because Picard puts dates in the wrong fields: it puts the re-release date of the album that should be in releasedate in date, which is a field that's supposed to hold the recording date. Because ND sorts by date (it's the most logical date to sort on), this means those albums get sorted wrong, often decades later.
• to correct for this, ND tries to 'intelligently' detect when this happens (originaldate < date, no releasedate), and re-maps date to releasedate, and originaldate to date (which is the one we sort on)
• in your case however, your files are not 'wrongly tagged like Picard' but the 'intelligence' makes the mistake to map your date to releasedate even though it shouldn't.

There's a few solutions I see:

• you manually tag your files with a releasedate (in any tag editor, mp3tag/Picard/Yate/kid3/etc)
• we introduce stricter detection of TaggedLikePicard: we could for example check if there's an MbzAlbumID tag, if there is none we don't do the date -> releasedate remapping
• remove the TaggedLikePicard logic entirely, and for those files rely on the grouping by Musicbrainz Album ID. But a) that doesn't fix the sorting issue: date still has the wrong date in it, and b) grouping by MBZ Album ID isn't implemented yet in Navidrome

Does it really come up that there are multiple releases within the same year?

Yes, a lot. Just have a look around in musicbrainz.org , you'll see tons of them. US/European releases with different tracklists, Japanese editions with bonustracks, various versions of singles with different remixes.

Remove the Picard Maneuver entirely. How many people are tagging their music with the expectation that it's going to group albums by releasedate without ACTUALLY adding a releasedate tag??

A lot, because if you tag with Picard, you have put the release date in date, and you would expect albums to be split by that. So you have to remap it.

from navidrome.

Does changing >= to > work? This would more accurately catch those who are blindly tagging with Picard (releasedate for this version is after the originaldate for the first time the album was released), but not people who are being more discerning like me (date for this recording is the same as the originaldate for the first time the album was released).

Another suggestion, add a setting that can be selected by admins, such as "Detect Picard Tagging" and add that as another condition for applying the Picard Maneuver. I'm happy for this to be on by default as long as I can disable it.

Also, what is the exact name of the tag that needs to be set? I'm not having any luck getting my tracks to combine into a single album by setting releasedate. The code above uses releaseYear, but that seems to be a variable name and not necessarily the tag name. I'm in the middle of tagging my music, so if I can get this right, I can at least have half my collection correct and then go back and re-tag the first half.

Your second suggestion of stricter detection won't work because my tracks all have a MusicBrainz Album ID as well. They're going to look just like MusicBrainz-tagged tracks except for the 'date' field, which I've corrected.

I like the third option. Obviously it would be up to the devs to implement group by MBZ ID, but this would seemingly be more accurate. And to your concern that it will give a wrong sorting date, if someone is putting a date in date, they should expect the song to sort by that date. That's the way every other system I've seen does it, and users can't really expect anything else. In fact, they would probably be unhappy if songs were sorting by something else.

But, that could be solved by allowing the web interface to add columns beyond the handful that are currently available. It would be a nice feature to be able to add other tags as columns entirely aside from this issue. So if you could pick virtually any tag to display as a column and then sort by it, you could display releasedate or originaldate and sort by that if that's the way your music is set up and what works best for you.

Another solution: prevent MBZ from setting an originaldate. If there's no originaldate, the Picard Maneuver doesn't kick in.

from navidrome.

I'm still not getting expected results, even after fixing tags:
I have set a releasedate tag for all songs, and it is the same for all songs:

This has been confirmed in both mp3tag (where the image is from) and Picard.
The date tag (Picard terminology)/year tag (mp3tag terminology) is different from song to song, and ND continues to split the album based on date/year.

Is there a separate release year tag I need to set (setting releaseyear, releaseYear, release_year, and release_Year did nothing)?

from navidrome.

if someone is putting a date in date, they should expect the song to sort by that date. That's the way every other system I've seen does it, and users can't really expect anything else. In fact, they would probably be unhappy if songs were sorting by something else.

Unfortunately not, this is the whole problem. For songs auto-tagged with Picard (there's billions out there), people will expect the sort date to be the value in originaldate and not date. Think of a 2005 reissue of the 1973 recorded and originally released album "Dark Side Of The Moon". Picard will tag this originaldate = 1973, date = 2005. Users expect this album and its songs to be sorted/filtered as 1973, not 2005.

Of course, Picard does it wrong and should tag this as originaldate = 1973, date = 1973, releasedate = 2005. You can set it do that in the config (settings -> scripting), for example with:

$set(releasedate,%date%) <-- this writes the (re)release date to releasedate
$set(date,%_recording_firstreleasedate%) <-- this writes individual track dates to date
$set(originaldate,%originaldate%) <-- this writes the album's original release date to originaldate

but that's not what Picard does by default. So for those billions of Picard default tagged files, we have to remap, otherwise we'll have tons of users complain that Navidrome sorts albums and songs wrong.

Also, what is the exact name of the tag that needs to be set?

TDRL in id3v2.4, RELEASEDATE in FLAC. These two are mapped by TagLib to the generic variable releasedate, which is what Navidrome looks for.

I'm still not getting expected results, even after fixing tags:
I have set a releasedate tag for all songs, and it is the same for all songs:

this is strange - can you send me (one of) the files (through Discord)?

from navidrome.

Further experimenting suggests the problem with songs not combining even after adding a releasedate might be an issue specific to the way ND handles m4a files.

from navidrome.

I figured out why there isn't an uproar about this issue that I seem to be the only one experiencing

Actually I've been having this issue too, for several weeks now; the problem started with release 0.50.0 and was so intractable that I reverted to an earlier release of Navidrome and deleted my database. I'm really glad to see it being worked on so diligently, as a fix will permit me to resume updating to newer releases. Thanks for persevering.

(Now that you-all have got this tracked down to the release date, at least in part, I understand what my problem likely was: I have a large collection of podcast episodes, each podcast spanning many years, and so this issue was probably splitting each of them up by release date even though each group of podcasts had the same "album" title.)

from navidrome.

I figured out why there isn't an uproar about this issue that I seem to be the only one experiencing

Actually I've been having this issue too, for several weeks now; the problem started with release 0.50.0 and was so intractable that I reverted to an earlier release of Navidrome and deleted my database. I'm really glad to see it being worked on so diligently, as a fix will permit me to resume updating to newer releases. Thanks for persevering.

I am affected by the issue too. Usually it's a mixed mode album where I have mp3 and m4a's of the same album but different tracks. I lost my library when a computer crashed a few years ago, but I was using iTunes Match at the time, so I was able to recover everything, but now some albums are fixed file formats. I didn't want to go through the hassle of re-ripping 1,000s of CD's. At first I used the ENV setting to merge them, but I figured it would be better for the long run to make the metadata match and have been using Picard to fix everything (also updating the year when it is wrong). I have been watching this thread.. so thanks all for reporting and tinkering...

from navidrome.

I have an album like this, where some songs have different artist names, but all these songs belong to the same album. So, I examined this piece of code and found it unable to successfully differentiate this situation. According to this code logic, as long as there are different artist names, they will be split into different albums. There is a logic issue here that needs to be addressed by fixing the code.

https://github.com/navidrome/navidrome/blob/6f7b48202e9135378f3696f28b8f49595f8a8f75/scanner/mapping.go#L130C1-L138C2

I think this piece of code should be modified as follows:

func (s MediaFileMapper) albumID(md metadata.Tags, releaseDate string) string {
	albumPath := s.mapAlbumName(md)
	if !conf.Server.Scanner.GroupAlbumReleases {
		if len(releaseDate) != 0 {
			albumPath = fmt.Sprintf("%s\\%s", albumPath, releaseDate)
		}
	} else {
		albumPath = strings.ToLower(fmt.Sprintf("%s\\%s", albumPath, s.mapAlbumArtistName(md)))
	}
	return fmt.Sprintf("%x", md5.Sum([]byte(albumPath)))
}

from navidrome.

Albums with multiple “artists” will sort together. You just need to make sure that the “album artist” is the same for the whole album. If you have a true compilation, you can use something like “Various Artists” for the album artist. I would think splitting by album artist (or artist if there is no album artist) would be the preferred method so that two “Greatest Hits” albums from two different artists don’t merge.

from navidrome.

Albums with multiple “artists” will sort together. You just need to make sure that the “album artist” is the same for the whole album. If you have a true compilation, you can use something like “Various Artists” for the album artist. I would think splitting by album artist (or artist if there is no album artist) would be the preferred method so that two “Greatest Hits” albums from two different artists don’t merge.

I understand. I apologize for not noticing the additional 'Album Artist' field, aside from the 'Album' and 'Artist' fields. I've observed that songs categorized incorrectly often only have information in the 'Album' and 'Artist' fields, while the 'Album Artist' field is empty. Is there a way to group these songs into the same album solely based on the similarity of the 'Album' field?

from navidrome.

You could try the GroupAlbumReleases setting mentioned above to see if it does what you need, but it may not. You may need to tag affected albums with an Album Artist. You could probably write a short script to take the artist from the first song in each folder (aka album) of your collection and populate it into the album artist field for all songs in that folder. Or if you really wanted to be crazy, you could use a tool like mp3tag or MusicBrainz Picard to add the same album artist to EVERY track in your collection.

from navidrome.

You could try the GroupAlbumReleases setting mentioned above to see if it does what you need, but it may not. You may need to tag affected albums with an Album Artist. You could probably write a short script to take the artist from the first song in each folder (aka album) of your collection and populate it into the album artist field for all songs in that folder. Or if you really wanted to be crazy, you could use a tool like mp3tag or MusicBrainz Picard to add the same album artist to EVERY track in your collection.

I understand, although modifying EVERY track would be quite troublesome, there isn't a better solution for now. I will try to do this. Thank you very much for your patient response.

from navidrome.

Is there a way to group these songs into the same album solely based on the similarity of the 'Album' field?

That won't work, you'll have every album called "Greatest Hits" grouped as one album.

As you can see from that code snippet, Navidrome uses Album Name, Album Artist and (optionally) Release Date as the splitting criteria.

from navidrome.

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

from navidrome.