Code Monkey home page Code Monkey logo

anime-offline-database's People

Contributors

beeequeue avatar manami-project avatar nattadasu avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

anime-offline-database's Issues

[Docs] Add a section on the readme to explain how the animes are match between sites

Hi,

I just found out about this project and, looks amazing and a hell lot of work. I am working on a personal project (To auto teach me Vue/.Net core) and until now I have been using Kitsu, but I would like to have links to AniDB and MAL.

I was thinking of using this DB, but my only concern is how trusty is this process, so this section in the Readme will help me and other possible future developers that want to use this DB.

Again, great work doing this massive job.

Some missing entries

Kitsu AniDB AniList MyAnimeList
46100 17352 148401 51772
44974 16533 137819 49590
46010 17286 146722 51367
46020 17239 146885 51263
44397 16081 130389 48375
44294 16187 131942 48661

Character database?

This seems infeasible given the amount of effort it would take to maintain it but would it be possible to add a separate database for characters too? This repo is extremely useful for me but an offline list of characters would be even more useful for what I'm trying to do.

Depending on how this is implemented, it could be possible to write a script to fetch characters from newly-added animes in the database. If this is something you're interested in I'd be happy to give it a shot and see how big the json for this could get if not, that's okay too.

2022-12-21 Data Entry ' Blue Dragon 2'

Last update 2022-12-21

$.data.[2681].synonyms.[1]
it's not 'Blue Dragon' but it should be only 'Blue Dragon 2'
Should you delete 'Blue Dragon' synonyms ?

Blue Dragon is from 2007 year
Blue Dragon 2 is from 2008 year

Happy Holidays

Add status

Add the airing status.
One of [FINISHED_AIRING, CURRENTLY_AIRING, NOT_YET_AIRED, UNKNOWN]

EDIT:
In my opinion the wording above is not taking releases of home video formats into consideration which are not airing.
The correct way would be to distinguish between home video releases and anime which have a broadcast / airing.
Something like this:
Home video formats: [RELEASED, RELEASING, NOT_YET_RELEASED, UNKNOWN]
Broadcast / airing: [FINISHED_AIRING, CURRENTLY_AIRING, NOT_YET_AIRED, UNKNOWN]
However that is not possible at the moment. So I tried to find a wording that would be applicable for both. I got inspired by the status wording of kitsu and notify.moe and came up with
[FINISHED, CURRENTLY, UPCOMING, UNKNOWN]

ANIDB/ANN links appear in both sources and relations

If a record contains ANDIDB/ANN, the link to ANIDB/ANN in sources will also appear in relations.

Example:

{
            "sources": [
                "https://anidb.net/a11606",
                "https://anilist.co/anime/21366",
                "https://animenewsnetwork.com/encyclopedia/anime.php?id=18166",
                "https://kitsu.io/anime/11380",
                "https://myanimelist.net/anime/31646"
            ],
            "type": "TV",
            "title": "3-gatsu no Lion",
            "picture": "https://myanimelist.cdn-dena.com/images/anime/6/82898.jpg",
            "relations": [
                "https://anidb.net/a11395",
                "https://anidb.net/a11606",
                "https://anidb.net/a12994",
                "https://anilist.co/anime/102293",
                "https://anilist.co/anime/21308",
                "https://anilist.co/anime/21587",
                "https://anilist.co/anime/98478",
                "https://animenewsnetwork.com/encyclopedia/anime.php?id=18166",
                "https://myanimelist.net/anime/28789",
                "https://myanimelist.net/anime/34611",
                "https://myanimelist.net/anime/34647",
                "https://myanimelist.net/anime/35180"
            ],
            "thumbnail": "https://myanimelist.cdn-dena.com/images/anime/6/82898t.jpg",
            "episodes": 22,
            "synonyms": [
                "3-gatsu no Lion",
                "3\u6708\u306e\u30e9\u30a4\u30aa\u30f3",
                "3\u6708\u7684\u72ee\u5b50",
                "3\uc6d4\uc758 \ub77c\uc774\uc628",
                "March Comes in Like a Lion",
                "March comes in like a lion",
                "March Comes In Like a Lion",
                "sangatsu (2016)",
                "Sangatsu no Lion",
                "Un marzo da leoni",
                "\u041c\u0430\u0440\u0442 \u043c\u0435\u0441\u0435\u0446 \u0432\u0440\u044a\u0445\u043b\u0438\u0442\u0430 \u043a\u0430\u0442\u043e \u043b\u044a\u0432",
                "\u041c\u0430\u0440\u0442 \u043c\u0435\u0441\u0435\u0446 \u0442\u0435 \u0432\u0440\u044a\u0445\u043b\u0438\u0442\u0430 \u043a\u0430\u0442\u043e \u043b\u044a\u0432",
                "\u041c\u0430\u0440\u0442\u043e\u0432\u0441\u043a\u0438\u0439 \u043b\u0435\u0432",
                "\uff13\u6708\u306e\u30e9\u30a4\u30aa\u30f3"
            ]
        },

"RahXephon Kansoukyoku: Kanojo to Kanojo Jishin to - Thatness and Thereness" broken title

First entry in the current JSON:

{
  "data": [
    {
      "sources": [
        "https://anilist.co/anime/559"
      ],
      "type": "OVA",
      "title": " \tRahXephon Kansoukyoku: Kanojo to Kanojo Jishin to - Thatness and Thereness",
      "picture": "https://myanimelist.cdn-dena.com/images/qm_50.gif",
      "relations": [
        "https://anilist.co/anime/165"
      ],
      "thumbnail": "https://myanimelist.cdn-dena.com/images/qm_50.gif",
      "episodes": 1,
      "synonyms": [
        "RahXephon Interlude: Her and Herself/Thatness and Thereness",
        "ラーゼフォン 間奏曲 / 彼女と彼女自身と 「Thatness And Thereness」"
      ]
    },

The title is broken and not merged with the kitsu.io 516 entry and/or the MAL 599, anidb a7969, ANN 3064 entry.

Universal ID

Hello!

First of all let me say great job!

How about adding an universal ID so that we can somehow relate two anime database sites

Thanks!

Divide the whole database into 4 mini-database

Background:
This database is awesome, but the anime-offline-database.json seems too large, 19.4 MB.

When I view the file, Github shows this:

2019-02-01 10 35 28

15444 entries from myanimelist.net
10472 entries from anidb.net
11270 entries from anilist.co
14201 entries from kitsu.io

Proposal:
Divide the whole database into 4 mini-database based on their source.

Decision on content addition

Current Situation

Content of the database

The database contains anime which do not fit into one of the following categories:

  • commercials/promotions
  • stop motion productions
  • music videos
  • pure CG productions
  • abstract animations
  • unrelated shorts
  • any non-japanese productions (korean, chinese, american...)
  • anime before 1970

The exclusion of specific titles is tool-supported, but not completely automated.

Benefits

  • The database is subjectively clean.
  • The database is smaller in size than it would be if contained other not fitting entries.

Problems

  • There is a lot of manual intervention necessary in the process.
    • This takes time and some anime might not be added to the database for a while.
  • There are always anime which should be excluded, but which are not identified.
    • That means that the database will never be as clean as it's supposed to be.

Merging

The base for all entries is always myanimelist.net (MAL). Links from other meta data provider are merged into these existig entries. That means that the meta data are also based off the entry
from MAL.

Problems

  • Over the past few months MAL has been down. This has proven to be a real problem, because
    • no new anime had been added during this time. That means that missing titles which were available on other meta data provider were not available in this database.
    • Very, VERY few changes/updates had been made on MAL during that time. So the database was outdated.
  • Meta data provider might provide different meta data. E.g. Both MAL and another meta data provider lists the same special with 2 episodes. Both are merges in one entry in this database. Another episodes comes out. MAL changes the existing entry and lists the anime now with 3 episodes whereas the other meta data provider creates a new entry. That would mean the following: The entry in this DB lists the link from the other meta data provider with data which is not correct for this meta data provider. And the entry of the other meta data provider containting the third episode will never be merged.
  • Anime entries which cannot be merged itno an existing MAl entry will never appear in the database. E.g. if MAL lists an anime as OVA and another meta data provider lists the same anime as special the link for the other meta data provider will never appear in the database.

Possible change

A possible solution to address the first problem would be to remove all rules to filter anime. To get rid of the second problem it would be best to stop being based on MAL. Instead of the current logic, a hit-first approach should be established in merge process. It is important to adapt both changes or none, becauses if you solely want to stop being based on MAL, but still maintain a subjectively clean database you drastically increase the manual effort for filtering anime, because you have to filter an anime for each meta data provider.

Functionality

  • An anime is added to the database if it does not already exists no matter which meta data provider it originated from
  • entries which already exists will be merged based on the current logic
  • All anime of a meta data provider will be added to the database. No exclusion will be applied.

Your Feedback

For me it makes totally sense to adapt these changes in populating the database. However I would love to hear from anyone interested in this project why you are interested in it. Do you like the apprach of a clean database based on the filter categories mentioned above or do you just like that there is an anime offline database and would you therefore prefer a database that strives for completeness? Please leave a comment on this topic on this issue.

Dub tag

could you add a tag if it’s dubbed or not

Please read the FAQ first.

Some entries are duplicate but with different meta data providers

Below is an example of such a situation. They are both the same anime but they aren't cross-reference-able because they are in different entries. The same goes for Aldnoah Zero 2 / Aldnoah.Zero (2015) / Aldnoah.Zero 2nd Season

I haven't noticed any other irregularities yet. I found this while trying to find matching Anilist.co URLs for the Kitsu.io variants in my own project.

{
    "sources": [
        "https://anilist.co/anime/20632"
    ],
    "type": "TV",
    "title": "Aldnoah Zero",
    "picture": "https://cdn.myanimelist.net/images/qm_50.gif",
    "relations": [
        "https://anilist.co/anime/20853"
    ],
    "thumbnail": "https://cdn.myanimelist.net/images/qm_50.gif",
    "episodes": 12,
    "synonyms": [
        "\u30a2\u30eb\u30c9\u30ce\u30a2\u30fb\u30bc\u30ed"
    ]
},
{
    "sources": [
        "https://kitsu.io/anime/8297",
        "https://myanimelist.net/anime/22729"
    ],
    "type": "TV",
    "title": "Aldnoah.Zero",
    "picture": "https://cdn.myanimelist.net/images/anime/7/60263.jpg",
    "relations": [
        "https://kitsu.io/anime/10159",
        "https://kitsu.io/anime/9136",
        "https://myanimelist.net/anime/27655",
        "https://myanimelist.net/anime/29223"
    ],
    "thumbnail": "https://cdn.myanimelist.net/images/anime/7/60263t.jpg",
    "episodes": 12,
    "synonyms": [
        "AZ",
        "\u30a2\u30eb\u30c9\u30ce\u30a2\u30fb\u30bc\u30ed"
    ]
} 

Can you update relation name ?

Relation names are 'sequel', 'prequel', 'adaption', ...
I want to use relation for my website, but it's hard to use when I don't have what is relation name?

any plans for manga-offline-database

i Believe a lot of manga meta aggregators have harsh rate limits so a offline db would help a lot of application liike komga , kavita adding meta to files. there is comicvine too
sources = manga updates , comicvine (volume based meta data} , anilist , mal
i know it was closed in past but are there any changes in plans for future*

Upvotes needed: Duplicated entries at meta data providers

Duplicated Entries

With this project we can identify duplicated entries very easily and I would love to use this instrument to contribute back to the community.
The following list contains duplicated entries at the meta data providers. Please support the database requests with an upvote.

LAST UPDATE: 2019-05-26

meta data provider database request
https://kitsu.io/anime/11371, https://kitsu.io/anime/12264, https://kitsu.io/anime/41448, https://kitsu.io/anime/41449, https://kitsu.io/anime/41744] request
MAL 33825, MAL 33831 to be created
https://myanimelist.net/anime/14949, https://myanimelist.net/anime/31551 to be created
https://myanimelist.net/anime/35279, https://myanimelist.net/anime/39143 to be created
https://anilist.co/anime/106750, https://anilist.co/anime/107880 to be created

Wrongful merges

The last duplication check showed that there are a lot of wrongful merges. I stopped extracting the real the duplicates and hope to provide a fix in the next update in order to only get the real dupliucates. So the following list contains all remaining duplicates. Mostly wrongful merges, but it might contain some real duplicates as well, because as I said I stopped midway extracting the real duplicates:

[https://myanimelist.net/anime/3269, https://myanimelist.net/anime/4469, https://anilist.co/anime/3269, https://anilist.co/anime/4469, https://kitsu.io/anime/2895, https://kitsu.io/anime/3625]
[https://anilist.co/anime/103759, https://anilist.co/anime/8953, https://kitsu.io/anime/13863, https://kitsu.io/anime/5546]
[https://myanimelist.net/anime/36113, https://myanimelist.net/anime/6535, https://anilist.co/anime/102034, https://anilist.co/anime/6535, https://kitsu.io/anime/13959, https://kitsu.io/anime/4602]
[https://anilist.co/anime/309, https://anilist.co/anime/491]
[https://myanimelist.net/anime/22729, https://myanimelist.net/anime/27655, https://kitsu.io/anime/8297, https://kitsu.io/anime/9136]
[https://kitsu.io/anime/12059, https://kitsu.io/anime/41804]
[https://myanimelist.net/anime/10030, https://myanimelist.net/anime/7674]
[https://kitsu.io/anime/5079, https://kitsu.io/anime/6714]
[https://myanimelist.net/anime/36078, https://myanimelist.net/anime/38551]
[https://kitsu.io/anime/11655, https://kitsu.io/anime/12569]
[https://myanimelist.net/anime/38581, https://myanimelist.net/anime/38595, https://myanimelist.net/anime/38598, https://myanimelist.net/anime/38607]
[https://kitsu.io/anime/12186, https://kitsu.io/anime/9225]
[https://myanimelist.net/anime/32417, https://myanimelist.net/anime/32418, https://kitsu.io/anime/11703, https://kitsu.io/anime/11704]
[https://anilist.co/anime/5202, https://anilist.co/anime/9119, https://kitsu.io/anime/3981, https://kitsu.io/anime/5607]
[https://kitsu.io/anime/10566, https://kitsu.io/anime/11238]
[https://kitsu.io/anime/13736, https://kitsu.io/anime/7826]
[https://anilist.co/anime/102613, https://anilist.co/anime/6055]
[https://anilist.co/anime/104545, https://anilist.co/anime/18457, https://kitsu.io/anime/10638, https://kitsu.io/anime/8234]
[https://myanimelist.net/anime/38317, https://myanimelist.net/anime/38318]
[https://anidb.net/a7045, https://anidb.net/a7260, https://animenewsnetwork.com/encyclopedia/anime.php?id=15516, https://animenewsnetwork.com/encyclopedia/anime.php?id=3444]
[https://anidb.net/a11419, https://anidb.net/a11504, https://anilist.co/anime/102707, https://anilist.co/anime/103497, https://anilist.co/anime/103770, https://kitsu.io/anime/10357, https://kitsu.io/anime/11327]
[https://anilist.co/anime/18045, https://anilist.co/anime/18053]
[https://myanimelist.net/anime/37139, https://myanimelist.net/anime/38344, https://myanimelist.net/anime/38345]
[https://myanimelist.net/anime/15487, https://myanimelist.net/anime/22673, https://kitsu.io/anime/7262, https://kitsu.io/anime/8717]
[https://anilist.co/anime/102357, https://anilist.co/anime/17819, https://kitsu.io/anime/10565, https://kitsu.io/anime/7742
[https://myanimelist.net/anime/13789, https://myanimelist.net/anime/20463, https://anilist.co/anime/101180, https://anilist.co/anime/13789, https://kitsu.io/anime/7029, https://kitsu.io/anime/8920]
[https://anilist.co/anime/1367, https://anilist.co/anime/7304]
[https://anilist.co/anime/7303, https://anilist.co/anime/9839]
[https://myanimelist.net/anime/21647, https://myanimelist.net/anime/23399, https://anidb.net/a10326, https://anidb.net/a10565, https://anilist.co/anime/20519, https://anilist.co/anime/21014, https://kitsu.io/anime/8135, https://kitsu.io/anime/8400]
[https://myanimelist.net/anime/37632, https://myanimelist.net/anime/37819]
[https://anilist.co/anime/102161, https://anilist.co/anime/16810, https://kitsu.io/anime/12715, https://kitsu.io/anime/7528]
[https://myanimelist.net/anime/36125, https://myanimelist.net/anime/36223, https://kitsu.io/anime/13716, https://kitsu.io/anime/41611]
[https://kitsu.io/anime/11290, https://kitsu.io/anime/11723]
[https://myanimelist.net/anime/16143, https://myanimelist.net/anime/8171, https://kitsu.io/anime/5225, https://kitsu.io/anime/7377]
[https://myanimelist.net/anime/1639, https://myanimelist.net/anime/5391]
[https://anilist.co/anime/104081, https://anilist.co/anime/104366]
[https://myanimelist.net/anime/30765, https://myanimelist.net/anime/37594, https://kitsu.io/anime/10995, https://kitsu.io/anime/41237]
[https://myanimelist.net/anime/1982, https://myanimelist.net/anime/3474, https://anidb.net/a3202, https://anidb.net/a3474, https://animenewsnetwork.com/encyclopedia/anime.php?id=3384, https://animenewsnetwork.com/encyclopedia/anime.php?id=3422, https://anilist.co/anime/1982, https://anilist.co/anime/3474, https://kitsu.io/anime/1784, https://kitsu.io/anime/3036]
[https://anidb.net/a13185, https://anidb.net/a8950, https://anilist.co/anime/16620, https://anilist.co/anime/16786]
[https://anilist.co/anime/10334, https://anilist.co/anime/12067, https://kitsu.io/anime/6132, https://kitsu.io/anime/6659]
[https://myanimelist.net/anime/29637, https://myanimelist.net/anime/32847, https://anidb.net/a13131, https://anidb.net/a13132, https://anilist.co/anime/102303, https://anilist.co/anime/104180, https://kitsu.io/anime/10491, https://kitsu.io/anime/13197]
[https://myanimelist.net/anime/3038, https://myanimelist.net/anime/3914]
[https://myanimelist.net/anime/3473, https://myanimelist.net/anime/38574]
[https://myanimelist.net/anime/38485, https://myanimelist.net/anime/38488]
[https://anilist.co/anime/19211, https://anilist.co/anime/20449]
[https://anidb.net/a2778, https://anidb.net/a4054]
[https://anilist.co/anime/10132, https://anilist.co/anime/102943]
[https://myanimelist.net/anime/35602, https://myanimelist.net/anime/36539, https://kitsu.io/anime/13522, https://kitsu.io/anime/13948]
[https://myanimelist.net/anime/31568, https://myanimelist.net/anime/32382, https://anidb.net/a11619, https://anidb.net/a11852, https://anilist.co/anime/104480, https://anilist.co/anime/21849, https://kitsu.io/anime/11616, https://kitsu.io/anime/12133]
[https://myanimelist.net/anime/28085, https://myanimelist.net/anime/33866, https://anilist.co/anime/103355, https://anilist.co/anime/97978, https://kitsu.io/anime/10075, https://kitsu.io/anime/12704]
[https://myanimelist.net/anime/38490, https://myanimelist.net/anime/38593]
[https://anilist.co/anime/102300, https://anilist.co/anime/98133]
[https://anilist.co/anime/103317, https://anilist.co/anime/104064, https://anilist.co/anime/21411]

Redistribution under MIT for a specific part of the database

Hey, I've been following this repository for a while. Great work and thank you for your hard work!

I'm interested in a specific part of this database that I would like to redistribute under MIT License. Will definitely provide credits where due.

I'm looking into creating a repository specifically for relational IDs for anime and manga - which would be integrated with my own project (API service). If you're okay with it, I'd like to utilize this database for that purpose. Though I will have my own service workers populating the repository as well, this database would save me a lot of time initially.

I'm only interested in this part;

            "sources": [
                "http://anilist.co/anime/1535",
                "https://anidb.net/a4563",
                "https://animenewsnetwork.com/encyclopedia/anime.php?id=6592",
                "https://kitsu.io/anime/1376",
                "https://myanimelist.net/anime/1535"
            ],

The data above would be restructured/optimized to something better suited for my API. e.g;

{
	"anilist": "1535",
	"anidb": "a4563"
}

Let me know if you're okay with that, thanks!

Introducing a merge lock?

With this weeks update I noticed another unjustified merge which resulted in an entry having two anidb links. This is a direct result of my decision making the merge logic fuzzy for the duration (#19).

https://anidb.net/anime/7045
https://anidb.net/anime/7260

The difference in the duration is a minute. So I can assume that there might have been more unjustified merges like this. The probability is very high, because if I remember correctly around 4000 entries were affected by that change.

First thing I want to do is revert the change and make the merge logic strict again. This will definitely take place within the next days.

However I also have your feedback in mind that you guys seek a higher merge density. I thought about it today and this is just an idea. I neither say that I will realize it nor that I have thought everything through, but here is what I thought about.

Currently the merge is done with each update. My idea is to introduce a 'merge lock'. This would be an extra meta file that tells the merge logic to just put specific entries together if they are in that list. The rest would be handled dynamically as it is now the case. This was inspired by the rest-api project by @BeeeQueue.
The merge-lock.json could look like this:

[
  [
    https://anidb.net/anime/2880,
    https://kitsu.io/anime/4761,
    https://myanimelist.net/anime/6868
  ]
]

It doesn't have to contain ALL entries. It just says that if entries are in one array together, then those will definitely be merged together. The rest will be handled using the default merge logic.
That is a lot of manual work and I cannot do that. So I would need you guys to create PRs and fill that file. With each weekly update I could provide another file which contains all anime that havent't been merged and the reason why merging has been prevented. Such a file could look like this:

Year [1993] does not match [1992]. Not merging [https://anidb.net/anime/3172, https://animenewsnetwork.com/encyclopedia/anime.php?id=4405] into [https://kitsu.io/anime/12337, https://myanimelist.net/anime/32628].
Duration [1800]seconds does not match [1500]seconds. Not merging [https://anilist.co/anime/6868] into [https://anidb.net/anime/2880, https://animenewsnetwork.com/encyclopedia/anime.php?id=1481, https://kitsu.io/anime/4761, https://myanimelist.net/anime/6868].

Additional thoughts and points:

  • Should the merge-lock contain only these 'override' style rules or should every merge be locked? (see EDIT1)
  • 1-on-1 array or all entries within one array? (has to be dealt with, unresolved)
  • What about duplicates? (see EDIT2)
  • What about entries that have been deleted? (see EDIT2)

We are talking about >7000 merge lock entries here. So I would really need help on this.
So I'd love to hear your thoughts on this whole concept.
And just to be clear:
I will definitely go back to strict merging rules and it is completely uncertain whether or not I will approach the merge density with an idea such as I state above or if I leave it as it was before.

EDIT1

To keep it simple we should start with only merge-locks for everything that is listed in the merge prevention list.

EDIT2

In my opinion it is fine to merge duplicates since the merge is correct. We cannot say for sure which of the duplicates will be removed and when. As soon as a duplicate is removed nothing will happen anyway. That brings me to the second point. I think that it wouldn't be necessary to handle removed source links. The respective entries will never pass the merge lock again. The meta file merge-lock.json might contain some dead links, but that is not critical in my opinion. The merge was correct at some point and it won't have any effect on future merges. There are only unnecessary entries in the file, but I doubt that dead entries would increase the file size tremendously. However removing dead entries could be done automatically later on.

Genres

Get anime genres with it.

The code

I just want to have the script code to help manami with update each day i think.

Raw data

This repo looks very interesting. Is there a possibility getting the raw data before merge to optimize the merge between the anime sites?

Database usage & AGPL license

Hey, I was a little confused during my research so thought it would be best to ask here. With the AGPL license this database is under, what am I allowed to do? I would like to query this database simply to map anime between different providers and possibly use the tags for a closed-source project. Is this fine?

manga

I would also need the same for mangas

Possible Extension: Character databases

Here are some websites that have data on Anime characters:

I would like to ask for pointers to other repos, or ways to implementing such a database with the AOD as a basis.

Synonym matching multiple entries from same website

I wanted to ask if this is something that the manimi project thinks should be fixed, or not.
I expect it's valid "data", so it would be more of decision if this is within the intent of what AOD is for or not.

I'm primarily interested in using AOD as a method for finding the synonyms for AniDB entries that other sites use (and AniDB doesn't want added to their site), so synonym entries that match multiple anidb entries are problematic for me.

I am just working around this by ceating a new file without the offending synonyms, but I wanted to know if this is something considered a data integrity issue or more of a factor of the data and should remain.

Here is one of (many) examples, the Synonym "Minna no Uta" matches several entries (and AniDB in this case isn't where the synonyms look to be originating, there are cases where AniDB also has this behavior)

    {
      "sources": [
        "https://anidb.net/anime/12585",
        "https://anilist.co/anime/8646",
        "https://anime-planet.com/anime/44-hiki-no-neko",
        "https://kitsu.io/anime/5415",
        "https://myanimelist.net/anime/8646",
        "https://notify.moe/anime/UAUmpKmig"
      ],
      "title": "44-hiki no Neko",
      "type": "Special",
      "episodes": 1,
      "status": "FINISHED",
      "animeSeason": {
        "season": "SPRING",
        "year": 1969
      },
      "picture": "https://cdn.myanimelist.net/images/anime/7/23825.jpg",
      "thumbnail": "https://cdn.myanimelist.net/images/anime/7/23825t.jpg",
      "synonyms": [
        "44 Cats",
        "44 Gatti",
        "44 Hiki no Neko",
        "44ひきのねこ",
        "Minna no Uta",
        "Minna no Uta: 44-hiki no Neko",
        "Shijushi-hiki no Neko"
      ],
      "relations": [],
      "tags": [
        "animal protagonists",
        "animals",
        "cats",
        "kids",
        "minna no uta",
        "music",
        "non-human protagonists"
      ]
    },
    {
      "sources": [
        "https://anidb.net/anime/12720",
        "https://anime-planet.com/anime/aikodesho",
        "https://kitsu.io/anime/11915",
        "https://myanimelist.net/anime/32421",
        "https://notify.moe/anime/Xl4epKiiR"
      ],
      "title": "Aikodesho",
      "type": "Special",
      "episodes": 1,
      "status": "FINISHED",
      "animeSeason": {
        "season": "SUMMER",
        "year": 1988
      },
      "picture": "https://cdn.myanimelist.net/images/anime/2/77796.jpg",
      "thumbnail": "https://cdn.myanimelist.net/images/anime/2/77796t.jpg",
      "synonyms": [
        "Minna no Uta",
        "Minna no Uta: Aikodesho",
        "あいこでしょ"
      ],
      "relations": [],
      "tags": [
        "kids",
        "minna no uta",
        "music"
      ]
    },
    {
      "sources": [
        "https://anidb.net/anime/12666",
        "https://anilist.co/anime/8731",
        "https://anime-planet.com/anime/akai-boushi",
        "https://kitsu.io/anime/5466",
        "https://myanimelist.net/anime/8731",
        "https://notify.moe/anime/xrqmtFmiR"
      ],
      "title": "Akai Boushi",
      "type": "Special",
      "episodes": 1,
      "status": "FINISHED",
      "animeSeason": {
        "season": "FALL",
        "year": 1984
      },
      "picture": "https://cdn.myanimelist.net/images/anime/8/24113.jpg",
      "thumbnail": "https://cdn.myanimelist.net/images/anime/8/24113t.jpg",
      "synonyms": [
        "Minna no Uta",
        "Red Hat",
        "赤い帽子"
      ],
      "relations": [],
      "tags": [
        "kids",
        "minna no uta",
        "music",
        "slice of life"
      ]
    },
    {
      "sources": [
        "https://anidb.net/anime/12798",
        "https://anilist.co/anime/109653",
        "https://anime-planet.com/anime/akai-jitensha",
        "https://kitsu.io/anime/9107",
        "https://myanimelist.net/anime/24999",
        "https://notify.moe/anime/5uVatFimR"
      ],
      "title": "Akai Jitensha",
      "type": "Special",
      "episodes": 1,
      "status": "FINISHED",
      "animeSeason": {
        "season": "SPRING",
        "year": 1999
      },
      "picture": "https://cdn.myanimelist.net/images/anime/12/64039.jpg",
      "thumbnail": "https://cdn.myanimelist.net/images/anime/12/64039t.jpg",
      "synonyms": [
        "Minna no Uta",
        "Minna no Uta: Akai Jitensha",
        "赤い自転車"
      ],
      "relations": [],
      "tags": [
        "family friendly",
        "kids",
        "minna no uta",
        "music"
      ]
    },
    {
      "sources": [
        "https://anidb.net/anime/13382",
        "https://anime-planet.com/anime/akisutozeneko",
        "https://kitsu.io/anime/40842",
        "https://myanimelist.net/anime/35786",
        "https://notify.moe/anime/yCvOhKmmR"
      ],
      "title": "Akisutozeneko",
      "type": "Special",
      "episodes": 1,
      "status": "FINISHED",
      "animeSeason": {
        "season": "SPRING",
        "year": 2001
      },
      "picture": "https://cdn.myanimelist.net/images/anime/9/86389.jpg",
      "thumbnail": "https://cdn.myanimelist.net/images/anime/9/86389t.jpg",
      "synonyms": [
        "Minna no Uta",
        "Minna no Uta: Akisutozeneko",
        "アキストゼネコ"
      ],
      "relations": [],
      "tags": [
        "minna no uta",
        "music"
      ]
    },

Multiple sources entries of the same meta data provider

The following 378 entries have multiple sources of the same meta data provider:

4d70dd25-5a3a-4faa-be79-1ffee5da7ef0
434f2d77-feea-449a-8f27-6a485a10bfc7
c9bd4295-4475-4a1d-980c-de480b3fc5a9
6bb8490f-5a41-4691-9a6a-34cc911c57d3
7c2be4c8-db8e-4597-b5ef-33c1e1a19df9
f4d9c3d1-1d35-4b8f-8861-65e2345334db
48a2b631-b382-465b-93e8-7e660ae9edcf
9957bc72-70bf-48df-8c6a-511bd519f33e
1a5ebe9f-d05a-4e10-8859-236a3426582e
d4cde38b-6109-4859-8124-54bae8f45c99
b59d6390-a888-4fb7-946a-5797ec86ffbf
8655e02c-6bb5-448a-b8b1-9a2b920eb8f2
b7a3293a-7292-48bd-b453-0b0fffff3346
a565b256-db24-4f0e-9fc7-d0911a554f51
015be63d-85ae-497a-9919-f18c3e3722fc
a8868895-510e-416a-b387-93b3ec235640
ee72f68e-5cfb-49bc-bb82-36689fa20c6a
fc6d8f23-ffaa-475f-bb91-be178df82819
89796dd8-cab5-4031-b26a-ca18b0d82a28
bb44c580-10e0-4a07-80f2-cde0ec6ea547
21e96764-0958-4ebc-a589-a907f45558c6
9b618a17-9295-4f42-9b76-5c08ad95632e
e2bb60f4-c5a1-4ed8-bd36-55f008f8b411
ba46db94-df19-4d5c-bd0a-1697b240b62d
2a17c828-591c-49bd-b938-bc7b2f9068e0
57eb5fc0-e5a9-4118-be99-e7a30608f346
68fac992-44fe-4a51-a746-9ed21fd417ca
79d4b4de-7e72-4608-8545-c20cefcaee5c
e925bbfb-bdd0-47bb-b327-2f1fd4a0491e
6e500704-65a4-43d1-a8c0-6b6a632459c6
83760686-9ee3-4335-98cb-9df54a69ff92
479f1b69-6d89-4a54-b3df-fd299c09645f
f70fa964-0aee-4f8d-8289-0718086421e1
737aac29-29e1-4833-9dfa-9b6d63acf810
4318e33d-beff-416a-b137-424a6d90e37b
fb7757bd-8002-4470-93aa-0ed75e23043b
0039c83f-e82e-448d-b84f-aea491b84bcb
baf0beab-d5de-4b65-9b40-90e7f318ef57
2ea395f3-db0c-4782-9bcb-a421b160181c
55a97f74-c9ad-48ac-a89e-4399ae74170a
3e20fa81-a31f-4f32-a388-7c61db66a4f6
fedfda62-026c-4912-aa2d-ab479199b9c1
296e5649-e5da-4d47-96dc-9906967ebdca
e25abaf9-9443-4114-8bf8-a091c2be73be
d3bf055d-db39-4e2f-bce4-9c2b2f6c9ba6
2d71ba8f-e59c-4e22-838c-7a9d37378242
328863bf-3478-4d95-8da3-9bb4321ed26b
a69f1066-b425-44c5-a002-2d0c850d4a5c
40eedbab-b614-466c-8a12-c129b482218f
234f0833-8e6d-4783-8ea3-5b15264be412
5234d261-5b07-40ee-9ec4-163d87b19628
76ab71cf-b477-4b3e-96de-ca6123bc6ab8
71b702bc-c30e-4eb5-aee5-b14bb9e192a7
7740b184-8d6f-4070-84bd-695a3606066d
d8ceeb59-5c77-4603-9335-492c478b474f
62f36a18-5f30-455c-8905-7e9962756fa5
5938e4c0-eb5e-4494-a36d-476803c5d769
dae9a535-b251-442d-8ced-fc44acf54897
def07104-04ab-4461-8526-9b9fd1366569
9b2f6e5c-5b23-4d53-a71f-f35c57c87e13
90c28005-65c6-4540-8f60-9ccbc53b7eb9
3fa2ff99-c11d-423b-9af7-216b68454b0a
df65ad28-7d23-4e90-8faa-a0a9dc639a1b
2fec1ea4-55f9-4dba-b0b7-a6d1c1a644ec
53cd2105-76e1-452a-811e-997f7b6a3d13
d312e696-57fd-4823-b16d-2d4cda8fe793
b5294b84-1141-42e6-8b5b-6aa171aabc0d
e8486eb6-17a5-42fc-85d7-ab7caca6105d
56e10e23-cfd8-4a01-b11b-61b6d8ab5b50
a73ee311-65e3-421b-b30a-87c8e655ab63
cadfb0c7-81c0-4735-8e0d-5599f1d60f3c
3c1d0894-b5b9-4198-8330-47d6ce969ca0
081075d0-7092-4167-9d14-0fa26b29601e
6f09c582-2fb3-46c9-9841-a51a7f986340
4885fe3f-d872-48ac-9891-bc5c8c42045f
f21f3ace-b845-4660-8de2-b8352f594149
7a414c7e-e83c-4abc-9a6a-14cf00ad9755
336da32e-6825-4f72-bfcf-3b74c961074d
7b28d1fd-d17b-4720-a149-e8e51b3c07a4
3980bda0-e218-4612-bce7-98d81f9332e8
ac1e335f-9cec-4e01-a9e1-9a211e3c5627
f4580e64-d1f9-49e6-8e0e-af38ca3d40b7
d5dec14c-adcc-4484-a201-cd3eba26054f
08aec410-7cdf-49b9-9ca6-57cfe60dbab8
f4f7bfc7-399b-4ae5-9ba4-7026155aa47e
0cf8f16e-57de-4c33-8782-f0f51b03d20a
4f10a644-4152-474d-9fb5-b2f53a3a8d2d
95e59590-8496-4df4-805a-c5b71052b3bc
ab34efea-3fd3-495c-a394-6372a1aaa783
dc7026f1-3138-4992-b45c-395e1f15d02f
58131df4-bd7e-4b62-a90d-fbefe0089ab4
3cb26639-ae23-4686-a176-fe6aad66f1b7
270da48e-0cb4-4704-b4e8-58d31f7a807d
a9e51a3c-a9cf-42d0-8491-c1878b0747dc
0688a1ba-23aa-484d-aba0-931348300723
bbc3195c-c3e7-4f3f-bef3-6c26d8579ca4
b2090dbc-9623-41a2-9818-194e92f33307
f945b7ec-61b0-4926-bb29-690fd8c85389
6c7f2246-5ff8-4f13-9030-e0e43f1a8307
e6d90f29-e776-4f0d-839a-e2b0b932c00c
f3fb3139-20d9-4eb9-97ec-252b12a707ad
2cd9ff44-b575-4ff1-ba6f-a96a2ea33e07
6c4d6c62-4f26-4ec0-a53f-314f331497da
3ea8934f-932a-4bf5-90b8-32c4747e738c
75568846-aecb-42fd-bf64-6aef70cc05e8
35e3728c-9fee-4bcb-8a3a-ca9de8b46205
a1f1fe84-16c3-4154-81e2-797ed9b04d34
c494c9e4-5aa4-47f3-9f82-ddbd32c1b244
fe3311e0-6256-49e4-909d-32893e9b58c1
0a51afe6-5baa-430a-8192-716a2c25df82
c98016bf-d9f0-464f-9f54-e8b98d96bd32
e90e125f-a27c-4846-a065-416682e8c5fa
5b604d68-291a-4fe9-a397-abe8f736d8e2
1adf24f4-5c44-4805-aa21-bf2b3de2093a
a09bc801-b9f8-4c79-a373-4aed0a003095
b6d0a1b6-4ac3-4f24-80ca-09c14a662756
b5285a8b-507c-4433-9887-d333b788d974
bf503928-0468-43ef-87cb-6272cfed4851
c813ab7d-0d0d-4cb0-ac3a-f884a34b7339
c1c0ccbb-2e10-48e8-8da3-67ee6db2471f
5d509c30-e4e8-4eb2-b91b-ffc6032a37c0
4cd11724-966f-4d0d-9291-163f2067d284
f14a081f-985c-4135-bcb1-7052044801fc
9153e474-78c9-45dd-9a69-6ec0ba9d30e0
bab2c69c-7397-47c9-977f-ed0473a2264b
a2e1342d-6c9f-40c4-84b5-f477008b28a4
b9b1b06a-9afe-4411-80f8-4a8481f5f5ea
ffee15f2-51e3-4baa-8067-8f7232e3126a
bd3d465c-47ee-4941-ac88-156b70b37fdc
108c1140-109a-42a8-a458-24612b0b3af5
20c77cf0-ca57-493f-a78a-d3a27bd7fec5
a902e5b0-f68d-495e-ab62-af791480acac
045260e4-16d8-4bcf-8f1f-15fd7df0cc2b
2a7ce1bc-7df8-441c-855a-1c5e220494e4
91c8bd28-035c-4f31-9eae-8e2fe06efda7
f82c64e6-ab6d-4240-8ae5-a01ed293eb8f
1f987d1a-1069-445a-8692-f9e57d67d11c
1e6ba075-1ff1-4f14-8bf3-d68d8a1b53d4
c89b8022-8529-4c47-8acc-cac3e887ff17
02e21625-03fd-40a6-afc4-16209aa73dd9
9e070d9d-b1e9-4553-aefa-c67a51f63c66
bd5d1d14-2015-4bce-8e28-e7089fd9851a
b00c2a0d-be02-4777-9c38-33d90568b7f9
57860fe0-a575-42af-a25f-4b2c0e699f8a
f4e896eb-baf1-461f-b6f2-28298e4a95f3
3858aeb5-a04d-489a-917a-73258a5a0c5d
22c016c4-c91c-4a44-b34c-e5269544fae0
54a313d2-e6e9-4f16-9a73-4dd9c9b69a48
a6597dcc-deb6-4184-b558-4923b4cfe0db
1fb7336b-0a2d-42fc-9e05-16037cca6dc5
da9f3b1e-77df-4390-bb8b-e71e5408581e
08f806f9-ab69-469d-a789-83d391ff4585
d8b3acae-0c70-41d2-b5cb-6fc383a107d7
f2ac9b5a-a0b6-4d67-90ff-bf99abdf95c1
7a6e3764-398b-4d87-bcba-66fb01597201
df502290-e5b5-4f1f-8b11-4662df15d309
afea8851-0b9e-4043-8fb3-1842688f46ed
4da40d15-e67b-4614-bdee-807420905256
57d9d026-7d3d-442c-8d6a-1020b627cc3a
5b92cefe-93d7-430d-9c41-c7d098d994d9
73b664a8-20f0-4c79-a4f2-27df0d818089
1f988083-e12a-4494-8ca4-4a74a4970b1c
7f7882d4-6bf2-4860-98f2-eac2386db9cc
807a7017-c98f-427f-82ee-bd95e4ac61e3
f3181c00-244e-4cf8-a916-4e20d63e3292
91ddd225-1b28-4c6b-a612-6b6bb99975e1
32bd40b6-c707-49a4-8764-3f41564fa74e
2478f337-819f-4789-a3d4-7c436fd6c7ba
d9c6fc77-a080-4766-8d2b-d7357a66628f
b1f191cf-5cfa-4510-ba4c-061fc71cbbb8
a874f961-a8be-43a0-bf3d-6636b7faf631
64e4f661-8231-4400-a366-6be7060753da
2d05687b-121a-4e53-9fe3-4e462e4a6ae4
73da5fb1-6caf-4d46-8bd5-0e5fc2adeee6
4ac72b80-90a5-4224-862e-f1f54245fdd8
54390865-4994-45f0-9f89-4f67a2533e6d
99350f1d-5871-43d6-a00f-e95c9dc25294
91cfd3c7-6de4-45b5-8ae1-cc4a1c91c96f
1f0f60a8-b249-458e-a66b-3d07c7c043ae
dbac61be-47ba-4c9b-a780-c77759a7f9bb
63c5576f-d80b-4d6b-bc5f-fba16601e62d
2d7d2ae1-b88d-4768-b3f5-081535850a1d
fcb0e3ab-e734-4818-8ea2-bf55940884aa
13b7dc71-54d3-405e-bf7b-1c0e32b9768d
c732e00b-5f0f-457d-ac7e-d048f7914314
eede41bf-71fe-4aa6-b6c5-f009e1a4522a
077f26d9-486b-4933-9f8c-4720b8fe4751
aaaeb984-7713-437e-a853-a9468f7e88fc
a91e50fc-b983-4939-bb4e-de6ebefd8db6
bec7048a-4def-462c-89d2-3640f93ca9ca
021a4734-f9e2-4803-9ecd-eaa7db492779
6ff3fb0a-fa2e-45e4-99c5-daba61c396e9
de6761b5-e2b0-4121-a948-33297c567c04
14922546-5f1e-4547-8216-e7a9ddaa9ceb
38d2758a-963e-4ce2-928f-9a8d9f31941d
95de9042-cd9c-463a-b60c-d27963ed83e6
9813a457-6118-4c27-8592-5f94597dcd0a
519d4572-d078-4990-9df3-25c35fbc3789
aa1ce59b-1f18-46f1-8aaf-3ea47a6958f0
58206a80-e6be-4a42-acc1-6f750a0c37da
c6490b9c-fd88-44b6-a942-98f1c2e374e1
f8b295a0-1ca8-465f-9d32-7cb07a9115ac
f0f0baea-e7d8-431e-88b6-203746ddbc1f
065bbe9c-7ef3-4e06-bd82-1ce9ab7b74aa
d9116d74-b77a-470f-8b2f-5647c2373d3e
cba33941-b662-463a-9a8b-646c67ffa8d5
8946f4c6-93b6-43ad-99f0-8c18b2ee8910
00dd16d9-52ed-4185-a34a-3927ea2c40e7
3fc1a0d8-6dcd-4f68-b5b0-6f20217eb371
566def5b-6569-4430-8b47-3aceac287441
7d78a453-2652-4157-a523-233db4e40da2
83b3f964-89aa-47ef-a57e-a8b572f5ee32
22bbfc1c-354b-42e5-9e5b-6ae941b25a99
1ae28ee3-b689-4a75-a45b-ecc076545550
95960dab-a060-429a-9cc3-9589a4df3701
bacd333c-8ec4-47e5-8d44-eac72909e37c
8a433a3d-51ea-4f63-88f0-fca28c9ece77
be2a3a7e-b499-47ea-a9cc-98078529c92e
d6a96557-d563-49ea-bbdf-781a1d38cf5d
945e88d0-bc89-4b8c-b5c1-cf3c1be8e488
a65236d6-44ab-4272-a768-d8bdfee3e653
eb4c0a1a-a905-44d6-b662-f2eeae349dff
a72b2906-279a-4109-a338-eb3cee502e05
6793a674-a874-4267-b9ee-db54aeefd9b0
b08f4194-03ae-4260-bfb8-a2b94783cea4
32a49248-b20f-4d12-854d-cb3102305caa
5f63f369-0282-4545-b906-41e2f35f1646
1d880312-91a7-4128-949b-c851e728b562
6ee92725-3546-433b-8d40-dc19078c6352
1bd811ed-7575-4bc0-a34c-8e502f3bae5d
29efd1c2-cd39-4d4d-a4df-ca575b7f4039
52aa50fc-5e1f-4765-a394-dcdc0949821a
0e2e1858-a2bd-400d-b1e4-02e406335214
2bc60926-d05e-4905-a9f8-7b61187165e0
848b5a66-dc52-4959-8754-ad23df9d12ed
ad407e04-5b62-4c86-a592-1f4c3ad5bd43
52c0e5b5-f3b6-4a9f-bf16-eef5caab07ad
46b959b7-d5d6-47ba-b558-9be7d59a5e4c
e780c5bf-bb77-43ab-a4d5-f2ebf620466a
e04e0428-20c8-4b28-857f-d4605871ad17
f57129a7-e24e-425d-856e-dd35267e448a
753d2dde-1a81-45b5-9d91-82af672a6560
d5f106ba-b3d8-4b13-8263-14eec787dc92
1f271222-9180-4572-9341-92246bb317b8
5e4d8ed1-5690-4359-a631-75c61a0e96d1
f812b974-2541-4f0e-86e9-a957eb5dbda3
15569add-4ba1-4ba3-87e1-d8fb518062f7
f25b02e5-0030-45a1-9c9e-66bbd3791248
50cb0fde-8750-48c0-99ba-90f463c7c6df
e62bdc88-df43-44c0-b974-4df21370e9dd
c77f31b2-b18d-4b20-8dfb-7223aeabd28e
c13f3dbe-5031-4f88-9d5b-9eb8a6786200
f3b71bd7-8dde-4dfd-829a-401bcbdaec13
b5d698aa-435d-4b29-9b23-a3f008b8c1c3
bac361e1-0210-422e-8ebc-4e92d684b8ac
3ed17de1-eccc-4a32-a3bc-a46049abb735
ec50229f-7a89-4e75-bed1-dd12d4b085a2
ade76783-4737-41e3-84d3-542b5fbed37f
97d8a6fc-d288-41a6-8fee-5631526235bc
c64c4950-25c7-4f50-a784-b396205c7149
0c900bf8-b775-4102-986b-1ad55fe9cf6c
d0fa3637-c06f-49bc-b83f-f93c8ca9d7fa
3be98d09-67da-404b-8445-bec315624ff8
588bd4a9-df7c-4d96-bb56-3584bad5e960
5562e602-fdc0-42b7-af8a-b251cdfc7515
fe534869-f7d2-4d38-abad-e97b59429b89
90f8da5f-098f-48f6-a93e-6570c398424a
95676b4e-0fe0-42f1-9ae2-0ccbfcb98331
a19175ff-aed3-42ce-a5e6-e926a76376ac
d35c3232-e541-40a7-a040-3d0fd724fab4
ac8ae8ab-2b9c-403e-96bc-d5a013b80b52
3ac9f96a-282b-48cc-b4aa-d8e0fa6bc7ba
4384775b-9b98-4139-b972-00eb41d63ebc
67eacbeb-3578-45ba-bbc8-9456b6941e65
a1990814-227c-41d6-a5a7-0927650f180e
323b4d57-2033-42a4-9db2-f4161fb68ea5
d8149166-df88-4e16-8cdf-2721067a90b6
5b6540d8-42a7-4ba8-9681-5579b94d53e5
81d63625-b648-41fb-b3ff-74c7f0530b29
010b87bc-4237-4fd0-ade6-a7120f86203f
88712b36-e10d-4e09-a6dc-d92d5721ad0c
0194062f-454e-4d61-813f-c1d708b8a134
34ca260f-63a6-45dc-b1c8-117379950a61
53756220-e260-4c24-86e2-24511c867eb9
71a23ce6-c30a-47ca-b21c-74a622e07e58
c0fbe73f-e10f-47d2-98e8-31754c0a39a5
fb7d6432-f94b-4e29-afd2-7ec6185e7987
79d2a844-321e-4c2d-a2a7-4bd0979f9991
fd2bf2e1-8983-4af4-b87a-f6ea5c2617b2
b62aafde-575f-4083-bbb5-b8265a28cb12
92e0fc62-ea4f-4248-996a-78cb320099a3
82f794f6-66ff-45ce-9ccc-d507451c1473
20d186b9-503a-4ab4-b3d7-c301f720f7ec
f94ad725-da7a-45b4-b6c6-bba2a1c5b8e8
3f17a276-1757-4e61-b9a7-f69d22c912e2
12b94cca-3cc3-4762-b472-204c2d25892d
35495a90-2047-47d6-bb53-24a292a02f05
e2ab8f36-55e7-4da0-8dad-ef11e998f914
b7fc69fb-ea45-40ac-94ce-e2fdded0acfa
c2ddc3cd-ec3d-45df-83e4-00ddb6cf1c58
2bb03d03-3091-4aa4-ad4a-2434b2dd29f2
17c2f468-8700-4edd-8d12-f53f50079d26
dfa6b5bb-c802-4dd7-aec2-11ab2dcbcc7d
531fb8c5-ce6a-4fc9-95ed-f9a2ee028d36
2ed2b25a-fcff-4103-9e69-76ad34aa82d8
9cdade7a-8e1b-40bb-b8fc-0d90e91000ce
c54c86da-3ee1-47c1-9102-f0e727294ac6
e07270dd-ff3d-48f7-b5d6-173d7b4edaaa
10969497-6ca7-48c0-83e4-adaed400ff90
edd870e1-52e4-4044-80bd-445bd750b0af
e182f3e3-895f-4506-861b-43a3c2956217
f7239999-7b4f-44a8-bc6d-f6e24d4e8ddd
790ae639-44cb-4d66-a902-23456e38a794
2a7804eb-7444-4abd-9912-ca2c1f5efe0a
d1c4e73d-44a6-497e-9993-06ea3f10a212
c505ef91-edf1-40f5-bd37-b52bf7c1722b
f9df91c1-9ddd-4e2c-b121-2dc54ab72fc9
927207d0-6fe3-4df2-b4ae-760575833095
ba770593-4684-48f6-ba79-1980c09a5714
a1b07291-d64e-47a3-98cf-3981609901c7
d5a8c6c0-a04f-4b46-b2c6-72ee4a81eb5d
369c2921-fb5a-4c03-9617-ead08c210464
fa9bb691-2490-48ad-b967-85e8bc66fccd
2719f01f-191b-4305-9af9-b0aa5b5bbbc9
050176ce-5244-43cb-ac7f-7fa5799ce9f6
ab9c194a-42ca-404c-870d-2fdd04599288
dd1f6c15-3296-4001-8622-5548e086c224
4bcab969-fb0b-4aed-9e1b-1bb5368b6800
238e0ca3-775d-4395-820e-7f35a397cf54
37a2363a-8ff4-4166-a148-a348169e2f5c
6e55c90a-2486-4cb5-aecf-a8e81ea98be7
b440d1bd-ec7b-404c-b94b-1e15572a81dc
0dd8d93d-6cb0-4bbd-9348-a3d2fdb576ef
7bab6ab3-75b1-44d7-b8ce-b32cc9d94a78
787fea7b-9bd1-4619-bc87-42907ac31756
bd34e907-a173-40d9-aaef-090f34a9dafb
6a9391af-1473-4037-9b41-27a26ab090ae
bc37ff35-ffe3-43d6-8f20-e5d8e33d44fd
4eea6845-ee77-436b-b1d9-c7d8094b69f4
a1f03a72-d327-4df3-9d9f-db3a610f280a
eae72e13-d3c5-4ac5-b475-740c368577dd
b183f678-fd40-4319-9575-8beda4983548
86b91aa4-1803-42fa-a760-3d21fa539655
82518ac6-28ff-4702-b851-c70f2436b7c5
ea7dd3c6-8f50-48da-91da-95685c4ffd27
307067ad-beb7-43c2-9670-745836a7065a
675bb7cd-c245-4d5e-8885-8ae5fe5a2180
31d6ba18-665a-4884-a29d-cd2f48b10225
cc9979e4-0396-471d-a573-4540b3fe969a
fd422b30-5037-429b-9bbb-772b16eff56d
18da931e-8561-4db1-bfcd-fc8759b85190
f224300f-5294-4484-bb46-674dfa8f0ff5
4a1ecb6c-a86f-4bbc-b0b2-2751e97f1804
6faad35d-52dd-42e3-bc44-ac22ccfecf27
ebffbbfd-bff9-47b2-9d33-aa3236f20201
bc1d2ebd-200e-4a38-bc4a-0d138ab88069
8fead479-51ce-48ec-a94e-b1efa9d47981
4ac6dcda-8209-4eff-8c04-b4110f8037ee
fc38adf8-10a4-454a-a2a1-5e8e8a7430ef
7dbd2224-c032-489b-a47f-187474e97adc
e3246107-85e3-454d-9a72-09e983253962
4bb326bc-d9b2-40d1-b9e2-36c48e4f9cc0
ee683d9e-6c7f-40bb-8882-12111e87b9d3
d04194b1-994c-4737-a70e-571bde7ffaed
7eb0c212-c0a9-4bbf-8432-f45449a53729
8030f657-7e03-4355-8089-c932f7a84dec
8a39b401-d784-45f1-89f0-d5ef7dffdb5f
60ca3dbf-d68e-4025-8568-988cd7d87121
b210c554-bdf4-4cea-8629-05384f354723
1ff0b835-2d08-4b40-ba08-59889b72ffec
50c76cf4-9e8b-45b2-9176-7bbcc98911a2
df2a3d67-d2c8-44a6-9cd5-871b80ace1a4
0610e834-ddf6-4864-ab41-da5a1b3a6813
2720ec9d-1d1d-46c8-aba9-64d4fb4ce78c
3c966565-8ba4-46b9-82f7-e4ecc65ea7dd
1d2042da-5e12-4e6d-8a7f-d0080fa70fd0
a1861f78-4547-4c13-abcf-d7e98dc9290b
c862d3ed-1df1-4acd-a8ec-796cfd371865

Duplicate Entry: Shirobako Special

I hope I'm posting in the right place.

There's a duplicate entry for Shirobako Specials which should be merged.
image

There are also three separate entries for the Shirobako Movie, which should all be merged into one.
image
image

Incorrect Entry

Looks like the following mapping is not done correctly:

{"anilist":null,"anidb":15335,"myanimelist":null,"kitsu":null}

and

{"anilist":114236,"anidb":null,"myanimelist":40956,"kitsu":42909}

despite them referring to the same show.

How is the json file updated?

Hey, I was wondering how the json file was updated? Is it done manually or through some scripts? I'd like to contribute to this project.

Typo

  • Which data is not extracted correctly? (e.g. title, episodes...)
    Download json on 24/12/2022

item[601] Title : 'Ai Tenshi Densetsu Wedding Peach'
synonyms[8] : 'Wedding Peach: I tanti segreti di cuore innamorato'

  • Can you provide an example entry?

  • Which value is expected?
    you should replace :

'Wedding Peach: I tanti segreti di cuore innamorato'

with:

'Wedding Peach: I tanti segreti di un cuore innamorato'

difference : 'un'

Happy Holidays and Thank you for your work !

AniDB Title Synonyms/Shorts are not populated

The Synonyms and Shorts from AniDB are not being populated in the offline database.

I am not knowledgeable in either the process used to generate the file nor the language, however it seems like the process is using a web scraper, and I noticed that on the AniDB website Synonym and Shorts are capitalized, while the search code seems to have all lower case. Is that web scrape case sensitive?

Two examples that have a number of synonyms and/or shorts to illustrate ..
https://anidb.net/anime/15146
https://anidb.net/anime/15283

Is the process used to generate the AOD using the generated title xml that AniDB regularly updates? https://wiki.anidb.net/API - File is http://anidb.net/api/anime-titles.xml.gz. I don't know if using that might help if it's not currently being used as it avoids the whole web scraping thing (which irritates the anidb mods from what I can tell), and might be faster then the normal http requests for all that stuff. It only has titles and AID, so it can't really help with any data beyond that.

Add KITSU relations

Since I added KITSU I always postponed adding the KITSU relations.
This should be added with the next update.

REST API

I created a REST server that returns the data in this repo in JSON form.
It fetches and caches the data in a database once per day, and is available at https://relations.yuna.moe (Github).

Would it be a good idea to add it to the readme of this repo?

How to report duplicates in Manami entries?

Hi,

As the "merge locks" are not defined anymore and as Manami IDs change over time, what is the best way to report to you duplicates? What do you internally use as unique identifier, e.g. "Berserk (2016)"?

Suggestions:

  • Sending you sets of titles that actually belong to the same work
  • Sending you sets of sources that point to the same work

I have a dozen of them.

A peculiar property of cross links to anidb

I've been mapping songs from anidb to mal anime ids, and discovered something about links to anidb entries. Some of them do not appear in manami, which is not the issue.

However, there seems to be an improvement potential, unmapped anidb anime titles all seem to be marked 18+. The thing is, anidb does not show them unless you are logged in and confirm you are over 18 y.o.

Here are some of those anidb ids for reference:
manami missing ids {'1640', '1847', '1166', '1278', '1546', '2803', '9549', '5358', '1492', '11194', '1426', '6002', '6765', '1876', '2748', '11182', '15258', '14102', '7984', '13384', '1556', '8814', '4544', '1844', '12157', '1522', '249', '6189', '1212', '1626', '8911', '4848', '5633', '1557', '307', '2566', '14252', '1629', '4981', '3289', '3982', '2747', '2552', '308', '2032', '5543', '13170', '4735', '2029', '7929', '2148', '1982', '2485', '2565', '549', '5203', '5359', '1333', '1328', '6484', '6817', '300', '9276', '851', '9469', '6561', '1088', '1396', '663', '664', '10942', '9699', '7259', '4347', '4443', '13652', '10504', '6208', '8994', '677', '4423', '1405', '174', '119', '15697', '12005', '7037', '907', '13761', '10263', '14340', '1047', '13411', '8262', '11085', '9933', '1603', '1987', '5594', '10296', '15510', '4078', '2254', '8402', '2141', '2145', '2571', '1875', '12547', '1624', '2573', '8473', '13550', '5205', '1628', '13285', '5541', '1627', '10964', '12512', '559', '12699', '700', '2567', '5031', '11047', '7030', '933', '1062', '1042', '12596', '1974', '849', '13489', '1168', '1420', '1510', '6154', '3527', '2373', '13356', '1843', '7915', '5200', '3999', '4493', '7489', '6675', '11476', '1338', '5362', '802', '1347', '8795', '2027', '10893', '1410', '10793', '7303', '2570', '267', '1642', '13251', '15630', '1108', '1720', '10839', '6781', '839', '1970', '8837', '1665', '4304', '2515', '3452', '1752', '2502', '2563', '6554', '667', '3531', '13834', '1337', '357', '1547', '7519', '1133', '1509', '5848', '7566', '243', '4155', '8508', '1159', '2268', '3537', '10221', '5889', '2804', '1346', '1694', '4075', '1207', '1331', '985', '5912', '5836', '2561', '870', '5193', '1433', '4823', '2701', '10189'}

Extra addition if possible: Synopsis and genres

Hey! Above all, thank you so much for the work here! You are awesome!
Is it possible to add synopsis and years to the list?
Also is adding genres also possible? I know it might be too much work than its worth, but if it would be possible to add I would be forever grateful.
Once again, thanks for the awesome work.

Clarification about merge preventions and locks

Hi, thanks for this awesome project!
Maybe this is just about vocabulary, but I don't get how preventions and locks are related.

Preventions means: should not merge, right?
And locks means: should merge, right?

By the way, thanks to our own database I think we managed to isolate merge suggestions for the anime offline database. See more in this notebook.

It's not perfect yet, as for example all Kara no Kyoukai movies correspond to the same AniDB ID, but you might want to consider them separately.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.