"Sign in to confirm you’re not a bot." error with YouTube videos about musicbot HOT 48 OPEN

As this is an issue that affects every project using YouTube, several communities have come across a few discoveries. Here's a summary of what I've seen:

1. This is an IP block.
2. Instead of just banning one IP, it looks YouTube is blocking whole ASNs/IP ranges. If you encounter this issue, it's likely a "neighboring" IP address was bombarding YouTube, causing them to block the ASN/IP range you're in.
3. The cobalt.tools developers seem to have found a workaround using OAuth2. This currently is not implemented in lavaplayer has a PR in the lavaplayer youtube source manager, however it seems a bit risky.
4. IPv6 rotation might also work, which lavaplayer supports & encourages using. We haven't put much thought about supporting IPv6 rotation on JMusicBot yet though.

In my opinion, I see three main ways JMusicBot could move forward:

1. Wait for OAuth2 support to be arrive in the lavaplayer youtube source manager & make use of that
2. Allow configuring IPv6 rotation for users on VPS' as they are most commonly affected by these blocks.
3. Implement support for an external tool (cobalt.tools) as a source, which have circumvented this block. I'm not sure if they would appreciate the additional traffic from JMusicBot users though.

Only my two cents though, no discussion around this has happened yet.

from musicbot.

This started affecting me on a Hetzner IP range yesterday. Not a solution per se, but managed to temporarily workaround it by routing my jmusicbot Docker container through my WireGuard Docker container running a config from a paid VPN service.

from musicbot.

@trodrigues This is what I'm using:

services:
  jmusicbot:
    container_name: jmusicbot
    image: bas0korver/jmusicbot:latest
    restart: unless-stopped
    network_mode: "container:wireguard"
    volumes:
     - ./config:/opt/jmusicbot/config

services:
  wireguard:
    container_name: wireguard
    image: linuxserver/wireguard
    restart: unless-stopped
    environment:
      - PUID=1000
      - PGID=1000
    cap_add:
      - NET_ADMIN
      - SYS_MODULE
    volumes:
      - ./config:/config
      - /lib/modules:/lib/modules
    privileged: true
    sysctls:
      - net.ipv4.conf.all.src_valid_mark=1
      - net.ipv4.ip_forward=1
      - net.ipv6.conf.all.disable_ipv6=1

from musicbot.

This other repo fixed the issue by adding support for Oauth2, essentially allowing people to create dummy accounts for the bots to log in with: imputnet/cobalt#551

Is this something we'd have to request support for in lavaplayer? Or is MusicBot responsible for querying the youtube API?

from musicbot.

As a test, on my server computer i created a new google account (so it appears that the registration is occurring from the banned ip) and logged in. I was able to play youtube video's in the browser! But when i log out in browser, once again im presented with the error. So if the bot can log into youtube, that should solve the problem for everyone.

from musicbot.

@Ruok2bu There is no difference between downloading and streaming a video from YouTube's point of view. It won't get around the ban.

from musicbot.

The solution might be to just log in. I checked it on my VPS and when i logged in, it started to work.
yt-dlp allow to login via parameters: https://github.com/yt-dlp/yt-dlp?tab=readme-ov-file#authentication-options
So I think you need to implement option to provide credentials for yt-dlp

from musicbot.

Also having this issue, if anyone has a solution or a quick and dirty workaround, I'd be down

from musicbot.

downloading youtube videos through tor might be a workaround for this problem. just tested this, it works

$ torsocks yt-dlp https://youtu.be/AL5vANu2gok                                                                                                                                                               
[youtube] Extracting URL: https://youtu.be/AL5vANu2gok
[youtube] AL5vANu2gok: Downloading webpage
[youtube] AL5vANu2gok: Downloading ios player API JSON
[youtube] AL5vANu2gok: Downloading player 4fc7f9fa
[youtube] AL5vANu2gok: Downloading m3u8 information
[info] AL5vANu2gok: Downloading 1 format(s): 302+251
[download] Destination: Black Holes Explained For Gen-Z [AL5vANu2gok].f302.webm
[download] 100% of   48.22MiB in 00:03:40 at 223.78KiB/s
[download] Destination: Black Holes Explained For Gen-Z [AL5vANu2gok].f251.webm
[download] 100% of   10.01MiB in 00:00:49 at 207.41KiB/s
[Merger] Merging formats into "Black Holes Explained For Gen-Z [AL5vANu2gok].webm"
Deleting original file Black Holes Explained For Gen-Z [AL5vANu2gok].f302.webm (pass -k to keep)
Deleting original file Black Holes Explained For Gen-Z [AL5vANu2gok].f251.webm (pass -k to keep)

from musicbot.

I'm facing the same issue:

Jun 17 23:49:14 env[441177]: [23:49:14] [ERROR] [AudioHandler]: Track BJhF0L7pfo8 has failed to play Jun 17 23:49:14 env[441177]: com.sedmelluq.discord.lavaplayer.tools.FriendlyException: Sign in to confirm you’re not a bot Jun 17 23:49:14 env[441177]: at dev.lavalink.youtube.clients.skeleton.Client.getPlayabilityStatus(Client.java:72) Jun 17 23:49:14 env[441177]: at dev.lavalink.youtube.clients.skeleton.NonMusicClient.loadTrackInfoFromInnertube(NonMusicClient.java:98) Jun 17 23:49:14 env[441177]: at dev.lavalink.youtube.clients.skeleton.StreamingNonMusicClient.loadFormats(StreamingNonMusicClient.java:35) Jun 17 23:49:14 env[441177]: at dev.lavalink.youtube.track.YoutubeAudioTrack.loadBestFormatWithUrl(YoutubeAudioTrack.java:175)

from musicbot.

I can confirm that with my VPS Server, it logs a "Sign in to confirm you’re not a bot." error everytime I want to play a youtube video on my server.

HOWEVER, just recently I found out my old instance of my bot in my local machine work perfectly with youtube.
Perhaps it's something to do with how the I.P of my local machine is tied to my youtube account perhaps? maybe youtube blocked requests from my server because of that.

from musicbot.

a bot called "Nero Bot" manages to still play YT videos- would be worth looking into how they manage past the whole thing with their "nodes".

from musicbot.

a bot called "Nero Bot" manages to still play YT videos- would be worth looking into how they manage past the whole thing with their "nodes".

1. Nero Bot is not open source
2. Minerea is also able to play YT. Not open source
3. Chances are they are paying for a YT API or idk

from musicbot.

Some people over lavaplayer suggests to use lavaplayer-fork to solve youtube issues

from musicbot.

Some people over lavaplayer suggests to use lavaplayer-fork to solve youtube issues

We're already using lavalink's lavaplayer fork, which is still receiving maintenance. For this issue specifically, a simple update is very unlikely to fix anything, because this block also affects all of YouTube's own apps/clients.

from musicbot.

So I guess this is a problem to any bot playing from YouTube as a source, right?

from musicbot.

See the various other issues on the internet on this topic for a rough history of this. The cobalt.tools developers seem to have found a workaround related to OAuth2, however this has not been implemented in any lavaplayer fork yet, and also seems a bit risky.

from musicbot.

The issue is fixed for me on version 0.4.2 🎉

from musicbot.

This same issue has started occurring for me, has an easy solution been found yet? (one that a novice like me could implement)

from musicbot.

There is no solution at the moment.

from musicbot.

This started affecting me on a Hetzner IP range yesterday

I'm also using Hetzner. Not sure if that's a coincidence or not.

from musicbot.

I'm also using Hetzner. Not sure if that's a coincidence or not.

This is expected. As I mentioned earlier, it seems like YouTube is banning entire ASNs/IP ranges, rather than just individual IP addresses. Chances are pretty high that all of Hetzner’s ASNs/IPs have been banned.

from musicbot.

So if the bot can log into youtube, that should solve the problem for everyone.

No, apparently this block affects both with or without an account, on the same IP.
See:
lighttube-org/LightTube#154
yt-dlp/yt-dlp#10085

Edit: Oops, I mentioned another problem without even paying much attention. Yes, you can "fix" it by connecting with an account (yt-dlp/yt-dlp#10128 (comment)), but I don't think it would be very ideal for a project like this (Not to mention that Google can easily end up discovering this repository and try to destroy it, I think). The only four solutions I've seen for now are:

• Add full support for IPv6 rotation, as stated earlier in this Issue.
• Set PO_token e visitordata ID (iv-org/invidious#4789; and it's still in WIP, and I don't know if their correction can be valid here or in lavaplayer)
• Wait for lavaplayer support to add OAuth2 (And yes, it is still risky and is a temporary solution)
• Change the default client player to web_creator, since it was one that was not affected by the new change. (yt-dlp/yt-dlp#10457; the only problem is that there will be audio loss, and it's also a fairly temporary solution, since YouTube can change this client as well)

While there is no 100% working solution, the only way to continue using the bot is to go to IPs, servers and/or VPNs that YouTube has not (yet) blocked.

from musicbot.

this also affects me
i used netrc file and add the credentials of youtube but still get the same error

from musicbot.

YT seems to want to shoot a bullet in his own foot.... I have the same issue from 4 servers in 4 different DC. well done YT, soon you are going to ban yourself as some of your servers are hosted in the same DC of ours!

from musicbot.

can confirm it's happening for me on hetzner also
not using musicbot though

from musicbot.

It’s also happening to me on Aruba (Italian hosting provider)

from musicbot.

Same. Not working for me either. Using a Hetzner Dedicated Server with Windows Server 2022 // Docker Desktop.

from musicbot.

i am just popping in to say that after a bunch of tinkering with my OVHcloud server yesterday i only found out that if you run jmusicbot and then a VPN (OpenVPN + Windscribe is what ive used) on top of it then will the bot finally be able to play youtube links without any issues, granted this is still a workaround and you may have to restart your vpn again each time if youtube happens to ban the one the vpn is running on but so far ive had 0 issues since ive went and used this solution.
just thought id let everyone know about this since ive seen a bunch of people still trying to figure out how to get their bot to play yt links so yeah i hope this helps

from musicbot.

same on OVH server.. any resolution is planned?

from musicbot.

same on OVH server.. any resolution is planned?

Using cloudflare warp on my racknerd vps fixed the issue BUT i lost accès to ssh and now use vnc because I didn't set up cloudflare warp correctly and I'm too lazy to reinstall it, I reccomend you to search on google stuff like "can't accès ssh after installing cloudflare warp" and you will probably find something to not have the same issue as me

Sorry if my English is bad

from musicbot.

Hello! I also encountered the same problem. Has anyone found a solution to this problem? My server is from Hentzer. Located in Finland!


Of course I have a nextcloud cloud. But maybe someone already knows the solution to this problem?

from musicbot.

But maybe someone already knows the solution to this problem?

No, it doesn't have a 100% effective solution so far. There are repositories with the same problem, and others have some pulls that can fix the problem, but many will fix it temporarily.

The best thing to do is to wait.

from musicbot.

well, why not to totally boycott the corpofascists?

from musicbot.

But maybe someone already knows the solution to this problem?

No, it doesn't have a 100% effective solution so far. There are repositories with the same problem, and others have some pulls that can fix the problem, but many will fix it temporarily.

The best thing to do is to wait.

Wait what? How long will the ban last? or until a complete solution comes out?

from musicbot.

One thing that can work and its not ideal. But what if the bot downloads the youtube video and then plays the audio through the bot? And optionally caches the downloaded video into a folder structure on the hard drive so it doesnt need to contact youtube servers again. I realize this method can bloat the hard disk with GB's of files, but if they are also converted from mp4 (m4v) to ogg (not mp3 because ogg offers better compression than mp3) then some disk space would be saved.

I realize this method wont unban everyone's IPs but at least it would allow us to get around the ban.

from musicbot.

@Ruok2bu There is no difference between downloading and streaming a video from YouTube's point of view. It won't get around the ban.

But at least adding that ability could let us schedule when the bot accepts songs to download (in other words it wont trigger the bans all the time). For example, allowing music requests for set time of the day or up to x amount per hour so youtube is less likely to block.

from musicbot.

This is highly unlikely to fix anything in a VPS/cloud service context (which are the people who get affected by this block the most.) Based on observations made by the community about this block, YouTube is very likely applying a ban to the whole ASN (IP range.) If only one bad actor within your "IP range" is misbehaving by downloading lots of videos in a short period, you will end up being affected through no fault of your own.

from musicbot.

Then there's only one way out of this. Some rich person needs to buy a ton of google stocks, get on the board and then use their veto power to change the rules.

from musicbot.

@MichailiK
well, imagine someone using yt-dlp in an airport which usually has an ip range, good luck to ban an entire airport without any consequences!

from musicbot.

@MichailiK well, imagine someone using yt-dlp in an airport which usually has an ip range, good luck to ban an entire airport without any consequences!

xD

from musicbot.

But maybe someone already knows the solution to this problem?

No, it doesn't have a 100% effective solution so far. There are repositories with the same problem, and others have some pulls that can fix the problem, but many will fix it temporarily.
The best thing to do is to wait.

Wait what? How long will the ban last? or until a complete solution comes out?

They also don't know how long this ban lasts, they say it's 72 hours or it's permanent. (yt-dlp/yt-dlp#10128 (comment))

from musicbot.

This started affecting me on a Hetzner IP range yesterday. Not a solution per se, but managed to temporarily workaround it by routing my jmusicbot Docker container through my WireGuard Docker container running a config from a paid VPN service.

@Snaacky would you happen to have a bit of guidance on you set up jmusicbot to go through the VPN? Is it a container level configuration?

from musicbot.

The lavaplayer oauth PR we are (maybe) waiting on has been making progress.

@MichailiK You mentioned a couple times that you think the idea of having JMusicBot users logging in with oauth seems "a bit risky." Could you explain that? I don't understand what the risks are.

from musicbot.

Google could figure out it is not a legitimate client playing back videos. As using JMusicBot technically violates YouTube's TOS, they could terminate your Google account. The OAuth2 PR also states at the end this should not be used on your main Google account.

(For clarity, the OAuth2 API that's being used here is unofficial & undocumented, it's only meant to be used by (Smart) TV makers as far as I'm aware.)

from musicbot.

Google could figure out it is not a legitimate client playing back videos. As using JMusicBot technically violates YouTube's TOS, they could terminate your Google account. The OAuth2 PR also states at the end this should not be used on your main Google account.

(For clarity, the OAuth2 API that's being used here is unofficial & undocumented, it's only meant to be used by (Smart) TV makers as far as I'm aware.)

You can make warning to dont use main account, just create new one :)

from musicbot.

how to fix this bug "Sign in to confirm you’re not a bot" while using the distube-ytdl package

from musicbot.

mark

from musicbot.