Code Monkey home page Code Monkey logo

Comments (3)

jagrosh avatar jagrosh commented on August 14, 2024 4

I don't like making a feature half-way, so while I appreciate the design, if I add a way to check links for redirects, it will check all links at a much larger depth. The exception to this might be things such as discord.me and other popular listing sites.

from vortex.

jagrosh avatar jagrosh commented on August 14, 2024

As cool of a feature it would be to follow redirects on all links,

  • it would not only use a lot more resources but also take a lot longer to evaluate if moderation is needed
  • it would expose the bot's IP to any link along the path, and thus be susceptible to attacks
  • some redirect paths can be insanely long to follow

I think a better solution to this problem is just to blacklist any redirect sites' urls once I add a blacklisting feature.

from vortex.

ZomoXYZ avatar ZomoXYZ commented on August 14, 2024

What about only following known shortlinks, such as goo.gl and bit.ly, they are the most common and both have APIs.

Google: https://developers.google.com/url-shortener/v1/getting_started#expand
Bitly: http://dev.bitly.com/links.html#v3_expand

Only following these links shouldn't take up too many more resources or time. These websites may show a generic location to where it was from, but not your actual IP. To stop redirect loops, just set a limit.

Here's some pseudocode to only follow Google and Bitly links:

//find every google/bitly link
links = content.match(/(goo.gl|bit.ly)\/[a-z0-9]+/i)

//function to expand a google link
function expandGoogle(link)
    body = GET "https://www.googleapis.com/urlshortener/v1/url?key=ACCESS_TOKEN&shortUrl=" + link
    body = parseJSON(body)
    return body.longUrl

//function to expand a bitly link
function expandBitly(link)
    body = GET "https://api-ssl.bitly.com/v3/expand?access_token=ACCESS_TOKEN&shortUrl="+ link
    body = parseJSON(body)
    return body.data.expand.long_url

//list of links the script has processed so it will not repeat the same link
processedLinks = []

//for each found link
for i = 0 links.length > i i++
    maxTime = 5 //most number of redirects it will look through in each link
    times = 0
    
    //loop for each redirect
    function loop(link)
        if !processedLinks.has(link)
            processedLinks.push(link)
            
            if link.match("goo.gl")
                link = expandGoogle(link)
                times++
                if times < maxTime
                    loop(link)
            else if link.match("bit.ly")
                link = expandBitly(link)
                times++
                if times < maxTime
                    loop(link)
            else if link.match("discord.gg")
                i = links.length //stop for loop
                message.delete() //replace this with whatever function you run for each discord.gg link
    
    loop(links[i])

This shouldn't loop too many times (maximum of 5*(number of unique bitly/google links), and can be cut off early).

A blacklist feature would make this easier, but it could delete links that are innocent.

from vortex.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.