Code Monkey home page Code Monkey logo

Comments (13)

yun17 avatar yun17 commented on May 21, 2024

你的数据库的login_info表中有没有账号?
使用爬虫之前需要把账号信息手动录入数据库当中。

或者
如果你忘记分配login_queue任务,那么也会导致这个问题
分配了login_queue的worker会在有登陆任务的时候进行登陆获取cookies
爬取任务在cookies池没有cookies的时候会尝试从数据库中获取账号信息并建立登陆任务(实际登陆交给有login_queue的worker)

所以你的问题的解决方法有:

  1. 确认login_info表中有账号
  2. 在任意节点的-Q参数后面添加login_queue,比如
    celery -A tasks.workers -Q login_queue,repost_crawler,repost_page_crawler worker

from weibospider.

shiyuzhu avatar shiyuzhu commented on May 21, 2024

image
账号问题 谢谢

from weibospider.

ResolveWang avatar ResolveWang commented on May 21, 2024

问题解决了吗?有问题提issue就是哈。大家懂的就帮忙答复一下

from weibospider.

ResolveWang avatar ResolveWang commented on May 21, 2024

如果账号登录需要验证码的,需要在数据库里把need_verify设置为1(默认也是1),设置后需要去云打码上注册一个账号,充值一块钱,用作识别验证码

from weibospider.

LymanLiuChina avatar LymanLiuChina commented on May 21, 2024

新买的一批号嘛, 跑了一会儿 就又出现这种情况,没有cookies了, 每个节点都有 login_queue

from weibospider.

ResolveWang avatar ResolveWang commented on May 21, 2024

你要确定一下,你的账号是否被封了,如果被封了,所有cookie都会被删除。如果长期单一ip,可能ip会有一些风险,意思就是,只要你用那个ip的所有账号,都会被要求验证或者别的一些措施

from weibospider.

LymanLiuChina avatar LymanLiuChina commented on May 21, 2024

我换个IP在试试

from weibospider.

ResolveWang avatar ResolveWang commented on May 21, 2024

嗯嗯,你可以手动用那些账号在有问题的那个ip试试,看是一个怎么样的效果,如果是被封了,然后你在该ip登一个没被封的账号(最好是一个正常账号,而不是买的低质量的号),再看看效果咋样

from weibospider.

LymanLiuChina avatar LymanLiuChina commented on May 21, 2024

我看到 数据里面 为0 的号 去手动登录了一下,是正常的没有被封,我调整数据库后,我换了一个外网IP后, 直接 login_queuw,home_crawler 命令 还是 显示 没有cookies , 感觉 login_queue这个任务没有执行.

from weibospider.

ResolveWang avatar ResolveWang commented on May 21, 2024

那是你的问题了,你需要在启动worker的时候指定login_queue,我看你写成了login_queue,不知道你启动命令行的时候是不是这样写的。还有,你可以在从数据库取账号的时候,把取得的账号打印一下,就知道到底取出来没了

from weibospider.

LymanLiuChina avatar LymanLiuChina commented on May 21, 2024

那个是我输入错误,但是, 在celery -A tasks.workers -Q login_queue,user_crawler worker -l info -c 1 上面写道 直接受到 user_crawler 的任务,只有在我手动 Python login_first.py 才会启动login这个任务,是不是每次当cookies池没有cookies ,都必须手动在重新执行一遍 这个文件后 login_queue 这个任务才会执行??

from weibospider.

ResolveWang avatar ResolveWang commented on May 21, 2024

不是,有个模块是定时任务,你好好读文档,好好理理思路。

from weibospider.

ResolveWang avatar ResolveWang commented on May 21, 2024

定时任务是有一定时间延迟的。

from weibospider.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.