怎么设置要爬取的用户 about cola HOT 4 CLOSED

是在weibo.yaml下设置uid

把配置贴出来看看，注意帐号密码隐藏一下。

from cola.

好的

job:
db: sina
mode: bundle # also can be bundle
size: 1000 # the destination (including bundle or url) size
speed:
max: -1 # to the cluster, -1 means no restrictions, if greater than 0, means webpages opened per minute
single: -1 # max restrictions to a single instance
adaptive: no
instances: 2
priorities: 13 # priorities queue count in mq
copies: 1 # redundant size of objects in mq
inc: yes
shuffle: no # only work in bundle mode, means the urls in a bundle will shuffle before fetching
error:
network:
retries: -1 # 0 means no retry, -1 means keeping on trying
span: 20 # seconds span to retry
ignore: no # only work under bundle mode, if True will ignore this url and move to the next after several tries, or move to the next bundle
server: # like 404 or 500 error returned by server
retries: 5
span: 10
ignore: no
components:
deduper:
cls: cola.core.dedup.FileBloomFilterDeduper
mongo:
host: localhost
port: 27017
login:
- username: ***
password: ***
starts:
- uid: 3045198074
- uid: 3969344084
fetch:
forward: no
comment: no
like: no
clear: no

发现总是会get同样的uid
比如：
get 1701018393 url: http://weibo.com/aj/mblog/mbloglist?count=15&pre_page=28&uid=1701018393&end_id=3907520622156990&_t=0&_k=1447243354911000&__rnd=1455415453462&pagebar=1&max_id=3618682338871099&page=28

from cola.

OK，可能是你之前跑过了，然后重设了uid以后会接着之前的跑。

可以把clear设置成yes看看行不行。（这个选项每次会重新开始，下次确认要接着抓取需要改回来）

from cola.

ok 解决了 非常感谢

from cola.