erisyoung / douyin_spider Goto Github PK
View Code? Open in Web Editor NEW🎨One simple and easy to use crawler for DouYin(一个简单易用的抖音爬虫,可下载指定用户,挑战,音乐的视频,音频和数据)
License: MIT License
🎨One simple and easy to use crawler for DouYin(一个简单易用的抖音爬虫,可下载指定用户,挑战,音乐的视频,音频和数据)
License: MIT License
macOS 10.15.1
python 3.6.5
在douyin_spider/douyin_spider中运行
python assign_share_url.py -u "https://v.douyin.com/QHt8K5/"
https://www.iesdouyin.com/share/user/76725372134?u_code=119ahghf6&sec_uid=MS4wLjABAAAA2jD45shuaphDnTULtCA3baR-xPXsD97pzSzgKAYwfss×tamp=1573702412&utm_source=copy&utm_campaign=client_share&utm_medium=android&share_app_name=douyin
Parse share-url to user share url...
Downloading videos from user, count:10...
<User<default,76725372134>>
internal/modules/cjs/loader.js:797
throw err;
^
Error: Cannot find module '/Users/macbook/anaconda3/lib/python3.6/site-packages/douyin_spider/utils/decryption/signture.js'
at Function.Module._resolveFilename (internal/modules/cjs/loader.js:794:15)
at Function.Module._load (internal/modules/cjs/loader.js:687:27)
at Function.Module.runMain (internal/modules/cjs/loader.js:1025:10)
at internal/main/run_main_module.js:17:11 {
code: 'MODULE_NOT_FOUND',
requireStack: []
}
Traceback (most recent call last):
File "assign_share_url.py", line 161, in <module>
run()
File "assign_share_url.py", line 157, in run
process_func(real_url, count)
File "assign_share_url.py", line 130, in download_videos_from_user
downloader.download(user.videos(max=count))
File "/Users/macbook/anaconda3/lib/python3.6/site-packages/douyin_spider/downloaders/parent.py", line 92, in download
for item in input_items:
File "/Users/macbook/anaconda3/lib/python3.6/site-packages/douyin_spider/models/user.py", line 35, in videos
'_signature': generate_signature(str(self.id)),
File "/Users/macbook/anaconda3/lib/python3.6/site-packages/douyin_spider/utils/decryption/signture.py", line 20, in generate_signature
return p.readlines()[0]
IndexError: list index out of range
可以根据手机号找到uid吗
怎么取的v1/feed/ 推荐列表的数据,那个签名不管用啊
比如:https://api.amemv.com/aweme/v1/feed/
使用的示例代码
`
from douyin_spider.downloaders.video import VideoDownloader
from douyin_spider.handler.video import VideoHandler
from douyin_spider.handler.music import MusicHandler
from douyin_spider.handler.mongodb import MongoHandler
from douyin_spider.enter.hot_top import hot_top20
video_handler = VideoHandler(folder='./videos')
music_handler = MusicHandler(folder='./musics')
mongo_handler = MongoHandler()
downloader = VideoDownloader([video_handler,music_handler,mongo_handler])
result = hot_top20()
for item in result.data:
downloader.download(item)
print("success")
`
执行命令结果报错
`A:douyin_spider mmm$ python douyin.py
Traceback (most recent call last):
File "douyin.py", line 2, in
from douyin_spider.handler.video import VideoHandler
File "/Users/mengqingyue/anaconda3/lib/python3.5/site-packages/douyin_spider/handler/video.py", line 1, in
from douyin_spider.handler.media import MediaHandler
File "/Users/mengqingyue/anaconda3/lib/python3.5/site-packages/douyin_spider/handler/media.py", line 54
media_save_path = Path(self.folder).joinpath(f"{item.id}.{extension}")
^
SyntaxError: invalid syntax
`
操作系统时MacOS, Python版本为3.5,请问该怎么办呢?
大佬,你好,你这个方法可以得到用户得城市这个字段吗?
估计是接口信息又做了修改,更换的频率有点高,up主若是有时间希望做下更新,有机会一起做交流,叩谢 ...
Error: Cannot find module '/Users/garry/anaconda3/lib/python3.7/site-packages/douyin_spider/utils/decryption/signture.js'
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.