This is an distrubuted spider system with Frontend which wrote by React. Spider contains web spider、weixin spider、weibo spider、app spider。Backend wrote by Tornado, cause I use it in my third company. This system also use Celery for destribute tasks to spiders, and use Redis to remove duplicate data such as URL, and use MongoDB to store pawed data, and use MySQL meeting business needs. U can use Docker to start a server in convenient ways, and use K8S to manage ur micro-sevrers. Hope it helps u. Hope ur suggestions.

写在最后一行的一点*话，事实上这是一个个人学习的项目，所以会有很多不规范的地方，如果有好的意见期待你们指出。