Code Monkey home page Code Monkey logo

lmlcspider_production's Introduction

基于 nodejs 的立马理财爬虫项目

要是觉得这个项目还不错,别忘记star哦

技术分析,详见我的博客

线上地址:点击这里

一、工程目录分析

data是爬虫爬取数据后存储的文件夹

dist是前端js、css打包的目录

public存放一些公共文件

server是后台代码目录

  • user.js是首页用户购买ajax接口爬虫代码
  • product.js是理财页产品ajax接口爬虫代码
  • record.js是详情页投资记录同步页面爬虫代码
  • getCookie.js是测试爬虫带cookie模拟登录代码
  • dataHandle.js是处理数据得到销售额的代码
  • lmlcSpider.js是将上述爬虫代码整合在一起的最终线上用的爬虫代码

spider存放爬虫代码的目录

src前端工程目录

views存放html模板

二、项目运行

  1. 下载chrome插件LiveReload

  2. 修改views文件夹里的index.html和detail.html文件。

  • 如果是本地调试,index.js都要改成http://localhost:8080/dist/index.js
  • 如果是其他设备访问,则改成your_ip_addr:5000/dist/index.js
  • index.css一样。
  1. 运行方式
npm run dev
npm run server // 启动后台
npm run spider your_username your_password // 启动爬虫
  1. 测试学习各页面的爬虫代码
npm run user  // 爬取立马理财首页的用户购买信息ajax接口
npm run product  // 爬取立马理财理财页所有产品的ajax接口
npm run record  // 爬取立马理财产品详情页的投资记录同步页面数据

注意

开启爬虫需要带网易立马理财登录的账号和密码参数。当然如果有网易邮箱,可以直接输入网易邮箱和密码即可。

lmlcspider_production's People

Contributors

tywei90 avatar

Watchers

xiao avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.