Code Monkey home page Code Monkey logo

spareroom-scraper's Introduction

spareroom-scraper

Filter SpareRoom listings and calculate commute times with CityMapper API to allow more comprehensive filtering. Outputs a .csv and can be re-run daily.

This code is pretty horrible as it was written in a mad rush to find an apartment - it was extremely useful as it narrowed SpareRoom's >1000 suggested Apartments to <20. Use with trepidation.

The barebones of the scraper were stolen and bastardised from https://github.com/davidmoremad/find-room-in-london

Usage

  1. Get an API key for citymapper from this link
  2. Copy and paste the API key into main.py along with a search link for your city of choice.
  3. Run the script and apply filters on the resulting spreadsheet as you see fit. For this I recommend google sheets.

Example Output

Fields: url id title desc Type Area Postcode Nearest station location_coords cycle_time transit_time Available Minimum term Maximum term room_0_price room_0_type Deposit room_1_price room_1_type Deposit(room 1) room_2_price room_2_type Deposit(room 2) rBills included? Furnishings Parking Garage Garden/terrace Balcony/patio Disabled access Living room Broadband included Flatmates Total rooms Age Smoker? Any pets? Language Nationality Occupation Gender Couples ok? Smoking ok? Pets ok? References? Min age Ages Interests Max age Housemates Orientation University Vegetarian date_scraped

And a filtered google sheet: image

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.