Code Monkey home page Code Monkey logo

google-takeout-location-parser's Introduction

Location History Parser

There are three scripts in this repo, semantic_location_parser.py, records_location_parser.py, and full_location_history_parser.py.

Each file scans through the Takeouts directory and parses the Google location data.

The resulting csv files can be used in ArcGIS for spatial analysis and visualization. If you want to see your data visually fast, then convert the csv to a macro enabled .xlms file and use Excel's mapping tool.

Fine Location data using records_location_parser.py

Requirements

  • Python 3.x

Usage

  1. Download and extract your Location History data from Google Takeout. Google will send you a link to download a .zip file. Unzip the file into a new directory.
  2. Save the python scripts in the same folder that you extracted the Takeouts folder.
  3. Open a terminal/command prompt and navigate to the location of the script and the data.
  4. Run the script using the command python semantic_location_parser.py.
  5. The resulting csv file will be saved in the same folder as the script and data with the name semantic_location_history.csv.
  6. The steps are the same for each script.

File Location

Data Format

Google Takeout Location History data consists of two types of data: raw location history data and semantic location history data.

E:.
|   location_history_parser.py
|
\---Takeout
    |   archive_browser.html
    |
    \---Location History
        |   Records.json
        |   Settings.json
        |   Tombstones.csv
        |
        \---Semantic Location History
            +---2022
            |       2022_DECEMBER.json
            |       2022_NOVEMBER.json
            |
            \---2023
                    2023_FEBRUARY.json
                    2023_JANUARY.json

The semantic_location_parser.py script is used to parse Google Takeout Location History data into a csv file. The script can extract the timestamp, address, placeId, name, latitudeE7 and longitudeE7 values from the semantic location history data and store them in a csv file.

The semantic location history data is found in the Semantic Location History folder and consists of more high-level and processed information compared to the raw location history data found in the records.json file. This data is partitioned by year in different subfolders and by month in different JSON files. Inside each semantic JSON file we can find a single flat timelineObjects array.

The records_location_parser.py is used to pull the less refined location data that is collected. This data can be used to highlight what roads you traveled along rather than just what location you visited.

The full_location_history_parser.pyparses the same data as the semantic_location_parser.py, but it also parses the records.json file in the same script. The two files are appended together on the timestamp, lat, and lon columns. It's not the best way to store this data and not adivsed to be used.

This data set will have the following columns:

  • epoch_time: Useful for APIs
  • timestamp: sample start time from device
  • date_str: timestamp to date
  • lat: latitudinal coordinate. The values need to be divided by 107 to be in the expected range.
  • lon: logitudinal coordinate. The values need to be divided by 107 to be in the expected range.
  • address: Best guess address by Google
  • placeid: Google place id
  • name: Name of location if it exists
  • alt: altitude
  • activity_type: Most confident activity type
  • platformtype: Platform used to access Google services
  • file_path: Which folder the data came from
  • sys_time: Comes from the timestamp key in the records.json, unique.

Limitations

The script only extracts specific values from the raw location history data and the semantic location history data and stores them in their respective csv files. If you need to extract additional information, you may need to modify the script accordingly.

google-takeout-location-parser's People

Contributors

dovarfalcone avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar

google-takeout-location-parser's Issues

KeyError: 'address'

Hi,
thank you for your script. It is exactly what I needed :-)
I got exception parsing some YYYY_MONTH.json files:
` File "/home/ts/RTS/googledata/semantic_location_parser.py", line 18, in process_file
address = obj['placeVisit']['location']['address']
KeyError: 'address'

So I added thisto semantic_location_parser.py in line 15: try:
address = obj['placeVisit']['location']['address']
except KeyError:
address = obj['placeVisit']['location']['name']`

regards
Thomas

Getting an issue while using the 'semantic_location_parser.py'

File "C:\Users\rakes\OneDrive\Documents\Repos\google-takeout-location-parser\semantic_location_parser.py", line 15, in process_file
lat = obj['placeVisit']['location']['latitudeE7'] / 107
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^
KeyError: 'latitudeE7'
PS C:\Users\rakes\OneDrive\Documents\Repos\google-takeout-location-parser> python semantic_location_parser.py
Traceback (most recent call last):
File "C:\Users\rakes\OneDrive\Documents\Repos\google-takeout-location-parser\semantic_location_parser.py", line 47, in
main()
File "C:\Users\rakes\OneDrive\Documents\Repos\google-takeout-location-parser\semantic_location_parser.py", line 44, in main
process_file(file_path, data_writer)
File "C:\Users\rakes\OneDrive\Documents\Repos\google-takeout-location-parser\semantic_location_parser.py", line 15, in process_file
lat = obj['placeVisit']['location']['latitudeE7'] / 10
7
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^
KeyError: 'latitudeE7'

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.