Code Monkey home page Code Monkey logo

nightlight's Introduction

Earth Observation Atlas

License: CC BY 4.0

Global Scale Nightlight Time Series Dataset

drawing

Use this code to reproduce graph.

Summary

  • Analysis-ready dataset of monthly nighttime lights derived from the VIIRS instrument aboard the SNPP satellite.

  • Nightlight is calculated at the administrative unit level (ADM 0~2) for 208 countries and territories, totaling 52,702 administrative units worldwide.

  • Administrative units are each represented by a time series (sequence of monthly observations) for the period between January 2012 and May 2024.

  • The dataset is updated regularly and hosted publicly on AWS.

Download

Click here to access the dataset.

Schema

Individual .csv files, one per administrative unit, adhere to the following schema:

Column Data type Description
year int Year of observation.
month int Month of observation.
shapeName string Name of administrative unit1.
shapeType string Type of administrative unit1.
shapeGroup string Group to which administrative unit belongs1.
min float Minimum nightlight.
max float Maximum nightlight.
mean float Mean nightlight.
q1 float First quartile.
q2 float Second quartile (Median).
q3 float Third quartile.
std float Standard deviation.
sum float Nightlight sum.
unit string Unit of nightlight.
count int Number of pixels within administrative unit boundary.
cloudFree float Average number of cloud-free observations per pixel.

1See geoBoundaries for details.

Pre-processing

(A) Raster data

  1. Monthly composites (DNB_npp_*-slcorr_v10_*.avg_rade9.tif) and cloud-free masks (DNB_npp_*-slcorr_v10_*.n_cf.tif) are sourced from Light Every Night (LEN) on AWS.
  2. nodata pixels (-1.5 and -999.3) are set to numpy.nan.

(B) Vector data

  1. Global boundaries with three levels of administrative divisions (ADM0~2) are sourced from geoBoundaries.
  2. Level 2 boundaries are mapped to their parent level 1 polygons via intersection.
  3. This vector layer is used below to extract zonal statistics from the raster layer.

(C) Time series generation

  1. Zonal stats (min, max, etc.) at the administrative unit level are extracted from the raster layer above using rasterstats library with the all_touched flag set to False.
  2. cloudFree column is calculated from the cloud-free mask using the same approach described in step 1.
  3. Extracted time series data are written to individual .csv files and uploaded to S3 on AWS.

Known Issues

  1. Orphan ADM2 boundaries listed under Null due to polygon intersection failure. See here for an example.

  2. Missing data: data missing from the source (03/2014, 08/2021, 06/2022 and 08/2022 are replaced with numpy.nan in the final dataset.

  3. Missing masks: cloud-free masks missing from the source (10/2017 and 11/2017) are replaced with numpy.nan accordingly.

Cite

This dataset is made available under the CC BY 4.0 license, requiring acknowledgment in any products that use it.

Written works:

Najjar, A. (2024). Global scale nightlight time series dataset. https://nightlight.eoatlas.org.

Online use:

Make sure to mention Earth Observation Atlas as the source and link to https://eoatlas.org.

nightlight's People

Contributors

eoameen avatar eoatlas avatar

Stargazers

 avatar  avatar Lukasz avatar

Watchers

 avatar

Forkers

eoameen

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.