Scraping Geographic Features from ArcGIS Server

Still many geographic data is delivered through ESRI's ArcGIS Server. It is not easy to utilize the geographic data in the GIS servers from data analysis platform like R or Pandas. This package enables users to scrape vector data in ArcGIS Server from R through the server's REST API. It download geographic features from ArcGIS Server and saves it as Simple Features.

This program sends a request to an ArcGIS Server and gets json responses containing coordinates of geometries of which format is not the same as geojson. So it converts the json into simple feature geometries from the response. Then it combines attribute data to the geometries to create sf dataframe. Often ArcGIS servers limits the maximum number of rows in the result set. So this program creates 500 features per request and automatically re-send requests until it gets all features in the dataset.

Use devtools to install this package. This package has dependency on dplyr, sf, httr, jsonlite

library(devtools)
install_github("yonghah/esri2sf")

What you need is the URL of REST service you want. You can get the URL by viewing the URL widget on the service's webpage (see image below), by asking a GIS admin, or looking at the javascript code of a webpage where it creates a feature layer.

library("esri2sf")
url <- "https://services.arcgis.com/V6ZHFr6zdgNZuVG0/arcgis/rest/services/Landscape_Trees/FeatureServer/0"
df <- esri2sf(url)
plot(df)

You can filter output fields. This may take a minute since it gets 18000 polylines.

url <- "https://services.arcgis.com/V6ZHFr6zdgNZuVG0/arcgis/rest/services/Florida_Annual_Average_Daily_Traffic/FeatureServer/0"
df <- esri2sf(url, outFields=c("AADT", "DFLG"))
plot(df)

You can filter rows as well by giving where condition.

url <- "https://sampleserver1.arcgisonline.com/ArcGIS/rest/services/Demographics/ESRI_Census_USA/MapServer/3"
df <- esri2sf(url, 
              where="STATE_NAME = 'Michigan'", 
              outFields=c("POP2000", "pop2007", "POP00_SQMI", "POP07_SQMI"))
plot(df)