project-open-data / esri2open Goto Github PK

this repo is an ESRI toolbox and tool(s) that export ESRI Feature Classes to open data formats, CSV, JSON, and GeoJSON

License: MIT License

Python 100.00%

esri2open's Introduction

esri2open

This repo is an ESRI toolbox and tool(s) that exports ESRI Feature Classes to open data formats, CSV, JSON, SQLite, and GeoJSON.

What Problem This Solves

Much of the data in government coffers is contained in spatial databases. A large percentage of government spatial data is created and managed using ESRI software. While the common interchange format, the ESRI Shapefile, is easily exported and imported by many other softwares, this data file format (the Shapefile) is not intrinsically part of the www ecology. Moreover, ESRI software does not provide an export of its generic 'feature class' (shapefile, file geodatabase, and personal geodatabase) to the most common open data file formats, CSV, JSON, and/or GeoJSON. Finally while open source tools easily transform ESRI shapefiles to open data, most government geospatial infrastructures only have ESRI tools. Lacking this basic export feature presented here, means the lion's share of government spatial data users cannot export their data to the most common open data formats.

How This Solves It

This repo contains two components that work inside ESRI ArcGIS. First is a python script that works at the lowest ESRI software license level to export ESRI "Feature Classes" to the the most common interchange formats; CSV, JSON and GeoJSON. Second, this repo has an ESRI toolbox (or .tbx file) that allows any ESRI user to easily connect this python script to native ESRI software. The toolbox points at the script. Users of this software need both files (the .tbx and the .py) to operate these functions. Once these files are downloaded, just add the .tbx file to the normal ESRI toolbox and run the .py script by double clicking on the script icon in the toolbox.

Requirements

Runs inside the ESRI ArcGIS desktop suite.

Usage

Copy the .tbx file and the .py file to any local directory
With ArcGIS desktop software running (e.g. ArcCatalog), add the .tbx file to your tool box by right clicking and choosing 'Add Toolbox'.
Double click on the which script you want to run which are:

ESRI To Open

Output one feature to an open format, arguments are:

Feature Class: the name of the Feature Class you want to export
Output Dataset: the output feature class, choose the format you want to output to here, choices are GeoJSON (default), TopoJSON, CSV, JSON, and SQLite.
Geometry Type: choices are Default, GeoJSON, WKT, and None, defaults to GeoJSON, ignored if the file is output to GeoJSON, TopoJSON, or SQLite.

ESRI To Open (multiple)

Output multiple features to an open format, arguments are:

Features: the names of the Feature Classes you want to export
Output Folder: the output folder.
Type: output data type, choices are want choices are GeoJSON (default), TopoJSON, CSV, JSON, SQLite
Geometry Type: choices are Default, GeoJSON, WKT, and None, defaults to GeoJSON, ignored if the file is output to GeoJSON, TopoJSON, or SQLite.

ESRI To Open (merge)

Merge several feature classes into one GeoJSON or TopoJSON file, useful for mixed geometry types:

Feature Classes: the names of the Feature Classes you want to export
Out File: the name of the output file extension determines whether the outfile is GeoJSON or TopoJSON.

License

MIT

Issues

Need to work on error trapping a bit more
This does not handle blob fields, or raster fields
Need to document python version; not sure how compatible it is with all current versions
Developed in ArcGIS 10.0

esri2open's People

Contributors

Stargazers

Watchers

Forkers

brightrain sgillies geobabbler spjika mzcoyle calvinmetcalf wlia chadcooper shakythesherpa nuttfive scw web5design rclark mosmos stephenott baltimorecounty fb98 opendata ldafner seansweeney ananse23 skincaid1 wangpeipei01 moutons xin-zheng gdmf ldanforth arieclaassens mhweber reyemtm lvdou2518 rkbiswal diversoft parselife geoholix govtmirror joshuaeveleth log-rational natalie-harper enterstudio dataportalindia rezagis fullstackenviormentss stevenconnorg nltgit tonasantiago michaeltroyer gaybro8777 fagan2888 thegitbuilder thiswildidea drroad jitshank elijahahianyo hassanmojeed

esri2open's Issues

It would be great to be able to specify the id field for GeoJSON

Currently, the OID field of a feature class is used when converting the feature class to a GeoJSON. However, most often this unique ID doesn't have any meaning and it would be great to choose the field that will become the id for features in GeoJSON. When doing plotting and map bindings with such Python libraries as folium and vincent, having a meaningful id is required.

I solve this for my needs by editing the line
https://github.com/project-open-data/esri2open/blob/master/Install/esri2open/parseRow.py#L16
with the field name I want to have as id in the output GeoJSON.

replaced
self.oid=getOID(self.fields)

with
self.oid='FIELD_NAME'

Tooblbox doesn't capture error messages

[Arcmap 10.1] When using the scripts from the toolbox messages from the tools (well I'm assuming it's all of them) are not passed back up to Arcmap. For example here is the message from the "ESRI to Open" tool, which runs "successfully" but creates no output:

Executing: esri2open T:\ENV.301\Bear_Incidents.gdb\Summer_2013 T:\ENV.301\dev-2013\geojson\bear_incidents_2013-Jun-14.geojson Default
Start Time: Thu Jun 27 10:22:30 2013
Running script esri2open...
Completed script esri2open...
Succeeded at Thu Jun 27 10:22:31 2013 (Elapsed Time: 1.00 seconds)

However running the script directly from a python enabled command prompt shows it failed to understand the input feature class:

D:\GitHub\esri2open>single.py t:\ENV.301\Bear_Incidents.gdb\Summer_2013 t:\env.301\dev-2013\geojson\test.geoson

I don't understand the format

10.3 Cannot add layer directly from a map

It worked in 10.2, but in ArcMap 10.3, I cannot add layers directly from the map. This means I have to export all my layers first - as I often use definition queries and turn fields off - then convert the exported files. Any ideas? It seems like it would be something simple.

One or more of the dropped items were invalid and will not be added to the control.

Options

Hi.
Is there any way to add the topojson option "-p" (-- properties) from the toolbox?
Thanks

Converts DateTime to Date only

An ESRI Feature class defines a "Date" Field as a DateTime field. When converting a datetime to geojson, this tool only outputs the "Date" portion of the field.

license

It might be worth discussing a more-permissive license the main issue is:

Do we want people writing proprietary software to be able to take this code, use it to implement export to GeoJSON and sell it with proprietary license?

I'd say yes, this would be a good thing, we'd want it as widely used as possible meaning a MIT license or a BSD Three clause might be a better fit.

no module named esri2open

do I have to install anything? All I did was download and unzip the package, but I saw a folder called Install and then I got this error.

sorry if i'm missing something really obvious; new to github

IDLE has also been saying that there's no module named arcpy?

Mongo

Anyone interested in writing an output which would be a mongoDB dump or similar? this might be a useful (and slightly challenging) task. right now mongoDB doesn't support multi features (on simple point, line and polygons) so the code would need to write out (perhaps) duplicate rows for multi features with new ids. also, for inserting into mongo, a single geojson file won't actually work, but a geojson file per feature kind of thing (not exactly sure, but something like that).

anyone interested in taking a crack at it?

consider adding TopoJSON

please consider adding https://github.com/mbostock/topojson as an output format

Support the simplestyle spec

GitHub and Mapbox make it easier to symbolize "pin drop" maps using GeoJSON with the simplestyle spec of additional feature properties.

Would this be a feature to include in this project? If so, I'd be happy to start working on it. I'm not sure what is the best way to implement it in a ArcGIS tool but would give it a shot.

Thanks!

This is half an issue and half me ~~trying to figure out if~~ paranoid I missed some downside of using the json library.