Last year we expanded our OSM Extracts service to produce what we call GeoJSONL. This is a data format that goes by many names: newline-delimited GeoJSON, line-oriented GeoJSON, GeoJSONSeq, or GeoJSON Text Sequences. Whatever its name, the format enables effficient reading and efficient writing of vector geo-data features.
For more background, see our previous blog post on GeoJSONL and see this new overview of the format’s many names and its defining properties.
Over the last year, there’s been even more progress on tools for writing and reading GeoJSONL (or whatever your preferred name is for this format).
HERE XYZ now supports GeoJSONL ingest.
Don’t know what HERE XYZ is? Here’s how we summarized the platform in a blog post last year:
Using the latest version of the HERE CLI, you can parse and upload GeoJSONL files to the HERE XYZ platform.
Here’s an update of our “Use Crowdsourced Data from Interline OSM Extracts in an XYZ Web Map” tutorial that uses the GeoJSONL format.
Download a GeoJSONL extract from OSM Extracts:
Upload to XYZ and set tags:
Previously you might hit memory limits on your computer if you tried to upload an extract for a big metro region. Now, thanks to GeoJSONL’s efficient parsing, you should be able to upload any extract of any size!
There’s a long history of tools that work with this format, and it’s been growing over the last year. Here are some more highlights:
Command line On the command line, read
.geojsons files using GDAL’s
Desktop GIS GDAL also powers QGIS, so now you can load
.geojsons files into our favorite desktop GIS sofware.
Parse in Python: Fiona is a Python library that wraps GDAL and provides a “neat and nimble” interface. The next release will include support for
Statistics and data science in R If you’re loading vector geo-data into R, try the geojson package on CRAN. See these instructions for how to load a
.geojsonl file directly from Interline OSM Extracts.
Convert from a shapefile Install the shapefile NPM package and run
shp2json --newline-delimited to convert a shapefile to GeoJSONL. See these instructions.
Convert to vector tiles Tippecanoe can take GeoJSONL as input and turn it in to MVT vector tiles. It also provides its own
tippecanoe-json-tool command to convert a standard GeoJSON file to GeoJSONL format (see these instructions).
If you like GeoJSON but find your data files are too big to hold in memory, give one of these tools a try.
If you need to run a bunch of vector geo-data through a MapReduce operation that, give this format a try. (It’s easy to divide a GeoJSONL file in to chunks before parsing its exact contents.)
If you want to download OSM Extracts from Interline and explore even the largest of extracts using HERE XYZ, give this a try.
* Shoutout to Sean Gillies, the lead creator of Fiona and instigator of RFC 8142 - GeoJSON Text Sequences. We may have differing views on the record separator character. Regardless, his packages like Fiona and Rasterio are fabulous and we’re happy users of them.
† Shoutout to Brett Camper, the author of this Observable notebook and a fellow alum of Mapzen.
Originally posted to the Interline blog on March 8, 2019 by: