M GIS A Short Introduction to Volunteered Geographic Information Presentation of the OpenStreetMap Project Sylvain Bouveret – LIG-STeamer / Universit´ e Grenoble-Alpes ´ Quatri` eme Ecole Th´ ematique du GDR Magis. S` ete, September 29 – October 3, 2014 Sources I Part of the presentation dedicated to OSM inspired from: I I I An old joint presentation with N. Petersen and Ph. Genoud Nicolas Moyroud: Several talks from 3rd MAGIS summer school 2012 Released under licence CC-BY-SA and downloadable here: http://libreavous.teledetection.fr. Guillaume All` egre: Cartographie libre du monde: OpenStreetMap Released under licence CC-BY-SA. I Reference book about VGI [Sui et al., 2013] I Other references cited throughout the presentation Sui, D. Z., Elwood, S., and Goodchild, M., editors (2013). Crowdsourcing geographic knowledge: Volunteered Geographic Information (VGI) in Theory and Practice. Springer. M GIS 2 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Outline 1. Introduction to Volunteered Geographic Information 2. Presentation of the OpenStreetMap Project 3. Using OpenStreetMap Data 4. Using Volunteered Geographic Information M GIS 3 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Outline 1. Introduction Introduction to to Volunteered VolunteeredGeographic GeographicInformation Information 2. Presentation of the OpenStreetMap Project 3. Using OpenStreetMap Data 4. Using Volunteered Geographic Information M GIS 3 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Outline 1. Introduction to Volunteered Geographic Information 2. Presentation Presentation of of the theOpenStreetMap OpenStreetMapProject Project 3. Using OpenStreetMap Data 4. Using Volunteered Geographic Information M GIS 3 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Outline 1. Introduction to Volunteered Geographic Information 2. Presentation of the OpenStreetMap Project 3. Using Using OpenStreetMap OpenStreetMapData Data 4. Using Volunteered Geographic Information M GIS 3 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Outline 1. Introduction to Volunteered Geographic Information 2. Presentation of the OpenStreetMap Project 3. Using OpenStreetMap Data 4. Using Using Volunteered VolunteeredGeographic GeographicInformation Information M GIS 3 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete 1 First part Introduction to Volunteered Geographic Information Beyond traditional GIS A new trend Some examples Features of participative datasets Volunteered vs Contributed Open vs Closed Sensing vs Thinking Volunteered Geographic Information Introduction to Volunteered Geographic Information Beyond traditional GIS A new trend Some examples Features of participative datasets Volunteered vs Contributed Open vs Closed Sensing vs Thinking Volunteered Geographic Information M GIS 5 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete A very brief history of GIS First users of GIS (at least in France): local authorities, department of defense M GIS I Early 90’s: paper maps (unprecise, to be regularly updated...) I 1990 → 2010: Digital transposition of data I 2000: Integration to enterprise IS (first spatial extensions to Oracle and Postgres) I 2002: Geospatial webservers + OGC standards I 2005: Mobility 6 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete A very brief history of GIS First users of GIS (at least in France): local authorities, department of defense M GIS I Early 90’s: paper maps (unprecise, to be regularly updated...) I 1990 → 2010: Digital transposition of data I 2000: Integration to enterprise IS (first spatial extensions to Oracle and Postgres) I 2002: Geospatial webservers + OGC standards I 2005: Mobility I 2004: Participative data, geographic crowdsourcing, volunteered geographic information, neogeographic datasets... 6 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete OpenStreetMap http://www.openstreetmap.org/ I Project started on 2004 I Open and collaborative geographical database of the world I Content generated by users (about 1.8M registered users) I Free license (initially CC-by-sa; ODbL since 2012) More about OpenStreetMap later M GIS 7 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Wikimapia http://wikimapia.org/ M GIS I Project started on 2006 I Aims at “marking all geographical objects in the world and providing a useful description of them” I Mostly provides a way for users to give annotations about places in the world, (initially) using Google Maps as a base layer. I Free license since 2010 (CC-by-SA). 8 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Google Map Maker http://www.google.com/mapmaker/ M GIS I Project started on 2008 I Equips Google Maps with a map edition interface I Every registered user can submit modifications I Modifications have to be approved before being published in Google Maps I Data released under proprietary license 9 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Ushahidi http://www.ushahidi.com/ M GIS I Free Software and platform for crisis management I Crowdsourcing-based mapping I Focuses on information flow (smartphones, SMS,...) I Web platform Based on OpenStreetMap and Google Maps for Geocoding (source: Wikipedia). 10 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Ligue de Protection des Oiseaux http://www.ornitho.fr/ M GIS I A network of “participatory science websites” dedicated to wildlife inventory I Anyone can participate by adding observations to the database I Requires some basic knowledge about different species I In general, no verification is made, except for outliers I Search engine and visualization tool (map) on the website 11 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Other examples M GIS I Geolabeled Flickr Images ( I Foursquare ( I UCrime ( I ... http://www.flickr.com/ http://foursquare.com/ http://ucrime.com/ ) ) ) 12 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Introduction to Volunteered Geographic Information Beyond traditional GIS A new trend Some examples Features of participative datasets Volunteered vs Contributed Open vs Closed Sensing vs Thinking Volunteered Geographic Information M GIS 13 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Features All these applications are examples of geographical crowdsourcing approaches M GIS 14 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Features All these applications are examples of geographical crowdsourcing approaches Some strong common points... I Data contributed on a voluntary basis by users I Geospatial nature of data (or at list a part of it) ...But very different features as well: M GIS I Aims I Geospatial as a first-class citizen or not I Skills required I Process for quality assessment (data verification) I Data license 14 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete To volunteer or to contribute? In all these examples, data is jointly produced by users volunteering to contribute (geographical crowdsourcing) M GIS 15 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete To volunteer or to contribute? In all these examples, data is jointly produced by users volunteering to contribute (geographical crowdsourcing) But... What about geolocalized data transmitted by a smartphone, (more or less) unbeknownst to its user? M GIS 15 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete To volunteer or to contribute? In all these examples, data is jointly produced by users volunteering to contribute (geographical crowdsourcing) But... What about geolocalized data transmitted by a smartphone, (more or less) unbeknownst to its user? ; An example of crowdsourced geospatial data, assuredly not volunteered! M GIS 15 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Opt-in vs opt-out Two approaches to crowdsourced geographic data [Harvey, 2013]: I Volunteered Geographic Information (opt-in): I I I I Clarity about purposes Control over data collection Some guarantees about data reuse Contributed Geographic Information (opt-out): I I I Unclear purposes No (or little) control over data collection No control over data reuse Harvey, F. (2013). To volunteer or to contribute locational information? Towards truth in labelling for crowdsourced geographic information. In Crowdsourcing Geographic Knowledge [...], chapter 3. Springer. M GIS 16 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Data reuse – licenses A key distinction between opt-in and opt-out: control over data reuse ...raises the following crucial questions: M GIS I Who owns the data jointly produced by users? I Under which license this data can be used? 17 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Data reuse – licenses A key distinction between opt-in and opt-out: control over data reuse ...raises the following crucial questions: I Who owns the data jointly produced by users? I Under which license this data can be used? Two antagonistic examples: M GIS I Google Map Maker: Google owns the data, releases it under proprietary license, whose conditions can change whenever it wants (cf April 2011) irrespective of whether the user is a regular contributor or not I OpenStreetMap: Data belongs to the contributors, and is released under a free license 17 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Geomatics and open (free) licenses I In geomatics, most datasets are released under proprietary licenses, even those who are funded by public money – IGN, Ordnance Survey,... I Even if it is free (cost-less), cannot be freely exploited (e.g for scientific projects). I M GIS ; Some VGI projects like OSM explicitly aim at providing free and open geospatial data. 18 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Geomatics and open (free) licenses I In geomatics, most datasets are released under proprietary licenses, even those who are funded by public money – IGN, Ordnance Survey,... I Even if it is free (cost-less), cannot be freely exploited (e.g for scientific projects). I ; Some VGI projects like OSM explicitly aim at providing free and open geospatial data. But... How to do it while still being compatible with author’s right → freely release data while protecting it and its authors? M GIS 18 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Geomatics and open (free) licenses I In geomatics, most datasets are released under proprietary licenses, even those who are funded by public money – IGN, Ordnance Survey,... I Even if it is free (cost-less), cannot be freely exploited (e.g for scientific projects). I ; Some VGI projects like OSM explicitly aim at providing free and open geospatial data. But... How to do it while still being compatible with author’s right → freely release data while protecting it and its authors? M GIS I The software community has a solution: use Free licenses I Initially dedicated to software (like GNU/GPL) → ill-suited for other kinds of intellectual stuff (music, books, pictures, information. . . ) 18 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Data and open licenses: Creative Commons First option for geomatics: Creative Commons licenses (artworks) http://creativecommons.org/ M GIS 19 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Data and open licenses: Creative Commons First option for geomatics: Creative Commons licenses (artworks) http://creativecommons.org/ 4 options Attribution (BY) the original author has to be credited Non Commercial (NC) no commercial profit allowed No Derivatives (ND) no derived work allowed Share Alike (SA) derivatives must be licensed under identical terms Six possible combinations CC-by, CC-by-sa, CC-by-nc, CC-by-nc-nd, CC-by-nc-sa, CC-by-nd M GIS 19 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Data and open licenses: ODbL I Initial OSM data was released under the terms of the CC-by-sa license. I However: it turned out that it was not very adapted... I I I I Combining OSM data with other datasets Share-Alike only applies to rendered maps (tiles), not to data itself Attribution ; too many contributors! Uncertainty about derived work I After two years of effort, OSM switched to ODbL in September 2012: Attribution, Share Alike, Redistribution (as long as one of the redistributed versions is kept open). I All the past contributors have been contacted... I I I most of them agreed with the new terms some of them explicitly disagreed (→ data erased) some of them did not answer (→ data erased) http://opendatacommons.org/licenses/odbl/ M GIS 20 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Sensing vs Thinking I Different applications ; different skills and levels of implication required from the users Haklay, M. (2013). Citizen science and volunteered geographic information: Overview and typology of participation. In Crowdsourcing Geographic Knowledge [...], chapter 7. Springer. M GIS 21 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Sensing vs Thinking I I Different applications ; different skills and levels of implication required from the users VGI as geographic citizen science (even if not every VGI application falls into this category) Haklay, M. (2013). Citizen science and volunteered geographic information: Overview and typology of participation. In Crowdsourcing Geographic Knowledge [...], chapter 7. Springer. M GIS 21 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Sensing vs Thinking I Different applications ; different skills and levels of implication required from the users I VGI as geographic citizen science (even if not every VGI application falls into this category) I The ladder of implication according to Haklay [2013] I I I I Level 1: Crowdsourcing (citizens as sensors, volunteered computing) Level 2: Distributed intelligence (basic interpreters, volunteered thinking) Level 3: Participatory science (implication in problem definition and data collection) Level 4: Extreme citizen science (problem definition, data collection and analysis) Haklay, M. (2013). Citizen science and volunteered geographic information: Overview and typology of participation. In Crowdsourcing Geographic Knowledge [...], chapter 7. Springer. M GIS 21 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Introduction to Volunteered Geographic Information Beyond traditional GIS A new trend Some examples Features of participative datasets Volunteered vs Contributed Open vs Closed Sensing vs Thinking Volunteered Geographic Information M GIS 22 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete What is VGI? As we have seen, crowdsourcing is just one feature of VGI M GIS 23 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete What is VGI? As we have seen, crowdsourcing is just one feature of VGI Volunteered Geographic Information [Goodchild, 2007] Volunteered geographic information is the harnessing of tools to create, assemble, and disseminate geographic data provided voluntarily by individuals. Goodchild, M. F. (2007). Citizens as sensors: the world of volunteered geography. GeoJournal, 69(4):211–221. M GIS 23 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Neogeographic datasets Coote and Rackham [2008] propose the following (complementary) characterization of neogeographic datasets: I Creation stimulated by lack of available data or restrictions, costs, limitations of conventional data sources I Involve geographic information provided voluntarily by individuals I Creation and management are not necessarily ruled by accepted standards I Data licensed using open-source approach Coote, A. and Rackham, L. (2008). Neogeographic data quality — is it an issue? In AGI Geocommunity conference, ConsultingWhere Ltd. M GIS 24 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete The big picture? M GIS 25 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete The big picture? Participative M GIS 25 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete The big picture? Participative Geospatial M GIS 25 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete The big picture? Participative Citizen science Geospatial M GIS 25 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete The big picture? Participative Open data Citizen science Geospatial M GIS 25 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete The big picture? Participative Citizen science VGI? Open data Geospatial M GIS 25 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete 2 Second part Presentation of the OpenStreetMap project OSM: History and principles What is OSM? History Technical considerations Data model The OSM ontology Contributing Presentation of the OpenStreetMap project OSM: History and principles What is OSM? History Technical considerations Data model The OSM ontology Contributing M GIS 27 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete OpenStreetMap: Genesis OpenStreetMap was created in July 2004 by Steve Coast, which was then studying at the University College of London. I He did not understand why the Ordnance Survey created massive geographical datasets but did not freely distribute them to those who had paid to create them (i.e happy tax payers). I NB: the same thing happens in almost every country in the world (except USA and the Netherlands) He then decided to start a mapping project whose aim would be to freely provide data to the users. M GIS 28 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete OpenStreetMap: principles I Collaborative: I I I Open: data freely usable without restriction (ODbL). I Data: OpenStreetMap is not a map, it is a database. I I M GIS mainly individual contributions the more contributors, the more complete the world coverage Online “map” only provided for visualization purposes No airborne or satellite view 29 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete One dataset, several maps Mapnik standard style M GIS 30 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete One dataset, several maps Transport map M GIS 30 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete One dataset, several maps Cycle map M GIS 30 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete One dataset, several maps MapQuest M GIS 30 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete One dataset, several maps FranceTopo.fr (enriched with other public datasets such as Nasa SRTM) M GIS 30 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete One dataset, several maps 3D-OSM (XNavigator – University of Bonn and Heidelberg) M GIS 30 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete History 1. Founding and Early History I 9th August 2004 - openstreetmap.org registered by Steve Coast I 20th August 2004 - Steve Coast presented his mapping idea at EuroFOO I 2nd September 2004 - First posting to the mailing list I 17th July 2005 - Map Limehouse the first Mapping Party I 22th January 2006 - Release of version 1.0 of the offline editor JOSM I 20th August 2006 - OpenStreetMap Foundation registered I 10th November 2006 - Mapnik rendered Slippy map makes its debut. I 4th December 2006 - Yahoo! aerial imagery sketching allowed http://wiki.openstreetmap.org/wiki/History M GIS 31 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete History 2. The Start of OSM’s Current Technology Stack I 5th May 2007 - 1st version of the Potlatch editor. I 14th-15th July 2007 - First conference, ”State Of The Map 2007”, held in Manchester. I September 2007 - TIGER data import for the US started I 20th September 2007 - AND Data for The Netherlands imported I - The French Direction g´en´erale des finances publiques January 2009 officially allows the OSM contributors to use the Cadastre as a source of data. http://wiki.openstreetmap.org/wiki/History M GIS 32 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete History 3. The Switch to API 0.6 and the explosion of User Growth I 21st April 2009 - Big switch to API version 0.6 I 1st April 2010 - Ordnance Survey Opendata releases. OSM partly responsible for bringing this about. I 30th November 2010 - Use of Bing vertical aerial imagery allowed I 25th November 2011 I 12th Sept 2012 - License switched over to ODbL - Association OpenStreetMap France registered http://wiki.openstreetmap.org/wiki/History M GIS 33 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Number of contributors 2e+06 Today 1.8e+06 Number of contributors 1.6e+06 1.4e+06 1.2e+06 1e+06 800000 600000 400000 200000 0 2000/01 2002/01 2004/01 2006/01 2008/01 2010/01 2012/01 2014/01 2016/01 Date http://wiki.openstreetmap.org/wiki/Statistics M GIS 34 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Paris Paris in OpenStreetMap september 2006 −→ october 2010. http://wiki.openstreetmap.org/wiki/Historical_Coverage M GIS 35 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Paris Paris in OpenStreetMap september 2006 −→ october 2010. http://wiki.openstreetmap.org/wiki/Historical_Coverage M GIS 35 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Paris Paris in OpenStreetMap september 2006 −→ october 2010. http://wiki.openstreetmap.org/wiki/Historical_Coverage M GIS 35 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Paris Paris in OpenStreetMap september 2006 −→ october 2010. http://wiki.openstreetmap.org/wiki/Historical_Coverage M GIS 35 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Paris Paris in OpenStreetMap september 2006 −→ october 2010. http://wiki.openstreetmap.org/wiki/Historical_Coverage M GIS 35 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Paris Paris in OpenStreetMap september 2006 −→ october 2010. http://wiki.openstreetmap.org/wiki/Historical_Coverage M GIS 35 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Paris Paris in OpenStreetMap september 2006 −→ october 2010. http://wiki.openstreetmap.org/wiki/Historical_Coverage M GIS 35 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Paris Paris in OpenStreetMap september 2006 −→ october 2010. http://wiki.openstreetmap.org/wiki/Historical_Coverage M GIS 35 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Paris Paris in OpenStreetMap september 2006 −→ october 2010. http://wiki.openstreetmap.org/wiki/Historical_Coverage M GIS 35 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Paris Paris in OpenStreetMap september 2006 −→ october 2010. http://wiki.openstreetmap.org/wiki/Historical_Coverage M GIS 35 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Paris Paris in OpenStreetMap september 2006 −→ october 2010. http://wiki.openstreetmap.org/wiki/Historical_Coverage M GIS 35 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Paris Paris in OpenStreetMap september 2006 −→ october 2010. http://wiki.openstreetmap.org/wiki/Historical_Coverage M GIS 35 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Paris Paris in OpenStreetMap september 2006 −→ october 2010. http://wiki.openstreetmap.org/wiki/Historical_Coverage M GIS 35 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Paris Paris in OpenStreetMap september 2006 −→ october 2010. http://wiki.openstreetmap.org/wiki/Historical_Coverage M GIS 35 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Paris Paris in OpenStreetMap september 2006 −→ october 2010. http://wiki.openstreetmap.org/wiki/Historical_Coverage M GIS 35 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Paris Paris in OpenStreetMap september 2006 −→ october 2010. http://wiki.openstreetmap.org/wiki/Historical_Coverage M GIS 35 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Paris Paris in OpenStreetMap september 2006 −→ october 2010. http://wiki.openstreetmap.org/wiki/Historical_Coverage M GIS 35 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Paris Paris in OpenStreetMap september 2006 −→ october 2010. http://wiki.openstreetmap.org/wiki/Historical_Coverage M GIS 35 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Paris Paris in OpenStreetMap september 2006 −→ october 2010. http://wiki.openstreetmap.org/wiki/Historical_Coverage M GIS 35 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Paris Paris in OpenStreetMap september 2006 −→ october 2010. http://wiki.openstreetmap.org/wiki/Historical_Coverage M GIS 35 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Paris Paris in OpenStreetMap september 2006 −→ october 2010. http://wiki.openstreetmap.org/wiki/Historical_Coverage M GIS 35 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Paris Paris in OpenStreetMap september 2006 −→ october 2010. http://wiki.openstreetmap.org/wiki/Historical_Coverage M GIS 35 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Paris Paris in OpenStreetMap september 2006 −→ october 2010. http://wiki.openstreetmap.org/wiki/Historical_Coverage M GIS 35 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Paris Paris in OpenStreetMap september 2006 −→ october 2010. http://wiki.openstreetmap.org/wiki/Historical_Coverage M GIS 35 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Paris Paris in OpenStreetMap september 2006 −→ october 2010. http://wiki.openstreetmap.org/wiki/Historical_Coverage M GIS 35 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Paris Paris in OpenStreetMap september 2006 −→ october 2010. http://wiki.openstreetmap.org/wiki/Historical_Coverage M GIS 35 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Paris Paris in OpenStreetMap september 2006 −→ october 2010. http://wiki.openstreetmap.org/wiki/Historical_Coverage M GIS 35 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Paris Paris in OpenStreetMap september 2006 −→ october 2010. http://wiki.openstreetmap.org/wiki/Historical_Coverage M GIS 35 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Paris Paris in OpenStreetMap september 2006 −→ october 2010. http://wiki.openstreetmap.org/wiki/Historical_Coverage M GIS 35 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Paris Paris in OpenStreetMap september 2006 −→ october 2010. http://wiki.openstreetmap.org/wiki/Historical_Coverage M GIS 35 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Paris Paris in OpenStreetMap september 2006 −→ october 2010. http://wiki.openstreetmap.org/wiki/Historical_Coverage M GIS 35 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Paris Paris in OpenStreetMap september 2006 −→ october 2010. http://wiki.openstreetmap.org/wiki/Historical_Coverage M GIS 35 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Paris Paris in OpenStreetMap september 2006 −→ october 2010. http://wiki.openstreetmap.org/wiki/Historical_Coverage M GIS 35 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Paris Paris in OpenStreetMap september 2006 −→ october 2010. http://wiki.openstreetmap.org/wiki/Historical_Coverage M GIS 35 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Paris Paris in OpenStreetMap september 2006 −→ october 2010. http://wiki.openstreetmap.org/wiki/Historical_Coverage M GIS 35 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Paris Paris in OpenStreetMap september 2006 −→ october 2010. http://wiki.openstreetmap.org/wiki/Historical_Coverage M GIS 35 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Paris Paris in OpenStreetMap september 2006 −→ october 2010. http://wiki.openstreetmap.org/wiki/Historical_Coverage M GIS 35 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Paris Paris in OpenStreetMap september 2006 −→ october 2010. http://wiki.openstreetmap.org/wiki/Historical_Coverage M GIS 35 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Paris Paris in OpenStreetMap september 2006 −→ october 2010. http://wiki.openstreetmap.org/wiki/Historical_Coverage M GIS 35 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Paris Paris in OpenStreetMap september 2006 −→ october 2010. http://wiki.openstreetmap.org/wiki/Historical_Coverage M GIS 35 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Paris Paris in OpenStreetMap september 2006 −→ october 2010. http://wiki.openstreetmap.org/wiki/Historical_Coverage M GIS 35 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Paris Paris in OpenStreetMap september 2006 −→ october 2010. http://wiki.openstreetmap.org/wiki/Historical_Coverage M GIS 35 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Paris Paris in OpenStreetMap september 2006 −→ october 2010. http://wiki.openstreetmap.org/wiki/Historical_Coverage M GIS 35 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Paris Paris in OpenStreetMap september 2006 −→ october 2010. http://wiki.openstreetmap.org/wiki/Historical_Coverage M GIS 35 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Paris Paris in OpenStreetMap september 2006 −→ october 2010. http://wiki.openstreetmap.org/wiki/Historical_Coverage M GIS 35 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Paris Paris in OpenStreetMap september 2006 −→ october 2010. http://wiki.openstreetmap.org/wiki/Historical_Coverage M GIS 35 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Paris Paris in OpenStreetMap september 2006 −→ october 2010. http://wiki.openstreetmap.org/wiki/Historical_Coverage M GIS 35 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Paris Paris in OpenStreetMap september 2006 −→ october 2010. http://wiki.openstreetmap.org/wiki/Historical_Coverage M GIS 35 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Paris Paris in OpenStreetMap september 2006 −→ october 2010. http://wiki.openstreetmap.org/wiki/Historical_Coverage M GIS 35 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Paris Paris in OpenStreetMap september 2006 −→ october 2010. http://wiki.openstreetmap.org/wiki/Historical_Coverage M GIS 35 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Current state Current state of the database (25 M GIS th September, 2014) Number of users 1,800,453 Number of uploaded GPS points 4,218,137,961 Number of nodes 2,535,804,643 Number of ways 253,523,371 Number of relations 2,818,286 36 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Presentation of the OpenStreetMap project OSM: History and principles What is OSM? History Technical considerations Data model The OSM ontology Contributing M GIS 37 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Basic OSM data model As in most GIS, each geographical entity is described in OSM using: M GIS I Geographical information (geometries) I Attributes (≈ semantics) 38 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Basic OSM data model As in most GIS, each geographical entity is described in OSM using: I Geographical information (geometries) I Attributes (≈ semantics) More precisely, each OSM entity has: I a numeric identifier: OSM ID I a geometry I a set of generic attributes present for every element I I I I I I M GIS uid, user: user id and user name timestamp: time of the last modification visible: if false then the element should only be returned by history calls version: edit version of the object (starts from 1) changeset: the changeset (group of edits made within a certain time by one user) in which the object was created or updated a set of tags (attributes): key-value pairs 38 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Geometries in OpenStreetMap Three kinds of geometries: 1. nodes : basic element. Geographic point: latitude & longitude (WGS84) → Point Of Interest (POIs) 2. ways : ordered interconnection of nodes open ways → linear features (roads, railways...) closed ways → areas 3. relations : group of any primitive with associated roles Relate nodes, ways and potentially other relations to each other, thereby forming complex objects (e.g. multipolygons). → relationship between objects and abstract objects M GIS 39 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Example of a way M GIS 40 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Example of a relation M GIS 41 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Tags in OpenStreetMap Attributes of geographical entities are described using tags M GIS 42 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Tags in OpenStreetMap Attributes of geographical entities are described using tags I A tag is a key-value pair I I A key broadly describes an element (e.g. highway, name) A value specifically describes its accompanying key http://wiki.openstreetmap.org/wiki/Tagging_samples/out_of_town M GIS 42 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Tags in OpenStreetMap Attributes of geographical entities are described using tags I A tag is a key-value pair I I A key broadly describes an element (e.g. highway, name) A value specifically describes its accompanying key http://wiki.openstreetmap.org/wiki/Tagging_samples/out_of_town I Use of keys and values is unrestricted (free text) I I M GIS the data model is infinitely extensible anyone can define and use its own keys and values 42 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Anarchy? Unrestricted tags M GIS 43 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Anarchy? Unrestricted tags lead to uncontrolled data production process. M GIS 43 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Anarchy? Unrestricted tags lead to uncontrolled data production process. Uncontrolled data production process leads to anarchy. M GIS 43 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Anarchy? Unrestricted tags lead to uncontrolled data production process. Uncontrolled data production process leads to anarchy. Anarchy leads to useless data. M GIS 43 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Anarchy? Unrestricted tags lead to uncontrolled data production process. Uncontrolled data production process leads to anarchy. Anarchy leads to useless data. Unrestricted tags are the path to the dark side. M GIS 43 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Anarchy? Unrestricted tags lead to uncontrolled data production process. Uncontrolled data production process leads to anarchy. Anarchy leads to useless data. Unrestricted tags are the path to the dark side. Actually, tagging in OSM is governed by an agile self-organizing community process defining the basic ontology of OSM M GIS 43 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Anarchy? Unrestricted tags lead to uncontrolled data production process. Uncontrolled data production process leads to anarchy. Anarchy leads to useless data. Unrestricted tags are the path to the dark side. Actually, tagging in OSM is governed by an agile self-organizing community process defining the basic ontology of OSM Resources to find an appropriate tag or explore tag usage: I I M GIS Map Features ( http://wiki.openstreetmap.org/wiki/Map_Features ) ; an extensive list of the most commonly used tags Taginfo ( http://taginfo.openstreetmap.org/ ) ; a useful site to exploring current tag usage, including tag values that are not necessarily documented 43 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Example of OSM tags I Tags to describe real world objects I I I I Tags to describe immaterial objects I I I boundary = administrative, national_park... ... Commons tags I I M GIS building = church, hotel, school, university... highway = motorway, primary, secondary... ... name = * source = * 44 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete OSM tags M GIS 45 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete A more formal OSM ontology? Simple tag structure and unrestricted tags are probably one reason for the success of OSM (easiness of contribution, flexibility, extensibility...) M GIS 46 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete A more formal OSM ontology? Simple tag structure and unrestricted tags are probably one reason for the success of OSM (easiness of contribution, flexibility, extensibility...) The price to pay is poor semantics I harder to detect and fix logical inconsistencies I less expressive queries I link to other datasets? (Linked Data) Some attempts to provide a formal (ontological) backbone to the OSM ontology: e.g. LinkedGeoData M GIS 46 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete OSM Editing tools How to contribute? M GIS 47 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete OSM Editing tools How to contribute? I Id ( I I I I ) JavaScript online editor, web application easy to use, recommended for beginners JOSM ( I M GIS http://ideditor.com/ http://josm.openstreetmap.de/ ) Java OpenStreetMap Editor desktop application written in Java, with a plugin architecture for advanced users, large set of features and tools 47 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete OSM Editing tools How to contribute? I Id ( I I I I I I http://wiki.openstreetmap.org/wiki/Potlatch_2 ) written in Flash, can be used directly from a web browser has been made obsolete by Id desktop application C++,Qt (Windows, GNU/Linux, MacOSX) Plugins for: I I M GIS ) Java OpenStreetMap Editor Merkaartor (http://merkaartor.be/) I I http://josm.openstreetmap.de/ desktop application written in Java, with a plugin architecture for advanced users, large set of features and tools Potlatch ( I ) JavaScript online editor, web application easy to use, recommended for beginners JOSM ( I I http://ideditor.com/ QGIS ( http://esriosmeditor.codeplex.com/ ) ArcGIS ( http://esriosmeditor.codeplex.com/ ) 47 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete The Id editor M GIS 48 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete The Id editor M GIS 48 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete The Id editor M GIS 48 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete The Id editor M GIS 48 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Typical JOSM edition session M GIS 49 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Typical JOSM edition session M GIS 49 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Typical JOSM edition session M GIS 49 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Typical JOSM edition session M GIS 49 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete 3 Third part Using OpenStreetMap data Exploiting the data Retrieving data (basic principles) OSM API Other querying tools End user applications Tutorial: create your own database Basic principles A step by step example Using OpenStreetMap data Exploiting the data Retrieving data (basic principles) OSM API Other querying tools End user applications Tutorial: create your own database Basic principles A step by step example M GIS 51 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete OSM and server applications I I I The OSM server cannot be queried for data directly (only a small amount). Instead, it can provide some dumps and regular diffs → application servers have to instantiate the DB locally. The OSM server can provide tiles updated on a regular basis. planet.osm (first import) OSM server http://www. regular diffs openstreetmap.org/ Application server (data) API or Web server Regular tile generation (Mapnik) tiles Client (e.g OpenLayers) M GIS Client 52 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Mapnik A (standalone) tile generator for OpenStreetMap: Mapnik OSM file OR Mapnik Tile (image) PostGIS database M GIS 53 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete OSM API v0.6 I RESTFul API to consult and edit OSM entities I I requests take the form of HTTP GET, PUT, POST, and DELETE messages requests return or expect the data for the entities in a XML format http://wiki.openstreetmap.org/wiki/OSM_Protocol_Version_0.6/DTD I Example : GET /api/0.6/[node|way|relation]/#id I M GIS returns the XML representation of the entity 54 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete OSM API v0.6 http://api.openstreetmap.org/api/0.6/way/23000671 < osm version = " 0.6 " generator = " OpenStreetMap server " copyright = " OpenStreetMap and contributors " attribution = " http: // www . openstreetmap . org / copyright " license = " http: // op end atac ommo ns . org / licenses / odbl /1 -0/ " > < way id = " 23000671 " visible = " true " timestamp = " 2013 -03 -10 T09:00:14Z " version = " 11 " changeset = " 15311804 " user = " Liberal " uid = " 667850 " > < nd ref = " 344548301 " / > < nd ref = " 247961085 " / > [...] < nd ref = " 344548275 " / > < nd ref = " 344548301 " / > < tag k = " amenity " v = " university " / > < tag k = " building " v = " yes " / > < tag k = " contact:webs ite " v = " http: // ensimag . grenoble - inp . fr / " / > < tag k = " name " v = " Ensimag - D " / > </ way > </ osm > M GIS 55 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete OSM API v0.6 http://api.openstreetmap.org/api/0.6/node/344548301 < osm version = " 0.6 " generator = " OpenStreetMap server " copyright = " OpenStreetMap and contributors " attribution = " http: // www . openstreetmap . org / copyright " license = " http: // op end atac ommo ns . org / licenses / odbl /1 -0/ " > < node id = " 344548301 " version = " 3 " changeset = " 4100388 " lat = " 45.1932723 " lon = " 5.7684104 " user = " FredB " uid = " 1626 " visible = " true " timestamp = " 2010 -03 -11 T19:09:04Z " > < tag k = " source " v = " cadastre - dgi - fr source : Direction G e ´n´ e rale des Imp ^ o ts Cadastre . Mise ` a jour : 2009 " / > </ node > </ osm > M GIS 56 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Overpass API http://wiki.openstreetmap.org/wiki/Overpass_API M GIS I an optimized read-only API that serves up custom selected parts of the OSM map data I a powerful query language with search criteria like e.g. location, type of objects, tag properties, proximity, or combinations of them. 57 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Overpass API: Example Overpass XML Overpass QL < query type = " node " > <has - kv k = " name " v = " Ensimag " / > < bbox - query e = " 5.78 " n = " 45.22 " s = " 45.13 " w = " 5.67 " / > </ query > < union > < item / > < recurse type = " down " / > </ union > < print / > relation [ " name " = " Ensimag " ] (45.13 ,5.67 ,45.22 ,5.78) ; ( ._; >; ); out OR Output in OSM XML or JSON M GIS 58 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Overpass API: Example http://overpass-api.de/api/interpreter?data=relation["name"="Ensimag"](45. 13,5.67,45.22,5.78);(._;>;);out; <? xml version = " 1.0 " encoding = " UTF -8 " ? > < osm version = " 0.6 " generator = " Overpass API " > < note > The data included in this document is from www . openstreetmap . org . The data is made available under ODbL . </ note > < meta osm_base = " 2013 -04 -11 T09:07:03Z " / > < node id = " 247961085 " lat = " 45.1933111 " lon = " 5.7690476 " / > [...] < node id = " 2148742026 " lat = " 45.1932049 " lon = " 5.7690445 " / > < node id = " 2148742029 " lat = " 45.1931915 " lon = " 5.7688690 " / > < way id = " 23000671 " > < nd ref = " 344548301 " / > [...] < nd ref = " 344548301 " / > < tag k = " amenity " v = " university " / > < tag k = " building " v = " yes " / > < tag k = " contact:website " v = " http: // ensimag . grenoble - inp . fr / " / > < tag k = " name " v = " Ensimag - D " / > </ way > [...] < relation id = " 2776018 " > < member type = " way " ref = " 31391089 " role = " " / > [...] M GIS 59 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Overpass API Applications http://wiki.openstreetmap.org/wiki/Overpass_API/Applications Overpass Turbo: A web based graphical user interface for Overpass API http://overpass-turbo.eu/ M GIS 60 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Overpass API Applications Public transport line generator http://www.overpass-api.de/public_transport.html M GIS 61 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Overpass API Applications Public transport line generator http://www.overpass-api.de/public_transport.html http://www.overpass-api.de/api/sketch-line?network=TAG&ref=B&correspondences=100& width=1600&force-rows=1 M GIS 61 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Overpass API Applications Public transport line generator http://www.overpass-api.de/public_transport.html http://www.overpass-api.de/api/sketch-line?network=TAG&ref=B&correspondences=100& width=1600&force-rows=1 ⇓ M GIS 61 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Nominatim I Nominatim: tool to search OSM data by name and address. I Accessible through the HTTP protocol with GET parameters → can be queried with a standard web browser or with command-line tools. I Three output formats: I I I Standard HTML (a web page with embedded openlayers map) XML JSON Example: [sylvain@msnordlys]~ $ curl "http://nominatim.openstreetmap.org/search.php?q=rue+ de+la+Passerelle%2C+Saint-Martin-d%27H%C3%A8res&polygon=1&format=xml" M GIS 62 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Nominatim: result of the query <? xml version = " 1.0 " encoding = " UTF -8 " ? > < searchresults timestamp = ’Thu , 21 Mar 13 08 :36:41 +0000 ’ attribution = ’ Data ( c ) OpenStreetMap contributors , ODbL 1.0. http: // www . openstreetmap . org / copyright ’ querystring = ’ rue de la Passerelle , Saint - Martin - d ' H ` e res ’ polygon = ’ true ’ exclude_place_ids = ’ 20901791 ,20955443 ,20237913 ’ more_url = ’ http: // nominatim . openstreetmap . org / search ? format = xml & amp ; exclude_place_ids =20901791 ,20955443 ,20237913& amp ; polygon =1& amp ; q = rue + de + la + Passerelle %2 C + Saint - Martin - d %27 H % C3 % A8res ’ > < place place_id = ’ 20901791 ’ osm_type = ’ way ’ osm_id = ’ 4014846 ’ place_rank = ’ 26 ’ boundingbox = " 4 5.1930046081543 ,45.1940269470215 ,5.76749229431152 ,5.76763010025024 " lat = ’ 45.1931335 ’ lon = ’ 5.7676136 ’ display_name = ’ Rue de la Passerelle , Saint Martin - d ' H ` e res , Domaine Universitaire , Saint - Martin - d ' H e ` res , Grenoble , Is ` e re , Rh ^ o ne - Alpes , 38402 , France m ´ e tropolitaine , European Union ’ class = ’ highway ’ type = ’ unclassified ’ importance = ’ 1.6 ’/ > < place place_id = ’ 20955443 ’ osm_type = ’ way ’ osm_id = ’ 4014844 ’ place_rank = ’ 26 ’ boundingbox = " 4 5.1941413879395 ,45.1945037841797 ,5.7674036026001 ,5.76747751235962 " lat = ’ 45.1941427 ’ lon = ’ 5.7674775 ’ display_name = ’ Rue de la Passerelle , Saint Martin - d ' H ` e res , Domaine Universitaire , Saint - Martin - d ' H e ` res , Grenoble , Is ` e re , Rh ^ o ne - Alpes , 38402 , France m ´ e tropolitaine ’ class = ’ highway ’ type = ’ unclassified ’ importance = ’ 1.6 ’/ > </ searchresults > M GIS 63 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete End user applications M GIS I MapOSMatic I WheelMap I OpenRouteService: I “Thematic” maps: http://openrouteservice.org/ I http://openpistemap.org/ – map of skiing/snowboarding pistes I http://opencyclemap.org/ – map of cycling routes I http://openseamap.org/ – map of sea navigation elements 64 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Using OpenStreetMap data Exploiting the data Retrieving data (basic principles) OSM API Other querying tools End user applications Tutorial: create your own database Basic principles A step by step example M GIS 65 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Data formats Where to download ? I http://download.geofabrik.de/osm/ : OSM files (XML format) by geographical area, city, country. . . : Garmin Map files, Shape files, TomTom POI, Adobe Illustrator, ..., by geographical area, city, ... I http://download.cloudmade.com/ I http://planet.osm.org/ : OSM files + changesets. What to download ? M GIS I OSM files: planet.osm for the entire planet (currently over 27GB compressed, over 300GB uncompressed), or regional extracts. I Diffs: changesets for regular database updates (example: weekly changesets in http://planet.osm.org/ ). 66 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Create your own database PostGIS database OSM file Several possible tools and schemas. . . I Osm2pgsql: Lossy translation (only converts some entities). Reconstructs polygons from relations (e.g for administrative boundaries) I Osmosis: OSM data general purpose processing tool I I I I M GIS converts OSM data to Postgres/PostGIS DB (sticks to OSM datamodel) generates planet dumps from a DB applies or generates changesets extracts data. . . 67 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete A small Example The problem I want to develop a little application that displays a map showing the French departments and regions, and the French road network. My initial configuration: a PC running Ubuntu 12.10 GNU/Linux OS 1st step. Install Postgresql: [sylvain@msnordlys]~ $ sudo apt-get install postgresql-9.1 M GIS 68 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete PostGIS installation 2nd step. Install PostGIS: [sylvain@msnordlys]~ $ sudo apt-get install postgis postgresql-9.1-postgis (A piece of cake... We have some packages for that in the distribution) M GIS 69 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Database creation 3rd step. Login as postgres user and create a new user and a new database (and set this new user to be the database’s owner): [sylvain@msnordlys]~ $ sudo -s Password: [root@msnordlys]~ # su postgres [postgresql@msnordlys] ~ $ psql postgres=# CREATE USER [username]; CREATE ROLE postgres=# ALTER USER [username] WITH ENCRYPTED PASSWORD ’[password]’; ALTER ROLE postgres=# CREATE DATABASE [dbname]; CREATE DATABASE postgres=# ALTER DATABASE [dbname] OWNER TO [username]; ALTER DATABASE M GIS 70 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete PostGIS Enabling 4th step. Enable PostGIS spatial functions into the database: [postgresql@msnordlys]~ $ cd /usr/share/postgresql/9.1/contrib/postgis-1.5 [postgresql@msnordlys]/.../postgis-1.5 $ createlang plpgsql mydb [...] [postgresql@msnordlys]/.../postgis-1.5 $ psql -d [dbname] -f postgis.sql [...] [postgresql@msnordlys]/.../postgis-1.5 $ psql -d [dbname] -f spatial_ref_sys.sql [...] [postgresql@msnordlys]/.../postgis-1.5 $ cd .. [postgresql@msnordlys]/.../postgis-1.5 $ psql -d [dbname] -f postgis_comments.sql [...] M GIS 71 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Data conversion 5th step. Run osm2pgsql to load Data into the database: [sylvain@msnordlys]~ $ osm2pgsql -U [username] -d [dbname] -r pbf --cache=4000 \ > -W france.osm.pbf Password: Using projection SRS 900913 (Spherical Mercator) Setting up table: planet_osm_point NOTICE: table "planet_osm_point_tmp" does not exist, skipping Setting up table: planet_osm_line [...] Completed planet_osm_polygon Osm2pgsql took 15962s overall M GIS 72 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete A test request We can now check whether the data has been correctly loaded... [sylvain@msnordlys]~ $ psql -U [username] -d [dbname] Password for user [username]: psql (9.1.8) Type "help" for help. [dbname]=# SELECT name, place, ST_XMin(way), ST_YMin(way) [dbname]-# FROM planet_osm_point WHERE name=’Grenoble’ AND place=’city’; name | place | st_xmin | st_ymin ----------+-------+------------------+-----------------Grenoble | city | 638583.190179611 | 5650917.09875511 (1 row) M GIS 73 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Boundaries and roads The SQL requests inside the program... Retrieve administrative boundaries SELECT name , S T _ S i m p l i f y P r e s e r v e T o p o l o g y ( way ,5000) , admin_level FROM pla net _os m_p oly g o n WHERE boundary = ’ administrative ’ AND admin_level <= ’6 ’; Retrieve the road network SELECT S T _ S i m p l i f y P r e s e r v e T o p o l o g y ( way ,5000) , highway FROM planet_osm_line WHERE highway IN ( ’ motorway ’ , ’ trunk ’ , ’ primary ’ , ’ secondary ’) ; M GIS 74 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete The result M GIS 75 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete 4 Fourth part Using Volunteered Geographic Information Authoritative data vs participative data Authoritative data, conventional data About metadata Can we trust OpenStreetMap? About data quality Some OSM data quality tools Case 1: Data quality for Location Based Services Case 2: Evaluating OSM data quality on the Department of Sarthe Using Volunteered Geographic Information Authoritative data vs participative data Authoritative data, conventional data About metadata Can we trust OpenStreetMap? About data quality Some OSM data quality tools Case 1: Data quality for Location Based Services Case 2: Evaluating OSM data quality on the Department of Sarthe M GIS 77 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete VGI vs authoritative “My project [put here the name of any serious research or industrial project] cannot use VGI data because we cannot control, trust, verify [put any other reason here] it...” (Anonymous quotation) M GIS 78 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete VGI vs authoritative “My project [put here the name of any serious research or industrial project] cannot use VGI data because we cannot control, trust, verify [put any other reason here] it...” (Anonymous quotation) “VGI is not a very important trend in GIS nowadays, so we do not consider this approach in our GIS department” (Head of GIS department of a big IT group — cited from memory) M GIS 78 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete VGI vs authoritative “My project [put here the name of any serious research or industrial project] cannot use VGI data because we cannot control, trust, verify [put any other reason here] it...” (Anonymous quotation) “VGI is not a very important trend in GIS nowadays, so we do not consider this approach in our GIS department” (Head of GIS department of a big IT group — cited from memory) M GIS I Are participative and authoritative data production so antagonistic? I Can we trust participative data? 78 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Authoritative data Authoritative data ≈ produced by professional mapping organizations M GIS 79 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Authoritative data Authoritative data ≈ produced by professional mapping organizations According to Van der Molen and Wubbe [2007] (cited by Coleman [2013]): I High-quality database I Explicit guarantees about quality assurance I Contains essential data about persons, institutions, issues, entities... I Designated by law as the sole officially recognized register to be used by government agencies Coleman, D. J. (2013). Potential contributions and challenges of VGI for conventional topgraphic base-mapping programs. In Crowdsourcing Geographic Knowledge [...], chapter 14. Springer. Van Der Molen, P. and Wubbe, M. (2007). E-government and e-land administration-as an example: The netherlands. In 6th FIG Regional Conference, San Jose, Costa Rica, pages 12–15. M GIS 79 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Conventional data The term authoritative is a bit restrictive Coote and Rackham [2008] prefer the term conventional data: I Collected for specific purpose and requirements I Usually not free (charged) I Use limited to some organizations or individuals I Copyrighted data I Managed by organizations established for the purpose I Collected by professional staff, paid for this I Based on standard and established methods and practices I Quality assessment at different levels, guarantees provided to the user Coote, A. and Rackham, L. (2008). Neogeographic data quality — is it an issue? In AGI Geocommunity conference, ConsultingWhere Ltd. M GIS 80 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Example – IGN, BD Topo http://professionnels.ign.fr/bdtopo M GIS I Restricted access, specific license (free for a sample, charged for the rest) I IGN keeps the ownership of data, and only gives utilization rights I Collected by professional staff at IGN, standard production techniques I Complete metadata following ISO-19115 standard I Well-documented dataset, detailed quality assessment 81 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete About metadata One important aspect in Geographical datasets is metadata. M GIS 82 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete About metadata One important aspect in Geographical datasets is metadata. M GIS I Metadata: ≈ data about the data (producer, owner, units, error bounds, geographical bounding box, production process, source of data...) I Usually, metadata is separated from data (ISO 19115) I Usually, metadata production is separated from data production 82 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete About metadata One important aspect in Geographical datasets is metadata. I Metadata: ≈ data about the data (producer, owner, units, error bounds, geographical bounding box, production process, source of data...) I Usually, metadata is separated from data (ISO 19115) I Usually, metadata production is separated from data production But in the context of VGI... M GIS 82 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete About metadata One important aspect in Geographical datasets is metadata. I Metadata: ≈ data about the data (producer, owner, units, error bounds, geographical bounding box, production process, source of data...) I Usually, metadata is separated from data (ISO 19115) I Usually, metadata production is separated from data production But in the context of VGI... M GIS I Some metadata elements make sense for the dataset as a whole: owner (?), description of the dataset... I But some don’t really: bounding box, producer, production process... 82 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Metadata embedded Let’s have a look at OpenStreetMap... M GIS 83 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Metadata embedded Let’s have a look at OpenStreetMap... Reminder: Each OSM entity has: I a numeric identifier: OSM ID I a geometry I a set of generic attributes present for every element I I I I I I M GIS uid, user: user id and user name timestamp: time of the last modification version: edit version of the object (starts from 1) changeset: the changeset (group of edits made within a certain time by one user) in which the object was created or updated comment: each changeset has an associated comment describing it a set of tags (attributes): key-value pairs 83 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Metadata embedded Let’s have a look at OpenStreetMap... Reminder: Each OSM entity has: I a numeric identifier: OSM ID I a geometry I a set of generic attributes present for every element I I I I I I uid, user: user id and user name timestamp: time of the last modification version: edit version of the object (starts from 1) changeset: the changeset (group of edits made within a certain time by one user) in which the object was created or updated comment: each changeset has an associated comment describing it a set of tags (attributes): key-value pairs Metadata, “embedded” in the description of the entity itself M GIS 83 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Metadata embedded Metadata embedded: user, timestamp, version, changeset, comment M GIS 84 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Metadata embedded Metadata embedded: user, timestamp, version, changeset, comment But tags can also contain metadata, e.g.: M GIS source = tiger:source survey:date = = cadastre-dgi-fr source : Direction G´ en´ erale des Imp^ ots - Cadastre. Mise ` a jour : 2009 tiger_import_dch_v0.6_20070829 2013-12-10 84 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Metadata embedded Metadata embedded: user, timestamp, version, changeset, comment But tags can also contain metadata, e.g.: source = tiger:source survey:date = = cadastre-dgi-fr source : Direction G´ en´ erale des Imp^ ots - Cadastre. Mise ` a jour : 2009 tiger_import_dch_v0.6_20070829 2013-12-10 Actually, some metadata is common to the whole OSM dataset: license, conditions of use, and OSM Wiki ( http://wiki.openstreetmap.org/ ) M GIS 84 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Centralized vs distributed approaches to metadata I Traditional approach to metadata: “centralized” standards (ISO-19115) I I I I I Distributed “user-centric” approach: I I I M GIS Clear standards, easy to navigate Heavy to produce ; sometimes incomplete even for authoritative data Does not always completely make sense Does not always completely reflects the production process Can be messy Lightweight and flexible approach Consistent with the production process 85 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete VGI and metadata Conclusion about VGI and metadata [Poore and Wolf, 2013]: the process of producing metadata has to be rethought for VGI, because the dataset is: I dynamically (continuously) generated I generated locally by hundreds, thousands, or millions of users ; Production of metadata has to be integrated to the data production process Poore, B. S. and Wolf, E. B. (2013). Metadata squared: Enhancing its usability for volunteered geographic information and the geoweb. In Crowdsourcing Geographic Knowledge [...], chapter 4. Springer. M GIS 86 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Using Volunteered Geographic Information Authoritative data vs participative data Authoritative data, conventional data About metadata Can we trust OpenStreetMap? About data quality Some OSM data quality tools Case 1: Data quality for Location Based Services Case 2: Evaluating OSM data quality on the Department of Sarthe M GIS 87 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete What is data quality? Data quality is a perception or an assessment of data’s fitness to serve its purpose in a given context and a subjective to various applications. It highly depends on the need of individuals on how to use datasets. [Caprioli et al., 2003] Two main objectives: I Is the dataset suitable for my needs? I How can the dataset be improved? Caprioli, M., Scognamiglio, A., Strisciuglio, G., and Tarantino, E. (2003). Rules and standards for spatial data quality in gis environments. In Proc. 21st Int. Cartographic Conf., pages 10–16, Durban, South Africa. M GIS 88 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Main dimensions I Accuracy Does it reflect reality? [Olson, 2003] I Completeness Is something missing? [Batini and Scannapieco, 2006], I Timeliness Is the data up-to-date? [Pipino et al., 2002] I Volatility How long stays the information valid? [Batini and Scannapieco, 2006] Batini, C. and Scannapieco, M. (2006). Data quality: concepts, methodologies and techniques. Springer. Olson, J. E. (2003). Data quality: the accuracy dimension. Morgan Kaufmann. Pipino, L. L., Lee, Y. W., and Wang, R. Y. (2002). Data quality assessment. Communications of the ACM, 45(4):211–218. M GIS 89 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Main dimensions I Consistency Does it involve inconsistent data? [Batini and Scannapieco, 2006] I Quality of information source Credibility? [Pipino et al., 2002] I Validity Inside a range? [Olson, 2003] I Understandability Clarity of information? [Pipino et al., 2002] Batini, C. and Scannapieco, M. (2006). Data quality: concepts, methodologies and techniques. Springer. Olson, J. E. (2003). Data quality: the accuracy dimension. Morgan Kaufmann. Pipino, L. L., Lee, Y. W., and Wang, R. Y. (2002). Data quality assessment. Communications of the ACM, 45(4):211–218. M GIS 90 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Geographical dimensions [ANZLIC, 2001] I Lineage Can we assess where the data comes from? I Positional accuracy Absolute Position I Attributive accuracy Does it contain all attributes to reflect reality? I Logical consistency Does it contain logical errors? I Completeness Does it contain all information needed for the task at hand? ANZLIC (2001). Anzlic metadata guidelines: Core metadata elements for geographic data in australia and new zealand. Technical report, ANZLIC. M GIS 91 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Assessing geographical data Quality I Ground truth I I M GIS Ground truth on sparse points Second dataset considered as ground truth I Logical rules, mainly for checking logical consistency (open polygons, non-crossing roads, etc.) or completeness (missing mandatory information) I Statistical approach using predefined metrics based e.g. on completeness of information or user activity 92 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Bug reporting http://www.openstreetmap.org/ I Anyone can add notes to any geographical place on the OSM website I Notes can be visualized on the map or downloaded as RSS feeds using OSM API http://www.openstreetmap.org/api/0.6/notes/feed?bbox=5.7393265,45.1798589,5. 7891083,45.206352 M GIS 93 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Error detection tools I Keep Right ( I I Osmose ( I I ) http://osmose.openstreetmap.fr/map/ ) Similar to KeepRight. Currently, it covers : I I I M GIS http://keepright.at/ Detects: non-closed areas, dead-ended one-ways, almost junctions, deprecated tags, missing tags, bridges/tunnels without layer (careful - not always an error), motorways without ref, places of worship without religion, POIs without name, ways without nodes, floating islands, un-tagged railway crossings, wrongly-used railway crossing tag, objects with FIXME tags, and relations without type. France some nearby countries in Europe: Belgium, Luxembourg, and Switzerland; some nearby countries in Africa/Indian Ocean: Madagascar, Cameroon 94 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Case 1: Data quality for Location Based Services Context: Data Quality Analysis of the OpenStreetMap Project Regarding Location Based Services, Bachelor Thesis by Niklas Petersen Approach: M GIS I Grid Analysis with 1km2 cells I Count the number of values in each cell I Merge with population data (reference ≈ ground truth) I Highlight differences 95 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete House numbers M GIS 96 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete House numbers M GIS 97 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Buildings M GIS 98 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Roads M GIS 99 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Shops M GIS 100 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Conclusion about the case study I Urban > rural regions I Strongest coverage: I I I M GIS Buildings: Netherlands → can be considered as complete Housenumbers: Czech Republic → coverage in general weak Shops: Switzerland → big difference between countries 101 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Case 2: Evaluating OSM data quality on the Department of Sarthe Context: a study of OSM data quality carried out by Petit, Billon and Follin [2012] on the French department of Sarthe OSM data analyzed from the point of view of: I geometrical accuracy I attributive accuracy I completeness Petit, O., Billon, P., and Follin, J.-M. (2012). ´ Evaluation de la qualit´ e des donn´ ees OpenStreetMap sur la Sarthe et r´ eflexion sur le processus de contribution. XYZ, (131):24–34. M GIS 102 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Methodology ´ Reference: R´ef´erentiel ` a Grande Echelle (RGE), IGN (authoritative data) M GIS I Preprossessing: automatic matching of OSM and RGE road networks I Completeness of road network: comparison of the total lengths OSM vs RGE, for each city I Geometrical accuracy of road network: Hausdorff distance (roads) and average Euclidian distance (nodes) I Attributive precision: comparison of lengths named (name = ...) roads between OSM and RGE + manual comparison to find the sources of mismatches (spelling errors, punctuation, incomplete names...) I + Evaluation of the precision for two methods of contribution (GPS, and imagery-based digitizing) 103 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Conclusions M GIS I Heterogeneous data quality (urban > rural) I Incompleteness is the main weakness (especially on rural areas) – to be compared with the homogeneity of RGE, whose mission is the complete cover of French territory I Rather good geometrical accuracy I A few attribute values but quite good accuracy I Both digitizing methods are quite precise 104 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete Conclusion To VGI or not to VGI? Conventional data M GIS VGI data 106 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete To VGI or not to VGI? Conventional data Quality of data M GIS 3 VGI data Random quality 106 / 107 7 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete To VGI or not to VGI? Conventional data Quality of data 3 Guarantees about quality M GIS VGI data Random quality 3 7 No guarantee about quality 106 / 107 7 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete To VGI or not to VGI? Conventional data Quality of data M GIS VGI data 3 Random quality 7 Guarantees about quality 3 No guarantee about quality Heavy production process 7 Lightweight production process 106 / 107 7 3 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete To VGI or not to VGI? Conventional data Quality of data 3 Random quality 7 Guarantees about quality 3 No guarantee about quality Heavy production process 7 Lightweight production process Poor reactivity M GIS VGI data 7 Timeliness 106 / 107 7 3 3 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete To VGI or not to VGI? Conventional data Quality of data 3 Random quality 7 Guarantees about quality 3 No guarantee about quality Heavy production process 7 Lightweight production process Poor reactivity 7 Restrictions (usage) M GIS VGI data 7 Timeliness 3 Free, open 3 106 / 107 7 3 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete To VGI or not to VGI? Conventional data Quality of data VGI data 3 Random quality 7 Guarantees about quality 3 No guarantee about quality Heavy production process 7 Lightweight production process Poor reactivity 7 Restrictions (usage) 7 Timeliness 3 Free, open 3 7 3 Conclusion: VGI is not the solution to everything, but certainly a solution M GIS I If guarantees about data quality (with someone who is liable for that) is important, use conventional datasets I If timeliness is more important (e.g. crisis management), consider using to VGI datasets 106 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete VGI and conventional data Actually, several traditional data producers consider integrating VGI in their data production process M GIS 107 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete VGI and conventional data Actually, several traditional data producers consider integrating VGI in their data production process Is data quality compatible with agility? M GIS 107 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete VGI and conventional data Actually, several traditional data producers consider integrating VGI in their data production process Is data quality compatible with agility? Three possible approaches: I I I M GIS Limit the number of authorized contributors (confidence network) ; Victoria Department of Sustainability and Environment, Australia Let the users annotate, then internally check the validity ; Google Map Maker, United States Geological Survey, 2001 Mix participative and conventional data, flag participative data and let the end users choose whether they want to integrate them or not ; TomTom MapShare 107 / 107 ´ GdR MAGIS – Ecole de G´ eomatique 29 septembre au 3 octobre 2014 – S` ete
© Copyright 2024 ExpyDoc
![[ArcGIS[Ver.10]キャンパスライセンスのインストールマニュアル] - 筑波大学](http://s3.expydoc.com/store/data/000575249_1-0129adc2f82d6df540ce53066bc9575c-250x500.png)
