The Second Interna.onal Conference on CyberGIS and Geodesign August 19th – 21st, 2014 Redlands, California Extending WorldMap to Make It Easier for Humanists and Others to Find, Use, and Publish Geospa@al Informa@on Peter K. Bol Benjamin G. Lewis Weihe Wendy Guan Essence of the NEH Grant Project • WorldMap is a web-‐based, open source, collabora@ve mapping plaPorm developed at Harvard CGA since 2010 • The Na@onal Endowment for the Humani@es’ Digital Humani@es Implementa@on Grant funded its enhancement for two years star@ng Sep. 2013 with $320K. • A key objec@ve of the grant is to create a comprehensive and sustainable map service registry which researchers and the public can use to discover, create and share any work that can be represented spa@ally. WorldMap In a Nutshell • Designed to lower barriers for researchers who wish to use spa@al technology • Web-‐based, cloud hosted • Harvard instance made available to the world to use • Service oriented architecture • Open source soZware -‐ available on GitHub h[ps://github.com/cga-‐harvard/cga-‐worldmap WorldMap allows researchers to… • Organize their own (large) raster or vector mapping datasets and share them • Visualize maps with data-‐driven symbology • Publish data to the world or to just a few collaborators in a variety of ways • Mashup or Combine one’s own data with data provided by others • Collaborate by leang several people edit the same map or layer Usage Sta@s@cs • • • • • 11,600 registered users 13,665 data layers added by users 3,771 map projects created 780,000 unique visitors, all countries About 1000 visitors per day As of August 2014 (Ver. 1.4) Since July 2011 (Beta) Broad Developer Community Contribu@ng Organiza@ons (WorldMap and GeoNode) • WorldMap • • • • • • Boston Area Research Ini.a.ve -‐ BostonMap UNICEF – Educa@on Access in Cameroon Cornell University – Global Health Map UN University – Wildlife Enforcement Monitoring Virtue Founda.on – Women in the World Amazon -‐ Hardware Others… • • • • GeoNode • • • World Bank – GFDRR, Dominode, Risiko U.S. State Dept. – ROGUE, HIU, Syria Damage Assessment NOAA -‐ GeoCloud UN World Food Program – WFP Geonode Australian Govt. -‐ AIFDR, TsuDAT MapStory Founda.on – MapStory App Others… Loosely Coupled Approach to System Integra@on Esri REST ArcGIS Server JavaScript WMS, JSON Dataverse WMS Elas@c Search Map Service Registry Neatline HarvardX Data Connectors to… • Google maps, Open Street Maps, Bing, MapQuest, ESRI • Picassa, YouTube, Flickr • Geonames, Google Places, Yahoo Places • Google Street View • Geo-‐tweets • GeoRSS • Google Earth • Social Explorer, Yelp Lowering the Barrier to Geodata Access, Across Disciplines Commercial Systems WorldMap Data and Service Registry ST, Esri RE WMS etc. S S R , , WFS Government Systems upload downloa d Internet local data Other Ins@tu@ons Key Problems We A[empt to Solve (with this NEH grant) • Discovery of the millions of web maps that are exposed but not easy to find (the “dark” geoweb) • To allow mashups of mapping content from many sources from within any mapping applica@on • Enable non-‐IT professionals to create their own map services without IT support • Crowd-‐sources map data cura@on in a metadata-‐ weak environment • Start building a @me-‐enable gaze[eer in a crowd-‐ sourced way Find, Add, Mashup and Share • Build and maintain a comprehensive, public, API-‐accessible registry of map service-‐enabled content from map servers around the world • Search and consume contents of any mapping systems, mash up, and overlay them within WorldMap, then share it with any applica@on • Allowing users to upload their data to WorldMap. All data in WorldMap can be consumed as map services in other applica@ons Deep Search, Smart Search Crowd Sourced Solu@ons • Enable search by @me, by place, and by feature level content within data layers • Build the “wisdom of crowds” into the search mechanism • Allow users to create custom historical place name databases (each with their own access point) and allow these to be added to a common crowd sourced temporal gaze[eer. Crea@ng a Global Service Registry -‐-‐ A fundamental piece of geo-‐infrastructure that, for whatever reason, doesn’t exist -‐ yet • Build registry of web map services (millions of map layers) • Make API available so any system can use it • Provide a fast, faceted search interface • Allow anyone to add new services to the registry • Maintain up@me sta@s@cs on each service • Use WorldMap usage sta@s@cs to improve search (eventually bring in stats from systems outside WorldMap which use API) How Many Services Are Out There? • We es@mate millions, each containing many map layers totaling petabytes of data which is currently VERY hard for the average researcher to find and use. • Try this to es@mate number of Esri REST servers (15million) – allinurl: h[p "arcgis rest services" mapserver -‐test -‐kml -‐ kmz -‐sitemap -‐query • Try this to es@mate number of WMS servers (47 thousand) – allinurl: h[p "?request getcapabili@es" -‐test Service Registry Challenges • • • • • Metadata -‐ tagging and usage sta@s@cs Projec@ons -‐ cascading Persistence -‐ caching Discovery -‐ central index, usage sta@s@cs Performance -‐ caching Open API to Registry • Public, RESTful API • Access all (public) map layers within WorldMap • Access all service layers outside WorldMap • Access all Maps (collec@ons of layers) within WorldMap • Search on informa@on: – Metadata – Usage sta@s@cs – A[ribute info (for local layers) Faceted Services Search UI Mockup (draZ) Distributed Users Find and bind to layers WorldMap Service Crawler OpenLayers, Leaflet S e r v Service i Registry c e s A P I Esri clients Any map client WorldMap Up@me Checker *Common Crawl Distributed Map Services Service caching, reprojec@on WorldMap Local Services WorldMap Core Crowd cura@on, user submi[ed services *Start with hadoop search of Common Crawl dataset hKp://commoncrawl.org/ Time-‐enable Geospa@al Data • Core Issues Encountered – Historical place name databases are sca[ered – Gaze[eer a[esta@on and disambigua@on is a challenge, par@cularly for historical entries – Spa@al data layers containing @me a[ributes are common, but hard to integrate together • Solu@ons – Allow users to create their own custom temporal gaze[eers – Op@onally also contribute to a common crowd-‐sourced gaze[eer – Users may tag historic maps with place names to verify or add to any of the temporal gaze[eers Serving the Humani@es • Lowering the cost of entry into spa@al humani@es • Elimina@ng duplica@on of effort for rou@ne spa@al tasks • Gran@ng a permanent life for the spa@al researches of any scholar worldwide • Encouraging humanists to share their data and interpre@ve organiza@on of the data • Allowing for the development of mul@ple and compe@ng spa@al visualiza@ons • Fostering collabora@on and cumula@vely work Evolving Defini@ons of CyberGIS • GIS on steroids (Michael Goodchild, 2010) • high-‐performance compu@ng • distributed compu@ng • the geospa@al Web • At Esri, we understand cyberGIS to essen@ally mean GIS detached from the desktop and deployed on the web, with the associated issues of hardware, soZware, data storage, digital networks, people, training and educa@on. (Dawn Wright, 2013) • Cyberinfrastructure-‐based GIS empowering extreme digital transforma@on of geographic research and educa@on (Shaowen Wang, 2013) Evolving Defini@ons of CyberGIS Nancy Wilkins-‐Diehr, Director of the San Diego Supercomputer Center h[p://hubzero.org/resources/784/watch?resid=958 (2012) • Applica@on of high-‐end cyberinfrastructure to GIS • Influence on mul@ple domains • Improved decision support • Goal is core set of composable, interoperable, manageable, reusable soZware components • Collabora@ve geospa@al problem solving environment Evolving Defini@ons of CyberGIS Excerpt from “Enhancing the CyberGIS Vision” (Michael Goodchild, September, 2013) • CyberGIS – a subset of cyberinfrastructure – specific to geospa@al ac@vi@es • The components of CI: – high-‐performance compu@ng • applied to geospa@al analysis, modeling • because such problems are oZen compute-‐intensive – distributed, collabora@ve systems • using geospa@al data and tools • strong emphasis on visualiza@on – community building WorldMap’s Relevance to CyberGIS … • Cyberinfrastructure-‐based – Hosted on Amazon EC2 – Service oriented architecture – Web-‐based UI for crea@ng and using spa@al informa@on • High-‐performance – On-‐the-‐fly caching to enable support of thousands of users rendering thousands of layers – Fast faceted search against global registry of map services … WorldMap’s Relevance to CyberGIS • Distributed – System will index and consume distributed map services – Search access provided to systems outside WorldMap via open API – Supports publishing of map services from raw data • Collabora@ve – – – – – – Crowd sourced cura@on Crows sourced data Shared map edi@ng and commen@ng Shared layer symboliza@on and edi@ng Temporal and deep data search augment metadata Temporal gaze[eer provides anchor for mapping non-‐spa@al materials WorldMap Objec@ve: Be an Open and Accessible CyberGIS Open registra@on Open access to data Open service protocols (WMS, WFS, ESRI Rest) Open data formats (Shape, GeoTIFF, GeoRSS, KML, JSON, CSV) • Open source code (GPL on github) • Usable on any opera@ng system • • • • Future Perspec@ves • Performance • Scalability • SoZware sustainability (version upgrades of underlying open source components) • Financial sustainability • Content preserva@on • Usage sta@s@cs contributed by the consuming clients Key Players in WorldMap Development Prof. Peter Bol Ben Lewis Prof. Suzanne Blier Ma[ Bertrand For More Informa@on • WorldMap: h[p://about.worldmap.harvard.edu • NEH Grant: h[p://www.neh.gov/divisions/odh/grant-‐ news/announcing-‐6-‐digital-‐humani@es-‐ implementa@on-‐grant-‐awards-‐july-‐2013 • Center for Geographic Analysis: h[p://gis.harvard.edu HarvardX Integra@on (Course Development Which Requires Maps) Neatline Integra@on (Map-‐based Story Telling) Dataverse Integra@on (Social Science Archive)
© Copyright 2024 ExpyDoc
