Cloud GIS, it’s all about sharing the map

Cloud GIS is obviously NOT just about sharing the map, but it is just cool to share a beautiful and useful map with others through blog, Facebook or Twitter.

Google Maps

View Trip to Washington D.C. in a larger map

ArcGIS.com

View Larger Map

Geocommons


GISCloud

Mashup WMS in GeoCommons

WMS, whether dying or not, still has its place in the world of GIS. Given that many organizations have published their spatial data through WMS, it is a very important data source especially in the era of cloud GIS today. GeoCommon’s is one of the first cloud GIS platform that allows users to add WMS as one type of data sources which can mashup on the base map. Follow the blog post “Adding WMS and Tiles in GeoCommons and GeoIQ” I gave it a try using one public WMS called “StatesCitiesRivers_USA”  from ArcGIS Online sample server and another from OSGeo.

After log in, select “Make a Map” and pick whatever base map as you like, which in my case is OSM.

Click “+Add” at top-right corner, and select “Upload” tab and “Add a URL Link” tab, and finally select “Web Map Service” in the format dropdown list, and type in the url to WMS’s capabilities files (the url must point to the capabilities XML file either static or dynamic).

Since ArcGIS Server supports WMS 1.3.0, so I tried this highest version of capabilities file, but unfortunate it doesn’t display the WMS layers on top of base layer although it is able to get the list of layers from that WMS.

Next I switched to version 1.1.1, which actually worked for me.

For the WMS overlay in GeoCommons, users are able to turn and off individual or all layers, and also change the opacity of the whole WMS overlay which obviously happens at client side (because WMS itself doesn’t support opacity). But none of other style or filter options are available to WMS layer. What is also observed is that WMS map is requested from server as multiple tiles instead of a single map image, which may be because GeoCommons platform currently doesn’t support any dynamic mapping service like WMS. Unless those tiles from WMS are cached at client-side, it’s not quite efficient.

One big problem I see, which is also quite obvious from the screenshot , is that GeoCommons requests WMS map in EPSG:4326 even though the base map is Web Mercator (EPSG:3857 or EPSG":900913 for most open source GIS community). This is caused by the fact that the public ArcGIS Server WMS service doesn’t list EPSG:900913 as one of the supported SRS.

I then added the public WMS from OSGeo, which supports EPSG:900913, which is side proof of what happened to ArcGIS Online Sample Server.

Since ArcGIS Server WMS doesn’t support EPSG:900913 (not in the way when it’s named as 900913) at this point, the only workaround is to choose one of the base map marked as (4326).

 

Now it overlays perfectly.

Conclusion

There are still a lot of data source published as WMS, and in my opinion there will only be more in the near future. So supporting WMS will give huge benefit to today’s Cloud GIS providers. It will be really nice to see similar WMS support in other major Cloud GIS player like ArcGIS.com, Google Maps and so on.

World-wide routing, MapQuest + OpenStreetMap

MapQuest was THE first and only web routing service I relied on. But once up on a time, I almost forgot its existence because of Google Maps, Yahoo, and even others. Recently MapQuest seems to be coming back really strong especially with its Open Initiative and integration with OpenStreetMap data. One of the most amazing things I have seen is this Dublin-to-Shanghai world-wide routing:

Although I don’t think it has much practical use to most people except for professional backpacker traveler, it is still amazing and seriously no one else can do it.

Choosing China’s location check-in platform

Like many other Chinese people (I am not talking about the those who were born or grown up here in States) here in Unite States, I’m facing a dilemma when choosing a location check-in game service to share location, pictures and tips with my friends. There seems to be quite a lot options. We have the popular Foursquares, Gowalla, and Facebook, and Google Latitude which jumped in recently, plus Yelp, Booyah, MyTown, and the list can go on and on. However, the problem is that many of my friends are in mainland China who want to see where I have been to and how I felt about those places, but they probably don’t want to install any of those clients on their mobile devices (actually even if they do, they can’t because many social networks applications are blocked in China). For the same reason I can’t see their move and tips either because obviously they don’t share it through any of the location check-in platform above.

The only workaround seems to be choosing one of China’s location check-in platform. Fortunately there is also a long list of those to choose, but none of them made a perfect solution. My top candidate list includes: Jiepang (街旁), Sina (微领地), Sifang (四方), and kaikai (开开). Which one to choose is certainly subjective to each individual person, but here is a list of things (ordered by priority) that I will base my choice on.

1. Size and accuracy of the existing foreign (U.S in this case) venue database;
2. Share check-in with friends on other check-in platform or social networks (Facebooks, Foursquares, Kaixin001, Sina Weibo etc);
3. Being able to add new venue.

To me both #1 and #2 are must for a check-in platform be to usable, and #3 is nice to have but also important.
Venue database

This almost dominates my decision. Sina has its decisive advantage over others in its complete and accurate U.S venue database, which even overcomes its shortage in all other aspects. Just as an example, I live in Rancho Cucamonga, a small size city in southern California, which is not famous and probably unknown to most people in China. No one would expect a Chinese location check-in service to have much venues in this area in its databases. But I was really surprised the first time when I checked in at home, Sina accurately lists closest everything nearby which include a KFC, a RiteAid pharmacy, a Ralphs grocery store, a US Bank (a small local bank I’ve been to), …, and a lot of other small places. None of others have a similar or even close size and accuracy of the venue database as Sina does.

Sharing check-in with others

This is the second most important thing because first I obviously want my check-in to be shared with my friends in other check-in platforms or social networks, and second I obviously don’t want to repeat my check-in on every check-in platform either for my friends in U.S or in China. This seems to be possible in two aspects: either (a) I check-in once on one of them (e.g. Sina) and my check-in will spread out through synchronization to Facebook, Foursquare, Twitter, Kaixin (开心网), Weibo (新浪微博) an so on; or (b) I check in through a 3rd party proxy app which takes my check in and broadcast to all my location check in services. Option #b is only possible when everyone has a free open API for venues and check-in which is NOT true for any of the location check-in platform in China, so it is ruled out. Option a is possible in one direction (China check-in spread to U.S platform), and almost all Chinese location check-in services provide you such capability except for Sina.

Adding new venues

This is also very important to both users and location check-in service providers. I am true believer of the fact that a real complete and accurate venue database can only be evolved from volunteered geographic information from those locals who know where they live well. Unfortunately Sina again doesn’t support this feature while all other check-in platforms supports it which is quite disappointing to me. As latest version of Sina (微领地) goes, users are allowed to create new venue and submit to service provide for reviewing and I guess if it is approved then it will be added into existing venue database. This is a big leap to me and makes Sina (微领地) outstands other options in my particular use case.

Conclusion

No one wins because neither Sina or any other candidates has both #1 and #2, but it provokes some interesting thinking on the current situation in the location check-in services market in China. Sina is close because it is the only one who is either interested in or can afford building a relatively complete and accurate foreign venue database. It is also easy to prove that those venues in sina’s database are not coming from any major open free sources (e.g. OpenStreetMap or Foursquare API etc.) So it’s probably from a commercial provider, which in turns logically explains why there are restrictions on adding new venues or modifying existing ones, and why check-in on vld.sina cannot be spread out to other platforms. Other location check-in platforms in China are still missing a solid venue database to start with, and importing from OpenStreetMap database or relying on the venue API from a popular U.S provider like Foursquares or so seem to be possible solutions.

Thoughts on OSM and the Bing automatic vector detect service

In Mike Dobson’s recent blog post OSM vs. the Mechanical Turk – A New Option For Mappers?, he brought a point, which I think is more than reasonable:

“…I guess that when I thought about the term Volunteered Geographic Information, I made the mental leap that these volunteers were providing content in the form of spatial information reflecting the geographic areas in which they lived or with which they were more than casually familiar. It has now occurred to me that there may not always be a direct beneficial relationship between geographical knowledge and Volunteered Geographic Information…”

And then he did an analogous comparison between OSM and Mechanical Turk, in which he points out that:

“…If the majority of OSM contributors to the UK database are spending their time digitizing imagery for the UK portion of the OSM database, as opposed to contributing GPS traces and attributes from paths along which they have traveled or know something about, how likely is it that the OSM effort in the UK benefits from local knowledge to the same extent that it benefits from “free” digitizing?…”

to him, the way how some people contribute to OSM database is similar to a modern version of Mechanical Turk. It looks like the data are contributed with valuable local geographical knowledge, but it is only artificially generated from aerial imagery.

Now this reminds me of another exciting post I came across from Bing, in which Steve Coast announced the experimental service from Bing which can automatically derive street vector data from Bing aerial imagery. I am pretty sure it will boost the popularity of OSM in short term, but in the long run is it really helping?

Create your public WMS through www.GISCloud.com

More and more cloud based GIS infrastructures are popping up these days. GISCloud is just one of those many and probably not the most famous one. But it does have some interesting features that attracts me. Today, with its free account, I was able to quickly publish a public WMS (OGC Web Map Service) simply with a few clicks. Although the service seems a little buggy in it beta but the features are promising.

To get started, simply go to http://www.giscloud.com/ and then sign up for a free account. After verification log in you will be able to things like upload/manage/share your dataset, create mashup maps with built-in layers (OSM, Google, etc.), your own dataset or public dataset shared by others, and of course your maps can be shared to other through different flavors. These are quite standard across different cloud GIS platforms nowadays.

The aspect I like most of www.GISCloud.com is the fact that it has a broad list of supported input/export formats, which by the way is very open source friendly. Especially the claim of “all OGC and GDAL formats”, which sounds quite powerful.

© 2011 GIS Cloud Ltd

The particular feature I tried is simply publishing a world boundary layer as a WMS. The user experience is quite smooth, since the world boundary is a default built-in layer, so I end up having a public WMS in literally just a few seconds.

Select “world” map on left, click Project->Share and Publish

In “Share and Publish” dialog, go to WMS tab and click “Enable”, and below that is the url endpoint of your public WMS.

The next thing is simply consume it in a free 3rd party client like Gaia

This is really fantastic start, and I really like it.

A few things to improve though:

• None of the built-in tiles layers (OSM, Google) come through as WMS, which I don’t think is technical issue but probably licensing issues;
• The world I configure in website seems to be in Mercator projection but the output WMS only claims EPSG:4326 in capabilities files;
• Still a little unstable when I enable/disable WMS back and forth with/without tiled layer like OSM, need to refresh the page to get rid of it.

Configure GDAL/OGR Python debug environment in Pydev on Windows

Recently I am involved in a project that requires some Geospatial development in Python. So I decide to take this chance to get familiar with using GDAL/OGR in Python because it is the most popular open source GIS library plus it has a python binding.

The first thing you need is obviously a Python development environment for GDAL/OGR. I know IDLE or even a plain text editor plus “print …” will do the job just fine, but I still prefer a decent IDE. Why not? Especially when it is free. So I choose the Eclipse based Python IDE Pydev. The only problem is that the process of configuring it with GDAL/OGR on Windows isn’t that obvious, or at least to me it is not. So in this post I will go through it step by step.

1. Install Python

Installing Python is quite straight forward, just go to http://www.python.org/, pick up the installer for your version and OS, and install it. I would recommend Python 2.6 or 2.7 because there are pre-compiled GDAL/OGR python binding for both versions.

2. Install GDAL/OGR Python binding

First of all, this website is really helpful for you to get pre-compiled GDAL/OGR Python binding. It has different stable released version and the version latest trunk, and the best part is that it also has the installer version and non-installer (zipped version) versions for both 32bit and 64bit.

Use installer version

Find the .exe or .msi installer file, which usually comes with name GDAL-1.x.0-win32-py2.x.msi () e.g. here is the download link to the one for GDAL 1.8.0 32bit for python 2.7 (GDAL-1.8.0.win32-py2.7.msi). So basically just follow the installation wizards to finish the installation. Under the hood, my understanding is that the installer just simply copy a bunch of files and folder into \Lib\site-packages folder of your Python install location.

Note: the installer version requires you to have an appropriate version of Python installed and registered in registry first. The normal installation of Python takes care of the registry, but if not (e.g. you want to use the Python that comes with ArcGIS Server), then you will have to manually install a standard version of Python at a different location, install the GDAL Python binding, then copy those GDAL related files and folders back to the site-packages folder of that unregistered Python.

Use non-installer version

Instead of using installer version, you can choose to use a non-installer version which avoid adding stuff upon your vanilla Python install and registry. So from the same website just download those zip files with name e.g. release-1500-x64-gdal-mapserver.zip (don’t worry it comes with  a copy of UNM MapServer), and unzip it on your local disk. The “information” link gives you very detailed information about the folder structure and versions of packages included.

After unzipping, I highly recommend you to take a look at the SDKShell.bat, which usually contains code similar to below:

:setenv2
SET PATH=%CD%\bin;%CD%\bin;%CD%\bin\gdal\python\osgeo;%CD%\bin\proj\apps;%CD%\bin\gdal\apps;%CD%\bin\ms\apps;%CD%\bin\gdal\csharp;%CD%\bin\ms\csharp;%CD%\bin\curl;%PATH%
SET GDAL_DATA=%CD%\bin\gdal-data
SET GDAL_DRIVER_PATH=%CD%\bin\gdal\plugins
SET PYTHONPATH=%CD%\bin\gdal\python\osgeo
SET PROJ_LIB=%CD%\bin\proj\SHARE

This basically tells you what environment variables and paths must be included in PATH system environment variable, which we need when later configuring Pydev.

3. Install Pydev Python IDE

Install Pydev IDE is also easy and straightforward. Either download it from sourceforge, or update it through Eclipse by adding http://pydev.org/updates in software update sites.

After installation, just launch it.

4. Configure GDAL/OGR Python debug environment in Pydev

This is the most trick part. Depend on whether you’re using the installer version or the non-installer version, the steps are different.

First of all, you need to specify the Python interpreter in Pydev. So just go to Windows->Preference. and navigate to “Pydev”

Click “New”, type in a name and specify the path to python.exe of your Python installation, and finally click ok.

If you install GDAL Python binding with installer, then you should be good to go.

If you install GDAL Python binding with non-installer zip file, then you will have to configure a few more things. Still in the “Python Interpreters” configure dialog box, at the bottom part, select “libraries” tab and click “New Folder” on the right, navigate and select the <GDAL_Python_Install>\bin\gdal\python, finally click ok.

Assume you unzip the GDAL Python package at E:\programs\gdal\1.8.0, then you should add E:\programs\gdal\1.8.0\bin\gdal\python to the list of libraries.

Next go to “Environment” table, and there you need to add a few environment variables to reflect the settings in SDKShell.bat (mentioned earlier) except for “PYTHONPATH”.

Assume again you unzip the GDAL Python package at E:\programs\gdal\1.8.0, then you should add following

Path=<your_original_path_variable>;E:\programs\gdal\1.8.0\bin;E:\programs\gdal\1.8.0\bin\gdal\python\osgeo;E:\programs\gdal\1.8.0\bin\proj\apps;E:\programs\gdal\1.8.0\bin\gdal\apps;E:\programs\gdal\1.8.0\bin\curl;
GDAL_DATA=E:\programs\gdal\1.8.0\bin\gdal-data
GDAL_DRIVER_PATH=E:\programs\gdal\1.8.0\bin\gdal\plugins
PROJ_LIB=E:\programs\gdal\1.8.0\bin\proj\SHARE

Note: these environment setting only applies within your Pydev IDE when you choose to use this particular Python interpreter, and it doesn’t affect your existing system environment settings.

Finally save the changes and you are good to go. To verify it, just launch a Python console window in Pydev, and execute following:

from osgeo import gdal

from osgeo import ogr

from osgeo import osr

They should just execute fine without any error message.