August Town

Back from ESRI User Conference 2011

2011-07-17T15:21:00.001-07:00

Finally done with the work and presentations of user conference this year, and I’m very glad to hear some positive feedback of my demo theatre presentation (“Leverage OGC capabilities of ArcGIS Server using Open Source GIS technology”) and technical workshop (“Leverage OGC capabilities of ArcGIS Server”) from users and colleagues. Now it’s time for a little break, then keep working and also look forward to the FOSS4G conference in Denver.

Google Earth Builder is really coming

2011-06-20T11:46:00.001-07:00

Just as many of others, I heard the announcement of Google Earth Builder (GEB) and also watched the informative GEB webinar on Directions Magazine very recently, so Google’s Enterprise Cloud GIS is really coming (Q3, 2011 as mentioned in the webinar).

Here are some of the the highlights I perceived from the the online webinar:

1. Google Earth Builder will have similar functionalities as other existing Cloud GIS solutions, e.g. ArcGIS.com, GeoCommons, SimpleGeo, GloudGIS etc. So basically user can upload and manage their own dataset, style the dataset and mash them up with Google’s basemap layers as well as dataset shared by others, and finally share your mash up maps with others. But it doesn’t mean Google Earth Builder is just another Cloud GIS service at all, a few thing I notice will potential make it outstand.

Supports both vector and raster dataset in their native projections;

Leverage the base map layers and data from Google Maps and Google Earth API;

Google’s superior and reliable cloud infrastructure;

Same user experience of authentication and search as other Google products and services.

2. Support for OGC output

KML is obviously supported and it’s now an OGC standard, but that is just a special case. Now WMS and WMTS are also mentioned to be supported soon which will make it possible for much more non-Google to consume data and map from Google Earth Builder. One thing great to have though is to allow users to upload/import data from various types of OGC services e.g. Web Coverage Service and Web Feature Service.

Things that are missing or unknown to me from the webinar though are:

1. Unknown about Metadata

OGC/ISO 19115/19139 mentioned in Q&A section but I don’t think it will ever be supported.

2. You won’t be able to directly access the data from Google Maps and Google Earth

Finally when Google comes into an area, there is always question regarding whether Google will killer everybody in the same area or not. I don’t know about that but I feel if you’re doing the same thing as Google does, then you’d better be more creative and do it a little better.

Cloud GIS, it’s all about sharing the map

2011-06-13T22:35:00.001-07:00

Cloud GIS is obviously NOT just about sharing the map, but it is just cool to share a beautiful and useful map with others through blog, Facebook or Twitter.

Google Maps

View Trip to Washington D.C. in a larger map

ArcGIS.com

View Larger Map

Geocommons

GISCloud

Mashup WMS in GeoCommons

2011-06-01T23:54:00.001-07:00

WMS, whether dying or not, still has its place in the world of GIS. Given that many organizations have published their spatial data through WMS, it is a very important data source especially in the era of cloud GIS today. GeoCommon’s is one of the first cloud GIS platform that allows users to add WMS as one type of data sources which can mashup on the base map. Follow the blog post “Adding WMS and Tiles in GeoCommons and GeoIQ” I gave it a try using one public WMS called “StatesCitiesRivers_USA” from ArcGIS Online sample server and another from OSGeo.

After log in, select “Make a Map” and pick whatever base map as you like, which in my case is OSM.

Click “+Add” at top-right corner, and select “Upload” tab and “Add a URL Link” tab, and finally select “Web Map Service” in the format dropdown list, and type in the url to WMS’s capabilities files (the url must point to the capabilities XML file either static or dynamic).

Since ArcGIS Server supports WMS 1.3.0, so I tried this highest version of capabilities file, but unfortunate it doesn’t display the WMS layers on top of base layer although it is able to get the list of layers from that WMS.

Next I switched to version 1.1.1, which actually worked for me.

For the WMS overlay in GeoCommons, users are able to turn and off individual or all layers, and also change the opacity of the whole WMS overlay which obviously happens at client side (because WMS itself doesn’t support opacity). But none of other style or filter options are available to WMS layer. What is also observed is that WMS map is requested from server as multiple tiles instead of a single map image, which may be because GeoCommons platform currently doesn’t support any dynamic mapping service like WMS. Unless those tiles from WMS are cached at client-side, it’s not quite efficient.

One big problem I see, which is also quite obvious from the screenshot , is that GeoCommons requests WMS map in EPSG:4326 even though the base map is Web Mercator (EPSG:3857 or EPSG":900913 for most open source GIS community). This is caused by the fact that the public ArcGIS Server WMS service doesn’t list EPSG:900913 as one of the supported SRS.

I then added the public WMS from OSGeo, which supports EPSG:900913, which is side proof of what happened to ArcGIS Online Sample Server.

Since ArcGIS Server WMS doesn’t support EPSG:900913 (not in the way when it’s named as 900913) at this point, the only workaround is to choose one of the base map marked as (4326).

Now it overlays perfectly.

Conclusion

There are still a lot of data source published as WMS, and in my opinion there will only be more in the near future. So supporting WMS will give huge benefit to today’s Cloud GIS providers. It will be really nice to see similar WMS support in other major Cloud GIS player like ArcGIS.com, Google Maps and so on.

World-wide routing, MapQuest + OpenStreetMap

2011-05-31T17:11:00.001-07:00

MapQuest was THE first and only web routing service I relied on. But once up on a time, I almost forgot its existence because of Google Maps, Yahoo, and even others. Recently MapQuest seems to be coming back really strong especially with its Open Initiative and integration with OpenStreetMap data. One of the most amazing things I have seen is this Dublin-to-Shanghai world-wide routing:

Although I don’t think it has much practical use to most people except for professional backpacker traveler, it is still amazing and seriously no one else can do it.

Choosing China’s location check-in platform

2011-05-27T11:00:00.001-07:00

Like many other Chinese people (I am not talking about the those who were born or grown up here in States) here in Unite States, I’m facing a dilemma when choosing a location check-in game service to share location, pictures and tips with my friends. There seems to be quite a lot options. We have the popular Foursquares, Gowalla, and Facebook, and Google Latitude which jumped in recently, plus Yelp, Booyah, MyTown, and the list can go on and on. However, the problem is that many of my friends are in mainland China who want to see where I have been to and how I felt about those places, but they probably don’t want to install any of those clients on their mobile devices (actually even if they do, they can’t because many social networks applications are blocked in China). For the same reason I can’t see their move and tips either because obviously they don’t share it through any of the location check-in platform above.

The only workaround seems to be choosing one of China’s location check-in platform. Fortunately there is also a long list of those to choose, but none of them made a perfect solution. My top candidate list includes: Jiepang (街旁), Sina (微领地), Sifang (四方), and kaikai (开开). Which one to choose is certainly subjective to each individual person, but here is a list of things (ordered by priority) that I will base my choice on.

1. Size and accuracy of the existing foreign (U.S in this case) venue database;
2. Share check-in with friends on other check-in platform or social networks (Facebooks, Foursquares, Kaixin001, Sina Weibo etc);
3. Being able to add new venue.

To me both #1 and #2 are must for a check-in platform be to usable, and #3 is nice to have but also important.
Venue database

This almost dominates my decision. Sina has its decisive advantage over others in its complete and accurate U.S venue database, which even overcomes its shortage in all other aspects. Just as an example, I live in Rancho Cucamonga, a small size city in southern California, which is not famous and probably unknown to most people in China. No one would expect a Chinese location check-in service to have much venues in this area in its databases. But I was really surprised the first time when I checked in at home, Sina accurately lists closest everything nearby which include a KFC, a RiteAid pharmacy, a Ralphs grocery store, a US Bank (a small local bank I’ve been to), …, and a lot of other small places. None of others have a similar or even close size and accuracy of the venue database as Sina does.

Sharing check-in with others

This is the second most important thing because first I obviously want my check-in to be shared with my friends in other check-in platforms or social networks, and second I obviously don’t want to repeat my check-in on every check-in platform either for my friends in U.S or in China. This seems to be possible in two aspects: either (a) I check-in once on one of them (e.g. Sina) and my check-in will spread out through synchronization to Facebook, Foursquare, Twitter, Kaixin (开心网), Weibo (新浪微博) an so on; or (b) I check in through a 3rd party proxy app which takes my check in and broadcast to all my location check in services. Option #b is only possible when everyone has a free open API for venues and check-in which is NOT true for any of the location check-in platform in China, so it is ruled out. Option a is possible in one direction (China check-in spread to U.S platform), and almost all Chinese location check-in services provide you such capability except for Sina.

Adding new venues

This is also very important to both users and location check-in service providers. I am true believer of the fact that a real complete and accurate venue database can only be evolved from volunteered geographic information from those locals who know where they live well. ~~Unfortunately Sina again doesn’t support this feature while all other check-in platforms supports it which is quite disappointing to me.~~ As latest version of Sina (微领地) goes, users are allowed to create new venue and submit to service provide for reviewing and I guess if it is approved then it will be added into existing venue database. This is a big leap to me and makes Sina (微领地) outstands other options in my particular use case.

Conclusion

No one wins because neither Sina or any other candidates has both #1 and #2, but it provokes some interesting thinking on the current situation in the location check-in services market in China. Sina is close because it is the only one who is either interested in or can afford building a relatively complete and accurate foreign venue database. It is also easy to prove that those venues in sina’s database are not coming from any major open free sources (e.g. OpenStreetMap or Foursquare API etc.) So it’s probably from a commercial provider, which in turns logically explains why there are restrictions on adding new venues or modifying existing ones, and why check-in on vld.sina cannot be spread out to other platforms. Other location check-in platforms in China are still missing a solid venue database to start with, and importing from OpenStreetMap database or relying on the venue API from a popular U.S provider like Foursquares or so seem to be possible solutions.

Thoughts on OSM and the Bing automatic vector detect service

2011-02-09T13:37:00.001-08:00

In Mike Dobson’s recent blog post OSM vs. the Mechanical Turk – A New Option For Mappers?, he brought a point, which I think is more than reasonable:

“…I guess that when I thought about the term Volunteered Geographic Information, I made the mental leap that these volunteers were providing content in the form of spatial information reflecting the geographic areas in which they lived or with which they were more than casually familiar. It has now occurred to me that there may not always be a direct beneficial relationship between geographical knowledge and Volunteered Geographic Information…”

And then he did an analogous comparison between OSM and Mechanical Turk, in which he points out that:

“…If the majority of OSM contributors to the UK database are spending their time digitizing imagery for the UK portion of the OSM database, as opposed to contributing GPS traces and attributes from paths along which they have traveled or know something about, how likely is it that the OSM effort in the UK benefits from local knowledge to the same extent that it benefits from “free” digitizing?…”

to him, the way how some people contribute to OSM database is similar to a modern version of Mechanical Turk. It looks like the data are contributed with valuable local geographical knowledge, but it is only artificially generated from aerial imagery.

Now this reminds me of another exciting post I came across from Bing, in which Steve Coast announced the experimental service from Bing which can automatically derive street vector data from Bing aerial imagery. I am pretty sure it will boost the popularity of OSM in short term, but in the long run is it really helping?

Create your public WMS through www.GISCloud.com

2011-02-04T16:38:00.001-08:00

More and more cloud based GIS infrastructures are popping up these days. GISCloud is just one of those many and probably not the most famous one. But it does have some interesting features that attracts me. Today, with its free account, I was able to quickly publish a public WMS (OGC Web Map Service) simply with a few clicks. Although the service seems a little buggy in it beta but the features are promising.

To get started, simply go to http://www.giscloud.com/ and then sign up for a free account. After verification log in you will be able to things like upload/manage/share your dataset, create mashup maps with built-in layers (OSM, Google, etc.), your own dataset or public dataset shared by others, and of course your maps can be shared to other through different flavors. These are quite standard across different cloud GIS platforms nowadays.

The aspect I like most of www.GISCloud.com is the fact that it has a broad list of supported input/export formats, which by the way is very open source friendly. Especially the claim of “all OGC and GDAL formats”, which sounds quite powerful.

The particular feature I tried is simply publishing a world boundary layer as a WMS. The user experience is quite smooth, since the world boundary is a default built-in layer, so I end up having a public WMS in literally just a few seconds.

Select “world” map on left, click Project->Share and Publish

In “Share and Publish” dialog, go to WMS tab and click “Enable”, and below that is the url endpoint of your public WMS.

The next thing is simply consume it in a free 3rd party client like Gaia

This is really fantastic start, and I really like it.

A few things to improve though:

None of the built-in tiles layers (OSM, Google) come through as WMS, which I don’t think is technical issue but probably licensing issues;
The world I configure in website seems to be in Mercator projection but the output WMS only claims EPSG:4326 in capabilities files;
Still a little unstable when I enable/disable WMS back and forth with/without tiled layer like OSM, need to refresh the page to get rid of it.

Configure GDAL/OGR Python debug environment in Pydev on Windows

2011-02-03T13:07:00.001-08:00

Recently I am involved in a project that requires some Geospatial development in Python. So I decide to take this chance to get familiar with using GDAL/OGR in Python because it is the most popular open source GIS library plus it has a python binding.

The first thing you need is obviously a Python development environment for GDAL/OGR. I know IDLE or even a plain text editor plus “print …” will do the job just fine, but I still prefer a decent IDE. Why not? Especially when it is free. So I choose the Eclipse based Python IDE Pydev. The only problem is that the process of configuring it with GDAL/OGR on Windows isn’t that obvious, or at least to me it is not. So in this post I will go through it step by step.

1. Install Python

Installing Python is quite straight forward, just go to http://www.python.org/, pick up the installer for your version and OS, and install it. I would recommend Python 2.6 or 2.7 because there are pre-compiled GDAL/OGR python binding for both versions.

2. Install GDAL/OGR Python binding

First of all, this website is really helpful for you to get pre-compiled GDAL/OGR Python binding. It has different stable released version and the version latest trunk, and the best part is that it also has the installer version and non-installer (zipped version) versions for both 32bit and 64bit.

Use installer version

Find the .exe or .msi installer file, which usually comes with name GDAL-1.x.0-win32-py2.x.msi () e.g. here is the download link to the one for GDAL 1.8.0 32bit for python 2.7 (GDAL-1.8.0.win32-py2.7.msi). So basically just follow the installation wizards to finish the installation. Under the hood, my understanding is that the installer just simply copy a bunch of files and folder into \Lib\site-packages folder of your Python install location.

Note: the installer version requires you to have an appropriate version of Python installed and registered in registry first. The normal installation of Python takes care of the registry, but if not (e.g. you want to use the Python that comes with ArcGIS Server), then you will have to manually install a standard version of Python at a different location, install the GDAL Python binding, then copy those GDAL related files and folders back to the site-packages folder of that unregistered Python.

Use non-installer version

Instead of using installer version, you can choose to use a non-installer version which avoid adding stuff upon your vanilla Python install and registry. So from the same website just download those zip files with name e.g. release-1500-x64-gdal-mapserver.zip (don’t worry it comes with a copy of UNM MapServer), and unzip it on your local disk. The “information” link gives you very detailed information about the folder structure and versions of packages included.

After unzipping, I highly recommend you to take a look at the SDKShell.bat, which usually contains code similar to below:

:setenv2
SET PATH=%CD%\bin;%CD%\bin;%CD%\bin\gdal\python\osgeo;%CD%\bin\proj\apps;%CD%\bin\gdal\apps;%CD%\bin\ms\apps;%CD%\bin\gdal\csharp;%CD%\bin\ms\csharp;%CD%\bin\curl;%PATH%
SET GDAL_DATA=%CD%\bin\gdal-data
SET GDAL_DRIVER_PATH=%CD%\bin\gdal\plugins
SET PYTHONPATH=%CD%\bin\gdal\python\osgeo
SET PROJ_LIB=%CD%\bin\proj\SHARE

This basically tells you what environment variables and paths must be included in PATH system environment variable, which we need when later configuring Pydev.

3. Install Pydev Python IDE

Install Pydev IDE is also easy and straightforward. Either download it from sourceforge, or update it through Eclipse by adding http://pydev.org/updates in software update sites.

After installation, just launch it.

4. Configure GDAL/OGR Python debug environment in Pydev

This is the most trick part. Depend on whether you’re using the installer version or the non-installer version, the steps are different.

First of all, you need to specify the Python interpreter in Pydev. So just go to Windows->Preference. and navigate to “Pydev”

Click “New”, type in a name and specify the path to python.exe of your Python installation, and finally click ok.

If you install GDAL Python binding with installer, then you should be good to go.

If you install GDAL Python binding with non-installer zip file, then you will have to configure a few more things. Still in the “Python Interpreters” configure dialog box, at the bottom part, select “libraries” tab and click “New Folder” on the right, navigate and select the <GDAL_Python_Install>\bin\gdal\python, finally click ok.

Assume you unzip the GDAL Python package at E:\programs\gdal\1.8.0, then you should add E:\programs\gdal\1.8.0\bin\gdal\python to the list of libraries.

Next go to “Environment” table, and there you need to add a few environment variables to reflect the settings in SDKShell.bat (mentioned earlier) except for “PYTHONPATH”.

Assume again you unzip the GDAL Python package at E:\programs\gdal\1.8.0, then you should add following

Path=<your_original_path_variable>;E:\programs\gdal\1.8.0\bin;E:\programs\gdal\1.8.0\bin\gdal\python\osgeo;E:\programs\gdal\1.8.0\bin\proj\apps;E:\programs\gdal\1.8.0\bin\gdal\apps;E:\programs\gdal\1.8.0\bin\curl;
GDAL_DATA=E:\programs\gdal\1.8.0\bin\gdal-data
GDAL_DRIVER_PATH=E:\programs\gdal\1.8.0\bin\gdal\plugins
PROJ_LIB=E:\programs\gdal\1.8.0\bin\proj\SHARE

Note: these environment setting only applies within your Pydev IDE when you choose to use this particular Python interpreter, and it doesn’t affect your existing system environment settings.

Finally save the changes and you are good to go. To verify it, just launch a Python console window in Pydev, and execute following:

from osgeo import gdal

from osgeo import ogr

from osgeo import osr

They should just execute fine without any error message.

Retire my T-Mobile G1

2011-01-09T23:15:00.001-08:00

I know this is not quite relevant to what I usually blogged, and I am also pretty sure that there are a lot better up-to-date articles on Internet which are related to the same topic. But I just feel I must write it down because it took me almost a day and half to finally retire my T-Mobile G1 phone, by that I mean two things:

Flash a custom Android 2.2 Froyo ROM on G1
Activate G1 without T-Mobile data plan

Here are detailed steps that worked for me, and the text are mostly copied and pasted from following two posts:

http://androidandme.com/2009/08/news/how-to-root-a-t-mobile-g1-and-mytouch-3g-android-phone/

http://istunner.com/?p=104

Downgrade to RC29

Format your phone’s microSD card to FAT32 mode
Download and unzip the RC29 or RC7 image file. Copy the DREAMIMG.nbh file to the SD card. (RC29 for US, RC7 is for UK)
Turn the device power off.
Hold Camera button, and press Power button to entry bootloader mode. You should see a gray/white screen with instructions to flash your phone with the update on your SD card. If you don’t see that, make sure you followed the instructions properly.
As per the on-screen instructions, press the Power button to start upgrade procedure. DO NOT DO ANYTHING TO INTERRUPT THIS PROCESS.
After it is finished, perform the restart your phone by pressing the Action key (press down on the scroll ball).

Root the phone - Automatically replace your recovery image with FlashRec

The purpose of this method is to use a bluetooth security exploit that existed in Android build CRC1 or lower. It downloads and installs the Cyanogen recovery image to your phone. Using this method is preferred because it does not overwrite your radio and SPL file like the manual install method. Future updates to Android are likely to block this hack, but you should always be able to revert to the official CRC1 build and gain access using this method.

1. Download and install the 1-click root app flashrec-20090815.apk

First we need to download the application file that was created by Zinx. This file is named flashrec-20090815.apk and can be located at Zen Thought or RyeBlog. I suggest downloading the file on your PC to avoid corruption. Copy the app to your SD card and install it using your favorite file manager like Astro or AppManager.

2. Launch the FlashRec application and backup your current recovery image

Before you can flash a new recovery image, you must backup your original once first. Press the large button that says “Backup Recovery Image”

Step 1. Press Backup Recovery Image

3. Flash the new Cyanogen recovery image

After backing up your recovery image, you should see a message that says “Backed up”. Now you should see the button “Flash Cyanogen Recovery 1.4″ light up. Click the button to perform the flash and do not touch your phone until you see “Flashed new recovery image.” When you see the flashed image below it is ok to reboot your phone and check that the new recovery image loaded.

Easiest root method to date.

Stop and Verify: Reboot your phone with the steps below to see if the new recovery image is loading correctly.

Power off your phone.
Press and hold the Home key and then power the phone back on.
You should be greeted by the new Cyanogen recovery image screen.

Cyanogen recovery image v1.4

Now that you have the Cyanogen recovery image, you can begin flashing custom builds of Android. Jump to the bottom for Part 2: Install a custom build of Android.

Upgrade the Radio Image to “ota-radio-2_22_19_26I.zip”

Since I downgraded G1 phone to RC29 using the DREAIMG.nbh method, the radio image was downgraded as well. So I have to update to the latest Radio Image so you can load the latest roms and flash a new SPL file.

1. The new radio image has been posted on HTC’s support website, Android Roms page, or Haykuro’s Sapphire Port page. In order to apply the Radio update, we must download the file “ota-radio-2_22_19_26I.zip”.

2. After the file is downloaded, it must be placed in the root directory of your SD card. This means to place it in the main directory and not inside any folder. When the radio image file is placed on your SD card perform the following steps:

Power off your phone.

Hold down the Home key and power the phone back on.

Select apply any zip from sd or press Alt+A

When prompted, hit Home to write the image file.

When the radio image is finished updating, it should reboot your phone to the recovery screen. You can select reboot system now or press Home+Back .

Stop and Verify: You can check the version of your radio image to make sure it updated. Go to Settings > About Phone > Baseband version. The version number should end in 2.22.19.26l just like the original name of the update file.

Flash the SPL

Flashing a new secondary program loader(SPL) will rewrite your internal partition tables and free up more space on your phone. This new SPL is required to load certain roms. You must have the latest radio installed before flashing the new SPL. If you do not have the latest radio loaded, flashing the new SPL can prevent your phone from booting.

Once again, this new SPL is only for:

Phone: US T-Mobile G1
Radio: 2.22.19.26I
Recovery Image: Recovery Image that allows you to flash custom roms (Cyanogen)

Flashing a new SPL will rewrite all your partition tables and will delete all data from your phone. The main reason people flash a new SPL is because it increases your system partition from 70 MB to 90 MB.

Download the latest SPL(1.33.2005) from Haykuro’s Sapphire-port page. It must be 1.33.2005!!!
Copy the SPL file(spl-signed.zip) to your SD card.
Reboot your phone into recovery mode. (Power off phone. Press and hold Home key then press power)
Wait for Cyanogen’s recovery image to load and select “apply any zip from sd”. Flash the spl-signed.zip file.
After flashing the SPL, you must flash a custom rom or your phone will not boot.

Flash Cyanogen Android 2.2 Custom ROM

Now that the hard parts are over, we are ready to install a custom build of Android. There are many to choose from and I will let you decide what to flash. For daily use, I suggest using the latest stable release from Cyanogen. For a complete list of ROM’s, visit the Android ROM Build Database.

All rom files come in zip format. Download the rom of your choice and place it in the root directory of you SD card.

Power off your phone.
Boot into recovery mode. (Press and hold the Home key, then hit the power button).
Before you flash a rom file, perform a wipe. Press Alt+W to wipe the data and cache folders. You must wipe when going form different builds of Android.
Wait for the wipe to finish and the recovery image to display again, then select “apply any zip from sd”. Flash the zip file of your choice.
After flashing any zip you should be able to reboot your system and watch it load to the home screen.

After changing Android builds, it came take several minutes for the first boot. If you phone hangs on the animated Android logo, this is because you did not perform a wipe. If you are stuck in a boot loop, return to the recovery image and make sure you wipe.

Activate wireless on the phone without data plan (only with a dead T-Mobile sim card)

Note: You must have latest Android SDK, adb tools and Android USB driver correctly installed first, so that your phone can be debugged through adb tools on your conputer.

After the restart, you will see the “android” icon telling you to “touch” it. I did NOT touch it. I just touch an “blank” area so that the “emergency” call is not highlighted anymore. (you can test to see if you have root by typing on the PHONE this: <enter>reboot<enter>)
[<enter> means press the enter key]
Plug the phone to you computer.
On the PHONE, type the following:
- <enter>setprop persist.service.adb.enable 1<enter>
- The computer will see a new device and try to install it. On the hardware setup, tell it to get the driver from where you unziped the Android USB Driver. This will enable the phone to “really” connect with the computer and mount the SD. If you don’t do this, the following steps will NOT work.
- IF it didn’t work, restart by holding down the power button and turning off and on again. And do the <enter>reboot<enter> test. If the test did not work, you did something wrong. If it the test did work, it means you press Something! DO NOT PRESS ANY KEYS.
You should be connected with the computer now. On the COMPUTER:
- Open command prompt (start>run>cmd)
- Type (all that are in bold):
  - cd C:\android-sdk-windows-1.1_r1 (assuming that you unziped to this location)
  - adb shell (This should give you “#” on the next line. If you get an error message “No device” or something, it means that your COMPUTER is not connected to the PHONE)
  - am start -a android.intent.action.MAIN -n com.android.settings/.Settings (this will force the “Setting” menu to popup on your PHONE)
Now on you PHONE, you should see the Settings menu. Activate the wifi setting and connect to your wifi.
Once you have wifi connection, you can sign into your Google account.

Using MapQuest OSM Tiles in OpenLayers

2011-01-04T22:58:00.001-08:00

With the recent launch of Open.Mapquest U.S, people now have one more way to view and contribute to OpenStreetMap data. Among those cool things of new Open Mapquest initiative, my favorite is the look and feel of those tiles generated from OSM data:

In my own opinion, it has much better looking than the original OSM Mapnik or Osmarender style. Since the structure of Open.Mapquest tiles are exactly the same as OSM tiles except for the base url, it can not be easier to use them as base layer in an OpenLayers application. Here is how to do it:

1: ...
2: // in place where you use to add OSM layer 
3: // var osm = new OpenLayers.Layer.OSM();
4: // map.addLayers([osm]);
5: 
6: // create a new class for Open.Mapquest tiles
7: OpenLayers.Layer.MapQuestOSM = OpenLayers.Class(OpenLayers.Layer.XYZ, {
8:     name: "MapQuestOSM",
9:     //attribution: "Data CC-By-SA by <a href='http://openstreetmap.org/'>OpenStreetMap</a>",
10:     sphericalMercator: true,
11:     url: ' http://otile1.mqcdn.com/tiles/1.0.0/osm/${z}/${x}/${y}.png',
12:     clone: function(obj) {
13:         if (obj == null) {
14:             obj = new OpenLayers.Layer.OSM(
15:             this.name, this.url, this.getOptions());
16:         }
17:         obj = OpenLayers.Layer.XYZ.prototype.clone.apply(this, [obj]);
18:         return obj;
19:     },
20:     CLASS_NAME: "OpenLayers.Layer.MapQuestOSM"
21: });
22:     
23: var mapquestosm = new OpenLayers.Layer.MapQuestOSM();
24: // map.addLayers([mapquestosm]);
25: ...

This is a screen shot of how Open.Mapquest tiles look like in OpenLayers app

Cool feature called “Destination Maps” from Bing Maps

2010-12-20T00:49:00.001-08:00

Recently I found this cool feature called “Destination Maps” from Bing Maps, which basically allows you to pick up a location, adjust an extent and then create a map out of it. The real cool thing though is the “sketchy” and “treasure” map style. BTW you can export the final map as PDF or JPEG format in different size. Here are the two maps I created:

Looks good, huh? It makes a good poster.

Hack OpenGeo Styler to work with generic WMS servers

2010-11-20T15:37:00.001-08:00

As part of the OpenGeo Suite, Styler is a web based WMS viewer plus SLD editor built upon OpenLayers and GeoExt. Although it uses WMS and SLD protocols, current release of Styler must be coupled with a running GeoServer instance. So it is impossible for Styler to work with a generic WMS server due to the following issues:

WMS layers must come from GeoServer instance on the same machine as “/geoserver/wms?”
Every WMS layer must have has a WFS feature type at back end which must come from “/geoserver/wfs?”
There is no where to configure proxy so it doesn’t work with a remote server
Instead of WMS GetStyle, Styler is using GeoServer REST API (not a standard) to load SLD from WMS
Instead of WMS PutStyle, Styler is using GeoServer REST API (not a standard) to push and persist styles back to WMS
Styler is not using either standard WMS parameter “SLD” or “SLD_BODY” to make stateless change of map styles
Styler requires the WMS to support DescribeLayer interface

By briefly looking through its source code, I found it’s not too hard to overcome those issues and make Styler talk to a generic WMS server (e.g. ArcGIS Server) as long as the target WMS meets following prerequisites:

WMS must support EPSG:900913 (EPSG:3857 or Esri’s EPSG:102110 is ok but requires a little tweak in OpenLayers)
WMS must support GetStyle interface
Each layer in WMS must have a WFS feature type at back end in order to get attributes list
Each style of a WMS layer must have a SLD definition which can be retrieved through GetStyle
Optionally WMS needs to support PutStyle interface if one wants to persist SLD styles at server side
Optionally WMS needs to support SLD_BODY to make stateless change of rendering and symbology

So in this article, I’m going to go through the basic steps to tweak the OpenGeo Styler to communicate with a non-GeoServer WMS:

1. Checkout Styler source code from http://svn.opengeo.org/suite/trunk/styler

2. Build Styler

sudo easy_install jstools

mkdir script

jsbuild build.cfg -o script

After a successful build there should be 4 js files in script folder.
Note: I only found out how I can build it on Linux, and I didn’t try on Windows

3. Setup debug environment for Styler
To launch Styler application simply follow the instructions here. By default, the index.html references merged and compressed js file in “script” folder, in order to modify and debug Styler you will need to reference those uncompressed js files in index.html as below:

1: ...
2: <script src="../styler/externals/openlayers/lib/OpenLayers.js"></script>
3: <script src="../styler/externals/ext/adapter/ext/ext-base.js"></script>
4: <script src="../styler/externals/ext/ext-all.js"></script>
5: <script src="../styler/externals/geoext/lib/GeoExt.js"></script>
6: <script src="../styler/script/gxp.js"></script>
7: <script src="../styler/lib/Styler.js"></script>
8: <script src="../styler/lib/Styler/dispatch.js"></script>
9: <script src="../styler/lib/Styler/ColorManager.js"></script>        
10: <script src="../styler/lib/Styler/SchemaManager.js"></script>
11: <script src="../styler/lib/Styler/SLDManager.js"></script>      
12: <script src="../styler/externals/ux/colorpicker/color-picker.ux.js"></script>
13: ...

Now you should be able to debug into OpenLayers and Styler in browser.

4. Hack the code
You don’t actually need to touch any code in OpenLayers or GeoExt, all changes are within three .js files in Styler itself, which are:

\lib\Styler.js
\lib\Styler\SLDManager.js
\lib\Styler\SchemaManager.js

Code changes in those files are explained in details below.

Code Changes in Styler.js

a. Add three properties for class Styler

1: // right before the “constructor”, and after other properties
2: ...
3: wmsUrl: "/geoserver/wms?",
4: wfsUrl: "/geoserver/wfs?",  
5: proxyUrl: "",
6: ...

b. Within Styler.js replace “/geoserver/wms?” or “/geoserver/ows?” with “this.proxyUrl + this.wmsUrl”, and replace “/geoserver/wfs?” with “this.proxyUrl + this.wfsUrl”. This is to allow user to point Styler to any WMS, either remote or local.

Note: there is one exception though, in method createLayers() you just need to replace “/geoserver/wms?” with this.wmsUrl because WMS layer doesn’t need a proxy to request map image.

c. Avoid using WMS DescribeLayer

1: //locate method:
2: describeLayers: function(callback) {
3:   ...
4: }
5: //Comment out original code and add following chunk of code
6: describeLayers: function(callback) {
7: for (var i=0, ii=this.wmsLayerList.length; i<ii; ++i) {
8:                   config = this.wmsLayerList[i];
9:                   // assume each WMS layer has a backend WFS layer
10:                   this.layerList.push(config);
11:             }
12:             callback();
13:   }
14: //here assumption is that every WMS layer has a WFS feature type at back end

d. If your WMS supports Google Mercator projection as something other than “EPSG:900913”, In method createLayers() you will need to add an extra parameter (e.g. “srs: ‘EPSG:3857’”) when creating each WMS layer

1: createLayers: function() {
2: ...
3:     layers.push(
4:     new OpenLayers.Layer.WMS(
5:         config.title, this.wmsUrl, {
6:         layers: config.name,
7:         styles: config.styles[0].name,
8:         transparent: true,
9:         format: "image/png",
10:         srs: "EPSG:3857"  // EPSG:3857, EPSG:102110 or EPSG:102113 etc.
11:         }, {
12:     }));
13: ...
14: }

e. Pass proxyUrl, wmsUrl and wfsUrl into SLDManager and SchemaManager

1: getSchemas: function(callback) {
2:             this.schemaManager = new Styler.SchemaManager(
3:             this.map,
4:             {
5:               proxyUrl: this.proxyUrl,
6:               wfsUrl: this.wfsUrl
7:             }
8:           );
9: 
10:   getStyles: function(callback) {
11:         this.sldManager = new Styler.SLDManager(this.map, {'proxyUrl': this.proxyUrl});
12:             this.sldManager.loadAll(callback);
13:       }
14: 
15:   initEditor: function() { {
16:   …
17:  this.sldManager = new Styler.SLDManager(this.map, {'proxyUrl': this.proxyUrl});
18:   ...
19: }
20: ...
21: 
22: // Note: the original constructors for SLDManager and SchemaManager takes a single input parameter
23: // but they will be modified to take 2nd input parameters in SLDManager.js and SchemaManager.js
24:

Changes in SLDManager.js

a. Add an extra parameter called “proxyUrl” right before the constructor

1: // right before the “constructor”, and after other properties
2: ...
3: proxyUrl: "",
4: ...

b. Modify SLDManager constructor to take 2nd input parameter

1: initialize: function(map, options) {
2:     this.map = map;
3:     var layer;
4:     this.layers = [];
5:     this.layerData = {};
6:     this.format = new OpenLayers.Format.SLD({multipleSymbolizers: true});
7:     for (var i=0; i<this.map.layers.length; ++i) {
8:         layer = this.map.layers[i];
9:         if(layer instanceof OpenLayers.Layer.WMS) {
10:             this.layers.push(layer);
11:         }
12:     }
13:     // this is important so user can pass in ‘proxyUrl’
14:     OpenLayers.Util.extend(this, options);  
15: },

c. Overwrite getUrl() so it sends GetStyle request to retrieve WMS SLD defintion, here is an example

1: getUrl: function(layer, styleName) {
2:     var url;
3:     if(layer instanceof OpenLayers.Layer.WMS) {
4:         url = layer.url.split("?")[0] + "?" + "version=1.3.0&request=GetStyles&layers=" + layer.params["LAYERS"];       
5:         OpenLayers.Console.log(url);
6:     }
7:     //TODO handle other layer types
8:     return url;
9: }
10:

d. In loadSld() method replace this.getUrl(layer, styleName) with this.proxyUrl + this.getUrl(layer, styleName)

1: ...
2: loadSld: function(layer, styleName, callback) {
3:         Ext.Ajax.request({
4:             url: this.proxyUrl + this.getUrl(layer, styleName),
5:             method: "GET",
6:             …              
7: ...
8: // obviously there may be more than one styles for each WMS layer, 
9: // in this case the first style is always used.
10:

e. In saveSld() method, replace original code with following to use “SLD_BODY” parameter to make stateless change of rendering and symbology

1: saveSld: function(layer, callback, scope) {
2:     this.layerData[layer.id].style.name = "__internalstyle__";
3:         var sldBody = this.format.write(this.layerData[layer.id].sld, {});
4:     layer.mergeNewParams({
5:       styles: "__internalstyle__", 
6:       sld_body: sldBody
7:     });
8:     callback.call(scope || this);  
9:   }
10: // By using “SLD_BODY” any change to the style is stateless which will not affect other users. 
11: // But if you want to persist modified styles on server side, 
12: // you must use WMS PutStyle operation and also make sure server supports it
13: 
14:

Changes in SchemaManager.js

a. Add two extra parameters called “wfsUrl” and “proxyUrl” right before the constructor

1: // right before the “constructor”, and after other properties
2: ...
3: proxyUrl: "",
4: wfsUrl: “”,
5: ...

b. Modify SchemaManager constructor to take 2nd input parameter

1: initialize: function(map, options) {
2:             // other code
3: ...
4:             OpenLayers.Util.extend(this, options);
5:        ...     
6:             for(var i=0; i<this.map.layers.length; ++i) {
7:                   layer = this.map.layers[i];
8:                   if(layer instanceof OpenLayers.Layer.WMS) {
9:                         this.attributeStores[layer.id] = new GeoExt.data.AttributeStore({
10:                               //url: layer.url.split("?")[0].replace("/wms", "/wfs"),
11:                           // TODO: need better logic to handle case where this.wfsUrl is empty or must be derived from this.wmsUrl
12:                           url: this.proxyUrl + this.wfsUrl, 
13:                               baseParams: {
14:                                     //version: "1.1.1",    // is there a WFS 1.1.1?
15:                             version: "1.1.0",
16:                                     request: "DescribeFeatureType",
17:                                     typename: layer.params["LAYERS"]
18:                               }
19:                         });
20:                   }
21:             }
22:       }

5. Rebuild Styler and test against a generic WMS server.

So far the OpenGeo Styler has been hacked to communicate with a generic WMS server. To test it you just need to pass in the urls for WMS, WFS and proxy into Styler constructor in the original index.html like below.

1: Ext.BLANK_IMAGE_URL = "theme/img/blank.gif";
2: Ext.onReady(function() {
3:     window.styler = new Styler({
4:         baseLayers: [                              
5:             new OpenLayers.Layer.OSM(
6:                 "Open Street Map",
7:                 "http://tile.openstreetmap.org/${z}/${x}/${y}.png",
8:                 {numZoomLevels: 19}
9:             )                                                             
10:         ],                        
11:         wmsUrl:"http://localhost:8399/<WMSServer_endpoint>?",
12:         wfsUrl:"http://localhost:8399/<WFSServer_endpoint>?",
13:         proxyUrl: "http://locahost:8080/openlayers-trunk/ApacheProxyServlet?url="
14:     });
15: });

Some screenshots:

Styler load SLD style definition through WMS GetStyle operation

Styler changes map symbology through SLD_BODY parameter

6. Things to improve

If a layer has more than one styles, provide GUI for users to select
Make OpenGeo Styler to persist SLD styles on server through WMS PutStyle operation

Using USGS High Resolution Orthoimagery in JOSM

2010-10-21T00:12:00.001-07:00

One of CarbonCloud’s recent article pointed me to USGS Seamless Data Warehouse, where many high resolution orthoimagery data is distributed as OGC WMS. It’s a perfect base map option for OSM contributors using JOSM because of the following reasons:

The super image quality and resolution (0.25 foot to 2.5 foot);
Distributed through open standard based interface (WMS), which can be directly used in JOSM;
Compatibility with OSM license makes it legal to be used to derive OSM data on. (features digitized based on Google Maps and Bings are actually considered as derivative work of them, which then is copyrighted)

An obvious disadvantage though is the data coverage because only very limited area are covered by those orthoimagery data. See data availability.
Here are some basic steps to use USGS high resolution orthoimagery in JOSM:
1. Install JOSM and WMS plugin
2. Find the WMS containing the orthoimagery data that covers the area you’re interested in
Here is the list of available WMS services and you can probably tell the covered area by service name. For example, I am interested in making some OSM edits in Redlands, CA, United States, so I pick the WMS USGS_EDC_Ortho_California, in which there is particular layer for Riverside and San Bernardino counties.

3. Start JOSM and load the WMS layer in JOSM WMS plugin
After you download the data from OSM in JOSM, you need to register WMS USGS_EDC_Ortho_California in JOSM through WMS Plugin Preference. So just go to Edit—>Preferences—>WMS tab;

Click “Add”, and type in a name and url as part of a WMS GetMap request

It’s a little tricky for WMS URL because you need to append a few GetMap request parameters manually. It seems to me that JOSM WMS plugin will only automatically populate “BBOX” parameter so you must manually complete other request parameters. Here is the URL that I came up with:
http://imsortho.cr.usgs.gov/wmsconnector/com.esri.wms.Esrimap/USGS_EDC_Ortho_California?VERSION=1.1.1&REQUEST=GetMap&SRS=EPSG:4326&LAYERS=Riverside-SanBernardinoCA_0.3m_Color_Mar_2008&STYLES=&EXCEPTIONS=application/vnd.ogc.se_xml&FORMAT=image/png&BGCOLOR=0xFEFFFF&TRANSPARENT=TRUE&
Notice that I indicate the layer “Riverside-SanBernardinoCA_0.3m_Color_Mar_2008”, which is the area I am interested in.
Note: you should request WMS map in WGS84 (EPSG:4326) because the OSM data overlay on it are all in longitude and latitude.
That’s it, enjoy the armchair mapping!
Below is a snapshot of JOSM with high resolution orthoimagery as base map

Same area with Yahoo Imagery as a comparison

Is that amazing! Only a little issue of JOSM WMS plugin itself though: if you zoom in after loading the WMS image you need to delete and add the same WMS again to refresh data from WMS, otherwise the image at previous zoom level will simply be resample and used.

This OSM wiki page has a lot of useful information regarding using USGS High Resolution Orthoimagery in JOSM too.

IP-based geolocation can also be accurate

2010-10-13T00:45:00.001-07:00

I always thought that IP-based geolocation is not as accurate as it should be until quite recently I ran into this HTML5 demo page (Geolocation API is one of the new features in HTML5), and now I have to admit that it was my prejudice.
If you open the demo page in browser and assume you don’t have a GPS device connected to your machine, then all this web app does is to detect your location based on your IP address and add a maker on Google Maps. The source code is also as simple as a “HelloWorld” sample of W3C Geolocation API:

if (navigator.geolocation) {
navigator.geolocation.getCurrentPosition(success, error);
} else {
error('not supported');
}
function success(position) {
// ...non-related code omitted...
var latlng = new google.maps.LatLng(position.coords.latitude, position.coords.longitude);  
var map = new google.maps.Map(document.getElementById("mapcanvas"), myOptions);
var marker = new google.maps.Marker({
position: latlng, 
map: map, 
title:"You are here!"
});
}
function error(msg) {
// ...non-related code omitted...
}

I tried this geolocation demo app twice at my bedroom, one using Firefox 3.6 and the other is using Google Chrome, and below are how the app locates me:

Firefox 3.6 Successfully locate me in my land lot

Google Chrome is a little off my land lot, but still quite accurate

The results of both are very accurate to where I actually was. That’s just amazing! And I guess the conclusion is that IP-based Geolocation can also be really accurate.

NASA Earth Observation (NEO) Website is awesome

2010-08-19T22:19:00.001-07:00

I recently came cross this NEO (NASA Earth Observation) website that contains a lot of recent and historical satellite image data related to environment and climate change. Unlike most other websites of the same type, NEO’s simple and pretty web GUI design and user experience really surprises me.

Screenshot taken from NASA Earth Observation site at http://neo.sci.gsfc.nasa.gov/Search.html

The dataset hosted by NEO are divided into categories like ocean, atmosphere , energy, land and life. For each of those categories users can browse, search, preview, download, and analysis which is pretty much all you may want to do with satellite data. What’s really awesome is that you can do all of those things in a single ajax-based web page (you see only one screenshot is enough).

Discover data

To find the data, you can select one of the main categories, and then either search or look through the full list. Once it’s found you will be able to see the brief description and preview it in an interactive web map (support zoom in/out)

Download option

Most of the data can be downloaded as PNG, JPEG, GeoTiff, CSV, KML/KMZ directly through NEO home page, and whenever applicable you can also download data in original format e.g. HDF on the same page of data search result.

Web Map Service

NEO also provides WMS access to its data, which is a super plus for those who don't want to download and host data themselves. NEO’s WMS links:

http://neowms.sci.gsfc.nasa.gov/wms/wms?version=1.1.1&service=WMS&request=GetCapabilities

http://neowms.sci.gsfc.nasa.gov/wms/wms?version=1.3.0&service=WMS&request=GetCapabilities

Consume NEO WMS in QGIS

NASA Earth Observation site is just awesome!!!

Mysterious “BBOX” parameter in Web Feature Service (WFS)

2010-08-11T17:14:00.001-07:00

“BBOX” is a standard parameter defined as part of the OGC Web Feature Service (WFS) keyword-value encoding which is used in GetFeature requests to request features within a specific extent. So “…&BBOX=-180.0,-90.0,180.0,90.0…” means getting all the features in the extent defined by lower corner coordinate (-180.0,-90.0) and upper corner coordinate (180.0,90.0). Not as straight forward as it seems to be to most people, I actually find it quite confusing when trying to answer this question below:

Is it possible to set BBOX to coordinate values in other projection?

The answer to this question depends on whether it’s WFS 1.0.0 (No) or WFS 1.1.0 (Yes), and the yes for WFS 1.1.0 even introduces some more complexity. So in the rest of this article I am going to clear things up based on the definition and statement in following OGC specifications:

BBOX in WFS 1.0.0

First of all, it is a little less confusing for WFS version 1.0.0 that you CAN’T specify BBOX in coordinates of any other projections. This is based on two facts in specs:

(1) WFS 1.0.0 spec (02-058), section 13.3.3, where it says:

The SRS of the bounding box must be the same as the SRS of the feature type(s) in a request. The SRS of a feature type is advertised by a WFS in the capabilities document. If more than one feature type is specified in a request, the feature types must all be in the same SRS and the BBOX must be specified in the common SRS as well.

(2) In WFS 1.0.0 schema, a WFS’s capabilities response can have only one “SRS” allowed for each feature type.

But I did see people get confused and thus argue about one thing: Can I specify BBOX in EPSG:4326?

Based on the statement above, if a WFS serves out feature types in a non-EPSG4326 then it should be valid and compliant to just reject GetFeature request with BBOX in EPSG:4326. But another paragraph in WFS 1.0.0 spec (same section 13.3.3) seems to indicate a yes on it:

If a request contains a Bounding Box whose area does not overlap at all with the LatLongBoundingBox(s) advertised in the Capabilities XML for the requested geodata object, the server should return empty content (e.g. null feature set) for that element. Any elements that are partly or entirely contained in the Bounding Box should be returned in the appropriate format.

In this case we don’t really have to strictly follow the spec, and I won’t be surprised to see most WFS 1.0.0 implementations supports BBOX in EPSG:4326 no matter what SRS its feature types are in.

BBOX in WFS 1.1.0

Now for WFS 1.1.0, things get a little complicated because of the fact that there is an optional “crsurl” value you can append at the end of the BBOX coordinates to explicitly specify what coordinate reference system the BBOX is in. For example, ”…&BBOX=minX,minY,maxX,maxY,EPSG:900913” means that those min/max X/Y values are in EPSG:900913. Another complexity is that in WFS 1.1.0 capabilities response there can be multiple supported SRS for each feature type. So based the section 14.3.3 of WFS spec 1.1.0:

The KVP encoding for a bounding box is defined in subclause 10.2.3 of normative reference [15]. The general form of the parameter is:
BBOX=lcc1,lcc2,…,lccN,ucc1,ucc2,…uccN[,crsuri]…

…If the crsuri is not specified then the 2-D coordinates shall be specified using decimal degrees and WGS84 as described in [15].

Note: The WFS spec is reference OWS 1.1.0 spec as [15] here

Here I can actually get two conclusions:

(1) You can specify BBOX in any coordinate reference system as long as you provide a “crsuri”.

Although not described in spec, I would believe you will get correct and useful response only if you request one of those supported coordinate reference system listed in a WFS’s capabilities.

(2) If “crsuri” is missing, then the coordinate values specified in BBOX should be considered as WGS84 as described in OWS 1.1.0 spec as below:

A WGS 84 bounding box shall be KVP encoded in a corresponding parameter value list, with the ordered listed values for the quantities:
LowerCorner longitude, in decimal degrees
LowerCorner latitude, in decimal degrees
UpperCorner longitude, in decimal degrees
UpperCorner latitude, in decimal degrees
crs URI = “urn:ogc:def:crs:OGC:1.3:CRS84” (optional)

In another word, if you set BBOX parameter in request without a “crsurl”, the BBOX value should always be in syntax “min_lon, min_lat, max_lon, max_lat”.

EPSG:4326 versus. CRS84

Now one finally thing people get confused very often is the order of longitude and latitude they should put in BBOX parameter for EPSG:4326 and CRS:84. Originally and as always, official EPSG:4326 defines coordinates in order of “latitude” followed by “longitude”, but unfortunately almost everyone speaks and uses the reverse order as “longitude” followed by “latitude”, which is why old spec like WMS 1.1.1 and WFS 1.0.0 also uses this order for EPSG:4326. But in most recent specifications OGC is trying to fix this common mistaken usage of EPSG:4326 by creating a brand new CRS:84, which based on my understanding is identical to EPSG:4326 except that it defines coordinates in order of “longitude” followed by “latitude”. So as far as WFS goes, always follow the rules below:

For WFS 1.0.0, uses EPSG:4326 and always set BBOX coordinate values as “longitude” followed by “latitude”. For WFS 1.1.0, specify coordinates in BBOX as longitude followed by latitude if you’re setting CRS:84 as the coordinate reference system for BBOX (e.g. “…&BBOX=–180.0,-90.0,180.0,90.0,urn:ogc:def:crs:OGC:1.3:CRS84…”), or set it the other way if you’re setting EPSG:4326 for BBOX (e.g. “…&BBOX=-90.0,-180.0,90.0,180.0,urn:x-ogc:def:crs:EPSG:6.9:4326…), but again if you’re omittng “crsuri” at the end please always stick to the order where longitude is ahead of latitude.

Summary

“BBOX” parameter looks easy and straight forward to use, but it also gets complicated when different versions of WFS implementations and EPSG:4326/WGS84 are involved. Hopefully this article clear things up a little.

Expo 2010 Shanghai China

2010-05-03T23:13:00.001-07:00

To celebrate the opening of Expo 2010 Shanghai China, I just had a peek at where the Expo Axis should locate in Shanghai in those popular online map services (Google Maps, OSM, Bing Maps, and Yahoo Maps). See what I get:

OSM Maps

Not bad, but that is actually what I expected to see.

Google Maps

It seems like Google Maps really needs to catch up with this big event. But if you’re laughing at Google Maps now, you better check out Bing Maps and Yahoo Maps below.

Bing Maps

What can I say…just like the map…

Yahoo Maps

You know what? I zoomed out on purpose because I can’t zoom in any more…

Several approaches to do Geolocation by IP address programmatically

2010-04-14T17:23:00.001-07:00

Geolocation is the process of identifying the real-world geographic location based on a device’s IP address (either connected through cable or WiFi access point), GSM/CDMA cells IDs or GPS and so on. The result of Geolocation is essentially a pair of coordinate (lat and lon) on earth, and if combined with other service like Geocoding the result can also be translated to a well-known place name or a physical address.
Geolocation by IP address although may not be as accurate as GPS sometimes, does have great value in many of today’s location aware web applications. One of the typical examples is that when you do search on internet the search engine will detect the location by your IP address and automatically filter and sort the results based on that information. In this article I will be looking at some existing approaches I’ve played with to do Geolocation by IP address.

1. W3C Geolocation API
If you’re in JavaScript environment, W3C Geolocation API is definitely the approach you should follow because it is the standard. Independent of the actual source of the location information which is not just limited to IP address but can also be WiFi and Bluetooth MAC address, GSM/CDMA cell IDs, GPS or so, it actually defines a high-level interface to location associated information with devices hosting the implementation. Other than that W3C Geolocation API also defines the way you request repeated position updates through callback mechanism as well as requesting an aged position.
Mozilla Firefox is a very good example of such implementation. The code example below illustrate the common work flow to get your current location in a JavaScript application:

if(navigator.geolocation) {  
/* geolocation is available */  
navigator.geolocation.getCurrentPosition(
function(position) { // success callback  
//alert(position.coords.latitude + "," + position.coords.longitude);
alert(position.address.city);                
},
function() { // failure callback
// error handling
},
{
// other options
timeout:30000
}
);    
} else {      
// browser does not support Geolocation API
}

In Firefox’s implementation, it gives you not only the lat and lon value of your location but also gives you the address associated with that coordinate. Under the hood Firefox takes advantage of Google Location Service which I believe provided all necessary infrastructure.

2. Google Geolocation API in Gears
Google Gears Geolocation API implements the same W3c Geolocation interfaces but in a more device independent, location information source independent, and browser independent way. It’s online doc is very good and concise for people who would like to use it in their web application.

3. Google Location Service
Google Location Service is far more than an interfaces that only provides geolocation function, but it’s totally valid to use that as a Geolocation service. Without even realizing, it is definitely behind a lot of its own services like Search and Latitude which sometimes you don’t even realize. It is also backing up many other Geolocation API implementations like Firefox. But unfortunately I was able to find out an explicit native Geolocation service API either in JavaScript or in REST flavor. So it’s probably impossible to directly communication with Google Location Service by feeding an IP address and getting a coordinate back.

4. Other Geolocation by IP service providers
HostIP, mentioned in Andrew Turner’s Introduction to Neogeography, HostIP has the largest free database and service for GeoIP (the term Andrew uses for geolocation by IP address). The best part of HostIP is that it freely distribute it’s database with regular updates. Since I don’t find any information regarding license and term of use for HostIP database I would assume it is in public domain. Programmatically HostIP provides a simple and lightweight RESTful API for people to locate themselves by IP. It doesn’t do much but it’s good at what it does. Because of its REST nature, it is not limited to any program environment except that you might need a proxy in your JavaScript web application. Below is an example:

http://api.hostip.info/country.php
US
http://api.hostip.info/get_html.php?ip=12.215.42.19
Country: UNITED STATES (US)
City: Sugar Grove, IL
IP: 12.215.42.19
http://api.hostip.info/get_html.php?ip=12.215.42.19&position=true
Country: UNITED STATES (US)
City: Sugar Grove, IL
Latitude: 41.7696
Longitude: -88.4588
IP: 12.215.42.19
http://api.hostip.info/?ip=12.215.42.19
[use the URL above for an example - XML too long to paste below]

Finally its website itself provides a simple browser based GUI for people who don’t want to bother coding their own app.

GeoIP: MaxMind has a free GeoIP database and API called GeoLite City. MaxMind and it GeoIP product suite does not have a publicly hosted RESTful service or API for directly use but it allows you to download their database in either binary format or CSV format (mainly for you to load data into SQL database) to host locally. The API for accessing the local database is available in many platforms (C, C#, MS COM, PHP, Java, Perl Python, Ruby etc.). The workflow is to first download the database to local disk and uncompress it, then use any of the API available the query the database. Below is an example I tried out using its Java API:

public void lookupServiceTest() {  
try {
LookupService cl = new LookupService("E:\\projects\\geoip\\geolite-city-database\\GeoLiteCity.dat", LookupService.GEOIP_MEMORY_CACHE);
Location location = cl.getLocation("xxx.xxx.xxx.xxx");
System.out.println(" countryCode: " + location.countryCode +
"\n countryName: " + location.countryName +
"\n region: " + location.region +
"\n regionName: " + regionName.regionNameByCode(location.countryCode, location.region) +
"\n city: " + location.city +
"\n postalCode: " + location.postalCode +
"\n latitude: " + location.latitude +
"\n longitude: " + location.longitude +
"\n metro code: " + location.metro_code +
"\n area code: " + location.area_code +
"\n timezone: " + timeZone.timeZoneByCountryAndRegion(location.countryCode, location.region));
cl.close();
} catch (IOException e) {
System.out.println("IO Exception");
}  
}

GeoIP API is free and open source under GPL/LGPL license, and the GeoLite City database itself is also free under its database license. MaxMind also provides an advanced commercial version called GeoIP City, which is functionally equal to the free one but has more accuracy, different redistribution license and more frequent updates.

Using Java Map Projection Library in Android

2010-03-28T23:07:00.001-07:00

Recently I was writing a little mapping application on Android, which basically takes Open Street Map tiles as base layer, and also has the ability to overlay some vector features on top. In this application there is need to convert coordinates back and forth between WGS84 and Mercator projection, so I need certain library to do the job. Given the popularity of PROJ.4, the pure Java port of it, Java Map Projection Library (name it “PROJ4Java” below), seems to be an obvious choice so I spend some hours during the weekend to see how it works.
First you can get the latest version (1.0.6) of PROJ4Java from this link. The size of the library is very small and lightweight with only 1.17MB for complete source code and 247KB for compiled jar file. If you’re familiar with PROJ.4 itself then this Java version should be quite straight forward, otherwise there isn’t much sample code or tutorial on its homepage. But the basic workflow is quite simple as described by its project homepage:

To use the library, you need to create a projection, either directly by calling its constructor or via one of the factory methods. Once you have a Projection, you can use it to convert between latitude/longitude and projected map units (by default, metres) by calling the transform and transformInverse methods.

A major issue of using PROJ4Java on Android is that out of box it can’t be run due to the fact that it is using some classes in java.awt.geom.* which is not available in Android SDK. But since the only major AWT class used by PROJ4Java is Point2D, it’s not difficult at all to work around by replacing it with your own 2D point class. In my case I use the point class from JTS library. After taking out all Java awt class references from source code and replacing them with JTS class, everything works pretty well on Android.
Another thing I modifies on the source code is to get rid of those projection implementations I don’t need. Mercator projection is the only one I need so far so that I am able to reduce the size of compile jar file to only 168KB. Here is a list of classes I extracted from original source code to support only Mercator projection:

com.jhlabs.map.*
com.jhlabs.map.proj.CylindricalProjection
com.jhlabs.map.proj.Ellipsoid
com.jhlabs.map.proj.MercatorProjection
com.jhlabs.map.proj.Projection
com.jhlabs.map.proj.ProjectionFactory
com.jhlabs.map.proj.ProjectionException
com.jhlabs.map.proj.Ellipsoid

ProjectionFactory class is the entry point for you to use PROJ4Java, which provides several static methods to create an instance of Projection. Below are some use cases I figured out by briefly reading through the source code:
(1) ProjectionFactory.getNamedPROJ4CoordinateSystem(String name) takes a well-known projection code like “epsg:4326” and returns a Projection:

1: String name = "epsg:3785";    
2: Projection proj = ProjectionFactory.getNamedPROJ4CoordinateSystem(name);

(2) ProjectionFactory.fromPROJ4Specification(String[] params) takes a set of PROJ.4 parameters and returns a Projection:

1: String[] params = {
2:     "proj=tmerc",
3:     "lat_0=37.5",
4:     "lon_0=-85.66666666666667",
5:     "k=0.999966667",
6:     "x_0=99999.99989839978",
7:     "y_0=249999.9998983998",
8:     "ellps=GRS80",
9:     "datum=NAD83",
10:     "to_meter=0.3048006096012192",
11:     "no_defs"
12: };
13: ProjectionFactory.fromPROJ4Specification(params);

(3) Create an instance of customized projection. All supported projection definition strings (PROJ.4 parameters for ) for PROJ4Java are listed in text files under folder “nad” that is coming with the library. These files are “epsg”, “esri”, “nad27”, “nad83”, and “world”, and a sample projection definition string is like below:

1: <2965> +proj=tmerc +lat_0=37.5 +lon_0=-85.66666666666667 +k=0.999966667 +x_0=99999.99989839978 +y_0=249999.9998983998 +ellps=GRS80 +datum=NAD83 +to_meter=0.3048006096012192 +no_defs
2:

To add a new customized projection just create a definition string (usually modify certain parameter values upon an existing projection) and add it into one of those files, or create a new text file under “nad” folder with the definition string, then use the code below:

1: Projection proj = null;
2: try {
3:     // assume that "epsg:900913" projection defintion 
4:     //   has been added in text file "others"
5:     proj = ProjectionFactory.readProjectionFile("others", "900913");    
6: } catch(IOException e) {
7:     e.printStackTrace();
8: }

(4) Transform between lat/lon and projection units with a Projection instance:

1: Projection epsg3785 = ProjectionFactory.getNamedPROJ4CoordinateSystem("epsg:3785");    
2:               
3: System.out.println("transform from latlon to epsg:3785");
4: System.out.println("latlon: -117.5931084, 34.1063989");
5: Point pEpsg3785 = epsg3785.transform(-117.5931084, 34.1063989);
6: System.out.println("epsg:3785: " + pEpsg3785.getX() + ", " + pEpsg3785.getY());
7:     
8: System.out.println("transform from epsg:3785 to latlon");
9: System.out.println("epsg:3785: " + pEpsg3785.getX() + ", " + pEpsg3785.getY());
10: Point latlon = epsg3785.inverseTransform(pEpsg3785);
11: System.out.println("latlon: " + latlon.getX() + ", " + latlon.getY());
12: 
13:

WMS TIME support in MapServer

2010-03-09T13:02:00.001-08:00

In context of my last blog, I’m looking at the WMS TIME support in OSGeo MapServer. MapServer has a very concise and helpful doc on the scope and usage of WMS TIME in its implementation. It is always a good idea to briefly go through the documentation first before you spend a lot time trying something that may not be even implemented.
My test data (in shapefile format) is very simple such that the picture below tells all about it:

Each point feature has a field “TIME” which has time value in syntax “YYYY-MM-DD HH:MM:SS”. This is valid ISO8601 time format and supported by MapServer. All points have a time stamp within day 2010-02-16. To enable time on WMS in MapServer is straight forward given that you have some experience of publishing shapefile as a WMS in MapServer. The only extra thing you need to configure is to specify time field, time extent and default time value in map file. That’s it and MapServer documentation is very clear on how to do that.

1: LAYER
2:     NAME 'points'
3:     METADATA
4:         "ows_title" "points"
5:         "gml_include_items"   "all"
6:         "wms_include_items"   "all"
7:         # TIME Related
8:         "wms_title"          "points"
9:         "wms_timeextent"     "2010-02-16T00:00:00Z/2010-02-17T00:00:00Z/P1H"
10:         "wms_timeitem"       "TIME"
11:         "wms_timedefault"     "2010-02-16T12:00:00Z,2010-02-16T18:00:00Z"
12:     END
13: ...
14: END

After saving the map file and refresh the GetCapabilities response you will notice information of time dimension is advertised.

1: <Dimension name="time" units="ISO8601" default="2010-02-16T12:00:00Z,2010-02-16T18:00:00Z" nearestValue="0">
2:     2010-02-16T00:00:00Z/2010-02-17T00:00:00Z/P1H
3: </Dimension>
4:

The time extent value in <Dimension> is whatever you specified in “wms_timeextent” so it doesn’t smart enough to calculate the extent from data, and neither does MapServer do any validation on the time extent string. A trick I did in map file is to provide a special default time value “2010-02-16T12:00:00Z, 2010-02-16T18:00:00Z,”, which will inform me if certain time string is not correctly processed by MapServer thus default value is used.

Now it’s time to try a few requests against this time enabled WMS. According to doc the supported request syntax for time are “t”, “t1,t2…”, “t1/t2”, and “t1/t2,t3/t4…” and “t” should be any valid ISO8601 time string. I skipped the time zone because there is nowhere indicating it is implemented (you can not configure time zone at server-side thus it makes no sense to request from client). Below is a list of different time values that have been tried:

…&TIME=2010 (not recognized, default time value is returned)
…&TIME=2010-02 (not recognized, default time value is returned)
…&TIME=2010-02-16 (recognized, only point of “2010-02-16 00:00:00” returned)
…&TIME=2010-02-16T12 (recognized, only point of “2010-02-16 12:00:00” returned)
…&TIME=2010-02-16T12:00 (recognized, only point of “2010-02-16 12:00:00” returned)
…&TIME=2010-02-16T12:00:00 (recognized, only point of “2010-02-16 12:00:00” returned)
…&TIME=2010-02-16T14:59:59 (recognized, only point of “2010-02-16 14:59:59” returned)
…&TIME=2010-02-16T15:00:01 (recognized, only point of “2010-02-16 15:00:01” returned)
…&TIME=2010-02-16T15:00:00.000 (recognized, only point of “2010-02-16 15:00:00” returned)
…&TIME=2009/2010 (not recognized, default time value is returned)
…&TIME=2010/2011 (not recognized, default time value is returned)
…&TIME=2010-02/2010-03 (not recognized, default time value is returned)
…&TIME=2010-02-16/2010-02-17 (recognized, all points returned)

The test results above actually proves the time interpretation described in the documentation:

When MapServer receives a request with a TIME parameter, it transforms the time requests into valid expressions that are assigned to the filter parameter on layers that are time-aware.

Here are some examples of how different types of requests are treated (wms_timeitem is defined here as being “time_field”):
single value (2004-10-12) transforms to (`[time_field]` eq `2004-10-12`)
multiple values (2004-10-12, 2004-10-13) transform to (`[time_field]` eq `2004-10-12` OR `[time_field]` eq `2004-10-13`)
single range : 2004-10-12/2004-10-13 transforms to ((`[time_field]` ge `2004-10-12`) AND (`[time_field]` le `2004-10-13`))
multiple ranges (2004-10-12/2004-10-13, 2004-10-15/2004-10-16) transform to ((`[time_field]` ge `2004-10-12` AND `[time_field]` le `2004-10-13`) OR (`[time_field]` ge `2004-10-15` AND `[time_field]` le `2004-10-16`))

As shown in the above examples, all fields and values are written inside back tics (`) - this is the general way of specifying time expressions inside MapServer.

Overall it’s doing a good job and the only thing missing point is that requests like “TIME=2010” and “TIME=2010/2011” should be properly handled.

If you read carefully, MapServer documentation has another slightly different time interpretation for PostGIS layer, to test that I followed up and applied the same requests on a PostGIS layer.

Things I need to configure before I can finally send the same set of WMS time requests to PostGIS layer are listed below:

Use “shp2pgsql” command line to convert shape file into pgsql file;
Use “psql” command line to load pgsql file into PostGIS database;
Add another new column of type “timestamp without time zone”;
Copy time value from original column to the new column;
Modify map file to use new column of of type “timestamp without time zone” for WMS time;
Modify the map file to point to PostGIS table instead of shape file;

All of above are straight forward except that a WMS layer based on PostGIS doesn’t seem to recognize the temporal field if it is of type “character varying”, which force me to add an extra column in type “timestamp without time zone” and copy over those original time values in string. After all of those it worked and below are the results of same WMS TIME requests on shapefile layer:

…&TIME=2010 (not recognized, default time value is returned)
…&TIME=2010-02 (not recognized, default time value is returned)
…&TIME=2010-02-16 (recognized, all points with time stamp within 2010-02-16 returned)
…&TIME=2010-02-16T12 (recognized, points between 12:00:00 and 13:00:00 on 2010-02-16 returned, not including points on 13:00:00)
…&TIME=2010-02-16T12:00 (recognized, only point of “2010-02-16 12:00:00” returned)
…&TIME=2010-02-16T12:00:00 (recognized, only point of “2010-02-16 12:00:00” returned)
…&TIME=2010-02-16T14:59:59 (recognized, only point of “2010-02-16 14:59:59” returned)
…&TIME=2010-02-16T15:00:01 (recognized, only point of “2010-02-16 15:00:01” returned)
…&TIME=2010-02-16T15:00:00.000 (recognized, only point of “2010-02-16 15:00:00” returned)
…&TIME=2009/2010 (not recognized, default time value is returned)
…&TIME=2010/2011 (not recognized, default time value is returned)
…&TIME=2010-02/2010-03 (not recognized, default time value is returned)
…&TIME=2010-02-16/2010-02-17 (recognized, all points returned)

So it proves what it says in the doc about PostGIS layer:

For PostGIS/PostgreSQL layers, the time expression built uses the date_trunc function available in PostgreSQL. For example, if the user passes a time value of ‘2004-10-12’, the expression set in the filter is date_trunc(‘day’, time_field) = ‘2004-10-12’. The use of the date_trunc function allows requests to use the concept of time resolution. In the example above, for a request of ‘2004-10-12’, MapServer determines that the resolution is “day” by parsing the time string and the result gives all records matching the date 2004-10-12 regardless of the values set for Hours/Minutes/Seconds in the database. For more information on the date_trunc function, please refer to the PostgreSQL documentation.

Two ambiguities about TIME in OGC WMS specification

2010-03-04T16:13:00.001-08:00

OGC WMS specification had included “TIME” dimension since version 1.1.0, in which it guides the WMS implementations on how “TIME” is involved in filtering feature that will finally be visible in either a result map or a query result (aka. GetFeatureInfo result). To me, what it defines about “TIME” dimension in WMS spec seems to be both accurate and ambiguous. It is accurate because it introduces the ISO 8601 standard to cover the exchange of date and time related parameters between a WMS server and a WMS client, which eliminate the risk of misinterpretation where numeric representation of dates and times are interchanged across national boundaries, and to avoid the confusion and other consequential errors or losses (quoted from wiki ISO8601 entry). But it’s also ambiguous in two places (1) where an interpretation is needed for an ISO standard time string; (2) where a temporal attribute has different meanings for different types of features. In this blog let’s skip the accurate part that I am happy with, but discuss the ambiguous part using a few examples.

The first ambiguity is actually not a problem of WMS spec itself but comes from the nature of the data when a temporal attribute value has different meanings for different types of features. Basically time values can ne divided into two different types: time as a individual time point and time as a time window. Time as a time point doesn’t have any relation to its history or future, one good example is a rocket feature with a time attribute called “launch” that has a value of “2000-03-04T00:00:00Z”. It means the rocket launched at Mar.4th 2000 at 12 am which has nothing to do with any time before it or after. But time as a time window instead expand backward or forward. One example is a house feature that has time attribute called “built” with a value of “2000”, it actually affects the time after 2000 because the house exists since then. Another example is an animal species with time attribute “extinct” of value “2000”, which actually affects the time before 2000 because the animal species had existed before 2000. Given the fact that all of those single time values are treated the same in database, it will potentially affect the result of a query on those features.

The second ambiguity in WMS spec, as I mentioned above is the interpretation for a time value. For example, when you send a WMS request like “http://…/…request=GETMAP&…&TIME=2010”, what does it exactly mean? It sounds like a stupid question because everyone knows “2010” meaning year 2010, which is clearly defined in ISO8601 as valid. But how do you interpret this request is ambiguous. Assume you’re a time-enabled WMS server and you have some features with time stamps between 2010 and 2011 (any month, day, hour, minute, or second within 2010), will you include those features in the map response to request “…&TIME=2010”? I bet you answer yes without hesitation because naturally “TIME=2010” means within 2010. But will you give it a second thought if I give a different request “http://…/…request=GETMAP&…&TIME=2010-12-10T12” on the same set of features? Do you still think you should include all the features between 12pm and 1pm on 2010-12-10? I don’t think so but that’s actually the same logic you used when you process request “http://…/…request=GETMAP&…&TIME=2010”.

If you’re implementing a TIME-enabled WMS at server-side you probably already encountered those two problems, and I don’t think the solution is simply throwing the TIME values to database for string comparison even if the database is aware of date/time type. The best way I can think of is always using time window internally (even though sometimes that time window is very small) for both feature temporal attributes and time parameter in the WMS request. To be more specific, when serving out you data if you have a rocket feature launched at “2000-03-04T00:00:00Z” then explicitly treat it as “2000-03-04T00:00:00.000Z/2000-03-04T00:00:00.999Z”; if you have a house built on “2010” then explicitly treat it as “2010/far future”; and finally if you have an animal species extinct in 2000 then explicitly treat it as “long ago/2000”. Now when it comes to time parameter in WMS request, the spec allows syntaxes like “TIME=time1”, “TIME=time_start/time_end”, “TIME=time1,time2” and “TIME=time_start1/time_end1,time_start2/time_end2” but no matter what client sends to you, my suggestion is to always treat each single time value as a time window based on its smallest interval. For example, if “TIME=2000” then treat it as “TIME=2000/2001” (the smallest time interval is year); if “TIME=2000-03-04T00:05Z” treat it as “2000-03-04T00:05Z/2000-03-04T00:06Z” (the smallest time interval is minute) and so on. By using this mechanism every time relate value is a time window, which makes the query more straight forward, just include the feature either in map or query results if the query time window touches, intersects or contains the time window of feature temporal attribute. Although it won’t totally solve the ambiguity above, it helps improve the accuracy.

Setup a http based SVN repository with TortoiseSVN and Apache http server

2010-02-22T13:37:00.001-08:00

It is very useful to setup a http based SVN repository on your local machine just for daily check in and out of source code. I am pretty sure there are plenty of materials over the web that explain how to do it for different combinations. Here is the one which I went through and it works pretty good for me.
1. Install and configure TortoiseSVN

Download TortoiseSVN from its official website at http://tortoisesvn.net/downloads; (.msi file)
Install it by following the standard wizards;

2. Install Apache HTTP Server

Download Apache HTTP Server from its' website at http://httpd.apache.org/download.cgi#apache22, and the version I’m using is 2.2.14 (either the one with or without ssl); try avoid using 1.x or 3.x which either doesn’t work well or hasn’t been tried by many people. So apache 2.2 is the best pick for now;
Install Apache http server by following the wizards;
Change the Apache HTTP Server listening port if necessary in conf.
After installation, just test it by launching http://localhost:port/ in browser and you should see a “It works!” page; (you can also monitor the apache http service in Apache Service Monitor)

3. Install Subversion

Download the latest version of the Subversion Win32 binaries for Apache. Be sure to get the right version to integrate with your version of Apache, otherwise you will get an obscure error message when you try to restart. If you have Apache 2.2.x go to http://subversion.tigris.org/servlets/ProjectDocumentList?folderID=8100;
!!!DON'T!!! use the msi version of subversion (it doesn't work somehow), instead use the zip version; (after unzip the subversion folder to your local disk, you also need to add the bin path to 'path' system environment variable)
Using the windows explorer, go to the installation directory of Subversion and find the files /httpd/mod_dav_svn.so and mod_authz_svn.so. Copy these files to the Apache modules directory;
Copy the file /bin/libdb*.dll and /bin/intl3_svn.dll from the Subversion installation directory to the Apache bin directory;

4. Configure Subversion and Apache HTTP Server

Edit Apache's configuration file (usually C:\Program Files\Apache Group\Apache2\conf\httpd.conf) with a text editor such as Notepad and make the following changes:

Uncomment (remove the '#' mark) the following lines:

#LoadModule dav_fs_module modules/mod_dav_fs.so
#LoadModule dav_module modules/mod_dav.so

Add the following two lines to the end of the LoadModule section.

LoadModule dav_svn_module modules/mod_dav_svn.so
LoadModule authz_svn_module modules/mod_authz_svn.so

b. At the end of the config file add the following lines:

    <Location /svn>
         DAV svn
         SVNListParentPath on
         SVNParentPath D:\SVN
         #SVNIndexXSLT "/svnindex.xsl"
         AuthType Basic
         AuthName "Subversion repositories"
         AuthUserFile passwd
         #AuthzSVNAccessFile svnaccessfile
         Require valid-user
     </Location>
c. This configures Apache so that all your Subversion repositories are physically located below D:\SVN. The repositories are served to the outside world from the URL: http://MyServer/svn/ . Access is restricted to known users/passwords listed in the passwd file.

d. To create the passwd file, open the command prompt (DOS-Box) again, change to the apache2 folder (usually c:\program files\apache group\apache2) and create the file by entering

bin\htpasswd -c passwd <username>

This will create a file with the name passwd which is used for authentication. Additional users can be added with

bin\htpasswd passwd <username>

e. Restart the Apache service again;

f. Point your browser to http://MyServer/svn/MyNewRepository (where MyNewRepository is the name of the Subversion repository you created before). If all went well you should be prompted for a username and password, then you can see the contents of your repository.

5. Create SVN repository and access it through HTTP

Create a folder on your local disk (under the folder where you specify in Apache http server to be the root folder of your SVN);
Right click on the folder and use the TortoiseSVN context menu “Create Repository Here”;
Access your SVN repository through HTTP as http://server:port/svn/repositoryName/ (you need to type in username and password).

Google Latitude finally locates me

2010-02-03T23:34:00.001-08:00

I joined Google Latitude more than half year ago but no matter whether I’m at home (Rancho Cucamonga, CA) or at my office (Redlands, CA), it always tells me that I’m in Hague Netherlands, which I assume is what my friends will see too. Recently it finally corrects the location to Rancho Cucamonga or Redlands. Since I am not using Google Latitude on my mobile device, the only way Google Latitude can rely on should be the IP address and associated ISP. Just wonder why there could be such a big mistake.

First experience of armchair mapping using Mapzen and JSOM WMS plug-in

2010-01-22T16:06:00.001-08:00

In previous post I shared my first experience of making changes to OSM database with Potlatch and JOSM. As I mentioned there both of them are good tools but neither is perfect so I’m still looking for some alternatives.

Mapzen caught my eye recently probably because it becomes quite a popular buzz word in blogs and news feedings so I decided to give it a try. At first glance, Mapzen looks quite similar to Potlatch (CloudMade who hosts Mapzen is founded by the same person who started OSM):

1. It requires an account (you can register it for free) to work with;

2. A peek at source of Mapzen front page shows it’s based on Flex too;

3. It uses OSM map as base layer in view mode;

4. It uses Yahoo Imagery as based layer in edit mode; (by the way, Yahoo Aerial Imagery is probably the only imagery available for free which has decent globe coverage. I never see other options, do you?)

Mapzen in view mode:

Mapzen in edit mode:

I like the way it categorize those pre-defined features in Points, Lines and Shapes and each one of the feature types has a meaningful icon associated. Of course it lacks some of the advanced editing functionalities that are available in Potlatch (it seems to have a tool to line up points) and JOSM, but it has more than enough for majority of the users. Another thing I like Mapzen over Potlatch is the look and feel when you draw the geometry on map. With Potlatch I always feel like using a big brush for wall painting, where all I actually need is pencil for sketch. The only weakness though is that I didn’t find a way to upload data as input for my editing, either through an osm file or a gpx track while Potlatch provides that option. Everything must be hand drawn. But anyway you can always combine the power of both.

After editing simply click “Save Map” button at corner (you might want to input some notes for this particular editing). The picture above shows the hand-draw building and foot way of my house using Mapzen, as well as a BBQ island garden “August Town” (named after my boy).

JOSM with its WMS plugin (with the Yahoo Aerial Imagery extension) is another option I tried after Mapzen, which I probably rank the highest among all three of them (Potlatch, Mapzen and JOSM). Like I mentioned before, the lack of a decent base imagery is a huge bottleneck which makes it almost impossible for armchair mapping, but the WMS plug-in with Yahoo Aerial Imagery extension overcomes it like magic. So let’s equip with it first by following the instructions here. A brief list of steps:

Install WMS plugin itself:

With the plugin manager

You can easily install plugins from within JOSM as follows

Start JOSM, open the preferences window (Edit->Preferences or use the toolbar icon) and select the plugins tab.

Click on "Download List" to download the list of available plugins.

Check the plugins you want installed.

Click the accept button. All new plugins should start downloading and installing.

Restart JOSM.

Manually

With older versions (up until 277), you have to install the plugins manually.

Download the plugin file from wherever the plugin is hosted. Look in the plugin page or the 'Plugins' page on the JOSM wiki site for the location of this file.

The file should have an ".jar" extension. If it doesn't, rename the downloaded file so that it ends with ".jar". Internet Explorer, for instance, may rename some files to ".zip".

Move the .jar file to the JOSM preferences directory ("%APPDATA%/JOSM" in windows, "~/.josm/plugins/" in Unix/MacOS.)

Start JOSM, open the preferences window (Edit->Preferences) and select the plugins tab.

Activate the plugin in the plugins tab.

Restart JOSM.

Install Yahoo! Aerial Imagery Downloader

On Windows use the WebKit based downloader called webkit-image as follows

Download webkit-image.zip

Unzip it.

Move the contents so that the DLL files and the EXE file are somewhere "on your system path" (eg. c:\windows ). The best way to achieve this might be to place them alongside josm-latest.jar Keep the 'imageformats' subfolder alongside too (so all the contents of the zip).

Restart JOSM

Do 'WMS' menu -> YAHOO (Webkit)

You should start to get Yahoo! imagery (may take up to 30 seconds to start showing). If not, it may not be finding the DLL files correctly.

Note: If you don't want to place webkit-image in your system path or the JOSM directory you don't have to. By editing the download program you can specify an absolute or relative path to the webkit-image executable. Examples:

webkit/webkit-image {0} - loads webkit-image.exe from the subdirectory webkit relative to the JOSM installation directory

D:/webkit/webkit-image {0} - uses webkit-image D:\webkit\webkit-image.exe

After a successful install, restart JOSM and you should a new menu called “WMS” like below:

Now download your data from OSM database as usual first, and after select the “WMS” menu and select “Yahoo Sat”. Wait for about 5, 6 seconds you will see a nice aerial imagery base layer underneath your OSM data.

Well done! And this is perfect for armchair mapping. Now enjoy your smooth mapping experience with powerful tools in JOSM. I didn’t try out the WMS layer and rectified image yet which can definitely be better base layer options for specific area.

After the editing simply click upload button (right next to download button) and upload the changes. (you may need to set your OSM account username and password)

So my conclusion is that JOSM with WMS plugin is an obvious winner due to the smooth and rich desktop UI experience if you’re doing armchair mapping at home. But Potlatch and Mapzen have their own advantages of being browser based and easy to use for non-professional users.

Install Apache HTTP Server + PHP5 on Windows 7 (64bit)

2010-01-20T09:45:00.001-08:00

Recent installation of dokuwiki (a simple web-based wiki engine) requires a combination of Apache HTTP Server + PHP5 on my 64-bit Windows 7 machine, but setup process didn’t seem to be as simple as “following the wizards”. So I decide to jot a few lines down such that I don’t have to search on the web again when I need to repeat the installation.

1. Install Apache HTTP Server; (just go to the official site and pick up .msi, and install in default path C:\Program Files(x86)\Apache Software Foundation\ seems to be ok)
2. Install PHP5;

This worth a short paragraph. My understanding of the install steps on PHP website is that it should be automatically taken care of by the installation wizards if it is for a popular http server like apache, and the install wizard UI does indicate that too. But unfortunately I didn’t get it to work in which I either got an general error message or no error message but just not working. So I switch to the zip version of PHP5 which is suggested by some article online and following the steps below makes it work properly:

2.1 Unzip the the zip version of PHP5 to your local disk; (unzip it to C:\Program Files(x86)\PHP5 seems to be working too)
2.2 Open httpd.conf file of Apache HTTP server in a text editor;
2.3 At the end of the http.conf file add following lines to make Apache HTTP Server be aware of PHP5; (adjust you path accordingly)

1: # Begin enable php5
2: LoadModule php5_module "C:/Program Files (x86)/PHP5/php5apache2_2.dll"
3: AddType application/x-httpd-php .php
4: PHPIniDir "C:/Program Files (x86)/PHP5"
5: # End enable php5

2.4 Save the change and restart Apache HTTP Server, and if successful you should see
“Apache/2.2.14(Win32) PHP/5.3.1” in the status bar of Apache Service Monitor.

Now your Apache HTTP Server should be able to display php content, at least my php based dokuwiki is running perfect.

Google Maps Navigation failed me again

2010-01-19T22:57:00.001-08:00

As an Android G1 mobile phone user and a fan of Google products and services, I was really excited when the GPS killer app Google Maps Navigation was ported to Android 1.6. But I don’t usually drive to a lot of new places that I have no idea how to get to, even when I do I will do my homework on desktop first, which leaves Google Maps Navigation on my Android phone very few chances to prove its value. But the only time I was trying to rely on it to help me out, it failed. That was about one month ago, I am on my way to metro station in Riverside to pick up my wife after work. Due to the heavy traffic on CA-91 I decided to switch to the station right after the original one (Riverside-La Sierra station), which as I recall later was the first time Google Maps Navigation on my G1 phone coming to rescue in a real situation rather than a fake drill. It actually did a very good job for the most part, which I even see the metro station I was looking for (the red place marker). But amazing thing happened when I passed by the destination (I should have followed the green dash line). First Google Maps Navigation didn’t tell me that I had arrived (I had a second thought that I should just turn into the parking lot, but I hesitated because Google never failed me before), instead it told me to get onto the highway that I just got off again in other direction (the solid green line). I followed although with a little doubt (still believe Google was right) but was totally pissed off after a few seconds when right in the middle of the ramp (where the red cross is) it said that I’ve arrived.

I followed the solid green line which Google Maps Navigation told me to, but the red cross is the place where it said that I’ve arrived.

Today the fancy free Google Maps Navigation failed me again. I was looking for Bed Bath & Beyond in Mira Loma, CA, which seems to be pretty new because my 3-year old Magellan GPS wasn’t able to find it at all. Searching in Google Maps Navigation (by typing “Bed Bath and Beyond” and click search) gave me a matching Bed Bath & Beyond in Mira Loma, which has the accurate telephone number and address. But obviously it put the spin at a wrong place which is about half mile north west of the actual location in a lake. Using the same searching pattern in Google Maps generates the same error, but if you type the result address back in both Google Maps and Navigation points to the right place.

Bed Bath & Beyond in Swan Lake

The red place marker is where Google Maps Navigation directed me to, while the green spin is the correct place:

Another thing I notice is that the streets around wrong Bed Bath & Beyond location in Google Maps hasn’t been updated at all, or at least not recent enough to reflect current name. An example is that the “Peachtree Dr” in red circle doesn’t exist any more. Looking at OpenStreetMap for the same area gives me the accurate result as below:

Customize WMS GetFeatureInfo response :: GeoServer 2.0 versus MapServer 5.4.2

2010-01-15T14:47:00.001-08:00

OGC Web Map Service (WMS) specification is a simple and popular protocol for serving maps over the web nowadays. But everything has two sides and the weakness of WMS is that it doesn’t define a standard format for its GetFeatureInfo operation, neither does it reference any existing standard like GML or so. The direct consequence of that is the truth that most of today’s WMS server implementations speaks different “languages” in their query results which results in the opposite of interoperability between servers and clients from different vendors.

Using HTML for WMS GetFeatureInfo somehow alleviate the situation, because without parsing clients can just throw HTML to browser. Some other implementations support GML to be more interoperable, but again those are just workarounds and the real issue should be solved in spec itself.
Anyway while waiting for the next version of spec (1.4.0?, 2.0? or else) recently, I spent some time with GeoServer 2.0 and MapServer 5.4.2 to customize the WMS GetFeatureInfo response in HTML, which is a feature available in both popular open source WMS server implementations.

What is the goal?

I have a simple vector dataset for San Francisco area in shape file format, which I published as WMS in both GeoServer and MapServer. Below is the map I will get when I send a WMS GetMap request:

http://<wms_service_url>?REQUEST=GetMap&SERVICE=WMS&VERSION=1.1.1&LAYERS=blockgroups,highways,pizzastores&STYLES=&FORMAT=image/png&BGCOLOR=0xEEEEEE&TRANSPARENT=false&SRS=EPSG:4326&BBOX=-122.545074509804,37.6736653056517,-122.35457254902,37.8428758708189&WIDTH=1020&HEIGHT=906

Now if I send a GetFeatureInfo request to query features on the map:

http://<wms_service_url>?REQUEST=GetFeatureInfo&SERVICE=WMS&VERSION=1.1.1&LAYERS=pizzastores,highways,blockgroups&STYLES=&FORMAT=image/png&BGCOLOR=0xFFFFFF&TRANSPARENT=TRUE&SRS=EPSG:4326&BBOX=-122.545074509804,37.6736653056517,-122.35457254902,37.8428758708189&WIDTH=1020&HEIGHT=906&QUERY_LAYERS=pizzastores,highways,blockgroups&X=652&Y=368&INFO_FORMAT=text/html

Zero or more features records will be returned (Note: this particular request actually returns records from all three layers: ‘pizzastores’, ‘highways’, and ‘blockgroups’). So what I am trying to achieve here is to have both GeoServer and MapServer output GetFeatureInfo results in html with styles like below:

It’s a very simple customization but it proves the ability in GeoServer and MapServer, and of course more rich HTML elements like charts & pies, audios and videos can be easily added.

MapServer 5.4.2
For MapServer (the ms4w.exe that contains MapServer 5.4.2) I finally made what I planned, but I have to say that the whole user experience is far less smooth than I expected. A non-guru user like me mostly replies on the doc and samples as a start point, but the information regarding to this topic is scattered all over piece by piece without links pointing to each other. So compared to this I liked the new GeoServer 2.0 user manual much better.

To make shape file layer queryable in MapServer WMS, I started from a sample map file that I modified from online sample, and here is a section for one of my layers

1: # ===============================================================================
2: # highways layer
3: # ===============================================================================
4: LAYER
5:     NAME "highways"    
6:     METADATA
7:         "ows_title" "highways"
8:         "ows_srs"   "EPSG:4326"
9:     END
10:     TYPE LINE
11:     STATUS ON    
12:     DATA "./sanfrancisco/shp/highways"    
13:     PROJECTION
14:         "init=epsg:4326"
15:     END    
16:     CLASS    
17:         NAME "highways"            
18:         STYLE        
19:             WIDTH 1
20:             COLOR 0 0 255
21:         END
22:     END      
23: END

But unfortunately, all WMS layers are defined unqueryable (you will see <Layer …queryable=”0”…> in WMS capabilities files) by default, and online documentation doesn’t say anything clear on how to enable query on WMS layers. I figured out in the end by searching through the forum, in which some others people are asking the similar questions. Basically you have to add following line in each layer definition in map file:

1: LAYER
2:     ...
3:     TEMPLATE "blank.html"
4:     ...
5: END

“TEMPLATE” is suppose to point to a html file used as a template for WMS query result, and it makes WMS layer queryable even though the html file you’re pointing to doesn’t exists (I just feel a little awkward about this)

Now I can actually get GetFeatureInfo response from WMS, but I encountered three more problems right way:

1. GetFeatureInfo response doesn’t support GML as it claims; by reading the MapServer WMS doc “Reference Section” it can be solved by adding “DUMP TRUE” into layer definition:

1: LAYER
2:     ...
3:     DUMP TRUE
4:     ...
5: END

2. Either GML response or plain text response of GetFeatureInfo doesn’t include any attribute of the result feature; by reading the doc, you can solve that for GML by adding “gml_include_items all” in metadata of layer definition. But I didn’t complete get rid of the problem until I searched through the forum again and found another undocumented “wms_include_items”. So what you need is:

1: LAYER
2:     ...
3:     METADATA
4:         ...
5:         "gml_include_items"   "all"
6:         "wms_include_items"   "all"
7:         ...
8:     END
9:     ...
10: END

3. “text/html” is not supported in GetFeatureInfo response; this is also documented but in a very obvious place. “wms_feature_info_mime_type text/html” in the web section of the map files fix the problem:

1: Map
2:     ...
3:     WEB
4:         ...
5:         "wms_feature_info_mime_type" "text/html"
6:         ...
7:     END
8:     ...
9: END

Until now I can finally start creating the html template for GetFeatureInfo response. Although not specific to WMS GetFeatureInfo response, there is a detailed documentation page on MapServer template. As expected, you can define a header template, a footer template and another template for the content.

1: LAYER
2:     ...
3:     HEADER "../template/getfeatureinfo_header.html" # header html template
4:     TEMPLATE "../template/getfeatureinfo_content.html" # content html template
5:     FOOTER "../template/getfeatureinfo_footer.html" # footer html template
6:     ...
7: END

All three html templates are specified at layer level which allows you to customize GetFeatureInfo response differently for individual layer. Two pros of MapServer template are (1) a lot of server related information are exposed in template which you can reference by operation “[]”, e.g. server host, port, map and layer metadata etc instead of just limited to query results of GetFeatureInfo; (2) since there is no other template language involved, I always feel easier to embed javascript code in template which is a big plus. But there are also many cons in this work flow too. If you have more than one included in WMS “query_layers” and the query result has multiple features then the the html content in header and footer template will be repeated for every feature in query result. It’s probably not too difficult to tweak the template to achieve what you want, but I don’t see a clean solution if I just want one header and footer template for all layers. Another thing I didn’t figure out is how to loop through each feature in query result in a more generic way instead of hard code the attribute name in each layer. I think it’s a very simple and typical work flow but I just don’t know how to do it in MapServer. Currently what I did for my “pizzastores” point layer (other two layers have similar but separate templates too) is like below:

header template for “pizzastores” layer:

1: <!-- MapServer Template -->
2: <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/transitional.dtd">
3: <html>
4:   <head>
5:     <!-- enforce the client to display result html as UTF-8 encoding -->  
6:     <meta http-equiv="content-type" content="text/html; charset=UTF-8"></meta>
7:     <style type="text/css">
8:       table, th, td {
9:         border:1px solid #e5e5e5;
10:         border-collapse:collapse;
11:         font-family: arial;          
12:         font-size: 80%;            
13:         color: #333333
14:       }             
15:       th, td {
16:         valign: top;
17:         text-align: center;
18:       }          
19:       th {
20:         background-color: #aed7ff
21:       }
22:       caption {
23:         border:1px solid #e5e5e5;
24:         border-collapse:collapse;
25:         font-family: arial;          
26:         font-weight: bold;
27:         font-size: 80%;      
28:         text-align: left;      
29:         color: #333333;        
30:       }
31:     </style>
32:     <title>GetFeatureInfo Response</title>
33:     
34:   </head>
35:   <body>
36:     <table>
37:       <caption>layer names: pizzastores</caption>
38:       <tbody>
39:         <th>Layer Name</th>
40:         <th>NAME</th>
41:         <th>ADDRESS</th>
42:         <th>TYPE</th>

content template for “pizzastores” layer

1: <!-- MapServer Template -->
2:     <tr>
3:       <td>Pizzastores</td>
4:       <td>[item name=NAME format=$value escape=none]</td>
5:       <td>[item name=ADDRESS format=$value escape=none]</td>
6:       <td>[item name=TYPE format=$value escape=none]</td>
7:     </tr>

footer layer for “pizzastores” layer

1: <!-- MapServer Template -->    
2:       </tbody>
3:     </table>
4:     <br/>
5:   </body>
6: </html>

You notice that in content template for the layer I can only access the current single query result which makes me think the templates will be repeatedly called for each feature in query results. I got the result below in browser but I have multiple <html> tags in the source:

The native GML format for WMS GetFeatureInfo response doesn’t have such issue though.

GeoServer 2.0

For GeoServer (latest released version 2.0.0), I found the work flow of customizing WMS GetFeatureInfo is very straight forward and smooth. All related information and samples are described in “GetFeatureInfo Templates” and “Freemaker Templates” sections of the online user manual, which is neat. Similar to MapServer, GeoServer also uses the concept of header, footer and content html template (templates are with suffix .ftl which is just html with freemaker engine tags). There are two things I really like about GeoServer: (1) templates can be set at different levels like global, workspace (not tested though), datastore, layer so common header and footer template can be shared; (2) the content template is repeatedly applied for each feature collection (meaning all the query results from one layer) instead of each feature such that I can loop through each feature in a generic way which in the end reduces the number of templates I need.

The only place I got trapped is that the online documentation is up to date enough the reflect the data folder structure change introduced in GeoServer 2.0.0. The old “featuretypes” folder is gone (“workspaces” folder is replacing it) but the online manual has a lot of places pointing to it.

Here is what I did for geoserver:

I created header.ftl and footer.ftl templates at global level and copy them to GEOSERVER_DATA_DIR\templates\ (create templates folder if it doesn’t exist):

1: <html>
2:   <head>
3:     <title>Geoserver GetFeatureInfo output</title>
4:   </head>
5:   <style type="text/css">
6:         table, th, td {
7:       border:1px solid #e5e5e5;
8:       border-collapse:collapse;
9:       font-family: arial;          
10:       font-size: 80%;            
11:       color: #333333
12:     }             
13:     th, td {
14:       valign: top;
15:       text-align: center;
16:     }          
17:     th {
18:       background-color: #aed7ff
19:     }
20:     caption {
21:       border:1px solid #e5e5e5;
22:       border-collapse:collapse;
23:       font-family: arial;          
24:       font-weight: bold;
25:       font-size: 80%;      
26:       text-align: left;      
27:       color: #333333;
28:       
29:     }
30:   </style>
31:   <body>

1:   </body>
2: </html>
3:

I created content.ftl template at datastore level which is copied to GEOSERVER_DATA_DIR\workspaces\<my_workspace>\<my_datastore>\

1: <table>
2:   <caption>layer names: ${type.name}</caption>
3:   <tr>
4:   <th>fid</th>
5:     <#list type.attributes as attribute>
6:       <#if !attribute.isGeometry>
7:       <th >${attribute.name}</th>
8:       </#if>
9:     </#list>
10:   </tr>
11: 
12:   <#assign odd=false>
13:   <#list features as feature>
14:     <#if odd>
15:       <tr class="odd">
16:     <#else>
17:       <tr>
18:     </#if>
19:     <#assign odd=!odd>
20:     <td>${feature.fid}</td>    
21:     <#list feature.attributes as attribute>
22:       <#if !attribute.isGeometry>
23:         <td>${attribute.value?string}</td>
24:       </#if>
25:     </#list>
26:     </tr>
27:   </#list>
28: </table>
29: <br/>

New templates seem to require a restart of GeoServer and after that I get GetFeatureInfo response displayed in browser like below:

Extend GeoServer with customized OWS service :: part 5

2010-01-12T08:56:00.001-08:00

In previous post of this series, I summarize the basic work flow of GeoServer OWS dispatcher dispatching OWS requests based on request parameters “service”, “version”, and “request”. We already know that in “ags-ows” service project, the “export” method will finally be called to process the request, and it takes a request bean (AgsOwsExportRequest) instance as input parameter. So in this post let’s pick up from that point and have a look at how response is produced.

We already know ows dispatcher dispatches requests, but after it finds and calls the appropriate method in appropriate service class with request bean it doesn’t stop it job. Instead it keeps doing its job to produce the response.

1: ...
2: // this is where OWS dispatcher finds the service and method and execute it
3: // execute it
4: Object result = execute(request, operation);
5: 
6: // this is where OWS dispatcher keeps its job to produce response
7: //write the response
8: if (result != null) {
9:     response(result, request, operation);
10: }
11: ...

The code snippet above is from OWS dispatcher, the “execute” method actually calls the service operation method (in case of “ags-ows” service, it is the export method in AarcGISServerOWSService class) and return whatever it returns, which will be assigned to “result”. Look at export method again

1: // export method in ArcGISServerOWSService class
2: ...
3: public AgsOwsExportResponse export(AgsOwsExportRequest request) {
4:     return new AgsOwsExportResponse(this.geoServer);
5: }
6: ...

the result is actually in type of AgsOwsExportResponse (you saw that in part 2 but I told you not to worry about then). AgsOwsExportResponse is a subclass of org.geoserver.ows.Response, which is the core element for an OWS service to produce responses. Before we jump into the implementation of AgsOwsExportResponse, let’s first look at how ows dispatcher uses response class. Go back to dispatcher code again and step into the “response(result, request, operation);” method we saw before:

1: ...
2: // response method in org.geoserver.ows.Dispatcher class
3: void response(Object result, Request req, Operation opDescriptor)
4:     throws Throwable {
5: 
6:     // loop through all registered subclass of org.geoserver.ows.Response 
7:     //   in application context and try to match a unique response type based on
8:     //   the class type of result and the binding class of Response subclass   
9: 
10:     ...
11:     ... 
12:     Response response = (Response) responses.get(0);
13:     ...
14:     // you need to implement getMimeType() in Response subclass 
15:     req.httpResponse.setContentType(response.getMimeType(result, opDescriptor));
16:     ...
17:     // you need to implement write() in Response subclass
18:     response.write(result, output, opDescriptor);
19:     ...
20:     // finally flush out the response to clients
21:     req.httpResponse.getOutputStream().flush();
22: }
23: ...

Dispatcher will loop through all the subclasses of org.geoserver.ows.Response registered in application context and find a unique response class whose binding class matches the class type of result object returned by service operation method. Take AgsOwsExportResponse as an example, since it’s binding class is itself which exactly matches the result object returned by export method of ArcGISServerOWSService class, it will be picked up by dispatcher.

After dispatcher locates the appropriate response class, two methods actually matters and thus must be implemented. (1) getMimeType(), which returns mime type string of your response, and it will be set directly in the header of http response back to clients; (2) write(), which writes response body in output stream of http response back to clients. Below is a sample implementation of AgsOwsExportResponse class and how it is registered in application context:

1: public class AgsOwsExportResponse extends Response {
2: 
3:     public AgsOwsExportResponse() {
4:         super(AgsOwsExportResponse.class);
5:     }
6: 
7:     @Override
8:     public String getMimeType(Object value, Operation operation)
9:         throws ServiceException {  
10:         // to simplify the problem, always return image/png
11:         // but in your own ows service, you can decided based on what you support 
12:         //    and what clients request
13:         return "image/png";
14:     }
15:     ...
16:     @Override
17:     public void write(Object value, OutputStream output, Operation operation)
18:         throws IOException, ServiceException {  
19:         ...
20:         // write response map image back to clients 
21:         ...
22:     }
23: }

1: <!-- service operation response -->
2: <bean id="agsOwsCapabilitiesResponse" 
3:     class="org.geoserver.ows.arcgisserver.responses.AgsOwsCapabilitiesResponse"
4:     singleton="false">     
5:     <constructor-arg ref="geoServer"/>   
6: </bean>

Notice the binding class is AgsOwsExportResponse itself but it could definitely be something else if the service operation method export chooses to return something else (org.geoserver.wcs.responses.GetCapabilitiesResponse is a good example of that). As I mentioned in the beginning, “image/png” is hard coded in getMimeType() to simplify the problem. And finally in write() I need to produce a map image and write it in output stream. The implementation of write() method deserves another separate paragraph of explanation which I will do right after, and so far ows dispatcher has finished the whole life cycle for an ows service request dispatching.

Implement Write() in AgsOwsExportResponse and AgsOwsMapProducer

Implementing Write() in AgsOwsExportResponse can be as simple as less than 10 lines of code if you only want to produce and output some simple static content back to clients.

1: private void write(Object value, OutputStream output, Operation operation) 
2:     throws IOException, ServiceException {
3:     String responseStr = "Static content";
4:     OutputStreamWriter writer = new OutputStreamWriter(output);
5:     writer.write(responseStr);
6:     writer.flush();
7: }

But if it is creating a map image, it can definitely become way more complicated (org.vfny.geoserver.wms.responses.GetMapResponse in wms project is a good example), which sometimes needs one or more helper classes (those map producer classes under package org.vfny.geoserver.wms.responses.map are very good example). In AgsOwsExportResponse of “ags-ows” service project I sort of borrowed the idea from GeoServer WMS GetMapResponse class and PNGMapProducer class, in which a helper class similar to PNGMapProducer called AgsOwsMapProducer is created to render map. Of course I omitted most of the details and keeps just enough code to create an map image in png format and write out to client.

In the write() method of AgsOwsExportResponse, I create an instance of GraphicEnhancedMapContext class (from GeoTools library) and pass it to AgsOwsMapProducer. The GraphicEnhancedMapContext instance basically stores all information (e.g. map projection, background color, transparent, layers, styles etc.) needed to produce a map image using GeoTools.

1: private void write(Object value, OutputStream output, Operation operation)
2:       throws IOException, ServiceException {    
3:     
4:     AgsOwsExportResponse exportResponse = (AgsOwsExportResponse)value;
5:     AgsOwsExportRequest exportRequest 
6:         = (AgsOwsExportRequest)OwsUtils.parameter(operation.getParameters(), AgsOwsExportRequest.class);
7:     
8:     /*
9:      * the main purpose here is to create mapContext {GraphicEnhancedMapContext} -
10:      * - and pass it to map producer to generate the map
11:      * - borrow the idea from GetMapResponse but omitted a lot details
12:      */    
13:     // requested image crs
14:     final CoordinateReferenceSystem imageSR = exportRequest.getImageSR();
15:     // requested layers
16:     final LayerInfo[] layers = exportRequest.getLayers();          
17:     // initialize mapContext
18:     this.mapContext = new GraphicEnhancedMapContext();    
19:     try {
20:       this.mapContext.setCoordinateReferenceSystem(imageSR);
21:     } catch(FactoryException e) {
22:       e.printStackTrace();
23:     } catch(TransformException e) {
24:       e.printStackTrace();
25:     }        
26:     // bbox and bbox crs
27:     final CoordinateReferenceSystem bboxSR = exportRequest.getBboxSR();
28:     final Envelope bbox = exportRequest.getBbox();
29:     if(bboxSR != null) {
30:       this.mapContext.setAreaOfInterest(bbox, bboxSR);
31:     } else {
32:       // TODO: should throw exception, no?
33:       this.mapContext.setAreaOfInterest(bbox, DefaultGeographicCRS.WGS84);
34:     }
35:     
36:     this.mapContext.setMapWidth(exportRequest.getWidth());
37:     this.mapContext.setMapHeight(exportRequest.getHeight());
38:     this.mapContext.setBgColor(exportRequest.getBgColor());
39:     this.mapContext.setTransparent(exportRequest.isTransparent());
40:     
41:     try {
42:       for (int i=0; i<layers.length; i++) {      
43:         final Style layerStyle = layers[i].getDefaultStyle().getStyle();        
44:         final MapLayer layer;                
45:         if(layers[i].getType().getCode() == LayerInfo.Type.VECTOR.getCode()) {
46:           FeatureSource<? extends FeatureType, ? extends Feature> featureSource;
47:           FeatureTypeInfo resource = (FeatureTypeInfo)layers[i].getResource();
48:           if(resource.getStore() == null || resource.getStore().getDataStore(null) == null) {
49:                   throw new IOException("");
50:               }
51:           Hints hints = new Hints(ResourcePool.REPROJECT, Boolean.valueOf(false));
52:           featureSource = resource.getFeatureSource(null, hints);
53:           
54:           layer = new FeatureSourceMapLayer(featureSource, layerStyle);
55:                     layer.setTitle(layers[i].getResource().getName());                    
56:                     // use default filter and version in DefaultQuery
57:                     final DefaultQuery definitionQuery = new DefaultQuery(featureSource.getSchema().getName().getLocalPart());                                      
58:                     layer.setQuery(definitionQuery);
59:                     mapContext.addLayer(layer);                    
60:         } else if(layers[i].getType().getCode() == LayerInfo.Type.RASTER.getCode()) {
61:           //
62:         }
63:         
64:         // default outputFormat to 'png' and mimeType to 'image/png'
65:         this.mapProducer = new AgsOwsMapProducer();
66:         this.mapProducer.setMapContext(this.mapContext);
67:         
68:         this.mapProducer.produceMap();
69:         this.mapProducer.writeTo(output);
70:       }
71:     } catch(Exception e) {
72:       e.printStackTrace();
73:     }
74:   }
75:

And AgsOwsMapProducer, which takes the map context, finally renders a map using GeoTools StreamingRenderer by calling produceMap() and write out image stream by calling writeTo().

1: ...
2: public void produceMap() throws Exception {  
3:     ...
4:     Rectangle paintArea 
5:         = new Rectangle(0, 0, mapContext.getMapWidth(), mapContext.getMapHeight());
6:     String antialias = "none";
7:     IndexColorModel palette = null;
8:     ...
9:     boolean useAlpha = true;
10:     final boolean transparent = mapContext.isTransparent();
11:     ...
12:     final Color bgColor = mapContext.getBgColor();
13:     ...
14:     final RenderedImage preparedImage 
15:         = ImageUtils.createImage(paintArea.width, paintArea.height, palette, useAlpha);
16:     ...
17:     final Map<RenderingHints.Key, Object> hintsMap 
18:         = new HashMap<RenderingHints.Key, Object>();      
19:     final Graphics2D graphic 
20:         = ImageUtils.prepareTransparency(transparent, bgColor, preparedImage, hintsMap);
21:     ...
22:     graphic.setRenderingHints(hintsMap);
23:     ...
24:     StreamingRenderer renderer = new StreamingRenderer();      
25:     renderer = new StreamingRenderer();        
26:     renderer.setContext(mapContext);
27:       
28:     RenderingHints hints = new RenderingHints(hintsMap);
29:     renderer.setJava2DHints(hints);
30:               
31:     Map<Object, Object> rendererParams = new HashMap<Object, Object>();
32:     rendererParams.put("optimizedDataLoadingEnabled", new Boolean(true));    
33:     rendererParams.put("maxFiltersToSendToDatastore",  DefaultWebMapService.getMaxFilterRules());
34:     rendererParams.put(ShapefileRenderer.SCALE_COMPUTATION_METHOD_KEY, ShapefileRenderer.SCALE_OGC);    
35:     ...
36:     rendererParams.put(StreamingRenderer.ADVANCED_PROJECTION_HANDLING_KEY, true);
37:     ...
38:     renderer.setRendererHints(rendererParams);
39:     try {
40:         final ReferencedEnvelope dataArea = mapContext.getAreaOfInterest();
41:         renderer.paint(graphic, paintArea, dataArea);        
42:     } finally {
43:         graphic.dispose();
44:     }
45:     ...
46:     this.image = preparedImage;
47: }
48: ...
49: public void writeTo(OutputStream out) throws ServiceException, java.io.IOException {
50:     ...
51:     final String format = getOutputFormat();
52:     ...
53:     new ImageWorker(image).writePNG(
54:         outStream, 
55:         "FILTERED", 
56:         100.0f,  
57:         true,
58:         image.getColorModel() instanceof IndexColorModel
59:     );
60:     ...
61: }

Note: the sample above for both produceMap() and writeTo() omitted a lot of details which can not be simply copied and paste to be compiled. But I will provide the link to download the sample code in the end of the series.

Wrap it up

So far I’ve finished everything I planned for this little “ags-ows” project. So just restart GeoServer and the customized OWS service “ags-ows” will be ready to serve out map. Here are two sample requests and result maps:

http://localhost:8080/geoserver/ows?service=agsows&request=export&version=1.0.0&bbox=-180,0,0,90&transparent=true&size=512,256&bboxSR=4326&imageSR=4326&layers=show:states

http://localhost:8080/geoserver/ows?service=agsows&request=export&version=1.0.0&bbox=-20037508.34,-20037508.34,20037508.34,20037508.34&transparent=true&size=512,512&bboxSR=900913&imageSR=900913&layers=show:states

This is the final part of the whole series and the source code covered in my little "ags-ows" project is packaged as geoserver-playground-ags-ows.zip, which can be downloaded from here. Feel free to leave any feedback and comments.

Extend GeoServer with customized OWS service :: part 4

2010-01-09T19:32:00.001-08:00

In previous post of this series I created request bean class AgsOwsExportRequest for “ags-ows” service and helper class AgsOwsExportRequestKvpReader to populate request bean with parsed value objects from kvp parsers. Now it’s time to have a close look at how OWS dispatcher exactly dispatches requests and produces responses.

In part 2 I briefly mentioned GeoServer’s OWS dispather (org.geoserver.ows.Dispatcher), which is also re-used by “ags-ows” service to handle requests. So what really happens under the hood? GeoServer as J2EE application is built upon Java Servlet technology and Spring framework (Wicket was introduced later for the web UI). No matter whether you start GeoServer in built Jetty web server or deploy it in other containers like Tomcat, the start point is always the DispatcherServlet (org.springframework.web.servlet.DispatcherServlet) declared in the “web-app” project of GeoServer. So if you look at the “web.xml” in “web-app” project, you will find following servlet definition:

1: <servlet>
2:       <servlet-name>dispatcher</servlet-name>
3:       <servlet-class>
4:           org.springframework.web.servlet.DispatcherServlet
5:       </servlet-class>
6: </servlet>

you will also find following servlet mapping for different GeoServer url patterns as below:

1:     <servlet-mapping>
2:         <servlet-name>dispatcher</servlet-name>
3:         <url-pattern>/web/*</url-pattern>
4:     </servlet-mapping>
5:     <servlet-mapping>
6:       <servlet-name>dispatcher</servlet-name>
7:       <url-pattern>/rest/*</url-pattern>
8:     </servlet-mapping>
9:     <servlet-mapping>
10:       <servlet-name>dispatcher</servlet-name>
11:       <url-pattern>/security/*</url-pattern>
12:     </servlet-mapping>    
13:     <servlet-mapping>
14:       <servlet-name>dispatcher</servlet-name>
15:       <url-pattern>/wms/*</url-pattern>
16:     </servlet-mapping>
17:     <servlet-mapping>
18:       <servlet-name>dispatcher</servlet-name>
19:       <url-pattern>/wcs/*</url-pattern>
20:     </servlet-mapping>
21:     <servlet-mapping>
22:       <servlet-name>dispatcher</servlet-name>
23:       <url-pattern>/wfs/*</url-pattern>
24:     </servlet-mapping>
25:     <servlet-mapping>
26:       <servlet-name>dispatcher</servlet-name>
27:       <url-pattern>/ows/*</url-pattern>
28:     </servlet-mapping>
29:     <servlet-mapping>
30:       <servlet-name>dispatcher</servlet-name>
31:       <url-pattern>/wfsv/*</url-pattern>
32:     </servlet-mapping>
33:     <servlet-mapping>
34:       <servlet-name>dispatcher</servlet-name>
35:       <url-pattern>/wcs111/*</url-pattern>
36:     </servlet-mapping>
37:     <servlet-mapping>
38:       <servlet-name>dispatcher</servlet-name>
39:       <url-pattern>/kml/*</url-pattern>
40:     </servlet-mapping>
41:     <servlet-mapping>
42:      <servlet-name>dispatcher</servlet-name>
43:      <url-pattern>/styles/*</url-pattern>
44:     </servlet-mapping>
45:     <servlet-mapping>
46:      <servlet-name>dispatcher</servlet-name>
47:      <url-pattern>/www/*</url-pattern>
48:     </servlet-mapping>
49:     <servlet-mapping>
50:      <servlet-name>dispatcher</servlet-name>
51:      <url-pattern>/temp/*</url-pattern>
52:     </servlet-mapping>
53:     <servlet-mapping>
54:      <servlet-name>dispatcher</servlet-name>
55:      <url-pattern>/history/*</url-pattern>
56:     </servlet-mapping>
57:     <servlet-mapping>
58:      <servlet-name>dispatcher</servlet-name>
59:      <url-pattern>/gwc/*</url-pattern>
60:     </servlet-mapping>
61:     <servlet-mapping>
62:      <servlet-name>dispatcher</servlet-name>
63:      <url-pattern>/pdf/*</url-pattern>
64:     </servlet-mapping>

almost all types of requests will be handled by that Spring DispatcherServlet at the first place. According to Spring framework documentation, DispatcherServlet is: “the central dispatcher for HTTP request handlers/controllers, e.g. for web UI controllers or HTTP-based remote service exporters. Dispatches to registered handlers for processing a web request, providing convenient mapping and exception handling facilities.” So HTTP requests that reach it will be dispatched to different sub-routine based on registered handlers, and as mentioned in part 2 OWS dispatcher is one of those registered handler (as a org.springframework.web.servlet.handler.SimpleUrlHandlerMapping):

1:   <bean id="dispatcher" class="org.geoserver.ows.Dispatcher"/>
2:   <bean id="dispatcherMapping" 
3:             class="org.springframework.web.servlet.handler.SimpleUrlHandlerMapping">
4:     <property name="alwaysUseFullPath" value="true"/>
5:     <property name="mappings">
6:       <props>
7:         <prop key="/ows">dispatcher</prop>
8:         <prop key="/ows/**">dispatcher</prop>
9:         <prop key="/styles/**">filePublisher</prop>
10:         <prop key="/www/**">filePublisher</prop>
11:       </props>
12:     </property>
13:   </bean>

When ows requests come, they will be captured and handled in handleRequestInternal() method of dispatcher.

1: protected ModelAndView handleRequestInternal(HttpServletRequest httpRequest,
2:         HttpServletResponse httpResponse) throws Exception {
3:     ...
4:     // handle OWS requests
5:     ...
6: }

Within the dispatcher, for each request parameter it loops through all registered kvp parsers and apply them when “service” and “version” match. After that dispatcher will try to find the correct service and operation among all those being registered in application context based on the value of request parameter “service”, “version” and “request”. Take my “ags-ows” service as an example, if a request comes in like :

http://localhost:8080/geoserver/ows?service=agsows&request=export&bbox=-115.8,30.4,-85.5,50.5&bboxSR=4326&layers=show:0,1,2&size=800,600&imageSR=4326&transparent=true

the dispatcher will find the instance of ArcGISServerOwsService and dispatch request to its export method because of what I’ve registered below in application context:

1: <bean id="ags-ows" 
2:     class="org.geoserver.ows.arcgisserver.ArcGISServerOWSService">
3:     <constructor-arg ref="geoServer"></constructor-arg>
4: </bean>   
5: <bean id="ags-ows-1.0.0" class="org.geoserver.platform.Service">
6:     <constructor-arg index="0" value="agsows"></constructor-arg>
7:     <constructor-arg index="1" ref="ags-ows"></constructor-arg>
8:     <constructor-arg index="2" value="1.0.0"></constructor-arg>
9:     <constructor-arg index="3">
10:         <list>            
11:             <value>Export</value>                            
12:         </list>
13:     </constructor-arg>
14: </bean>

After dispatcher locate the appropriate service and operation, it is aware which method in which service class to dispatch request to (in this case it’s the export() method in ArcGISServerOWSService class), it also knows number of types of input parameters for that method. So it will create the request bean by using an appropriate kvp request reader, and finally calls the operation method with request bean. You will see following in ows dispatcher.

1: ...
2: //dispatch the operation
3: Operation operation = dispatch(request, service);
4: 
5: //execute it
6: Object result = execute(request, operation);
7: ...

If you follow previous posts closely, you probably notice all necessary parts so far in this work flow have been covered such that the OWS dispatcher is able to find ArcGISServerOWSService class and call its export method

1: /**
2:   * a sample request:  
3:   * 
4:   * 
5:   */
6: public AgsOwsExportResponse export(AgsOwsExportRequest request) {
7:     return new AgsOwsExportResponse(this.geoServer);
8: }
9:

What happens next is the most import part in this “ags-ows” service sample. In next post of this series I will focus on extending org.geoserver.ows.Response class to produce map response for “ags-ows” services and sends back to clients.

Extend GeoServer with customized OWS service :: part 3

2010-01-08T09:22:00.001-08:00

In previous post of this series, I created the application context for “ags-ows” service project and also register a java bean of type org.geoserver.platform.Service in it (a bean instance of ArcGISServerOWSService along with the service name ‘ags-ows’ and version ‘1.0.0’ are passed in into constructor) so that GeoServer’s ows dispatcher can be reused to redirect requests to “ags-ows” service. A set of KvpParsers were also created and registered in application context to parse request parameters. So in this post, I will move onto AgsOwsExportRequest which is the request bean class for “ags-ows” service, and AgsOwsExportRequestKvpReader (a subclass of KvpRequestReader) which is used to to create request bean from parsed objects of request parameters.

Request bean and KvpRequestReader

A request bean is basically a java object carrying all parsed parameter key-value pairs (as well as their getter and setter methods) of a specific service operation’s request. For example GeoServer’s WMS service has GetMap operation so which uses GetMapRequest (org.vfny.geoserver.wms.requests.GetMapRequest) bean, and same idea GetCapabilities operation has WMSCapabilitiesRequest bean. Now for “ags-ows” service, since “export” is the only operation declared in ArcGISServerOWSService, I will create one request bean class and call it AgsOwsExportRequest (below is a sample implementation that omits getters and setters):

1: public class AgsOwsExportRequest {
2:     
3:   private LayerInfo[] layers;
4:   private Envelope bbox;
5:   private AgsOwsExportSize size;  
6:   private CoordinateReferenceSystem imageSR;  
7:   private CoordinateReferenceSystem bboxSR;
8:   private boolean transparent;
9:   private Color bgColor = Color.WHITE;
10:   
11:         public AgsOwsExportRequest() {}
12:        /*
13:         * omitted getter and setter methods
14:         * you need to add getter and setter in your own code 
15:         */
16: }

you probably notice that each request bean parameter represents one request parameter of “ags-ows” service, and the type of the parameter matches the binding class of parameter’s kvp parser. (e.g. AgsOwsKvpLayersParser binds to List<LayerInfo>, AgsOwsKvpSizeParser binds to AgsOwsExportSize class etc.)

Usually an OWS service’s operation logic takes its own request bean as input parameter (that’s why you saw AgsOwsExportRequest in previous post which I didn’t explain much then). Before the dispatcher redirects request the a service operation logic for further processing, it will create an instance of appropriate type of request bean and pass it in, and it’s actually a KvpRequestReader subclass’s job to create such request bean and populate it with parsed parameter objects from kvp parsers. So I created AgsOwsExportRequestKvpReader as below, which extends KvpRequestReader:

1: public class AgsOwsExportRequestKvpReader extends KvpRequestReader {
2: 
3:   
4:   public AgsOwsExportRequestKvpReader() {
5:     super(AgsOwsExportRequest.class);    
6:   }
7: 
8:   @Override
9:   public Object createRequest() throws Exception {
10:     AgsOwsExportRequest requestBean = new AgsOwsExportRequest();    
11:     return requestBean;
12:         }
13:   
14:   @Override
15:   public Object read(Object request, Map kvp, Map rawKvp) throws Exception {
16:     AgsOwsExportRequest exportRequest = (AgsOwsExportRequest)super.read(request, kvp, rawKvp);
17:     /*
18:      * do any specific setting for AgsOwsExportRequest request bean instance - 
19:      * - based on kvp and rawKvp 
20:      */
21:     // TODO: here shall I set default value for certain parameters -
22:     // - if they are missing from kvp?    
23:     return exportRequest;
24:   }
25: }

To extend KvpRequestReader, you basically override its createRequest() and read() method, and optionally set attribute “filter”.

createRequest() creates and returns a new instance of appropriate request bean object. Default creatRequest() does that by creating the new instance reflectively using KvpRequestReader.requestBean class (you set that class in constructor), but if you need to set extra attribute of request bean object then you should probably override it.
read() is usually called in ows dispather, and it takes a request bean instance created by createReques() as well as kvp (java.util.Map) parsed by KvpParsers and unparsed raw kvp (java.util.Map), and it is suppose to loop through each key-value pair of parsed kvp map and set the applicable value in request bean instance if there is a setter defined for the key.
“filter” is an optional attribute (through setFilter(Set<String> filter)) you can set so that certain keys in kvp can be filtered out without setting the its value to request bean instance.

Finally as always, you need register your KvpRequestReader subclass in application context (but you don’t need do so for request bean class):

1:   <bean id="agsOwsExportKvpRequestReader" 
2:     class="org.geoserver.ows.argisserver.kvp.AgsOwsExportRequestKvpReader">
3:   </bean>

Now it’s time to validate what I’ve added in “ags-ows” service project in this post so far. Restart GeoServer in debug mode, and set a break point in org.geoserver.ows.Dispatcher’s parseRequestKvp method (see highlighted line below), and send a “ags-ows” service request:

http://localhost:8080/geoserver/ows/agsows?service=agsows&request=export&version=1.0.0&bbox=-124.731,24.956,-66.97,49.372&transparent=true&size=512,384&bboxSR=4326&imageSR=4326&layers=show:states

1:     Object parseRequestKVP(Class type, Request request)
2:         throws Exception {
3:         KvpRequestReader kvpReader = findKvpRequestReader(type);
4: 
5:         if (kvpReader != null) {
6:             //check for http request awareness
7:             if (kvpReader instanceof HttpServletRequestAware) {
8:                 ((HttpServletRequestAware) kvpReader).setHttpRequest(request.httpRequest);
9:             }
10: 
11:             Object requestBean = kvpReader.createRequest();
12: 
13:             if (requestBean != null) {
14:                 requestBean = kvpReader.read(requestBean, request.kvp, request.rawKvp);
15:             }
16: 
17:             return requestBean;
18:         }
19: 
20:         return null;
21:     }

when it stops at the break point, check if variable “requestBean” is of class AgsOwsExportRequest, and also check if those attributes of AgsOwsExportRequest (e.g. “layers”, “bbox”, “imageSR” etc.) are populated with correct type of object values from kvp parsers. If so, it means AgsOwsExportRequest and AgsOwsExportRequestKvpReader are working as expected.

By now enough logics and functions have been added into “ags-ows” service to enable it to parse the request from client, in next post of this series, I will discuss more on the whole picture of how an ows request is processed by GeoServer OWS dispatcher, which is also a good half-way summary of what I've done so far.