Monday, December 20, 2010

Cool feature called “Destination Maps” from Bing Maps

Recently I found this cool feature called “Destination Maps” from Bing Maps, which basically allows you to pick up a location, adjust an extent and then create a map out of it. The real cool thing though is the “sketchy” and “treasure” map style. BTW you can export the final map as PDF or JPEG format in different size. Here are the two maps I created:



Looks good, huh? It makes a good poster.

Saturday, November 20, 2010

Hack OpenGeo Styler to work with generic WMS servers

As part of the OpenGeo Suite, Styler is a web based WMS viewer plus SLD editor built upon OpenLayers and GeoExt. Although it uses WMS and SLD protocols, current release of Styler must be coupled with a running GeoServer instance. So it is impossible for Styler to work with a generic WMS server due to the following issues:
  • WMS layers must come from GeoServer instance on the same machine as “/geoserver/wms?”
  • Every WMS layer must have has a WFS feature type at back end which must come from “/geoserver/wfs?”
  • There is no where to configure proxy so it doesn’t work with a remote server
  • Instead of WMS GetStyle, Styler is using GeoServer REST API (not a standard) to load SLD from WMS
  • Instead of WMS PutStyle, Styler is using GeoServer REST API (not a standard) to push and persist styles back to WMS
  • Styler is not using either standard WMS parameter “SLD” or “SLD_BODY” to make stateless change of map styles
  • Styler requires the WMS to support DescribeLayer interface
By briefly looking through its source code, I found it’s not too hard to overcome those issues and make Styler talk to a generic WMS server (e.g. ArcGIS Server) as long as the target WMS meets following prerequisites:
  • WMS must support EPSG:900913 (EPSG:3857 or Esri’s EPSG:102110 is ok but requires a little tweak in OpenLayers)
  • WMS must support GetStyle interface
  • Each layer in WMS must have a WFS feature type at back end in order to get attributes list
  • Each style of a WMS layer must have a SLD definition which can be retrieved through GetStyle
  • Optionally WMS needs to support PutStyle interface if one wants to persist SLD styles at server side
  • Optionally WMS needs to support SLD_BODY to make stateless change of rendering and symbology 
So in this article, I’m going to go through the basic steps to tweak the OpenGeo Styler to communicate with a non-GeoServer WMS:

1. Checkout Styler source code from

2. Build Styler
sudo easy_install jstools
mkdir script
jsbuild build.cfg -o script
After a successful build there should be 4 js files in script folder.
Note: I only found out how I can build it on Linux, and I didn’t try on Windows

3. Setup debug environment for Styler
To launch Styler application simply follow the instructions here. By default, the index.html references merged and compressed js file in “script” folder, in order to modify and debug Styler you will need to reference those uncompressed js files in index.html as below:
1: ...
2: <script src="../styler/externals/openlayers/lib/OpenLayers.js"></script>
3: <script src="../styler/externals/ext/adapter/ext/ext-base.js"></script>
4: <script src="../styler/externals/ext/ext-all.js"></script>
5: <script src="../styler/externals/geoext/lib/GeoExt.js"></script>
6: <script src="../styler/script/gxp.js"></script>
7: <script src="../styler/lib/Styler.js"></script>
8: <script src="../styler/lib/Styler/dispatch.js"></script>
9: <script src="../styler/lib/Styler/ColorManager.js"></script>        
10: <script src="../styler/lib/Styler/SchemaManager.js"></script>
11: <script src="../styler/lib/Styler/SLDManager.js"></script>      
12: <script src="../styler/externals/ux/colorpicker/color-picker.ux.js"></script>
13: ...
Now you should be able to debug into OpenLayers and Styler in browser.

4. Hack the code
You don’t actually need to touch any code in OpenLayers or GeoExt, all changes are within three .js files in Styler itself, which are:
Code changes in those files are explained in details below.

Code Changes in Styler.js

a. Add three properties for class Styler
1: // right before the “constructor”, and after other properties
2: ...
3: wmsUrl: "/geoserver/wms?",
4: wfsUrl: "/geoserver/wfs?",  
5: proxyUrl: "",
6: ...

b. Within Styler.js replace “/geoserver/wms?” or “/geoserver/ows?” with “this.proxyUrl + this.wmsUrl”, and replace “/geoserver/wfs?” with “this.proxyUrl + this.wfsUrl”. This is to allow user to point Styler to any WMS, either remote or local.

Note: there is one exception though, in method createLayers() you just need to replace “/geoserver/wms?” with this.wmsUrl because WMS layer doesn’t need a proxy to request map image.

c. Avoid using WMS DescribeLayer
1: //locate method:
2: describeLayers: function(callback) {
3:   ...
4: }
5: //Comment out original code and add following chunk of code
6: describeLayers: function(callback) {
7: for (var i=0, ii=this.wmsLayerList.length; i<ii; ++i) {
8:                   config = this.wmsLayerList[i];
9:                   // assume each WMS layer has a backend WFS layer
10:                   this.layerList.push(config);
11:             }
12:             callback();
13:   }
14: //here assumption is that every WMS layer has a WFS feature type at back end

d. If your WMS supports Google Mercator projection as something other than “EPSG:900913”, In method createLayers() you will need to add an extra parameter (e.g. “srs: ‘EPSG:3857’”) when creating each WMS layer
1: createLayers: function() {
2: ...
3:     layers.push(
4:     new OpenLayers.Layer.WMS(
5:         config.title, this.wmsUrl, {
6:         layers:,
7:         styles: config.styles[0].name,
8:         transparent: true,
9:         format: "image/png",
10:         srs: "EPSG:3857"  // EPSG:3857, EPSG:102110 or EPSG:102113 etc.
11:         }, {
12:     }));
13: ...
14: }

e. Pass proxyUrl, wmsUrl and wfsUrl into SLDManager and SchemaManager
1: getSchemas: function(callback) {
2:             this.schemaManager = new Styler.SchemaManager(
3:   ,
4:             {
5:               proxyUrl: this.proxyUrl,
6:               wfsUrl: this.wfsUrl
7:             }
8:           );
10:   getStyles: function(callback) {
11:         this.sldManager = new Styler.SLDManager(, {'proxyUrl': this.proxyUrl});
12:             this.sldManager.loadAll(callback);
13:       }
15:   initEditor: function() { {
16:   …
17:  this.sldManager = new Styler.SLDManager(, {'proxyUrl': this.proxyUrl});
18:   ...
19: }
20: ...
22: // Note: the original constructors for SLDManager and SchemaManager takes a single input parameter
23: // but they will be modified to take 2nd input parameters in SLDManager.js and SchemaManager.js

Changes in SLDManager.js

a. Add an extra parameter called “proxyUrl” right before the constructor
1: // right before the “constructor”, and after other properties
2: ...
3: proxyUrl: "",
4: ...

b. Modify SLDManager constructor to take 2nd input parameter
1: initialize: function(map, options) {
2: = map;
3:     var layer;
4:     this.layers = [];
5:     this.layerData = {};
6:     this.format = new OpenLayers.Format.SLD({multipleSymbolizers: true});
7:     for (var i=0; i<; ++i) {
8:         layer =[i];
9:         if(layer instanceof OpenLayers.Layer.WMS) {
10:             this.layers.push(layer);
11:         }
12:     }
13:     // this is important so user can pass in ‘proxyUrl’
14:     OpenLayers.Util.extend(this, options);  
15: },

c. Overwrite getUrl() so it sends GetStyle request to retrieve WMS SLD defintion, here is an example
1: getUrl: function(layer, styleName) {
2:     var url;
3:     if(layer instanceof OpenLayers.Layer.WMS) {
4:         url = layer.url.split("?")[0] + "?" + "version=1.3.0&request=GetStyles&layers=" + layer.params["LAYERS"];       
5:         OpenLayers.Console.log(url);
6:     }
7:     //TODO handle other layer types
8:     return url;
9: }

d. In loadSld() method replace  this.getUrl(layer, styleName) with this.proxyUrl + this.getUrl(layer, styleName)
1: ...
2: loadSld: function(layer, styleName, callback) {
3:         Ext.Ajax.request({
4:             url: this.proxyUrl + this.getUrl(layer, styleName),
5:             method: "GET",
6:             …              
7: ...
8: // obviously there may be more than one styles for each WMS layer, 
9: // in this case the first style is always used.

e. In saveSld() method, replace original code with following to use “SLD_BODY” parameter to make stateless change of rendering and symbology
1: saveSld: function(layer, callback, scope) {
2:     this.layerData[] = "__internalstyle__";
3:         var sldBody = this.format.write(this.layerData[].sld, {});
4:     layer.mergeNewParams({
5:       styles: "__internalstyle__", 
6:       sld_body: sldBody
7:     });
8: || this);  
9:   }
10: // By using “SLD_BODY” any change to the style is stateless which will not affect other users. 
11: // But if you want to persist modified styles on server side, 
12: // you must use WMS PutStyle operation and also make sure server supports it

Changes in SchemaManager.js

a. Add two extra parameters called “wfsUrl” and “proxyUrl” right before the constructor
1: // right before the “constructor”, and after other properties
2: ...
3: proxyUrl: "",
4: wfsUrl: “”,
5: ...

b. Modify SchemaManager constructor to take 2nd input parameter
1: initialize: function(map, options) {
2:             // other code
3: ...
4:             OpenLayers.Util.extend(this, options);
5:        ...     
6:             for(var i=0; i<; ++i) {
7:                   layer =[i];
8:                   if(layer instanceof OpenLayers.Layer.WMS) {
9:                         this.attributeStores[] = new{
10:                               //url: layer.url.split("?")[0].replace("/wms", "/wfs"),
11:                           // TODO: need better logic to handle case where this.wfsUrl is empty or must be derived from this.wmsUrl
12:                           url: this.proxyUrl + this.wfsUrl, 
13:                               baseParams: {
14:                                     //version: "1.1.1",    // is there a WFS 1.1.1?
15:                             version: "1.1.0",
16:                                     request: "DescribeFeatureType",
17:                                     typename: layer.params["LAYERS"]
18:                               }
19:                         });
20:                   }
21:             }
22:       }

5. Rebuild Styler and test against a generic WMS server.

So far the OpenGeo Styler has been hacked to communicate with a generic WMS server. To test it you just need to pass in the urls for WMS, WFS and proxy into Styler constructor in the original index.html like below.
1: Ext.BLANK_IMAGE_URL = "theme/img/blank.gif";
2: Ext.onReady(function() {
3:     window.styler = new Styler({
4:         baseLayers: [                              
5:             new OpenLayers.Layer.OSM(
6:                 "Open Street Map",
7:                 "${z}/${x}/${y}.png",
8:                 {numZoomLevels: 19}
9:             )                                                             
10:         ],                        
11:         wmsUrl:"http://localhost:8399/<WMSServer_endpoint>?",
12:         wfsUrl:"http://localhost:8399/<WFSServer_endpoint>?",
13:         proxyUrl: "http://locahost:8080/openlayers-trunk/ApacheProxyServlet?url="
14:     });
15: });

Some screenshots:

Styler load SLD style definition through WMS GetStyle operation

Styler changes map symbology through SLD_BODY parameter

6. Things to improve
  • If a layer has more than one styles, provide GUI for users to select 
  • Make OpenGeo Styler to persist SLD styles on server through WMS PutStyle operation

Thursday, October 21, 2010

Using USGS High Resolution Orthoimagery in JOSM

One of CarbonCloud’s recent article pointed me to USGS Seamless Data Warehouse, where many high resolution orthoimagery data is distributed as OGC WMS. It’s a perfect base map option for OSM contributors using JOSM because of the following reasons:
  • The super image quality and resolution (0.25 foot to 2.5 foot);
  • Distributed through open standard based interface (WMS), which can be directly used in JOSM;
  • Compatibility with OSM license makes it legal to be used to derive OSM data on. (features digitized based on Google Maps and Bings are actually considered as derivative work of them, which then is copyrighted)
An obvious disadvantage though is the data coverage because only very limited area are covered by those orthoimagery data. See data availability.
Here are some basic steps to use USGS high resolution orthoimagery in JOSM:
1. Install JOSM and WMS plugin
2. Find the WMS containing the orthoimagery data that covers the area you’re interested in
Here is the list of available WMS services and you can probably tell the covered area by service name. For example, I am interested in making some OSM edits in Redlands, CA, United States, so I pick the WMS USGS_EDC_Ortho_California, in which there is particular layer for Riverside and San Bernardino counties.
3. Start JOSM and load the WMS layer in JOSM WMS plugin
After you download the data from OSM in JOSM, you need to register WMS USGS_EDC_Ortho_California in JOSM through WMS Plugin Preference. So just go to Edit—>Preferences—>WMS tab;
Click “Add”, and type in a name and url as part of a WMS GetMap request
It’s a little tricky for WMS URL because you need to append a few GetMap request parameters manually. It seems to me that JOSM WMS plugin will only automatically populate “BBOX” parameter so you must manually complete other request parameters. Here is the URL that I came up with:
Notice that I indicate the layer “Riverside-SanBernardinoCA_0.3m_Color_Mar_2008”, which is the area I am interested in.
Note: you should request WMS map in WGS84 (EPSG:4326) because the OSM data overlay on it are all in longitude and latitude.
That’s it, enjoy the armchair mapping!
Below is a snapshot of JOSM with high resolution orthoimagery as base map
Same area with Yahoo Imagery as a comparison
Is that amazing! Only a little issue of JOSM WMS plugin itself though: if you zoom in after loading the WMS image you need to delete and add the same WMS again to refresh data from WMS, otherwise the image at previous zoom level will simply be resample and used.

This OSM wiki page has a lot of useful information regarding using USGS High Resolution Orthoimagery in JOSM too.

Wednesday, October 13, 2010

IP-based geolocation can also be accurate

I always thought that IP-based geolocation is not as accurate as it should be until quite recently I ran into this HTML5 demo page (Geolocation API is one of the new features in HTML5), and now I have to admit that it was my prejudice.
If you open the demo page in browser and assume you don’t have a GPS device connected to your machine, then all this web app does is to detect your location based on your IP address and add a maker on Google Maps. The source code is also as simple as a “HelloWorld” sample of W3C Geolocation API:
if (navigator.geolocation) {
navigator.geolocation.getCurrentPosition(success, error);
} else {
error('not supported');
function success(position) {
// ...non-related code omitted...
var latlng = new google.maps.LatLng(position.coords.latitude, position.coords.longitude);  
var map = new google.maps.Map(document.getElementById("mapcanvas"), myOptions);
var marker = new google.maps.Marker({
position: latlng, 
map: map, 
title:"You are here!"
function error(msg) {
// ...non-related code omitted...

I tried this geolocation demo app twice at my bedroom, one using Firefox 3.6 and the other is using Google Chrome, and below are how the app locates me:

Firefox 3.6 Successfully locate me in my land lot

Google Chrome is a little off my land lot, but still quite accurate

The results of both are very accurate to where I actually was. That’s just amazing! And I guess the conclusion is that IP-based Geolocation can also be really accurate.

Thursday, August 19, 2010

NASA Earth Observation (NEO) Website is awesome

I recently came cross this NEO (NASA Earth Observation) website that contains a lot of recent and historical satellite image data related to environment and climate change. Unlike most other websites of the same type, NEO’s simple and pretty web GUI design and user experience really surprises me.


Screenshot taken from NASA Earth Observation site at

The dataset hosted by NEO are divided into categories like ocean, atmosphere , energy, land and life. For each of those categories users can browse, search, preview, download, and analysis which is pretty much all you may want to do with satellite data. What’s really awesome is that you can do all of those things in a single ajax-based web page (you see only one screenshot is enough).

Discover data

To find the data, you can select one of the main categories, and then either search or look through the full list. Once it’s found you will be able to see the brief description and preview it in an interactive web map (support zoom in/out)

Download option

Most of the data can be downloaded as PNG, JPEG, GeoTiff, CSV, KML/KMZ directly through NEO home page, and whenever applicable you can also download data in original format e.g. HDF on the same page of data search result.

Web Map Service

NEO also provides WMS access to its data, which is a super plus for those who don't want to download and host data themselves.  NEO’s WMS links:


Consume NEO WMS in QGIS

NASA Earth Observation site is just awesome!!!

Wednesday, August 11, 2010

Mysterious “BBOX” parameter in Web Feature Service (WFS)

“BBOX” is a standard parameter defined as part of the OGC Web Feature Service (WFS) keyword-value encoding which is used in GetFeature requests to request features within a specific extent. So “…&BBOX=-180.0,-90.0,180.0,90.0…” means getting all the features in the extent defined by lower corner coordinate (-180.0,-90.0) and upper corner coordinate (180.0,90.0). Not as straight forward as it seems to be to most people, I actually find it quite confusing when trying to answer this question below:

Is it possible to set BBOX to coordinate values in other projection?

The answer to this question depends on whether it’s WFS 1.0.0 (No) or WFS 1.1.0 (Yes), and the yes for WFS 1.1.0 even introduces some more complexity. So in the rest of this article I am going to clear things up based on the definition and statement in following OGC specifications:

BBOX in WFS 1.0.0

First of all, it is a little less confusing for WFS version 1.0.0 that  you CAN’T specify BBOX in coordinates of any other projections. This is based on two facts in specs:

(1) WFS 1.0.0 spec (02-058), section 13.3.3, where it says:

The SRS of the bounding box must be the same as the SRS of the feature type(s) in a request. The SRS of a feature type is advertised by a WFS in the capabilities document. If more than one feature type is specified in a request, the feature types must all be in the same SRS and the BBOX must be specified in the common SRS as well.

(2) In WFS 1.0.0 schema, a WFS’s capabilities response can have only one “SRS” allowed for each feature type.

But I did see people get confused and thus argue about one thing: Can I specify BBOX in EPSG:4326?

Based on the statement above, if a WFS serves out feature types in a non-EPSG4326 then it should be valid and compliant to just reject GetFeature request with BBOX in EPSG:4326. But another paragraph in WFS 1.0.0 spec (same section 13.3.3) seems to indicate a yes on it:

If a request contains a Bounding Box whose area does not overlap at all with the LatLongBoundingBox(s) advertised in the Capabilities XML for the requested geodata object, the server should return empty content (e.g. null feature set) for that element. Any elements that are partly or entirely contained in the Bounding Box should be returned in the appropriate format.

In this case we don’t really have to strictly follow the spec, and I won’t be surprised to see most WFS 1.0.0 implementations supports BBOX in EPSG:4326 no matter what SRS its feature types are in.

BBOX in WFS 1.1.0

Now for WFS 1.1.0, things get a little complicated because of the fact that there is an optional “crsurl” value you can append at the end of the BBOX coordinates to explicitly specify what coordinate reference system the BBOX is in. For example, ”…&BBOX=minX,minY,maxX,maxY,EPSG:900913” means that those min/max X/Y values are in EPSG:900913. Another complexity is that in WFS 1.1.0 capabilities response there can be multiple supported SRS for each feature type.  So based the section 14.3.3 of WFS spec 1.1.0:

The KVP encoding for a bounding box is defined in subclause 10.2.3 of normative reference [15]. The general form of the parameter is:

…If the crsuri is not specified then the 2-D coordinates shall be specified using decimal degrees and WGS84 as described in [15].

Note: The WFS spec is reference OWS 1.1.0 spec as [15] here

Here I can actually get two conclusions:

(1) You can specify BBOX in any coordinate reference system as long as you provide a “crsuri”.

Although not described in spec, I would believe you will get correct and useful response only if you request one of those supported coordinate reference system listed in a WFS’s capabilities.

(2) If “crsuri” is missing, then the coordinate values specified in BBOX should be considered as WGS84 as described in OWS 1.1.0 spec as below:

A WGS 84 bounding box shall be KVP encoded in a corresponding parameter value list, with the ordered listed values for the quantities:
LowerCorner longitude, in decimal degrees
LowerCorner latitude, in decimal degrees
UpperCorner longitude, in decimal degrees
UpperCorner latitude, in decimal degrees
crs URI = “urn:ogc:def:crs:OGC:1.3:CRS84” (optional)

In another word, if you set BBOX parameter in request without a “crsurl”, the BBOX value should always be in syntax “min_lon, min_lat, max_lon, max_lat”.

EPSG:4326 versus. CRS84

Now one finally thing people get confused very often is the order of longitude and latitude they should put in BBOX parameter for EPSG:4326 and CRS:84. Originally and as always, official EPSG:4326 defines coordinates in order of “latitude” followed by “longitude”, but unfortunately almost everyone speaks and uses the reverse order as “longitude” followed by “latitude”, which is why old spec like WMS 1.1.1 and WFS 1.0.0 also uses this order for EPSG:4326. But in most recent specifications OGC is trying to fix this common mistaken usage of EPSG:4326 by creating a brand new CRS:84, which based on my understanding is identical to EPSG:4326 except that it defines coordinates in order of “longitude” followed by “latitude”. So as far as WFS goes, always follow the rules below:

For WFS 1.0.0, uses EPSG:4326 and always set BBOX coordinate values as “longitude” followed by “latitude”. For WFS 1.1.0, specify coordinates in BBOX as longitude followed by latitude if you’re setting CRS:84 as the coordinate reference system for BBOX (e.g. “…&BBOX=–180.0,-90.0,180.0,90.0,urn:ogc:def:crs:OGC:1.3:CRS84…”), or set it the other way if you’re setting EPSG:4326 for BBOX (e.g. “…&BBOX=-90.0,-180.0,90.0,180.0,urn:x-ogc:def:crs:EPSG:6.9:4326…), but again if you’re omittng “crsuri” at the end please always stick to the order where longitude is ahead of latitude.


“BBOX” parameter looks easy and straight forward to use, but it also gets complicated when different versions of WFS implementations and EPSG:4326/WGS84 are involved. Hopefully this article clear things up a little.

Monday, May 3, 2010

Expo 2010 Shanghai China

To celebrate the opening of Expo 2010 Shanghai China, I just had a peek at where the Expo Axis should locate in Shanghai in those popular online map services (Google Maps, OSM, Bing Maps, and Yahoo Maps). See what I get:

OSM Maps
Not bad, but that is actually what I expected to see.

Google Maps
It seems like Google Maps really needs to catch up with this big event. But if you’re laughing at Google Maps now, you better check out Bing Maps and Yahoo Maps below.

Bing Maps
What can I say…just like the map…

Yahoo Maps
You know what? I zoomed out on purpose because I can’t zoom in any more…

Wednesday, April 14, 2010

Several approaches to do Geolocation by IP address programmatically

Geolocation is the process of identifying the real-world geographic location based on a device’s IP address (either connected through cable or WiFi access point), GSM/CDMA cells IDs or GPS and so on. The result of Geolocation is essentially a pair of coordinate (lat and lon) on earth, and if combined with other service like Geocoding the result can also be translated to a well-known place name or a physical address.
Geolocation by IP address although  may not be as accurate as GPS sometimes, does have great value in many of today’s location aware web applications. One of the typical examples is that when you do search on internet the search engine will detect the location by your IP address and automatically filter and sort the results based on that information. In this article I will be looking at some existing approaches I’ve played with to do Geolocation by IP address.

1. W3C Geolocation API
If you’re in JavaScript environment, W3C Geolocation API is definitely the approach you should follow because it is the standard. Independent of the actual source of the location information which is not just limited to IP address but can also be WiFi and Bluetooth MAC address, GSM/CDMA cell IDs, GPS or so, it actually defines a high-level interface to location associated information with devices hosting the implementation. Other than that W3C Geolocation API also defines the way you request repeated position updates through callback mechanism as well as requesting an aged position.
Mozilla Firefox is a very good example of such implementation. The code example below illustrate the common work flow to get your current location in a JavaScript application:
if(navigator.geolocation) {  
/* geolocation is available */  
function(position) { // success callback  
//alert(position.coords.latitude + "," + position.coords.longitude);
function() { // failure callback
// error handling
// other options
} else {      
// browser does not support Geolocation API
In Firefox’s implementation, it gives you not only the lat and lon value of your location but also gives you the address associated with that coordinate. Under the hood Firefox takes advantage of Google Location Service which I believe provided all necessary infrastructure.

2. Google Geolocation API in Gears
Google Gears Geolocation API implements the same W3c Geolocation interfaces but in a more device independent, location information source independent, and browser independent way. It’s online doc is very good and concise for people who would like to use it in their web application.

3. Google Location Service
Google Location Service is far more than an interfaces that only provides geolocation function, but it’s totally valid to use that as a Geolocation service. Without even realizing, it is definitely behind a lot of its own services like Search and Latitude which sometimes you don’t even realize. It is also backing up many other Geolocation API implementations like Firefox. But unfortunately I was able to find out an explicit native Geolocation service API either in JavaScript or in REST flavor. So it’s probably impossible to directly communication with Google Location Service by feeding an IP address and getting a coordinate back.

4. Other Geolocation by IP service providers
HostIP, mentioned in Andrew Turner’s Introduction to Neogeography, HostIP has the largest free database and service for GeoIP (the term Andrew uses for geolocation by IP address). The best part of HostIP is that it freely distribute it’s database with regular updates. Since I don’t find any information regarding license and term of use for HostIP database I would assume it is in public domain. Programmatically HostIP provides a simple and lightweight RESTful API for people to locate themselves by IP. It doesn’t do much but it’s good at what it does. Because of its REST nature, it is not limited to any program environment except that you might need a proxy in your JavaScript web application. Below is an example:
City: Sugar Grove, IL
City: Sugar Grove, IL
Latitude: 41.7696
Longitude: -88.4588
[use the URL above for an example - XML too long to paste below]
Finally its website itself provides a simple browser based GUI for people who don’t want to bother coding their own app.

GeoIP: MaxMind has a free GeoIP database and API called GeoLite City. MaxMind and it GeoIP product suite does not have a publicly hosted RESTful service or API for directly use but it allows you to download their database in either binary format or CSV format (mainly for you to load data into SQL database) to host locally. The API for accessing the local database is available in many platforms (C, C#, MS COM, PHP, Java, Perl Python, Ruby etc.). The workflow is to first download the database to local disk and uncompress it, then use any of the API available the query the database. Below is an example I tried out using its Java API:
public void lookupServiceTest() {  
try {
LookupService cl = new LookupService("E:\\projects\\geoip\\geolite-city-database\\GeoLiteCity.dat", LookupService.GEOIP_MEMORY_CACHE);
Location location = cl.getLocation("");
System.out.println(" countryCode: " + location.countryCode +
"\n countryName: " + location.countryName +
"\n region: " + location.region +
"\n regionName: " + regionName.regionNameByCode(location.countryCode, location.region) +
"\n city: " + +
"\n postalCode: " + location.postalCode +
"\n latitude: " + location.latitude +
"\n longitude: " + location.longitude +
"\n metro code: " + location.metro_code +
"\n area code: " + location.area_code +
"\n timezone: " + timeZone.timeZoneByCountryAndRegion(location.countryCode, location.region));
} catch (IOException e) {
System.out.println("IO Exception");
GeoIP API is free and open source under GPL/LGPL license, and the GeoLite City database itself is also free under its database license. MaxMind also provides an advanced commercial version called GeoIP City, which is functionally equal to the free one but has more accuracy, different redistribution license and more frequent updates.

Sunday, March 28, 2010

Using Java Map Projection Library in Android

Recently I was writing a little mapping application on Android, which basically takes Open Street Map tiles as base layer, and also has the ability to overlay some vector features on top. In this application there is need to convert coordinates back and forth between WGS84 and Mercator projection, so I need certain library to do the job. Given the popularity of PROJ.4, the pure Java port of it, Java Map Projection Library (name it “PROJ4Java” below), seems to be an obvious choice so I spend some hours during the weekend to see how it works.
First you can get the latest version (1.0.6) of PROJ4Java from this link. The size of the library is very small and lightweight with only 1.17MB for complete source code and 247KB for compiled jar file. If you’re familiar with PROJ.4 itself then this Java version should be quite straight forward, otherwise there isn’t much sample code or tutorial on its homepage. But the basic workflow is quite simple as described by its project homepage:
To use the library, you need to create a projection, either directly by calling its constructor or via one of the factory methods. Once you have a Projection, you can use it to convert between latitude/longitude and projected map units (by default, metres) by calling the transform and transformInverse methods.
A major issue of using PROJ4Java on Android is that out of box it can’t be run due to the fact that it is using some classes in java.awt.geom.* which is not available in Android SDK. But since the only major AWT class used by PROJ4Java is Point2D, it’s not difficult at all to work around by replacing it with your own 2D point class. In my case I use the point class from JTS library. After taking out all Java awt class references from source code and replacing them with JTS class, everything works pretty well on Android.
Another thing I modifies on the source code is to get rid of those projection implementations I don’t need. Mercator projection is the only one I need so far so that I am able to reduce the size of compile jar file to only 168KB. Here is a list of classes I extracted from original source code to support only Mercator projection:
ProjectionFactory class is the entry point for you to use PROJ4Java, which provides several static methods to create an instance of Projection. Below are some use cases I figured out by briefly reading through the source code:
(1) ProjectionFactory.getNamedPROJ4CoordinateSystem(String name) takes a well-known projection code like “epsg:4326” and returns a Projection:
1: String name = "epsg:3785";    
2: Projection proj = ProjectionFactory.getNamedPROJ4CoordinateSystem(name);
(2) ProjectionFactory.fromPROJ4Specification(String[] params) takes a set of PROJ.4 parameters and returns a Projection:
1: String[] params = {
2:     "proj=tmerc",
3:     "lat_0=37.5",
4:     "lon_0=-85.66666666666667",
5:     "k=0.999966667",
6:     "x_0=99999.99989839978",
7:     "y_0=249999.9998983998",
8:     "ellps=GRS80",
9:     "datum=NAD83",
10:     "to_meter=0.3048006096012192",
11:     "no_defs"
12: };
13: ProjectionFactory.fromPROJ4Specification(params);
(3) Create an instance of customized projection. All supported projection definition strings (PROJ.4 parameters for ) for PROJ4Java are listed in text files under folder “nad” that is coming with the library. These files are “epsg”, “esri”, “nad27”, “nad83”, and “world”, and a sample projection definition string is like below:
1: <2965> +proj=tmerc +lat_0=37.5 +lon_0=-85.66666666666667 +k=0.999966667 +x_0=99999.99989839978 +y_0=249999.9998983998 +ellps=GRS80 +datum=NAD83 +to_meter=0.3048006096012192 +no_defs
To add a new customized projection just create a definition string (usually modify certain parameter values upon an existing projection) and add it into one of those files, or create a new text file under “nad” folder with the definition string, then use the code below:
1: Projection proj = null;
2: try {
3:     // assume that "epsg:900913" projection defintion 
4:     //   has been added in text file "others"
5:     proj = ProjectionFactory.readProjectionFile("others", "900913");    
6: } catch(IOException e) {
7:     e.printStackTrace();
8: }
(4) Transform between lat/lon and projection units with a Projection instance:
1: Projection epsg3785 = ProjectionFactory.getNamedPROJ4CoordinateSystem("epsg:3785");    
3: System.out.println("transform from latlon to epsg:3785");
4: System.out.println("latlon: -117.5931084, 34.1063989");
5: Point pEpsg3785 = epsg3785.transform(-117.5931084, 34.1063989);
6: System.out.println("epsg:3785: " + pEpsg3785.getX() + ", " + pEpsg3785.getY());
8: System.out.println("transform from epsg:3785 to latlon");
9: System.out.println("epsg:3785: " + pEpsg3785.getX() + ", " + pEpsg3785.getY());
10: Point latlon = epsg3785.inverseTransform(pEpsg3785);
11: System.out.println("latlon: " + latlon.getX() + ", " + latlon.getY());