Solr 4.0 - solid geospatial now comes standard

I have a fairly large corpus of events with spatial data which are indexed in Solr.  LatLonType won't cut it for me.  I'm a huge fan of David Smiley's work with SOLR-2155 which I used to scale my app in Solr 3.  David's embraced spatial grid implementation and I've found his modules outperform LatLonType in my use cases - other benchmarks support this.

Where SOLR-2155 required an additional JAR, Solr 4.0 ships with a solid geospatial solution.  I've been using solr.SpatialRecursivePrefixTreeFieldType with pretty much standard options.

There was a lot of weirdness in Solr 3 which is now fixed.  Namely, bounding boxes were actually circles - now we get simple bounding rectangles.  In fact, polygons are within our reach.  If you don't mind some LGPL in your app, you can include JTS and get it with Solr 4.

I don't have metrics to quantify performance yet.  Once I get more data indexed, I'll try to post some numbers.  Thanks David!

Using geohashing in Solr

LatLonType might be sufficient for many use-cases regarding Solr spatial searching.  It's a pretty good option and performs decently in benchmarking I've seen.  However, I'm interested in geohashing.  While there isn't a great option out of the box (yet), SOLR-2155 is interesting and I've given it a try.  It was written by David Smiley, whose Solr 3 book I use almost every day.

I grabbed the v1.0.5 JAR from SOLR-2155's github repo.  It's a little confusing where to drop it but I decided on /usr/local/solr/solr/lib/ based on David's advice (see bottom of thread).  The instructions on how to configure schema.xml and solrconfig.xml on the github page are spot-on.  Obviously you have to restart Solr for these changes to take effect.

Then feeding in data is a snap.  I'm using Lily so I created a STRING field in Lily and wired it up to point to the solr2155.solr.schema.GeoHashField Solr field.  When creating Lily records, I send in "lat + ',' + lon" values.  I haven't been able to benchmark Solr (or Lily) indexing but it's no noticable drop-off from my previous attempts with double, tdouble, or LatLonType.

Searching performance is great.  Again, no benchmarks but everything feels snappy (great technical term, I know).  Bounding box queries like this come back as quick as text or string searches which is really thrilling:

http://s1:8983/solr/select?q=*:*&fq=geoloc:[1,-60 TO 5,-55]&shards=s2:8983/solr,s1:8983/solr,s3:8983/solr,s4:8983/solr

One issue I've seen is precision values in Solr result sets.  It looks like a Java double (ie, 3.89999233 is really 3.9).  The underlying data (3.9 in this case) gets preserved but Solr returns the modified value (like 3.89999233).  So just be careful about what you display.

I'll try to get some benchmarks but here's a good brief with some numbers.