How to use geo_point with geohash in Elasticsearch with a Java class?

elasticsearch geohashing
Dani picture Dani · Apr 8, 2015 · Viewed 11.9k times · Source

I have a Java class which looks as follows (GeoPoint is an Elasticsearch type):

private Long id;
private Integer genre;
private String cityName;
private GeoPoint geoPoint;
private Date lastUpdate;
private Double lat;
private Double lon;

The Elasticsearch mapping I use is:

{
    "location": {
        "properties": {
            "id": {"type": "long"},
            "genre": {"type": "integer"},
            "cityName": {"type": "string"},
            "geoPoint": {
                "type": "geo_point",
                "geohash": true,
                "geohash_prefix": true,
                "geohash_precision": 7
            },
            "lastUpdate": {"type": "date", format: "yyyy/MM/dd HH:mm:ss"}
        }
    }
}

When trying to index it, I get the following exception:

org.elasticsearch.ElasticsearchParseException: field must be either lat/lon or geohash

The exception is thrown from line 381 of the GeoUtils class. It happens right after a check for Double lat and lon fields in the mapping class, just like GeoPoint properties are.

I don't understand why it doesn't work given that I set the geoPoint's field type as geo_point as the ElasticSearch documentation suggests.

UPDATE 1

The Java class.

public class UserLocation implements Serializable {

    private Long id;
    private Integer genre;
    private String cityName;
    private GeoPoint geoPoint;
    private Date lastUpdate;

    public UserLocation () {
    }

    public UserLocation (UserLocationLite userLocationLite) {

        this.id = userLocationLite.getId();
        this.genre = userLocationLite.getGenre();
        this.cityName = userLocationLite.getCity();
        this.geoPoint =
                new GeoPoint(userLocationLite.getLatitude(), userLocationLite.getLongitude());
        this.lastUpdate = userLocationLite.getActivity();
    }

    public Long getId() {
        return id;
    }

    public void setId(Long id) {
        this.id = id;
    }

    public Integer getGenre() {
        return genre;
    }

    public void setGenre(Integer genre) {
        this.genre = genre;
    }

    public String getCityName() {
        return cityName;
    }

    public void setCityName(String cityName) {
        this.cityName = cityName;
    }

    public GeoPoint getGeoPoint() {
        return geoPoint;
    }

    public void setGeoPoint(GeoPoint geoPoint) {
        this.geoPoint = geoPoint;
    }

    public Date getLastUpdate() {
        return lastUpdate;
    }

    public void setLastUpdate(Date lastUpdate) {
        this.lastUpdate = lastUpdate;
    }

}

Method to index.

@Override
public boolean save(String id, R r) {

    try {

        return this.transportClient.prepareIndex(this.index, this.type, id)
                .setSource(this.objectMapper.writeValueAsString(r))
                .execute().actionGet().isCreated();
    } catch (JsonProcessingException e) {

        e.printStackTrace();
    }

    return false;
}

Where R is one generic type implemented for another class, in this case with the UserLocation class, which is serialized as follows.

{"id":40,"genre":1,"cityName":"Madrid","geoPoint":{"lat":42.626595,"lon":-0.488439,"geohash":"ezrm5c0vx832"},"lastUpdate":1402144560000}

UPDATE 2

Right now the Java class structure works fine.

public class UserLocationSearch implements Serializable {

    private Long id;
    private Integer genre;
    private String cityName;
    @JsonIgnore
    private GeoPoint geoPoint;
    private Date lastUpdate;

    public UserLocationSearch() {

        this.geoPoint = new GeoPoint();
    }

    public UserLocationSearch(UserLocationLite userLocationLite) {

        this.id = userLocationLite.getId();
        this.genre = userLocationLite.getGenre();
        this.cityName = userLocationLite.getCity();
        this.geoPoint =
                new GeoPoint(userLocationLite.getLatitude(), userLocationLite.getLongitude());
        this.lastUpdate = userLocationLite.getActivity();
    }

    public Long getId() {
        return id;
    }

    public void setId(Long id) {
        this.id = id;
    }

    public Integer getGenre() {
        return genre;
    }

    public void setGenre(Integer genre) {
        this.genre = genre;
    }

    public String getCityName() {
        return cityName;
    }

    public void setCityName(String cityName) {
        this.cityName = cityName;
    }

    public GeoPoint getGeoPoint() {
        return geoPoint;
    }

    public void setGeoPoint(GeoPoint geoPoint) {
        this.geoPoint = geoPoint;
    }

public String getGeohash() {
    return this.geoPoint.getGeohash();
}

public void setGeohash(String geohash) {
    this.geoPoint.resetFromGeoHash(geohash);
}

public Date getLastUpdate() {
    return lastUpdate;
}

public void setLastUpdate(Date lastUpdate) {
    this.lastUpdate = lastUpdate;
}

}

But now I've another question.

If get the document.

GET /user/location/40

{
   "_index": "user",
   "_type": "location",
   "_id": "40",
   "_version": 7,
   "found": true,
   "_source": {
      "id": 40,
      "genre": 1,
      "cityName": "Madrid",
      "lastUpdate": 1402144560000,
      "geohash": "ezrm5c28d9x0"
   }
}

The geohash has 12 chars, but in the mapping geohash precision is set to 7...

GET /user/location/_mapping

{
   "user": {
      "mappings": {
         "location": {
            "properties": {
               "cityName": {
                  "type": "string"
               },
               "genre": {
                  "type": "integer"
               },
               "geoPoint": {
                  "type": "geo_point",
                  "geohash": true,
                  "geohash_prefix": true,
                  "geohash_precision": 7
               },
               "geohash": {
                  "type": "string"
               },
               "id": {
                  "type": "long"
               },
               "lastUpdate": {
                  "type": "date",
                  "format": "yyyy/MM/dd HH:mm:ss"
               }
            }
         }
      }
   }
}

This mean that it's working wrong?

UPDATE 3

Current class.

public class UserLocationSearch implements Serializable {

    private Long id;
    private Integer genre;
    private String cityName;
    private Location location;
    private GeoPoint geoPoint;
    private Date lastUpdate;

    public UserLocationSearch() {

    }

    public UserLocationSearch(UserLocationLite userLocationLite) {

        this.id = userLocationLite.getId();
        this.genre = userLocationLite.getGenre();
        this.cityName = userLocationLite.getCity();
        this.location = new Location(userLocationLite.getLatitude(), userLocationLite.getLongitude());
        this.geoPoint = new GeoPoint(this.location.getGeohash());
        this.lastUpdate = userLocationLite.getActivity();
    }

    public Long getId() {
        return id;
    }

    public void setId(Long id) {
        this.id = id;
    }

    public Integer getGenre() {
        return genre;
    }

    public void setGenre(Integer genre) {
        this.genre = genre;
    }

    public String getCityName() {
        return cityName;
    }

    public void setCityName(String cityName) {
        this.cityName = cityName;
    }

    public Location getLocation() {
        return location;
    }

    public void setLocation(Location location) {
        this.location = location;
    }

    public GeoPoint getGeoPoint() {
        return geoPoint;
    }

    public void setGeoPoint(GeoPoint geoPoint) {
        this.geoPoint = geoPoint;
    }

    public Date getLastUpdate() {
        return lastUpdate;
    }

    public void setLastUpdate(Date lastUpdate) {
        this.lastUpdate = lastUpdate;
    }

    public static class GeoPoint{

        private String geohash;

        public GeoPoint() {

        }

        public GeoPoint(String geohash) {

            this.geohash = geohash;
        }

        public String getGeohash() {
            return geohash;
        }

        public void setGeohash(String geohash) {
            this.geohash = geohash;
        }
    }

    public static class Location{

        private Double lat;
        private Double lon;

        public Location() {

        }

        public Location(Double lat, Double lon) {

            this.lat = lat;
            this.lon = lon;
        }

        public Double getLat() {
            return lat;
        }

        public void setLat(Double lat) {
            this.lat = lat;
        }

        public Double getLon() {
            return lon;
        }

        public void setLon(Double lon) {
            this.lon = lon;
        }

        @JsonIgnore
        public String getGeohash(){

            return new org.elasticsearch.common.geo.GeoPoint(this.lat, this.lon).getGeohash();
        }

    }

}

Its mapping.

{
    "location": {
        "properties": {
            "id": {"type": "long"},
            "genre": {"type": "integer"},
            "cityName": {"type": "string"},
            "location": {
                "type": "geo_point",
                "geohash": false
            },
            "geoPoint": {
                "type": "geo_point",
                "geohash": true,
                "geohash_prefix": true,
                "geohash_precision": 7
            },
            "lastUpdate": {"type": "date", format: "yyyy/MM/dd HH:mm:ss"}
        }
    }
}

Searchs.

By distance (works fine).

GET /user/location/_search

{
  "query": {
    "match_all": {}
  },
  "filter": {
    "geo_distance": {
      "distance": "100km",
      "location": {
        "lat": 42.5,
        "lon": -0.49
      }
    }
  }
}

By geohash (works fine too, more than 7 of precision is ignored, because it's mapped with "geohash_precision: 7").

GET /user/location/_search

{
  "query": {
    "filtered": {
      "filter": {
        "geohash_cell": {
          "geoPoint": {
            "geohash": "ezrm5c0y5rh8"
          },
          "neighbors": true, 
          "precision": 7
        }
      }
    }
  }
}

CONCLUSION

I don't understand why reason org.elasticsearch.common.geo.GeoHashUtils.GeoPoint class is not compatible with the actual Elastic version.

But, following the tracks provided by @jkbkot I've determinate to implement my own GeoPoint and Location classes to get full compatibility.

Answer

Jakub Kotowski picture Jakub Kotowski · Apr 8, 2015

The following Sense script should give you an idea what to do:

DELETE location
PUT location
PUT location/location/_mapping
{
"location": {
    "properties": {
        "id": {"type": "long"},
        "genre": {"type": "integer"},
        "cityName": {"type": "string"},
        "geoPoint": {
            "type": "geo_point",
            "geohash": true,
            "geohash_prefix": true,
            "geohash_precision": 7
        },
        "lastUpdate": {"type": "date", format: "yyyy/MM/dd HH:mm:ss"}
    }
}
}
GET location/location/_mapping

PUT location/location/1
{"id":40,"genre":1,"cityName":"Madrid","geoPoint":{"geohash":"ezrm5c0vx832"},"lastUpdate":1402144560000}

GET /location/location/_search
{
    "query" : {
         "match_all": {}
    },
    "filter" : {
        "geo_distance" : {
            "distance" : "40km",
            "geoPoint" : {
                "lat" : 42.5,
                "lon" : -0.49
            }
        }
    }
}

You have to transform the Java class that holds the document to a JSON with a structure like either:

{
    "id": 40,
    "genre": 1,
    "cityName": "Madrid",
    "geoPoint": {
        "geohash": "ezrm5c0vx832"
    },
    "lastUpdate": 1402144560000
}

or

{
    "id": 40,
    "genre": 1,
    "cityName": "Madrid",
    "geoPoint": {
        "lat": 42.626595,
        "lon": -0.488439
    },
    "lastUpdate": 1402144560000
}

So either you have to remove e.g. private GeoPoint geoPoint; from your Java class and leave lat and lon there (or the other way around), or you have to change how you serialize the class to a JSON string - to omit either geoPoint or both lat and lon.

Related questions

Elasticsearch query to return all records

I have a small database in Elasticsearch and for testing purposes would like to pull all records back. I am attempting to use a URL of the form... http://localhost:9200/foo/_search?pretty=true&q={'matchAll':{''}} Can someone …

Read More
Making a request to a RESTful API using python

I have a RESTful API that I have exposed using an implementation of Elasticsearch on an EC2 instance to index a corpus of content. I can query the search by running the following from my terminal (MacOSX): curl -XGET 'http://…

Read More
Removing Data From ElasticSearch

I'm new to ElasticSearch. I'm trying to figure out how to remove data from ElasticSearch. I have deleted my indexes. However, that doesn't seem to actually remove the data itself. The other stuff I've seen points to the Delete by …

Read More