Code Monkey home page Code Monkey logo

geoip2-java's Introduction

GeoIP2 Java API

Description

This distribution provides an API for the GeoIP2 and GeoLite2 web services and databases.

Installation

Maven

We recommend installing this package with Maven. To do this, add the dependency to your pom.xml:

    <dependency>
        <groupId>com.maxmind.geoip2</groupId>
        <artifactId>geoip2</artifactId>
        <version>4.2.0</version>
    </dependency>

Gradle

Add the following to your build.gradle file:

repositories {
    mavenCentral()
}
dependencies {
    compile 'com.maxmind.geoip2:geoip2:4.2.0'
}

JAR Files

If you are unable to use Maven or Gradle, you may include the geoip2.jar file and its dependencies in your classpath. Download the JAR files from the GitHub Releases page.

IP Geolocation Usage

IP geolocation is inherently imprecise. Locations are often near the center of the population. Any location provided by a GeoIP2 database or web service should not be used to identify a particular address or household.

Web Service Usage

To use the web service API, you must create a new WebServiceClient using the WebServiceClient.Builder. You must provide the Builder constructor your MaxMind accountId and licenseKey. To use the GeoLite2 web services instead of GeoIP2, set the host method on the builder to geolite.info. To use the Sandbox GeoIP2 web services intead of the production GeoIP2 web services, set the host method on the builder to sandbox.maxmind.com. You may also set a timeout or set the locales fallback order using the methods on the Builder. After you have created the WebServiceClient, you may then call the method corresponding to a specific web service, passing it the IP address you want to look up.

If the request succeeds, the method call will return a model class for the end point you called. This model in turn contains multiple record classes, each of which represents part of the data returned by the web service.

If the request fails, the client class throws an exception.

The WebServiceClient object is safe to share across threads. If you are making multiple requests, the object should be reused so that new connections are not created for each request.

See the API documentation for more details.

Web Service Example

Country Service

// This creates a WebServiceClient object that is thread-safe and can be
// reused across requests. Reusing the object will allow it to keep
// connections alive for future requests.
//
// Replace "42" with your account ID and "license_key" with your license key.
// To use the GeoLite2 web service instead of the GeoIP2 web service, call the
// host method on the builder with "geolite.info", e.g.
// new WebServiceClient.Builder(42, "license_key").host("geolite.info").build()
// To use the Sandbox GeoIP2 web service instead of the production GeoIP2
// web service, call the host method on the builder with
// "sandbox.maxmind.com", e.g.
// new WebServiceClient.Builder(42, "license_key").host("sandbox.maxmind.com").build()
WebServiceClient client = new WebServiceClient.Builder(42, "license_key")
    .build();

InetAddress ipAddress = InetAddress.getByName("128.101.101.101");

// Do the lookup
CountryResponse response = client.country(ipAddress);

Country country = response.getCountry();
System.out.println(country.getIsoCode());            // 'US'
System.out.println(country.getName());               // 'United States'
System.out.println(country.getNames().get("zh-CN")); // '美国'

City Plus Service

// This creates a WebServiceClient object that is thread-safe and can be
// reused across requests. Reusing the object will allow it to keep
// connections alive for future requests.
//
// Replace "42" with your account ID and "license_key" with your license key.
// To use the GeoLite2 web service instead of the GeoIP2 web service, call the
// host method on the builder with "geolite.info", e.g.
// new WebServiceClient.Builder(42, "license_key").host("geolite.info").build()
// To use the Sandbox GeoIP2 web service instead of the production GeoIP2
// web service, call the host method on the builder with
// "sandbox.maxmind.com", e.g.
// new WebServiceClient.Builder(42, "license_key").host("sandbox.maxmind.com").build()
WebServiceClient client = new WebServiceClient.Builder(42, "license_key")
    .build();

InetAddress ipAddress = InetAddress.getByName("128.101.101.101");

// Do the lookup
CityResponse response = client.city(ipAddress);

Country country = response.getCountry();
System.out.println(country.getIsoCode());            // 'US'
System.out.println(country.getName());               // 'United States'
System.out.println(country.getNames().get("zh-CN")); // '美国'

Subdivision subdivision = response.getMostSpecificSubdivision();
System.out.println(subdivision.getName());       // 'Minnesota'
System.out.println(subdivision.getIsoCode());    // 'MN'

City city = response.getCity();
System.out.println(city.getName());       // 'Minneapolis'

Postal postal = response.getPostal();
System.out.println(postal.getCode());       // '55455'

Location location = response.getLocation();
System.out.println(location.getLatitude());        // 44.9733
System.out.println(location.getLongitude());       // -93.2323

Insights Service

// This creates a WebServiceClient object that is thread-safe and can be
// reused across requests. Reusing the object will allow it to keep
// connections alive for future requests.
//
// Replace "42" with your account ID and "license_key" with your license key.
// Please note that the GeoLite2 web service does not support Insights.
// To use the Sandbox GeoIP2 web service instead of the production GeoIP2
// web service, call the host method on the builder with
// "sandbox.maxmind.com", e.g.
// new WebServiceClient.Builder(42, "license_key").host("sandbox.maxmind.com").build()
WebServiceClient client = new WebServiceClient.Builder(42, "license_key")
    .build();

InetAddress ipAddress = InetAddress.getByName("128.101.101.101");

// Do the lookup
InsightsResponse response = client.insights(ipAddress);

Country country = response.getCountry();
System.out.println(country.getIsoCode());            // 'US'
System.out.println(country.getName());               // 'United States'
System.out.println(country.getNames().get("zh-CN")); // '美国'
System.out.println(country.getConfidence());         // 99

Subdivision subdivision = response.getMostSpecificSubdivision();
System.out.println(subdivision.getName());       // 'Minnesota'
System.out.println(subdivision.getIsoCode());    // 'MN'
System.out.println(subdivision.getConfidence()); // 90

City city = response.getCity();
System.out.println(city.getName());       // 'Minneapolis'
System.out.println(city.getConfidence()); // 50

Postal postal = response.getPostal();
System.out.println(postal.getCode());       // '55455'
System.out.println(postal.getConfidence()); // 40

Location location = response.getLocation();
System.out.println(location.getLatitude());        // 44.9733
System.out.println(location.getLongitude());       // -93.2323
System.out.println(location.getAccuracyRadius());  // 3
System.out.println(location.getTimeZone());        // 'America/Chicago'

System.out.println(response.getTraits().getUserType()); // 'college'

Database Usage

To use the database API, you must create a new DatabaseReader using the DatabaseReader.Builder. You must provide the Builder constructor either an InputStream or File for your GeoIP2 database. You may also specify the fileMode and the locales fallback order using the methods on the Builder object.

After you have created the DatabaseReader, you may then call one of the appropriate methods, e.g., city or tryCity, for your database. These methods take the IP address to be looked up. The methods with the try prefix return an Optional object, which will be empty if the value is not present in the database. The method without the prefix will throw an AddressNotFoundException if the address is not in the database. If you are looking up many IPs that are not contained in the database, the try method will be slightly faster as they do not need to construct and throw an exception. These methods otherwise behave the same.

If the lookup succeeds, the method call will return a response class for the GeoIP2 lookup. The class in turn contains multiple record classes, each of which represents part of the data returned by the database.

We recommend reusing the DatabaseReader object rather than creating a new one for each lookup. The creation of this object is relatively expensive as it must read in metadata for the file. It is safe to share the object across threads.

See the API documentation for more details.

Caching

The database API supports pluggable caching (by default, no caching is performed). A simple implementation is provided by com.maxmind.db.CHMCache. Using this cache, lookup performance is significantly improved at the cost of a small (~2MB) memory overhead.

Usage:

new DatabaseReader.Builder(file).withCache(new CHMCache()).build();

Packaging Database in a JAR

If you are packaging the database file as a resource in a JAR file using Maven, you must disable binary file filtering. Failure to do so will result in InvalidDatabaseException exceptions being thrown when querying the database.

Database Example

City

// A File object pointing to your GeoIP2 or GeoLite2 database
File database = new File("/path/to/GeoIP2-City.mmdb");

// This creates the DatabaseReader object. To improve performance, reuse
// the object across lookups. The object is thread-safe.
DatabaseReader reader = new DatabaseReader.Builder(database).build();

InetAddress ipAddress = InetAddress.getByName("128.101.101.101");

// Replace "city" with the appropriate method for your database, e.g.,
// "country".
CityResponse response = reader.city(ipAddress);

Country country = response.getCountry();
System.out.println(country.getIsoCode());            // 'US'
System.out.println(country.getName());               // 'United States'
System.out.println(country.getNames().get("zh-CN")); // '美国'

Subdivision subdivision = response.getMostSpecificSubdivision();
System.out.println(subdivision.getName());    // 'Minnesota'
System.out.println(subdivision.getIsoCode()); // 'MN'

City city = response.getCity();
System.out.println(city.getName()); // 'Minneapolis'

Postal postal = response.getPostal();
System.out.println(postal.getCode()); // '55455'

Location location = response.getLocation();
System.out.println(location.getLatitude());  // 44.9733
System.out.println(location.getLongitude()); // -93.2323

Anonymous IP

// A File object pointing to your GeoIP2 Anonymous IP database
File database = new File("/path/to/GeoIP2-Anonymous-IP.mmdb");

// This creates the DatabaseReader object. To improve performance, reuse
// the object across lookups. The object is thread-safe.
try (DatabaseReader reader = new DatabaseReader.Builder(database).build()) {
    InetAddress ipAddress = InetAddress.getByName("85.25.43.84");

    AnonymousIpResponse response = reader.anonymousIp(ipAddress);

    System.out.println(response.isAnonymous()); // true
    System.out.println(response.isAnonymousVpn()); // false
    System.out.println(response.isHostingProvider()); // false
    System.out.println(response.isPublicProxy()); // false
    System.out.println(response.isResidentialProxy()); // false
    System.out.println(response.isTorExitNode()); //true
}

ASN

// A File object pointing to your GeoLite2 ASN database
File database = new File("/path/to/GeoLite2-ASN.mmdb");

// This creates the DatabaseReader object. To improve performance, reuse
// the object across lookups. The object is thread-safe.
try (DatabaseReader reader = new DatabaseReader.Builder(database).build()) {

    InetAddress ipAddress = InetAddress.getByName("128.101.101.101");

    AsnResponse response = reader.asn(ipAddress);

    System.out.println(response.getAutonomousSystemNumber());       // 217
    System.out.println(response.getAutonomousSystemOrganization()); // 'University of Minnesota'
}

Connection-Type

// A File object pointing to your GeoIP2 Connection-Type database
File database = new File("/path/to/GeoIP2-Connection-Type.mmdb");

// This creates the DatabaseReader object. To improve performance, reuse
// the object across lookups. The object is thread-safe.
DatabaseReader reader = new DatabaseReader.Builder(database).build();

InetAddress ipAddress = InetAddress.getByName("128.101.101.101");

ConnectionTypeResponse response = reader.connectionType(ipAddress);

// getConnectionType() returns a ConnectionType enum
ConnectionType type = response.getConnectionType();

System.out.println(type); // 'Corporate'

Domain

// A File object pointing to your GeoIP2 Domain database
File database = new File("/path/to/GeoIP2-Domain.mmdb");

// This creates the DatabaseReader object. To improve performance, reuse
// the object across lookups. The object is thread-safe.
DatabaseReader reader = new DatabaseReader.Builder(database).build();

InetAddress ipAddress = InetAddress.getByName("128.101.101.101");

DomainResponse response = reader.domain(ipAddress);

System.out.println(response.getDomain()); // 'umn.edu'

Enterprise

// A File object pointing to your GeoIP2 Enterprise database
File database = new File("/path/to/GeoIP2-Enterprise.mmdb");

// This creates the DatabaseReader object. To improve performance, reuse
// the object across lookups. The object is thread-safe.
try (DatabaseReader reader = new DatabaseReader.Builder(database).build()) {
    InetAddress ipAddress = InetAddress.getByName("128.101.101.101");

    //  Use the enterprise(ip) method to do a lookup in the Enterprise database
    EnterpriseResponse response = reader.enterprise(ipAddress);

    Country country = response.getCountry();
    System.out.println(country.getIsoCode());            // 'US'
    System.out.println(country.getName());               // 'United States'
    System.out.println(country.getNames().get("zh-CN")); // '美国'
    System.out.println(country.getConfidence());         // 99

    Subdivision subdivision = response.getMostSpecificSubdivision();
    System.out.println(subdivision.getName());           // 'Minnesota'
    System.out.println(subdivision.getIsoCode());        // 'MN'
    System.out.println(subdivision.getConfidence());     // 77

    City city = response.getCity();
    System.out.println(city.getName());       // 'Minneapolis'
    System.out.println(city.getConfidence()); // 11

    Postal postal = response.getPostal();
    System.out.println(postal.getCode()); // '55455'
    System.out.println(postal.getConfidence()); // 5

    Location location = response.getLocation();
    System.out.println(location.getLatitude());  // 44.9733
    System.out.println(location.getLongitude()); // -93.2323
    System.out.println(location.getAccuracyRadius()); // 50
}

ISP

// A File object pointing to your GeoIP2 ISP database
File database = new File("/path/to/GeoIP2-ISP.mmdb");

// This creates the DatabaseReader object. To improve performance, reuse
// the object across lookups. The object is thread-safe.
DatabaseReader reader = new DatabaseReader.Builder(database).build();

InetAddress ipAddress = InetAddress.getByName("128.101.101.101");

IspResponse response = reader.isp(ipAddress);

System.out.println(response.getAutonomousSystemNumber());       // 217
System.out.println(response.getAutonomousSystemOrganization()); // 'University of Minnesota'
System.out.println(response.getIsp());                          // 'University of Minnesota'
System.out.println(response.getOrganization());                 // 'University of Minnesota'

Exceptions

For details on the possible errors returned by the web service itself, see the GeoIP2 web service documentation.

If the web service returns an explicit error document, this is thrown as an AddressNotFoundException, an AuthenticationException, an InvalidRequestException, or an OutOfQueriesException.

If some sort of transport error occurs, an HttpException is thrown. This is thrown when some sort of unanticipated error occurs, such as the web service returning a 500 or an invalid error document. If the web service request returns any status code besides 200, 4xx, or 5xx, this also becomes an HttpException.

Finally, if the web service returns a 200 but the body is invalid, the client throws a GeoIp2Exception. This exception also is the parent exception to the above exceptions.

Values to use for Database or Map Keys

We strongly discourage you from using a value from any getNames method as a key in a database or map.

These names may change between releases. Instead we recommend using one of the following:

  • com.maxmind.geoip2.record.City - City.getGeoNameId
  • com.maxmind.geoip2.record.Continent - Continent.getCode or Continent.getGeoNameId
  • com.maxmind.geoip2.record.Country and com.maxmind.geoip2.record.RepresentedCountry - Country.getIsoCode or Country.getGeoNameId
  • com.maxmind.geoip2.record.Subdivision - Subdivision.getIsoCode or Subdivision.getGeoNameId

Multi-Threaded Use

This API fully supports use in multi-threaded applications. When using the DatabaseReader or the WebServiceClient in a multi-threaded application, we suggest creating one object and sharing that across threads.

What data is returned?

While many of the location databases and web services return the same basic records, the attributes poplulated can vary. In addition, MaxMind does not always have every piece of data for any given IP address.

Because of these factors, it is possible for any web service to return a record where some or all of the attributes are unpopulated.

See our web-service developer documentation for details on what data each web service may return.

The only piece of data which is always returned is the ip_address available at lookup.getTraits().getIpAddress().

Integration with GeoNames

GeoNames offers web services and downloadable databases with data on geographical features around the world, including populated places. They offer both free and paid premium data. Each feature is uniquely identified by a geonameId, which is an integer.

Many of the records returned by the GeoIP2 web services and databases include a getGeoNameId() method. This is the ID of a geographical feature (city, region, country, etc.) in the GeoNames database.

Some of the data that MaxMind provides is also sourced from GeoNames. We source things like place names, ISO codes, and other similar data from the GeoNames premium data set.

Reporting data problems

If the problem you find is that an IP address is incorrectly mapped, please submit your correction to MaxMind.

If you find some other sort of mistake, like an incorrect spelling, please check the GeoNames site first. Once you've searched for a place and found it on the GeoNames map view, there are a number of links you can use to correct data ("move", "edit", "alternate names", etc.). Once the correction is part of the GeoNames data set, it will be automatically incorporated into future MaxMind releases.

If you are a paying MaxMind customer and you're not sure where to submit a correction, please contact MaxMind support for help.

Other Support

Please report all issues with this code using the GitHub issue tracker.

If you are having an issue with a MaxMind service that is not specific to the client API, please contact MaxMind support.

Requirements

MaxMind has tested this API with Java 11 and above.

Contributing

Patches and pull requests are encouraged. Please include unit tests whenever possible.

Versioning

The GeoIP2 Java API uses Semantic Versioning.

Copyright and License

This software is Copyright (c) 2013-2023 by MaxMind, Inc.

This is free software, licensed under the Apache License, Version 2.0.

geoip2-java's People

Contributors

2shortplanks avatar aktasfatih avatar andyjack avatar autarch avatar borisz avatar chenzhangg avatar danielkaneider avatar dependabot-preview[bot] avatar dependabot[bot] avatar eilara avatar faktas2 avatar horgh avatar kevcenteno avatar lookfirst avatar martijnvg avatar najiobeid avatar nchelluri avatar nitinvijay94 avatar oalders avatar oschwald avatar patrickcronin avatar phraktle avatar rafl avatar rocketraman avatar shadromani avatar snyk-bot avatar tehshrike avatar tjmather avatar ugexe avatar wesrice avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

geoip2-java's Issues

Can I get timezone from City or ISP?

Hi,
I have already used ok City and ISP API, work very well. Thanks so much!
But I'm having issue about getting timezone data. Would you please show me the way to get it?
Thanks!

PROBLEM, GeoIP2 with Wowza Streaming Engine :(

Hello, I can not integrate GeoIP2 with Wowza, what could be happening? The GeoIP version (1) worked perfectly.

Wowza Code:

import com.maxmind.geoip2.*;
import com.maxmind.geoip2.exception.GeoIp2Exception;
import com.maxmind.geoip2.model.CityResponse;
import com.maxmind.geoip2.record.City;
import com.maxmind.geoip2.record.Country;
import com.maxmind.geoip2.record.Location;
import com.maxmind.geoip2.record.Postal;
import com.maxmind.geoip2.record.Subdivision;
import com.maxmind.db.*;

    public static final String DATABASE_CITY_PATH = "C:/wowza/GeoIP2-City.mmdb";

    public void onAppStart(IApplicationInstance appInstance) {
        appInstance.setStreamNameAliasProvider(this);
        this.applicationInstance = appInstance;
        this.vHostName = appInstance.getVHost();

        try {
            Class.forName("com.mysql.jdbc.Driver").newInstance(); 
            // A File object pointing to your GeoLite2 database
               File dbFile = new File(DATABASE_CITY_PATH);
               getLogger().info("DATABASE_CITY_PATH: "+DATABASE_CITY_PATH);
               // This creates the DatabaseReader object,
               // which should be reused across lookups.

               DatabaseReader reader = new DatabaseReader.Builder(dbFile).build();

               // A IP Address
               InetAddress ipAddress = InetAddress.getByName("128.101.101.101");

               // Get City info
               CityResponse response = reader.city(ipAddress);

               // Country Info
               Country country = response.getCountry();
               System.out.println("Country IsoCode: "+ country.getIsoCode()); // 'US'
               System.out.println("Country Name: "+ country.getName()); // 'United States'
               System.out.println(country.getNames().get("zh-CN")); //

               Subdivision subdivision = response.getMostSpecificSubdivision();
               System.out.println("Subdivision Name: " +subdivision.getName()); // 'Minnesota'
               System.out.println("Subdivision IsoCode: "+subdivision.getIsoCode()); // 'MN'

               // City Info.
               City city = response.getCity();
               System.out.println("City Name: "+ city.getName()); // 'Minneapolis'

               // Postal info
               Postal postal = response.getPostal();
               System.out.println(postal.getCode()); // '55455'

               // Geo Location info.
               Location location = response.getLocation();

               // Latitude
               System.out.println("Latitude: "+ location.getLatitude()); // 44.9733

               // Longitude
               System.out.println("Longitude: "+ location.getLongitude()); // -93.2323

        } catch (Exception e) { 
            getLogger().error("Error loading plugins: "+e.toString());
        }
    }

Here is a link with more explanations. http://pt.stackoverflow.com/q/180286/8984
Because the old version works, but the new one, GeoIP2 did not work @oschwald ?

Not able read from db file from ByteArrayInputstream

When database file is read into ByteArrayInputstream and used in Reader. Getting below exception.

DatabaseReader reader = new DatabaseReader.Builder(stream).withCache(new CHMCache()).build();

error
Could not find a MaxMind DB metadata marker in this file (). Is this a valid MaxMind DB file?

Read Timeout

Hi,

I have seen that you have the possibility of setup a connection timeout which is working fine. I have opened a service request few days ago about the possibility of adding a read timeout.

I got as a response this branch: https://github.com/maxmind/GeoIP2-java/tree/bz/add-read-timeout which I have used but seems not to work.

Although I setup a read timeout of 50 msecs (just for testing purposes) the call to your web service lasts for few seconds. Using the debugger looks like the read timeout is being completely ignored.

We need this feature as the response time is pretty critical to our company.

Can someone give a look to this?

Thanks

InetAddress with Hostname returns IOException

When we try to pass a valid InetAddress created from a hostname to the DatabaseReader, we receive a IOException back from the library.

// This creates the DatabaseReader object, which should be reused across
// lookups.
DatabaseReader reader = new DatabaseReader.Builder(database).build();

CityResponse response = reader.city(InetAddress.getByName("www.maxmind.com"));
System.out.println(response.getCountry().getIsoCode()); // 'US'

Would it be possible to get also for this cases a valid result, because the InetAddress receives the correct address.

thx Tommy

Jackson 2.1 / com.fasterxml.jackson.databind.node

com.fasterxml.jackson.databind.node was introduced to Jackson with 2.2 but tag 0.4.0 of com.maxmind.geoip2:geoip2 depends on 2.1 (due to com.google.http-client:google-http-client-jackson2:jar:1.15.0-rc:compile), so I am getting java.lang.NoClassDefFoundError: com/fasterxml/jackson/databind/node/FloatNode

Error when call GeoIP2-java

I put dependency GeoIP2-java in maven project, but i have this error:

2016-10-08 17:20:26,482 ERROR [dispatcherServlet]: Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Handler processing failed; nested exception is java.lang.NoSuchMethodError: com.fasterxml.jackson.databind.node.ArrayNode.(Lcom/fasterxml/jackson/databind/node/JsonNodeFactory;Ljava/util/List;)V] with root cause
java.lang.NoSuchMethodError: com.fasterxml.jackson.databind.node.ArrayNode.(Lcom/fasterxml/jackson/databind/node/JsonNodeFactory;Ljava/util/List;)V
at com.maxmind.db.Decoder.decodeArray(Decoder.java:272) ~[maxmind-db-1.2.1.jar:na]
at com.maxmind.db.Decoder.decodeByType(Decoder.java:156) ~[maxmind-db-1.2.1.jar:na]
at com.maxmind.db.Decoder.decode(Decoder.java:147) ~[maxmind-db-1.2.1.jar:na]
at com.maxmind.db.Decoder.decodeMap(Decoder.java:281) ~[maxmind-db-1.2.1.jar:na]
at com.maxmind.db.Decoder.decodeByType(Decoder.java:154) ~[maxmind-db-1.2.1.jar:na]
at com.maxmind.db.Decoder.decode(Decoder.java:147) ~[maxmind-db-1.2.1.jar:na]
at com.maxmind.db.Decoder.decode(Decoder.java:87) ~[maxmind-db-1.2.1.jar:na]
at com.maxmind.db.Reader.(Reader.java:132) ~[maxmind-db-1.2.1.jar:na]
at com.maxmind.db.Reader.(Reader.java:116) ~[maxmind-db-1.2.1.jar:na]
at com.maxmind.geoip2.DatabaseReader.(DatabaseReader.java:35) ~[geoip2-2.8.0.jar:2.8.0]
at com.maxmind.geoip2.DatabaseReader.(DatabaseReader.java:23) ~[geoip2-2.8.0.jar:2.8.0]
at com.maxmind.geoip2.DatabaseReader$Builder.build(DatabaseReader.java:129) ~[geoip2-2.8.0.jar:2.8.0]

please help

InvalidDatabaseException: pointer larger than the database

We are using paid maxmind account, GeoIP2-Country binary db, DatabaseReader (not webservice) api.

Operation used: country search
Ip being searched: 65.208.151.118
Exception is thrown:
com.maxmind.db.InvalidDatabaseException: The MaxMind DB file's data section contains bad data: pointer larger than the database.
at com.maxmind.db.Decoder.decode(Decoder.java:80)
at com.maxmind.db.Decoder.decode(Decoder.java:101)
at com.maxmind.db.Reader.resolveDataPointer(Reader.java:205)
at com.maxmind.db.Reader.get(Reader.java:103)
at com.maxmind.geoip2.DatabaseReader.get(DatabaseReader.java:148)
at com.maxmind.geoip2.DatabaseReader.country(DatabaseReader.java:193)

Max mind api version in maven:
<dependency> <groupId>com.maxmind.geoip2</groupId> <artifactId>geoip2</artifactId> <version>2.1.0</version> </dependency>

Code for country search used:
InetAddress ipAddress = InetAddress.getByName(ip);
CountryResponse response = database.country(ipAddress);
com.maxmind.geoip2.record.Country country = response.getCountry();
String result = country.getIsoCode();

Code for database loading:
InputStream data = ExternalIpMappingServiceImpl.class.getClassLoader().getResourceAsStream("GeoIP2-Country.mmdb");
database = new DatabaseReader.Builder(data).build();

toJson()

Does this API support taking a root level object and serializing the full graph to JSON (and back and forth)? If not, could it be added? This would be useful when taking output from your library and dumping into a document store such as mongo

Is there any option exist for finding location from Longitude and Latitude?

I want to use GeoIP2 Java API for location detection. I found a way for detecting location from IP. But it is not a comprehensive way if user is behind of a proxy server. Is there any way exist for detecting location from Longitude and Latitude using GeoIP2 Java API? I want to avoid using Google API for detecting location as it is not fully free.

I am expecting a quick response or resolution regarding this.

Thanks.

Region missing

Hi, I was using the old legacy library and it had the region codes and names hard coded in.

Is there any plan to add them back?
Why were they added as a class instead of in the database itself?

Regards

Maven dependency is wrong

In the Readme

   <dependency>
        <groupId>com.maxmind.geoip2</groupId>
        <artifactId>geoip2</artifactId>
        <version>v2.3.0</version>
    </dependency>

The version bit should be just '2.3.0' and not 'v.2.3.0'

Performance issue GeoIP2 vs GeoIP Legacy

I'm testing GeoIP2 vs GeoIP and getting a very strange result, it seems the new api is way slower.
maxMind1 - Legacy API
maxMind2 - new API, FileMode.MEMORY_MAPPED
maxMind2v2 - new API, FileMode.MEMORY
Caches are on;

Here are my benchmark results:

Benchmark                      Mode     Cnt       Score      Error  Units
MindMaxBenchmark.maxMind1     thrpt      50  176225.276 ± 1603.768  ops/s
MindMaxBenchmark.maxMind2     thrpt      50     825.766 ±   12.428  ops/s
MindMaxBenchmark.maxMind2v2   thrpt      50     767.863 ±   35.938  ops/s
MindMaxBenchmark.maxMind1      avgt      50      ≈ 10⁻⁵              s/op
MindMaxBenchmark.maxMind2      avgt      50       0.001 ±    0.001   s/op
MindMaxBenchmark.maxMind2v2    avgt      50       0.001 ±    0.001   s/op
MindMaxBenchmark.maxMind1    sample  131466      ≈ 10⁻⁵              s/op
MindMaxBenchmark.maxMind2    sample   38187       0.001 ±    0.001   s/op
MindMaxBenchmark.maxMind2v2  sample   34064       0.001 ±    0.001   s/op
MindMaxBenchmark.maxMind1        ss      50      ≈ 10⁻⁴              s/op
MindMaxBenchmark.maxMind2        ss      50       0.028 ±    0.005   s/op
MindMaxBenchmark.maxMind2v2      ss      50       0.026 ±    0.004   s/op

Services are initialized at the test start like that:

final DatabaseReader mindMaxDB = new DatabaseReader.Builder(...)
                    .withCache(new CHMCache())
                    .fileMode(Reader.FileMode.MEMORY)
                    .build();

and the test code is like (some null checks are ommited):

        public LocationData resolve(InetAddress inetAddress) {

                CityResponse cityResponse = mindMaxDB.city(inetAddress);
                Location location = cityResponse.getLocation();
                return new LocationData(
                        inetAddress, cityResponse.getCountry().getIsoCode(), cityResponse.getCity().getName()
                );
            }

The Legacy API:
Service created like that

LookupService lookupService = new LookupService(file, LookupService.GEOIP_MEMORY_CACHE);

and test code is

 com.maxmind.geoip.Location location = lookupService.getLocation(inetAddress);
                            return new LocationData(
                                    inetAddress, location.countryCode,
                                    location.city);

I'd like to know if I'm missing something?

Missing jackson.databind in jar?

I'm trying to use this library, but get the error below from the following line:

val file = new File("src/main/resources/GeoLite2-City.mmdb")
val db = new DatabaseReader.Builder(file).withCache(new CHMCache()).build()

yields:

An exception or error caused a run to abort: com.fasterxml.jackson.databind.node.ArrayNode.<init>(Lcom/fasterxml/jackson/databind/node/JsonNodeFactory;Ljava/util/List;)V 
java.lang.NoSuchMethodError: com.fasterxml.jackson.databind.node.ArrayNode.<init>(Lcom/fasterxml/jackson/databind/node/JsonNodeFactory;Ljava/util/List;)V
    at com.maxmind.db.Decoder.decodeArray(Decoder.java:272)
    at com.maxmind.db.Decoder.decodeByType(Decoder.java:156)
    at com.maxmind.db.Decoder.decode(Decoder.java:147)
    at com.maxmind.db.Decoder.decodeMap(Decoder.java:281)
    at com.maxmind.db.Decoder.decodeByType(Decoder.java:154)
    at com.maxmind.db.Decoder.decode(Decoder.java:147)
    at com.maxmind.db.Decoder.decode(Decoder.java:87)
    at com.maxmind.db.Reader.<init>(Reader.java:132)
    at com.maxmind.db.Reader.<init>(Reader.java:116)
    at com.maxmind.geoip2.DatabaseReader.<init>(DatabaseReader.java:39)
    at com.maxmind.geoip2.DatabaseReader.<init>(DatabaseReader.java:27)
    at com.maxmind.geoip2.DatabaseReader$Builder.build(DatabaseReader.java:133)
    at com.sift.etl.processors.IpToLatLong$.<init>(IpToLatLong.scala:22)

Does this indicate the source is missing the jackson.databind.jar? Is it possible I have two conflicting copies of this dependency?

DatabaseReader doesn't correctly close

Hi,

it looks like the DatabaseReader.close() of geoip2-2.0.0 doesn't work as excpeted. Looking deeper it seems that just an AtomaticReference is set to null. The problem is that, closing the reader, moving the db file, and opening it again, fails. This is demonstrated in the JUnit test case below. Moreover a System.gc should not be necessary.

import java.io.File;
import java.io.IOException;
import java.net.InetAddress;
import java.net.UnknownHostException;

import org.junit.Assert;
import org.junit.Test;

import com.maxmind.db.Reader.FileMode;
import com.maxmind.geoip2.DatabaseReader;
import com.maxmind.geoip2.exception.GeoIp2Exception;

public class TestDatabaseReaderClose {
    @Test
    public void test1() throws Exception {
        executeTest(false, true);
    }
    @Test
    public void test2() throws Exception {
        executeTest(true, false);
    }
    @Test
    public void test3() throws Exception {
        executeTest(false, true);
    }
    @Test
    public void test4() throws Exception {
        executeTest(true, false);
    }
    private void executeTest(boolean memoryMapped, boolean gc) throws IOException, GeoIp2Exception, UnknownHostException {
        String fileName = "C:\\Data\\GeoIP2-City.mmdb";

        File file1 = new File(fileName);
        File file2 = new File(fileName + ".temp");

        DatabaseReader reader1 = new DatabaseReader.Builder(file1).fileMode(memoryMapped ? FileMode.MEMORY_MAPPED : FileMode.MEMORY).build();
        Assert.assertEquals("GB", reader1.city(InetAddress.getByName("80.1.1.1")).getCountry().getIsoCode());
        reader1.close();
        if (gc)
            System.gc();

        Assert.assertTrue(file1.renameTo(file2));
        Assert.assertTrue(file2.renameTo(file1));

        DatabaseReader reader2 = new DatabaseReader.Builder(file1).build();
        Assert.assertEquals("GB", reader2.city(InetAddress.getByName("80.1.1.1")).getCountry().getIsoCode());
        reader2.close();
    }
}

Using legacy databases with the Java API?

I had a look at:
http://dev.maxmind.com/geoip/legacy/geolite/

The GeoLite databases are our free IP geolocation databases. They are updated on the first Tuesday of each month. These databases are offered in the same binary and csv formats as our subscription databases. Any code which can read the subscription databases can also read the GeoLite databases.

So I downloaded http://geolite.maxmind.com/download/geoip/database/GeoIPv6.dat.gz
and unzipped into a .dat file.

Have the following code:

        File file = new File("C:\\GeoIPv61.dat");
        DatabaseReader reader = new DatabaseReader.Builder(file)
                .withCache(new CHMCache(100)).build();

        InetAddress ipAddress = InetAddress.getByName("128.101.101.101");
        CityResponse response = reader.city(ipAddress);
        Country country = response.getCountry();
        System.out.println(country.getIsoCode());
        System.out.println(country.getName());
        System.out.println(country.getNames().get("zh-CN"));

Result:

Exception in thread "main" com.maxmind.db.InvalidDatabaseException: Could not find a MaxMind DB metadata marker in this file (GeoIPv61.dat). Is this a valid MaxMind DB file?

How should one use the java api here to access the legacy databases?

Using FileMode.MEMORY_MAPPED leaves the file locked, even after closing the reader

When using FileMode.MEMORY_MAPPED, the .mmdb file on the file system is not available for manipulation (renaming, moving, deleting) even after the reader is closed.

Using FileMode.MEMORY doesn't suffer from this problem. See sample unit test below. Could be OS-related (using JDK7 on Win7, MaxMindDB: v.0.3.0, GeoIP2: v.0.6.0).
Ran into the issue while working on automatically updating the .mmdb file once a month in a long-running process.

import java.io.File;
import java.io.IOException;
import org.junit.Assert;
import org.junit.Test;
import com.maxmind.db.Reader.FileMode;
import com.maxmind.geoip2.DatabaseReader;
import com.maxmind.geoip2.DatabaseReader.Builder;

public class _GeolocationFileUpdate
{
    private static final String GEO_LITE2_CITY_MMDB = "GeoLite2-City.mmdb";
    private static final String GEO_LITE2_CITY2_MMDB = "GeoLite2-City2.mmdb";
    private static final String DIR = "c://Temp/"; //adjust if on a diff. OS. Place the .mmdb file here
    private static final File f1 = new File( DIR + GEO_LITE2_CITY_MMDB );
    private static final File f2 = new File( DIR + GEO_LITE2_CITY2_MMDB );

    @Test
    public void testSwitchingMem()
    {
        try
        {
            // make sure the file name is reverted after renaming:
            f2.renameTo( f1 );

            Builder builder = new DatabaseReader.Builder( f1 );
            builder.fileMode( FileMode.MEMORY );
            DatabaseReader reader = builder.build();
            reader.close();

            // make sure file is not in use by a process:
            Assert.assertTrue( f1.renameTo( f2 ) );
        }
        catch ( IOException e )  {  e.printStackTrace();  }
    }

    @Test
    public void testSwitchingMemMapped()
    {
        try
        {
            // make sure the file name is reverted after renaming:
            f2.renameTo( f1 );

            Builder builder = new DatabaseReader.Builder( f1 );
            builder.fileMode( FileMode.MEMORY_MAPPED );
            DatabaseReader reader = builder.build();
            reader.close();

            // make sure file is not in use by a process:
            // ! test fails on the line bellow, file is in use !:
            Assert.assertTrue( f1.renameTo( f2 ) );
        }
        catch ( IOException e ) { e.printStackTrace(); }
    }
}

NoSuchMethodError in Jackson

there is library class conflict or something if you are using a different version of jackson already in the developer's app?

java.lang.NoSuchMethodError: com.fasterxml.jackson.core.JsonParser.getValueAsString()Ljava/lang/String;
at com.fasterxml.jackson.databind.deser.std.StringDeserializer.deserialize(StringDeserializer.java:29)
at com.fasterxml.jackson.databind.deser.std.StringDeserializer.deserialize(StringDeserializer.java:11)
at com.fasterxml.jackson.databind.deser.std.MapDeserializer._readAndBindStringMap(MapDeserializer.java:432)
at com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:314)
at com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:26)
at com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:449)
at com.fasterxml.jackson.databind.deser.impl.FieldProperty.deserializeAndSet(FieldProperty.java:107)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:295)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:121)
at com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:449)
at com.fasterxml.jackson.databind.deser.impl.FieldProperty.deserializeAndSet(FieldProperty.java:107)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:295)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:121)
at com.fasterxml.jackson.databind.ObjectMapper._readValue(ObjectMapper.java:2860)
at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:1569)
at com.fasterxml.jackson.databind.ObjectMapper.treeToValue(ObjectMapper.java:1902)
at com.maxmind.geoip2.DatabaseReader.get(DatabaseReader.java:96)
at com.maxmind.geoip2.DatabaseReader.city(DatabaseReader.java:118)

java.lang.ClassNotFoundException: com.maxmind.geoip2.DatabaseReader$Builder

Hi am trying to write a UDF for pig and i have using this for getting ip details.

my code is new DatabaseReader.Builder(db).build();

which generate error stack like this.

Error: java.lang.ClassNotFoundException: com.maxmind.geoip2.DatabaseReader$Builder
    at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    at com.XXXXXX.XXXXXXX.pig.ParseGeolocation.getDBReader(ParseGeolocation.java:86)

I am using maven for dependency.

Java version is 1.8

also if i trying to use withcache() then i always get com/maxmind/db/NodeCache not found. I have no idea why this happening.

Version I am using is 2.7.0

thanks and regards
Mithun Mandal

DatabaseReader together with scheduled weekly updates

Hi there

Can you give some guidance on how DatabaseReader is intended to be used in combination with weekly scheduled updates via the geoipupdate tool?

Should DatabaseReader "just work" in this case, do you need to close it and open a new one every week or otherwise prod it to check for an updated DB? Something in the docs about this would be good. And of course strong preference for "just works" :)

Cheers!
-Matt

dmacode and state code question from new GeoIP2 java API

  1. In the old API, the Location class provided the dmacode. Is there any API to get the DMA code from the new API ?
  2. In the old API, the Location class provided the region which maps to the state. ? Is the multi-level subdivision attributes provides the state info in the new API ?

Thanks,
Yogesh

Error loading database from InputStream

Hi,

When I try to create a DatabaseReader from an InputStream instead of a File (something like):

InputStream is = getClass().getClassLoader().getResourceAsStream(filename);
cities = new DatabaseReader.Builder(is).build();

I get the following:

Caused by: java.lang.NoClassDefFoundError: com/fasterxml/jackson/databind/node/FloatNode
    at com.maxmind.db.Reader.<init>(Reader.java:89)
    at com.maxmind.db.Reader.<init>(Reader.java:65)
    at com.maxmind.geoip2.DatabaseReader.<init>(DatabaseReader.java:33)
    at com.maxmind.geoip2.DatabaseReader$Builder.build(DatabaseReader.java:122)

PS: This is using version 0.8.0:

        <dependency>
            <groupId>com.maxmind.geoip2</groupId>
            <artifactId>geoip2</artifactId>
            <version>0.8.0</version>
        </dependency>

Enhancement request: Would be very helpful if the Metadata (com.maxmind.db.Metadata) is exposed via a getter in the com.maxmind.geoip2.DatabaseReader

For a scenario with a long-running process doing automatic downloads of the GeoLite2 DB from the MaxMind's web site it would be very helpful to be able to get some metadata of the DB.
Currently the com.maxmind.db.Metadata is not exposed neither through the com.maxmind.geoip2.DatabaseReader nor through the com.maxmind.db.Reader classes. Would be nice to be able to get it through another getter in the com.maxmind.geoip2.DatabaseReader class.

Ideally there should be a way to check the metadata even before downloading (maybe through a GET at a different URL than the DB) so a decision can be made if the DB should be downloaded or not at this time (for example if there is a major revision update, it would become incompatible with the code reading it) but this would involve web site changes.

Error when creating databse reader

I am new to GeoIP and am trying to set up a test app to try it out on the free databases. I have imported the DatabaseReader class as well as the DatabaseReader.Builder class, and when I create a DatabaseReader object the I get an error that says:

Exception in thread "main" java.lang.NoClassDefFoundError: com/maxmind/db/Reader$FileMode
at com.maxmind.geoip2.DatabaseReader$Builder.(DatabaseReader.java:76)
at GeoIP.main(GeoIP.java:55)
Caused by: java.lang.ClassNotFoundException: com.maxmind.db.Reader$FileMode
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 2 more
Java Result: 1

I am not sure why this is, I have a File object pointing to my database and have no errors there. And I have properly added the GeoIP2.jar file to my project. Has anyone seen this error before?

untitled

City is always empty

I tried the following:

File database = new File("/path/to/GeoLite2-City.mmdb");
boolean memoryMapped = true    
    DatabaseReader reader = new DatabaseReader.Builder(file).fileMode(memoryMapped ? FileMode.MEMORY_MAPPED : FileMode.MEMORY).build();
InetAddress ipAddress = InetAddress.getByName("78.54.245.106")
CityResponse cResponse = reader.city(ipAddress)
City city = cResponse.getCity()
println("city2: "+city.getName())

it turns out that city is always null.

Use Apache HttpClient directly rather than Google's HTTP Client

Google's client wraps the Apache client but often lags a few version behind and has been a source of problems. The new minFraud Java API already uses the Apache client API directly.

We may also be able to share code between the GeoIP2 and minFraud APIs.

Class not found:

import com.maxmind.db.Metadata;
import com.maxmind.db.Reader;
import com.maxmind.db.Reader.FileMode;

It can't find these files.
So it can't run - class not found.

Subdivions field is missing

Hello,

I'm using your API (v 2.5) through this library and requesting as:

new WebServiceClient.Builder(userId, licenseKey).city(InetAddress.getByName(anIpAddress))

In the documentation I see the field subdivisions and I don't see any indication that it is optional.

Debugging I see that the CityResponse returned by this library has an empty ArrayList as the value for it's subdivisions field but when serialized into JSON the field isn't there which leads me to assume that when a collection is empty the key-value pair is not even serialized (in my opinion this is wrong, since an empty collection would also indicate that there are no subdivisions but wouldn't force the clients to check for the field's existence).

So, is the subdivisions field optional? Where does it say if it is or not?

Performance issues when working with several threads

Hi

I have ran a simple test to check the performance of the library when working with several threads. I am working currently with the GeoLite2-City.mmdb database.
What I did was running 4 threads, each thread doing 1 million requests to the API (DatabaseReader.city). Test ran ~80 seconds. I did the similar test with 1 thread - test ran ~30 seconds. In order to make sure this is not a machine resource issue I ran the first test again (4 threads) - only this time I created a different instance of DatabaseReader per thread. The test ran 30 seconds (same result as 1 thread). In this case the memory consumption was obviously 4 times higher.
By the way, I have set the FileMode to both MEMORY and MEMORY_MAPPED... no real difference between the two in all cases.

I tried to look into the code a bit. I found one suspected issue - I don't think that the method BufferHolder.get() should be synchronized... all it does, I think, is just duplicating a "prototype" (read only). However, I tried changing it and it did not help, so I do not believe this is the problem.

When working with high performance services, this issue is really important... I would appreciate your response..:)

Thanks!
Doron.

Transitive dependency on jackson-databind pulls in -SNAPSHOT

The transitive dependency on com.maxmind.db:maxmind-db contains an open-ended version clause for jackson-databind: [2.7.0,). This causes gradle to pull in the 2.9.0-SNAPSHOT of jackson-databind into the project.

+--- com.maxmind.geoip2:geoip2:2.8.0
|    +--- com.maxmind.db:maxmind-db:1.2.1
|    |    \--- com.fasterxml.jackson.core:jackson-databind:[2.7.0,) -> 2.9.0-SNAPSHOT (*)
|    +--- org.apache.httpcomponents:httpclient:4.5.2 (*)
|    \--- com.fasterxml.jackson.core:jackson-databind:2.8.2 -> 2.9.0-SNAPSHOT (*)

This of course is a big problem -- the runtime now fails when the rest of the project uses a 2.8 release version of jackson.

I'm not sure why this has caused problems in our builds only recently because AFAICT this is not a new thing in maxmind-db. Still, it would be better if maxmind-db declared the version dependency explicitly instead of giving an open-ended range, which I think is the issue.

Using geoip2-java without granting the 'suppressAccessChecks' permission

The way jackson-databind is being used, is that it always uses reflection to suppress access checking in order to deserialise into the geoip2 model classes. This doesn't have to be way and jackson-databind can also be used without suppressing access checks. The jackson-databind's ObjectMapper needs be configured to not overwrite access modifiers ObjectMapper#configure(MapperFeature.CAN_OVERRIDE_ACCESS_MODIFIERS, false) and the model classes would then also need to have public getters/setters or public constructors. I'm wondering if using reflection to suppress access checking was chosen for a particular reason. It would be great if geoip2-java can be used without the suppressAccessChecks permission.

We're currently developing a pipeline framework to enrich data before indexing:
elastic/elasticsearch#14049

and are using geoip for geo lookups: https://github.com/elastic/elasticsearch/blob/feature/ingest/plugins/ingest/src/main/java/org/elasticsearch/ingest/processor/geoip/GeoIpProcessor.java#L47

Elasticsearch runs with the security manager enabled and doesn't allow suppressing access checks. Because this pipeline framework is a plugin we can specifically allow it in specifically privilege code blocks, but we prefer to use geoip2 without setting the suppressAccessChecks privilege at all.

Use geoip2-java in spark streaming

Hi there,

I want to use geoip2 to process nging log with Spark Streaming, but I could not find any example about how to use it with Spark Streaming. Do I use it in a wrong way ?

Better separation between API code and data

This is one thing that has been bugging me for years as a user of GeoIP City.

For some inexplicable reason the Country codes and country names were not part of the data file, but instead hard coded in an array in the API lookup code. This of course meant a code upgrade every time a new country appears. For a while it even meant array index out of bounds exceptions if we were too late updating our client library.

These days I see the same folly repeated for time zone names and region names. Why on earth can those not be placed in the data file? At 40+ MB the added cost of a few hundred extra names should be negligible.

If the concern is speed it should be easy enough to read all the lookup tables into memory on init.

If the concern is compatibility there is still no reason to hard code more lookups into the API code. Without knowing the full details of the data file layout I still imagine there would be room to add slots for lookup tables without breaking existing clients.

New transport created unnecessarily for http requests

This line creates a new transport on each request:

https://github.com/maxmind/GeoIP2-java/blob/master/src/main/java/com/maxmind/geoip2/WebServiceClient.java#L281

As per the comment, this is done because the transport is not thread-safe. However, according to Google the transport is thread-safe:

"Implementation is thread-safe. For maximum efficiency, applications should use a single globally-shared instance of the HTTP transport."

From:

https://developers.google.com/api-client-library/java/google-http-java-client/reference/1.20.0/com/google/api/client/http/javanet/NetHttpTransport

The importance of DatabaseReader.close() method

Hi

We are trying to synchronize a DatabaseReader update (after mmdb file changed) with reads happening a-synchronically without using locks (code MUST be clean of locks).
We are working in MEMORY mode (not MEMORY_MAPPED).
The only way to do that, as I see it, is if we first load the new file into a new DatabaseReader - and then replace the reader we are using in the reads (as long as the reader is a volatile member, this should be a safe operation). The problem is that in such flow we cannot call close() on the DatabaseReader we are dumping, because we may close a reader being currently used (simple test created such scenario and exceptions).
How important is the close() operation when using MEMORY mode? I tried profiling a bit and did not see any "memory leak" when I am not closing. Do I miss anything? It seems like the "set null" done there is meant for garbage collection that is important only in MEMORY_MAPPED mode.

Thanksl

JAR Conflict

Hi,
In our project we are using older version of following jars and unfortunately we cannot replace with the jar which comes with maxmind geoip2-2.8.0 libs.
We are using
commons-codec-1.7
jackson-annotations-2.4.2
jackson-core-2.4.2
jackson-databind-2.4.2

maxmind geoip2-2.8.0
commons-codec-1.9
jackson-annotations-2.8.0
jackson-core-2.8.2
jackson-databind-2.8.2

Due to this I am always below getting run time error

java.lang.NoSuchMethodError: com.fasterxml.jackson.databind.node.ArrayNode.<init>(Lcom/fasterxml/jackson/databind/node/JsonNodeFactory;Ljava/util/List;)V
	com.maxmind.db.Decoder.decodeArray(Decoder.java:272)
	com.maxmind.db.Decoder.decodeByType(Decoder.java:156)
	com.maxmind.db.Decoder.decode(Decoder.java:147)
	com.maxmind.db.Decoder.decodeMap(Decoder.java:281)
	com.maxmind.db.Decoder.decodeByType(Decoder.java:154)
	com.maxmind.db.Decoder.decode(Decoder.java:147)
	com.maxmind.db.Decoder.decode(Decoder.java:87)
	com.maxmind.db.Reader.<init>(Reader.java:132)
	com.maxmind.db.Reader.<init>(Reader.java:116)
	com.maxmind.geoip2.DatabaseReader.<init>(DatabaseReader.java:35)
	com.maxmind.geoip2.DatabaseReader.<init>(DatabaseReader.java:23)
	com.maxmind.geoip2.DatabaseReader$Builder.build(DatabaseReader.java:129)

Any suggestion ?
thanks in advance

#2.6.0 com/maxmind/db/Decoder.java:272:in `decodeArray': java.lang.NoSuchMethodError: com.fasterxml.jackson.databind.node.ArrayNode

With the version 2.6.0, downloaded at https://github.com/maxmind/GeoIP2-java/releases/download/v2.6.0/geoip2-2.6.0-with-dependencies.zip, I came across this issue:

com/maxmind/db/Decoder.java:272:in `decodeArray': java.lang.NoSuchMethodError: com.fasterxml.jackson.databind.node.ArrayNode.(Lcom/fasterxml/jackson/databind/node/JsonNodeFactory;Ljava/util/List;)

When trying to instance a DatabaseReader, with jRuby (this is to build a logstash filter plugin):

db_file = JavaIO::File.new(@database)
@parser = DatabaseReader::Builder.new(db_file).build();

It looks like an issue with Jackson lib?

where do i store the pom.xml file ?

you are saying to use and create pom.xml file
where do i store it ?

<dependency>
    <groupId>com.maxmind.geoip2</groupId>
    <artifactId>geoip2</artifactId>
    <version>2.2.0</version>
</dependency>

jar file?

Can you please provide a JAR file for those of us who do not use Maven?

Thanks.

Not able to get region name every time

I was making use of legacy Geolite City api. But for every IP I am not able to get region. This was very frustrating for me. Of course its a free, It should return country name and region code at least.

response.getHeaders().getContentLength() fails to cast String to Long

Hi,

I'm using GeoIP2 in my Android app and I'm getting this crash with my users. I think the crash rate is about 50% although I cannot replicate it. There is no particular version of Android, though all my users are Android 4+. No particular language or device. Can you please help? I don't know what it is they are getting as a response that is causing the casting of a String to Long to fail.

Cheers, Duane.

java.lang.RuntimeException: An error occured while executing doInBackground()
at android.os.AsyncTask$3.done(AsyncTask.java:300)
at java.util.concurrent.FutureTask.finishCompletion(FutureTask.java:355)
at java.util.concurrent.FutureTask.setException(FutureTask.java:222)
at java.util.concurrent.FutureTask.run(FutureTask.java:242)
at android.os.AsyncTask$SerialExecutor$1.run(AsyncTask.java:231)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1112)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:587)
at java.lang.Thread.run(Thread.java:848)
Caused by: java.lang.ClassCastException: java.lang.String cannot be cast to java.lang.Long
at com.google.api.client.http.HttpHeaders.java.lang.Long getContentLength()(SourceFile:326)
at com.maxmind.geoip2.WebServiceClient.java.lang.Object responseFor$296315a2(java.lang.String,java.lang.Class)(SourceFile:258)
at com.maxmind.geoip2.WebServiceClient.com.maxmind.geoip2.model.CityResponse city()(SourceFile:225)
at com.enzuredigital.weatherbomb.WeatherActivity$InitPlaceTask.java.lang.String doInBackground$4af589aa()(SourceFile:2459)
at com.enzuredigital.weatherbomb.WeatherActivity$InitPlaceTask.java.lang.Object doInBackground(java.lang.Object[])(SourceFile:2405)
at android.os.AsyncTask$2.call(AsyncTask.java:288)
at java.util.concurrent.FutureTask.run(FutureTask.java:237)
... 4 more

This is the code in my Android app:

WebServiceClient client = new WebServiceClient.Builder(MAXMIND_ID, MAXMIND_KEY).build();
CityResponse response = null;
try {
    response = client.city();  // <---- Offending line.
} catch (IOException e) {
    e.printStackTrace();
} catch (GeoIp2Exception e) {
    e.printStackTrace();
}

Update httpclient artifact

Hi Folks,
There are newever versions of the Apache httpclient artifact which this library should consider updating to.
Clients consuming geoip2 have classloading issues due to geoip2 using an old httpclient version.
I'll see if I can get something together and submit a PR.

dependencies issue started appearing.

The below issue was not there before two days.

Failed to execute goal on project ABC: Could not resolve dependencies for project com:war:1.0.0: Failed to collect dependencies at com.maxmind.geoip2:geoip2:jar:2.7.0 -> com.maxmind.db:maxmind-db:jar:1.2.1 -> com.fasterxml.jackson.core:jackson-databind:jar:2.9.0-SNAPSHOT: Failed to read artifact descriptor for com.fasterxml.jackson.core:jackson-databind:jar:2.9.0-SNAPSHOT: Could not find artifact com.fasterxml:oss-parent:pom:28

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.