Code Monkey home page Code Monkey logo

maxminddb's Introduction

maxminddb

Pure Ruby GeoIP2 MaxMind DB reader, which doesn't require libmaxminddb.

You can find more information about the GeoIP2 database here.

Gem Version Build Status Code Climate

Installation

Add this line to your application's Gemfile:

gem 'maxminddb'

And then execute:

$ bundle

Or install it yourself as:

$ gem install maxminddb

Usage

db = MaxMindDB.new('./GeoLite2-City.mmdb')
ret = db.lookup('74.125.225.224')

ret.found? # => true
ret.country.name # => 'United States'
ret.country.name('zh-CN') # => '美国'
ret.country.iso_code # => 'US'
ret.city.name(:fr) # => 'Mountain View'
ret.subdivisions.most_specific.name # => 'California'
ret.location.latitude # => -122.0574
ret.to_hash # => {"city"=>{"geoname_id"=>5375480, "names"=>{"de"=>"Mountain View", "en"=>"Mountain View", "fr"=>"Mountain View", "ja"=>"マウンテンビュー", "ru"=>"Маунтин-Вью", "zh-CN"=>"芒廷维尤"}}, "continent"=>{"code"=>"NA", "geoname_id"=>6255149, "names"=>{"de"=>"Nordamerika", "en"=>"North America", "es"=>"Norteamérica", "fr"=>"Amérique du Nord", "ja"=>"北アメリカ", "pt-BR"=>"América do Norte", "ru"=>"Северная Америка", "zh-CN"=>"北美洲"}}, "country"=>{"geoname_id"=>6252001, "iso_code"=>"US", "names"=>{"de"=>"USA", "en"=>"United States", "es"=>"Estados Unidos", "fr"=>"États-Unis", "ja"=>"アメリカ合衆国", "pt-BR"=>"Estados Unidos", "ru"=>"Сша", "zh-CN"=>"美国"}}, "location"=>{"latitude"=>37.419200000000004, "longitude"=>-122.0574, "metro_code"=>807, "time_zone"=>"America/Los_Angeles"}, "postal"=>{"code"=>"94043"}, "registered_country"=>{"geoname_id"=>6252001, "iso_code"=>"US", "names"=>{"de"=>"USA", "en"=>"United States", "es"=>"Estados Unidos", "fr"=>"États-Unis", "ja"=>"アメリカ合衆国", "pt-BR"=>"Estados Unidos", "ru"=>"Сша", "zh-CN"=>"美国"}}, "subdivisions"=>[{"geoname_id"=>5332921, "iso_code"=>"CA", "names"=>{"de"=>"Kalifornien", "en"=>"California", "es"=>"California", "fr"=>"Californie", "ja"=>"カリフォルニア州", "pt-BR"=>"Califórnia", "ru"=>"Калифорния", "zh-CN"=>"加利福尼亚州"}}]}

Even if no result could be found, you can ask for the attributes without guarding for nil:

db = MaxMindDB.new('./GeoLite2-City.mmdb')
ret = db.lookup('127.0.0.1')
ret.found? # => false
ret.country.name # => nil
ret.to_hash # => {}

For testing or other purposes, you might wish to treat localhost IP addresses as some other address - an external one. You can do this by assigning the desired external IP address to the attribute local_ip_alias:

db = MaxMindDB.new('./GeoLite2-City.mmdb')
ret = db.local_ip_alias = '74.125.225.224'
ret = db.lookup('127.0.0.1')
ret.found? # => true
ret.country.name # => 'United States'
ret.to_hash.empty? # => false

It's also possible to access the database metadata.

db = MaxMindDB.new('./GeoLite2-City.mmdb')
db.metadata['build_epoch'] # => 1493762948
db.metadata # => {"binary_format_major_version"=>2, "binary_format_minor_version"=>0, "build_epoch"=>1493762948, "database_type"=>"GeoLite2-City", "description"=>{"en"=>"GeoLite2 City database"}, "ip_version"=>6, "languages"=>["de", "en", "es", "fr", "ja", "pt-BR", "ru", "zh-CN"], "node_count"=>3678850, "record_size"=>28}

Regarding thread safety

A MaxMindDB instance doesn't do any write operation after it is created. So we can consider it as an immutable object which is 'thread-safe'.

File reading strategies

By default, MaxMinDB.new will read the entire database into memory. This makes subsequent lookups fast, but can result in a fairly large memory overhead.

If having a low memory overhead is important, you can use the LowMemoryReader by passing a file_reader argument to MaxMindDB.new. For example:

db = MaxMindDB.new('./GeoLite2-City.mmdb', MaxMindDB::LOW_MEMORY_FILE_READER)
ret = db.lookup('74.125.225.224')

The LowMemoryReader will not load the entire database into memory. It's important to note that for Ruby versions lower than 2.5.0, the LowMemoryReader is not process safe. Forking a process after initializing a MaxMindDB instance can lead to unexpected results. For Ruby versions >= 2.5.0, LowMemoryReader uses File.pread which works safely in forking environments.

Contributing

  1. Fork it ( http://github.com/yhirose/maxminddb/fork )
  2. Create your feature branch (git checkout -b my-new-feature)
  3. Commit your changes (git commit -am 'Add some feature')
  4. Push to the branch (git push origin my-new-feature)
  5. Create new Pull Request

maxminddb's People

Contributors

aaron-stripe avatar andreaso avatar andrusha avatar anishavasandani avatar f3ndot avatar henrik avatar marvs avatar meagar avatar metaskills avatar olleolleolle avatar sandstrom avatar tinogomes avatar tommay avatar v-kolesnikov avatar vvangemert avatar xnt avatar yanthefawn avatar ydakuka avatar yhirose avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

maxminddb's Issues

Thread safety and performance

Hi,

I am trying to figure out the most efficient way to use this gem in a Rails app.

Will it be more efficient if I cache the instance as a class-level constant? For example:

GEO_DB = MaxMindDB.new('./GeoLite2-City.mmdb')

def detect_country
ret = GEO_DB.lookup('74.125.225.224')
ret.country.name
end

This is so that all requests can use the same instance.

Does this gem perform any write operation? I assume if it doesn't then it must be thread safe, right?

Thanks

Name should be maxminddb or something similar

Hi, Yuji,

First, thank you for creating this package. It's always nice to see folks out in the community supporting our products like this.

It'd be great if you could rename this package to something like "maxminddb".

There's nothing GeoLite2 or GeoIP2 specific about your code. If you wanted to make a high-level GeoIP2 specific API on top of this code, that might also make sense, but that would be best done as a separate package that depends on this code. We've done this for the APIs we maintain for Perl, Python, Java, and others. Those would be good examples for you to see where the separation is. See http://dev.maxmind.com/geoip/geoip2/downloadable/ for links to these APIs.

Thanks,

Dave Rolsky
Director of Software Engineering
MaxMind, Inc.
www.maxmind.com

represented_country is not supported any more in v0.1.3?

In GeoIP2 database: "we also include a represented_country key for some records. This is used when the IP address belongs to something like a military base. The represented_country is the country that the base represents. This can be useful for managing content licensing, among other uses."

most_general method on subdivisions

We often need to know only the first subdivision, if any, returned by the lookup. I love that you've implemented a most_specific method so that it isn't necessary to guard for nil in that case. Would a most_general (or least_specific) method be a common enough use case to merge into the core project as well?

@raw result returning an integer?

Hi! I'm using this library on prod, and noticing that we're getting errors whereby the MaxMindDB::Result we get from performing an i.p. lookup has its @raw instance variable equal to an integer instead of the intended hash.
e.g =>#<MaxMindDB::Result:0x007f889a1dcb38 @raw=2129834001930874614633>}

Has anyone ever encountered this before? It seems like it may be a bug. Thanks!

Use value objects for responses

I would prefer if you return not hashes, but value objects. One example how this could look like would be:

ret = db.lookup('74.125.225.224')
ret.class # => MaxMindDB::Result
ret.country.class # => MaxMindDB::Result::Country
ret.country.name # => 'Germany'
ret.country.name(:de) # => 'Deutschland'

It's probably just a wrapper around the hashes that you currently have, but it provides a cleaner API which also could easily documented using rdoc. This API is also easier to keep stable if the underlaying database changes for some reason and you can have deprecation warning etc.

If you like it, you could use the Null Object pattern, which would allow you to access for example the country information even if you don't have it:

ret = db.lookup('127.0.0.1')
ret.class # => MaxMindDB::Result
ret.found? # => false
ret.country.name # => nil

ret = db.lookup('74.125.225.224') # with the country DB
ret.country.name # => 'United States'
ret.city.name # => nil

Currently, if the result might be nil (or some information on it), you have to do something like this to get the English name of a city without the risk of hitting a NoMethodError on nil:

ret = db.lookup('127.0.0.1')
ret &&
  ret['city'] &&
  ret['city']['names'] &&
  ret['city']['names']['en']

No BTC/ETH address

Hello, there does not seem to be a BTC/ETH address for me to send a thank you donation.

NoMethodError (undefined method `Hash' for #<MaxMindDB::Result:0x007f96f559b468>):

Using your example I setup the gem, downloaded a paid geoIP2 city db, and tried using it:

db = MaxMindDB.new('/maxminddbGeoIP2-City.mmdb')
ret = db.lookup('74.125.225.224')

which results in
NoMethodError (undefined method `Hash' for #MaxMindDB::Result:0x007f96f559b468):

I even tried the lite db used in the example:
GeoLite2-City.mmdb

This is after calling db.lookup .

same error. Any possible fix for this?
Thanks.

Getting 'invalid file format' errors when a lookup for an ip does not find a value

Hey there,

I use your gem to do lookups for the city, postal code, country code and lat and long coordinates from a purchased version of the MaxMind GeoIP2-City database.

It seems as though when a lookup for a certain IP fails, the gem raises an "invalid file format" error, when it should simply return nil. Here's the line I'm talking about: https://github.com/yhirose/maxminddb/blob/master/lib/maxminddb.rb#L49

Any ideas here?

Thanks!

RubyGems: yanked old releases

Hi,

we'd a Gemfile constraint to maxmind 0.1.2 and it appearently broke on deployment because you yanked the old versions in rubygems.org.

Is there any reasion why you did that?

Thanks

Create a new release

The latest version 0.1.12 doesn't support the metadata method. May you please release a new version so that it has the latest features described in the readme ?

UTF-8 corrupted values for country.iso_code

Hi.

We use maxminddb (0.1.19) and we're seeing the occasional issue where the country.iso_code that is returned is corrupted.

maxminddb.lookup('220.255.100.153')
"SG #\xE8BdeHSingapurBe"
maxminddb.lookup('83.10.231.26')
"\xB8\xE3"

We can never reproduce the issue, but we get an exception notification when the garbled UTF-8 is returned and we try to insert an invalid country.iso_code.

I was wondering it could be an issue with #38?

My plan is to:

  1. Upgrade to v0.1.22
  2. Add a rescue to attempt to do another lookup if it detects invalid UTF-8 encoding

Note: we have an initializer to load one shared instance across the app.

Any other suggestions?

Thanks!
Owen

MaxMind-DB-Writer-perl create mmdb can't read

cat 1.rb
require 'maxminddb'
db = MaxMindDB.new('/tmp/MaxMind_DB_Writer_perl_create.mmdb')
ret = db.lookup('1.0.16.1')
print ret.found?
[zhangrui@RuiZhangdeMacBook-Air-2 ~ ]$ ruby 1.rb
false%

cat 2.rb
require 'maxminddb'
db = MaxMindDB.new('/tmp/GeoLite2-Country.mmdb')
ret = db.lookup('1.0.16.1')
print ret.found?
[zhangrui@RuiZhangdeMacBook-Air-2 ~ ]$ ruby 1.rb
true%

but python maxminxddb two mmdb files can read

import maxminddb
import json
reader = maxminddb.open_database('/tmp/MaxMind_DB_Writer_perl_create.mmdb')
result = reader.get('1.0.16.1')
reader.close()
print json.dumps(result, indent=2)

Read in batches

I have a bunch of IP addresses. It is too expensive to get geographical information for them all one by one. Is there a way to read them in a batch?

client = ::MaxMindDB.new(Rails.root.join('geolite.mmdb'))
client.lookup(*[192.158.1.38, 192.158.1.39]) # splat args
client.lookup([192.158.1.38, 192.158.1.39]) # maybe use and array directly

Finding incorrect geonames

Hi,

I am using some test IP addresses with this library and am seeing it return random geonames back.

I am using the MaxMind DB binary, gzipped found here:
http://dev.maxmind.com/geoip/geoip2/geolite2/

Environment:

  • Ruby 2.2.4
  • Rails 4.2.6

eg. IP address 199.241.208.206 (which should be a U.S. one) comes back with

"city" => {
    "geoname_id" => 3173435,
           "names" => {
               "de" => "Mailand",
               "en" => "Milan",
              "es" => "Milán",
             "fr" => "Milan",
             "ja" => "ミラノ",
            "pt-BR" => "Milão",
               "ru" => "Милан"
 }

This is my ruby code.

    db = MaxMindDB.new('./db/GeoLite2-City.mmdb')
    ret = db.lookup(ip_address)
    ap ret

Note: if I use the hive_geoip2 gem, it works fine.

Am I missing something?

Does not accept `IPAddr` object as input

Devise now defaults to store IP addresses as inet columns, which are automatically available as IPAddr instances in Rails. It would be very helpful if maxminddb accepted such instance as an input param.

Example:

MAXMIND_DB = MaxMindDB.new(GeoLite2City::DB_PATH)
MAXMIND_DB.lookup(IPAddr.new('153.19.48.1'))
=> IPAddr::AddressFamilyError: address family must be specified

This method should be updated:
https://github.com/yhirose/maxminddb/blob/master/lib/maxminddb.rb#L180

I would be able to provide a patch if you agree with the above reasoning.

time zone info

In first version of geoip there was available time zone information.
Is it provided in geoip2? And if yes, would it be possible to provide it in this gem?

Error during lookup with latest GeoLite2-City

Hello!

Recently I had to upgrade the download path of the DB and re-deploy my app which uses this gem, due to Maxmind licensing changes.

However, I am now getting errors doing lookups (both with regular and low-memory reader):

irb(main):016:0> GEOIP = MaxMindDB.new(Rails.root.join(ENV['maxmind2_city_db_path']), MaxMindDB::LOW_MEMORY_FILE_READER)
=> #<MaxMindDB::Client: DBPath:'/app/GeoLite2-City.mmdb'>
irb(main):017:0> GEOIP.metadata
=> {"binary_format_major_version"=>2, "binary_format_minor_version"=>0, "build_epoch"=>1580229618, "database_type"=>"GeoLite2-City", "description"=>{"en"=>"GeoLite2 City database"}, "ip_version"=>6, "languages"=>["de", "en", "es", "fr", "ja", "pt-BR", "ru", "zh-CN"], "node_count"=>3786198, "record_size"=>28}
irb(main):018:0> GEOIP.lookup '89.73.253.152'
Traceback (most recent call last):
        1: from (irb):18
EOFError (end of file reached)
irb(main):019:0> GEOIP = MaxMindDB.new(Rails.root.join(ENV['maxmind2_city_db_path']))
=> #<MaxMindDB::Client: DBPath:'/app/GeoLite2-City.mmdb'>
irb(main):020:0> GEOIP.lookup '89.73.253.152'
Traceback (most recent call last):
        1: from (irb):20
NoMethodError (undefined method `ord' for nil:NilClass)

The MMDB file looks fine. Any ideas on what could have gone wrong? Thanks for the help.

Btw, I am using the latest gem version of maxminddb (0.1.22).

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.