Code Monkey home page Code Monkey logo

geoip2-python's Introduction

MaxMind GeoIP2 Python API

Description

This package provides an API for the GeoIP2 and GeoLite2 web services and databases.

Installation

To install the geoip2 module, type:

$ pip install geoip2

If you are not able to use pip, you may also use easy_install from the source directory:

$ easy_install .

Database Reader Extension

If you wish to use the C extension for the database reader, you must first install the libmaxminddb C API. Please see the instructions distributed with it.

IP Geolocation Usage

IP geolocation is inherently imprecise. Locations are often near the center of the population. Any location provided by a GeoIP2 database or web service should not be used to identify a particular address or household.

Web Service Usage

To use this API, you first construct either a geoip2.webservice.Client or geoip2.webservice.AsyncClient, passing your MaxMind account_id and license_key to the constructor. To use the GeoLite2 web service instead of the GeoIP2 web service, set the optional host keyword argument to geolite.info. To use the Sandbox GeoIP2 web service instead of the production GeoIP2 web service, set the optional host keyword argument to sandbox.maxmind.com.

After doing this, you may call the method corresponding to request type (e.g., city or country), passing it the IP address you want to look up.

If the request succeeds, the method call will return a model class for the endpoint you called. This model in turn contains multiple record classes, each of which represents part of the data returned by the web service.

If the request fails, the client class throws an exception.

Sync Web Service Example

>>> import geoip2.webservice
>>>
>>> # This creates a Client object that can be reused across requests.
>>> # Replace "42" with your account ID and "license_key" with your license
>>> # key. Set the "host" keyword argument to "geolite.info" to use the
>>> # GeoLite2 web service instead of the GeoIP2 web service. Set the
>>> # "host" keyword argument to "sandbox.maxmind.com" to use the Sandbox
>>> # GeoIP2 web service instead of the production GeoIP2 web service.
>>> with geoip2.webservice.Client(42, 'license_key') as client:
>>>
>>>     # Replace "city" with the method corresponding to the web service
>>>     # that you are using, i.e., "country", "city", or "insights". Please
>>>     # note that Insights is not supported by the GeoLite2 web service.
>>>     response = client.city('203.0.113.0')
>>>
>>>     response.country.iso_code
'US'
>>>     response.country.name
'United States'
>>>     response.country.names['zh-CN']
u'美国'
>>>
>>>     response.subdivisions.most_specific.name
'Minnesota'
>>>     response.subdivisions.most_specific.iso_code
'MN'
>>>
>>>     response.city.name
'Minneapolis'
>>>
>>>     response.postal.code
'55455'
>>>
>>>     response.location.latitude
44.9733
>>>     response.location.longitude
-93.2323
>>>
>>>     response.traits.network
IPv4Network('203.0.113.0/32')

Async Web Service Example

>>> import asyncio
>>>
>>> import geoip2.webservice
>>>
>>> async def main():
>>>     # This creates an AsyncClient object that can be reused across
>>>     # requests on the running event loop. If you are using multiple event
>>>     # loops, you must ensure the object is not used on another loop.
>>>     #
>>>     # Replace "42" with your account ID and "license_key" with your license
>>>     # key. Set the "host" keyword argument to "geolite.info" to use the
>>>     # GeoLite2 web service instead of the GeoIP2 web service. Set the
>>>     # "host" keyword argument to "sandbox.maxmind.com" to use the Sandbox
>>>     # GeoIP2 web service instead of the production GeoIP2 web service.
>>>     async with geoip2.webservice.AsyncClient(42, 'license_key') as client:
>>>
>>>         # Replace "city" with the method corresponding to the web service
>>>         # that you are using, i.e., "country", "city", or "insights". Please
>>>         # note that Insights is not supported by the GeoLite2 web service.
>>>         response = await client.city('203.0.113.0')
>>>
>>>         response.country.iso_code
'US'
>>>         response.country.name
'United States'
>>>         response.country.names['zh-CN']
u'美国'
>>>
>>>         response.subdivisions.most_specific.name
'Minnesota'
>>>         response.subdivisions.most_specific.iso_code
'MN'
>>>
>>>         response.city.name
'Minneapolis'
>>>
>>>         response.postal.code
'55455'
>>>
>>>         response.location.latitude
44.9733
>>>         response.location.longitude
-93.2323
>>>
>>>         response.traits.network
IPv4Network('203.0.113.0/32')
>>>
>>> asyncio.run(main())

Web Service Client Exceptions

For details on the possible errors returned by the web service itself, see https://dev.maxmind.com/geoip/docs/web-services?lang=en for the GeoIP2 web service docs.

If the web service returns an explicit error document, this is thrown as a AddressNotFoundError, AuthenticationError, InvalidRequestError, or OutOfQueriesError as appropriate. These all subclass GeoIP2Error.

If some other sort of error occurs, this is thrown as an HTTPError. This is thrown when some sort of unanticipated error occurs, such as the web service returning a 500 or an invalid error document. If the web service returns any status code besides 200, 4xx, or 5xx, this also becomes an HTTPError.

Finally, if the web service returns a 200 but the body is invalid, the client throws a GeoIP2Error.

Database Usage

To use the database API, you first construct a geoip2.database.Reader using the path to the file as the first argument. After doing this, you may call the method corresponding to database type (e.g., city or country), passing it the IP address you want to look up.

If the lookup succeeds, the method call will return a model class for the database method you called. This model in turn contains multiple record classes, each of which represents part of the data for the record.

If the request fails, the reader class throws an exception.

Database Example

City Database

>>> import geoip2.database
>>>
>>> # This creates a Reader object. You should use the same object
>>> # across multiple requests as creation of it is expensive.
>>> with geoip2.database.Reader('/path/to/GeoLite2-City.mmdb') as reader:
>>>
>>>     # Replace "city" with the method corresponding to the database
>>>     # that you are using, e.g., "country".
>>>     response = reader.city('203.0.113.0')
>>>
>>>     response.country.iso_code
'US'
>>>     response.country.name
'United States'
>>>     response.country.names['zh-CN']
u'美国'
>>>
>>>     response.subdivisions.most_specific.name
'Minnesota'
>>>     response.subdivisions.most_specific.iso_code
'MN'
>>>
>>>     response.city.name
'Minneapolis'
>>>
>>>     response.postal.code
'55455'
>>>
>>>     response.location.latitude
44.9733
>>>     response.location.longitude
-93.2323
>>>
>>>     response.traits.network
IPv4Network('203.0.113.0/24')

Anonymous IP Database

>>> import geoip2.database
>>>
>>> # This creates a Reader object. You should use the same object
>>> # across multiple requests as creation of it is expensive.
>>> with geoip2.database.Reader('/path/to/GeoIP2-Anonymous-IP.mmdb') as reader:
>>>
>>>     response = reader.anonymous_ip('203.0.113.0')
>>>
>>>     response.is_anonymous
True
>>>     response.is_anonymous_vpn
False
>>>     response.is_hosting_provider
False
>>>     response.is_public_proxy
False
>>>     response.is_residential_proxy
False
>>>     response.is_tor_exit_node
True
>>>     response.ip_address
'203.0.113.0'
>>>     response.network
IPv4Network('203.0.113.0/24')

ASN Database

>>> import geoip2.database
>>>
>>> # This creates a Reader object. You should use the same object
>>> # across multiple requests as creation of it is expensive.
>>> with geoip2.database.Reader('/path/to/GeoLite2-ASN.mmdb') as reader:
>>>     response = reader.asn('203.0.113.0')
>>>     response.autonomous_system_number
1221
>>>     response.autonomous_system_organization
'Telstra Pty Ltd'
>>>     response.ip_address
'203.0.113.0'
>>>     response.network
IPv4Network('203.0.113.0/24')

Connection-Type Database

>>> import geoip2.database
>>>
>>> # This creates a Reader object. You should use the same object
>>> # across multiple requests as creation of it is expensive.
>>> with geoip2.database.Reader('/path/to/GeoIP2-Connection-Type.mmdb') as reader:
>>>     response = reader.connection_type('203.0.113.0')
>>>     response.connection_type
'Corporate'
>>>     response.ip_address
'203.0.113.0'
>>>     response.network
IPv4Network('203.0.113.0/24')

Domain Database

>>> import geoip2.database
>>>
>>> # This creates a Reader object. You should use the same object
>>> # across multiple requests as creation of it is expensive.
>>> with geoip2.database.Reader('/path/to/GeoIP2-Domain.mmdb') as reader:
>>>     response = reader.domain('203.0.113.0')
>>>     response.domain
'umn.edu'
>>>     response.ip_address
'203.0.113.0'

Enterprise Database

>>> import geoip2.database
>>>
>>> # This creates a Reader object. You should use the same object
>>> # across multiple requests as creation of it is expensive.
>>> with geoip2.database.Reader('/path/to/GeoIP2-Enterprise.mmdb') as reader:
>>>
>>>     # Use the .enterprise method to do a lookup in the Enterprise database
>>>     response = reader.enterprise('203.0.113.0')
>>>
>>>     response.country.confidence
99
>>>     response.country.iso_code
'US'
>>>     response.country.name
'United States'
>>>     response.country.names['zh-CN']
u'美国'
>>>
>>>     response.subdivisions.most_specific.name
'Minnesota'
>>>     response.subdivisions.most_specific.iso_code
'MN'
>>>     response.subdivisions.most_specific.confidence
77
>>>
>>>     response.city.name
'Minneapolis'
>>>     response.country.confidence
11
>>>
>>>     response.postal.code
'55455'
>>>
>>>     response.location.accuracy_radius
50
>>>     response.location.latitude
44.9733
>>>     response.location.longitude
-93.2323
>>>
>>>     response.traits.network
IPv4Network('203.0.113.0/24')

ISP Database

>>> import geoip2.database
>>>
>>> # This creates a Reader object. You should use the same object
>>> # across multiple requests as creation of it is expensive.
>>> with geoip2.database.Reader('/path/to/GeoIP2-ISP.mmdb') as reader:
>>>     response = reader.isp('203.0.113.0')
>>>     response.autonomous_system_number
1221
>>>     response.autonomous_system_organization
'Telstra Pty Ltd'
>>>     response.isp
'Telstra Internet'
>>>     response.organization
'Telstra Internet'
>>>     response.ip_address
'203.0.113.0'
>>>     response.network
IPv4Network('203.0.113.0/24')

Database Reader Exceptions

If the database file does not exist or is not readable, the constructor will raise a FileNotFoundError or a PermissionError. If the IP address passed to a method is invalid, a ValueError will be raised. If the file is invalid or there is a bug in the reader, a maxminddb.InvalidDatabaseError will be raised with a description of the problem. If an IP address is not in the database, a AddressNotFoundError will be raised.

AddressNotFoundError references the largest subnet where no address would be found. This can be used to efficiently enumerate entire subnets:

import geoip2.database
import geoip2.errors
import ipaddress

# This creates a Reader object. You should use the same object
# across multiple requests as creation of it is expensive.
with geoip2.database.Reader('/path/to/GeoLite2-ASN.mmdb') as reader:
    network = ipaddress.ip_network("192.128.0.0/15")

    ip_address = network[0]
    while ip_address in network:
        try:
            response = reader.asn(ip_address)
            response_network = response.network
        except geoip2.errors.AddressNotFoundError as e:
            response = None
            response_network = e.network
        print(f"{response_network}: {response!r}")
        ip_address = response_network[-1] + 1  # move to next subnet

Values to use for Database or Dictionary Keys

We strongly discourage you from using a value from any ``names`` property as a key in a database or dictionaries.

These names may change between releases. Instead we recommend using one of the following:

  • geoip2.records.City - city.geoname_id
  • geoip2.records.Continent - continent.code or continent.geoname_id
  • geoip2.records.Country and geoip2.records.RepresentedCountry - country.iso_code or country.geoname_id
  • geoip2.records.subdivision - subdivision.iso_code or subdivision.geoname_id

What data is returned?

While many of the models contain the same basic records, the attributes which can be populated vary between web service endpoints or databases. In addition, while a model may offer a particular piece of data, MaxMind does not always have every piece of data for any given IP address.

Because of these factors, it is possible for any request to return a record where some or all of the attributes are unpopulated.

The only piece of data which is always returned is the ip_address attribute in the geoip2.records.Traits record.

Integration with GeoNames

GeoNames offers web services and downloadable databases with data on geographical features around the world, including populated places. They offer both free and paid premium data. Each feature is uniquely identified by a geoname_id, which is an integer.

Many of the records returned by the GeoIP web services and databases include a geoname_id field. This is the ID of a geographical feature (city, region, country, etc.) in the GeoNames database.

Some of the data that MaxMind provides is also sourced from GeoNames. We source things like place names, ISO codes, and other similar data from the GeoNames premium data set.

Reporting Data Problems

If the problem you find is that an IP address is incorrectly mapped, please submit your correction to MaxMind.

If you find some other sort of mistake, like an incorrect spelling, please check the GeoNames site first. Once you've searched for a place and found it on the GeoNames map view, there are a number of links you can use to correct data ("move", "edit", "alternate names", etc.). Once the correction is part of the GeoNames data set, it will be automatically incorporated into future MaxMind releases.

If you are a paying MaxMind customer and you're not sure where to submit a correction, please contact MaxMind support for help.

Requirements

Python 3.8 or greater is required. Older versions are not supported.

The Requests HTTP library is also required. See <https://pypi.org/project/requests/> for details.

Versioning

The GeoIP2 Python API uses Semantic Versioning.

Support

Please report all issues with this code using the GitHub issue tracker

If you are having an issue with a MaxMind service that is not specific to the client API, please contact MaxMind support for assistance.

geoip2-python's People

Contributors

2shortplanks avatar andyjack avatar arturro avatar autarch avatar borisz avatar browniebroke avatar dependabot-preview[bot] avatar dependabot[bot] avatar deronnax avatar dlieou avatar faktas2 avatar hongquan avatar horgh avatar illes avatar jnozsc avatar kecaps avatar kevcenteno avatar marselester avatar nchelluri avatar nkinkade avatar oalders avatar onovy avatar oschwald avatar patrickcronin avatar pcmanticore avatar rafl avatar shadromani avatar ugexe avatar wesrice avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

geoip2-python's Issues

ReadTimeout from https://geoip.maxmind.com/geoip/v2.1/city/{ip}

I've noticed a couple of instances of unhanded ReadTimout exceptions from geoip2.webservice.Client._response_for (line 157 in version 3.0.0).

Here's the sentry issue: https://sentry.io/share/issue/dea40ba4e7044dc5b70273b5f15b62e9/

I don't know this library and its public API well enough to know whether there's a nicer way to deal with these exceptions.

If you think it's the library user's responsibility to deal with these exceptions (as has happened in the codebase I'm working on) , feel free to close this.

GeoLite2-City.mmdb no such file

How can I fix the following error which is related to GeoLite2-City.mmdb ?


Traceback (most recent call last):
  File "distance.py", line 15, in <module>
    reader=geoip2.database.Reader('GeoLite2-City.mmdb')
  File "/usr/lib/python2.7/site-packages/geoip2/database.py", line 86, in __init__
    self._db_reader = maxminddb.open_database(fileish, mode)
  File "/usr/lib/python2.7/site-packages/maxminddb/__init__.py", line 41, in open_database
    return maxminddb.reader.Reader(database, mode)
  File "/usr/lib/python2.7/site-packages/maxminddb/reader.py", line 52, in __init__
    with open(database, 'rb') as db_file:
IOError: [Errno 2] No such file or directory: 'GeoLite2-City.mmdb'

I see on the web that I have to download GeoLite2-City.mmdb. However, I can not find the download procedure? Any help?

pip install of geoip2 fails

I am unable to install geoip2 with the following error:

Installing collected packages: multidict, async-timeout, typing-extensions, yarl, idna-ssl, aiohttp, geoip2
Running setup.py install for multidict ... error
Complete output from command /usr/bin/python3 -u -c "import setuptools, tokenize;file='/tmp/pip-build-xd1kaob1/multidict/setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, file, 'exec'))" install --record /tmp/pip-wsso88ig-record/install-record.txt --single-version-externally-managed --compile:
*********************
* Accelerated build *
*********************
running install
running build
running build_py
creating build
creating build/lib.linux-x86_64-3.6
creating build/lib.linux-x86_64-3.6/multidict
copying multidict/init.py -> build/lib.linux-x86_64-3.6/multidict
copying multidict/_abc.py -> build/lib.linux-x86_64-3.6/multidict
copying multidict/_compat.py -> build/lib.linux-x86_64-3.6/multidict
copying multidict/_multidict_base.py -> build/lib.linux-x86_64-3.6/multidict
copying multidict/_multidict_py.py -> build/lib.linux-x86_64-3.6/multidict
running egg_info
writing multidict.egg-info/PKG-INFO
writing dependency_links to multidict.egg-info/dependency_links.txt
writing top-level names to multidict.egg-info/top_level.txt
reading manifest file 'multidict.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no previously-included files matching '.pyc' found anywhere in distribution
warning: no previously-included files found matching 'multidict/_multidict.html'
warning: no previously-included files found matching 'multidict/
.so'
warning: no previously-included files found matching 'multidict/.pyd'
warning: no previously-included files found matching 'multidict/
.pyd'
no previously-included directories found matching 'docs/_build'
writing manifest file 'multidict.egg-info/SOURCES.txt'
copying multidict/init.pyi -> build/lib.linux-x86_64-3.6/multidict
copying multidict/_multidict.c -> build/lib.linux-x86_64-3.6/multidict
copying multidict/py.typed -> build/lib.linux-x86_64-3.6/multidict
creating build/lib.linux-x86_64-3.6/multidict/_multilib
copying multidict/_multilib/defs.h -> build/lib.linux-x86_64-3.6/multidict/_multilib
copying multidict/_multilib/dict.h -> build/lib.linux-x86_64-3.6/multidict/_multilib
copying multidict/_multilib/istr.h -> build/lib.linux-x86_64-3.6/multidict/_multilib
copying multidict/_multilib/iter.h -> build/lib.linux-x86_64-3.6/multidict/_multilib
copying multidict/_multilib/pair_list.h -> build/lib.linux-x86_64-3.6/multidict/_multilib
copying multidict/_multilib/views.h -> build/lib.linux-x86_64-3.6/multidict/_multilib
running build_ext
building 'multidict._multidict' extension
creating build/temp.linux-x86_64-3.6
creating build/temp.linux-x86_64-3.6/multidict
gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/usr/include/python3.6m -c multidict/_multidict.c -o build/temp.linux-x86_64-3.6/multidict/_multidict.o -O2 -std=c99 -Wall -Wsign-compare -Wconversion -fno-strict-aliasing -pedantic
multidict/_multidict.c:1:20: fatal error: Python.h: No such file or directory
#include "Python.h"
^
compilation terminated.
error: command 'gcc' failed with exit status 1

----------------------------------------

Command "/usr/bin/python3 -u -c "import setuptools, tokenize;file='/tmp/pip-build-xd1kaob1/multidict/setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, file, 'exec'))" install --record /tmp/pip-wsso88ig-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-build-xd1kaob1/multidict/

Context manager support

It would be more Pythonic if you guys support Python's context manager for geoip2.database.Reader, so that we can call:

with geoip2.database.Reader('GeoLite2-City.mmdb') as reader:
    r = reader.city(ip_addr)
    data = {
        'country_code': r.country.iso_code,
        'country_name': r.country.name
    }

and reader.close() is called automatically when exiting with.

Installation of only one component of the package

I use only the database functionality of this package, and do all web management myself using aiohttp. It would be great if this package didn't pull requests into my dependency graph as my application is entirely async, (and also didn't force my aiohttp version to be less than 4 in case that ever gets updated).

I think you can use the extras_require argument to setup() (with setuptools) to add optional dependencies. Maybe something like:

extras_require={
    "all": ["requests>=2.24.0,<3.0.0", "aiohttp>=3.6.2,<4.0.0"],
    "requests": "requests>=2.24.0,<3.0.0",
    "aiohttp": "aiohttp>=3.6.2,<4.0.0",
}

For reference: https://setuptools.readthedocs.io/en/latest/userguide/dependency_management.html#id7
And example: https://github.com/palantir/python-language-server/blob/0fa74bae6fbb331498dbc39b6257d74357edea2f/setup.py#L51

Additionally, I see that urllib3 is listed in the requirements, but I don't see it being imported anywhere. Maybe that can be removed?

Bug in returned object

I think there is a bug in the object returned by this:
omni = client.omni('24.24.24.24')

If I want to access the longitude, the only variable that exists is
omni.location.longitudemetro_code, while omni.location.latitude works fine.

IP with country None

Hi,
I found an IP that returns country None.
This is the IP - "66.249.93.24"

g = GeoIP2()
g.country('66.249.93.24')
returns {'country_code': None, 'country_name': None}

Im using geoip2==3.0.0

I have checked the IP in other sources,
https://whatismyipaddress.com/ip/66.249.93.24 - "Country" field is also empty
https://www.iplocation.net/ - Country field returns United States
I'm not sure what it should return.

It seems that the IP belongs to Google. From what I understand it's used for Google webmail in Europe.

Should I report this issue here? https://support.maxmind.com/geoip-data-correction-request/correct-a-geoip-location/
In the form its mandatory to select a country, but I'm not sure which country it should return.

how to get queries_remaining without doing an IP lookup

I'm currently doing this:

from geoip2 import webservice
client = webservice.Client(12345, '************')
city = client.city()
city.maxmind.queries_remaining

But the problem is every time I issue that query, my number of queries remaining goes down by 1, even though I don't actually want to do an IP lookup - I just want to see my account status.

Is there any way to get the queries_remaining without doing an IP lookup?

thanks

Pypi rename?

Hi folks,

I'm packaging it into Fedora for preparing GeopIP2 landing in Fedora.

However I think this one on pypi has a not good name "geoip2", will you have plan to change it to "GeoIP2" like GeoIP on pypi?

I hope before I build it for Fedora you can change the name to match the name it really is, and this will save our time of renaming packages and blahblahblah in the future(really painful)...

Thank you.

IP range not found by ASN reader

I am using the following python code:

import geoip2
import geoip2.database

rasn = geoip2.database.Reader('GeoLite2-ASN_20180206/GeoLite2-ASN.mmdb')
rloc = geoip2.database.Reader('GeoLite2-City_20180206/GeoLite2-City.mmdb')
print(rloc.city("103.63.30.101"))
print(rasn.asn("103.63.30.101"))

The asn query fails with an address not found error. Nonetheless, the prefix exists in the equivalent CSV database and I can successfully retrieve the IP from https://www.maxmind.com/en/geoip-demo.

Am I using the API incorrectly?

problem installing on MacOS

Hi,

When i run the pip install geoip[DB] command, I get the following error message:

building 'maxminddb' extension

creating build

creating build/temp.macosx-10.9-intel-2.7

cc -fno-strict-aliasing -fno-common -dynamic -arch x86_64 -arch i386 -g -Os -pipe -fno-common -fno-strict-aliasing -fwrapv -mno-fused-madd -DENABLE_DTRACE -DMACOSX -DNDEBUG -Wall -Wstrict-prototypes -Wshorten-64-to-32 -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch x86_64 -arch i386 -pipe -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c maxminddb.c -o build/temp.macosx-10.9-intel-2.7/maxminddb.o -Wall -Wextra

clang: warning: argument unused during compilation: '-mno-fused-madd'

maxminddb.c:2:10: fatal error: 'maxminddb.h' file not found

include <maxminddb.h>

     ^

1 error generated.

error: command 'cc' failed with exit status 1

What am I missing here? I "think" i have installed libmaxminddb, but I am not even sure of that. Is there a way to verify the correct installation of libmaxminddb? Or is my problem somewhere else?

Also, is there any specific directory, where I should be running the "pip install" command from? Because, I know the maxminddb.h file actually exists somewhere on my hard disk.

Thanks in advance for your replies

TypeError: ord() expected string of length 1, but int found

Hey, not sure exactly what is happening here. Thank you in advance.

MAC 10.12.2

import geoip2
import geoip2.database
reader = geoip2.database.Reader('GeoLite2-City.mmdb')
response = reader.city('128.101.101.101')
TypeError Traceback (most recent call last)
in ()
2 import geoip2.database
3 reader = geoip2.database.Reader('GeoLite2-City.mmdb')
----> 4 response = reader.city('128.101.101.101')

/Users/username/.pyenv/versions/venv27/lib/python2.7/site-packages/geoip2/database.pyc in city(self, ip_address)
108
109 """
--> 110 return self._model_for(geoip2.models.City, 'City', ip_address)
111
112 def anonymous_ip(self, ip_address):

/Users/username/.pyenv/versions/venv27/lib/python2.7/site-packages/geoip2/database.pyc in _model_for(self, model_class, types, ip_address)
178
179 def _model_for(self, model_class, types, ip_address):
--> 180 record = self._get(types, ip_address)
181 record.setdefault('traits', {})['ip_address'] = ip_address
182 return model_class(record, locales=self._locales)

/Users/username/.pyenv/versions/venv27/lib/python2.7/site-packages/geoip2/database.pyc in _get(self, database_type, ip_address)
171 "%s database" %
172 (caller, self.metadata().database_type))
--> 173 record = self._db_reader.get(ip_address)
174 if record is None:
175 raise geoip2.errors.AddressNotFoundError(

/Users/username/.pyenv/versions/venv27/lib/python2.7/site-packages/maxminddb/reader.pyc in get(self, ip_address)
102 'an IPv6 address in an IPv4-only database.'.format(
103 ip_address))
--> 104 pointer = self._find_address_in_tree(address)
105
106 return self._resolve_data_pointer(pointer) if pointer else None

/Users/username/.pyenv/versions/venv27/lib/python2.7/site-packages/maxminddb/reader.pyc in _find_address_in_tree(self, ip_address)
115 if node >= self._metadata.node_count:
116 break
--> 117 bit = 1 & (int_from_byte(packed[i >> 3]) >> 7 - (i % 8))
118 node = self._read_node(node, bit)
119 if node == self._metadata.node_count:

TypeError: ord() expected string of length 1, but int found

geoip2.database imports maxminddb.MODE_FD which no longer exists

Python 3.6.6 (default, Aug  7 2018, 11:27:19)
[GCC 4.2.1 Compatible FreeBSD Clang 4.0.0 (tags/RELEASE_400/final 297347)] on freebsd11
Type "help", "copyright", "credits" or "license" for more information.
>>> import geoip2
>>> import geoip2.database
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.6/site-packages/geoip2/database.py", line 11, in <module>
    from maxminddb import (MODE_AUTO, MODE_MMAP, MODE_MMAP_EXT, MODE_FILE,
ImportError: cannot import name 'MODE_FD'
>>> import maxminddb
>>> maxminddb.
maxminddb.InvalidDatabaseError(  maxminddb.MODE_MMAP_EXT          maxminddb.errors                 maxminddb.os
maxminddb.MODE_AUTO              maxminddb.Reader(                maxminddb.extension              maxminddb.reader
maxminddb.MODE_FILE              maxminddb.compat                 maxminddb.file
maxminddb.MODE_MEMORY            maxminddb.const                  maxminddb.maxminddb
maxminddb.MODE_MMAP              maxminddb.decoder                maxminddb.open_datab

what's wrong with response.country.names?

Imgur
why it‘s not print United States?

python 2.7.9
cffi (0.8.6)
cryptography (0.7.1)
dnslib (0.9.3)
geoip2 (2.1.0)
gevent (1.0.1)
gfwlist2pac (1.1.2)
greenlet (0.4.5)
maxminddb (1.1.1)
pip (1.5.6)
pycparser (2.10)
pycrypto (2.6.1)
pygeoip (0.3.2)
pyOpenSSL (0.13.1)
pywin32 (219)
setuptools (11.3.1)
six (1.9.0)

No module named database

I create a python script with geoip2, but system report error saying "No module named database"

Traceback (most recent call last):
File "geoip2.py", line 2, in
import geoip2
File "/home/ubuntu/google/geoip2.py", line 3, in
import geoip2.database
ImportError: No module named database

My code:

!/usr/bin/env python

import geoip2.database
reader = geoip2.database.Reader('/usr/share/GeoIP2/GeoLite2-City.mmdb')
response = reader.city('78.31.107.133')
print response.country.iso_code

generate the IP addresses from the ASN database

Hello and thanks for this wonderful package.

I know I can use this:


with geoip2.database.Reader('/path/to/GeoLite2-ASN.mmdb') as reader:
 response = reader.asn('1.128.0.0')
 response.autonomous_system_organization

to get the autonomous_system_organization for a given IP.

Something I was not able to figure out is whether this package can generate all the possible IPs for each autonomous_system_organization in the maxmind ASN database. For instance, can we get the IPs by iterating over the IP networks in CIDR format that are in the data? Is there a method for that?

Thanks!

Inquiry: Thread safety and use in prefork uwsgi process

I know that the Java library for GeoIP2 is thread safe (the documentation specifically recommends creating a single DatabaseReader and sharing it among multiple threads). That's perfect for Java web applications.

However, the documentation for this Python library doesn't say anything about threading. What I specifically need to know is, if I create a Reader in the uwsgi parent process, and then workers prefork to handle incoming HTTP(S) requests, will those workers be able to use the Reader simultaneously? Or does each worker need its own Reader?

Thanks!

How to get JSON?

I need to get full JSON that contain all data for specified IP. How to do it?

Multiple IP addresses lookup?

Hi! Thank you for your wonderful package!

Is there an option to query the database for multiple IP addresses at once? Would it be more performant query in case of multiple IP lookup?

AttributeError: 'Subdivisions' object has no attribute 'names'

Hi

When i get the name of Subdivisions get the following issue:

AttributeError: 'Subdivisions' object has no attribute 'names'

method

ipreader = geoip2.database.Reader('GeoLite2-City.mmdb') ipdetails = ipreader.city(ips) ipdetails.subdivisions.names.get('zh-CN','')

How can i get the name of Subdivisions?

Thanks

Input Country, Output Subnet Blocks

Is there an option to get the subnet blocks associated with a country?

I'm really hoping I don't need to parse the CSV to get this information...

pathlib.Path support

I try to construct Reader with pathlib.Path object:

from geoip2.database import Reader
from pathlib import Path

# db_path = '/tmp/GeoLite2-Country.mmdb'
db_path = Path('/tmp') / 'GeoLite2-Country.mmdb'

reader = Reader(db_path, locales=['en'])
print(reader.country('8.8.8.8').country.name)

And I get the error:

Traceback (most recent call last):
  File "/home/xxx/test/1.py", line 6, in <module>
    reader = Reader(db_path, locales=['en'])
  File "/usr/lib/python3.9/site-packages/geoip2/database.py", line 106, in __init__
    self._db_reader = maxminddb.open_database(fileish, mode)
  File "/usr/lib64/python3.9/site-packages/maxminddb/__init__.py", line 48, in open_database
    return maxminddb.extension.Reader(database)
TypeError: argument 1 must be str, not PosixPath

The solution to the problem is to use str(db_path):
reader = Reader(str(db_path), locales=['en'])

However, it would be great if the GeoIP2-python package would support the pathlib.Path objects.

Pre-4.0 versions no longer compatible with Python 2

After release of version 4.0, earlier versions of geoip2 tries to install minmaxddb 2.0.0, which is not compatible with python 2.
I think you need to pin minmaxddb to <2.0 in pre-4.0 versions of geoip2.

ASN get network

As far as I can see it is only possible to get the IP back in the model for most query types.
I would be interested to get the whole range back in which the IP was found.

As example if I would query the ip address "1.0.0.12" to the ASN database I would expect an answer with following fields:

  • autonomous_system_number -> 13335
  • autonomous_system_organization -> Cloudflare, Inc.
  • network -> 1.0.0.0/24

Is something like this archivable?

IPv6 with Scope-ID Support

When running GeoIP().city() on an IPv6 address that includes scope information it fails to validate the address returned a serious of errors from ipaddress.py. Rather than return the validation error, it would make more sense if GeoIP dropped the scope id for address determination.

Based on the IPv6 specifications (https://en.wikipedia.org/wiki/IPv6_address#Scoped_literal_IPv6_addresses, IPv6 addresses can include scope-id information and still be valid. The scope id is expressed as a % and a descriptor at the end of the IPv6 address.

This was tested on Python 3.7 running GeoIP2.

Passing IP to Reader.Country causing AddressNotFoundError exception

I am grabbing IPs from my lighttpd log to put them in a Counter object so I can for one, count all occurrences of the IP, and two - to pass that IP to the reader.country lookup so I get the location of the address. When I do this, it throws a AddressNotFoundError exception. To try and found out the problem, I print the type of the value and it tells me it is infact a string. I even use the strip() function to see if there are leading and trailing spaces but it still throws this exception.

If I pass a literal IP to the function which is one of the IPs in this list (ex: '142.93.25.225') the function works fine (See last two lines in the code below). Any ideas?

See below:

#!/usr/bin/python
  
# Count all IPs that are hitting webserver 

import re
from collections import Counter
import geoip2.database

logfile = "/var/log/lighttpd/lighttpd.log"

# regex to Grab IPs
myregex = r'(^\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})' #.*"(.*)"'
ipcount = Counter()

with open(logfile) as f:
    for line in f:
        my_iplist = re.findall(myregex,line)
        # Add the ip to the Counter obj
        ipcount.update(my_iplist)
    # Loop through the Counter obj and output all the stuff
    reader = geoip2.database.Reader("/home/alarm/geolite2/GeoLite2-Country_20181023/GeoLite2-Country.mmdb")
    for k, v in ipcount.most_common():
        try:
            geo = reader.country(k.strip())
            #geo.insights('128.101.101.101')
            # Forget the IPs that hit webserver once..
            if (v > 1): print("Count: " + str(v) + " IP Address: " + str(k) + " Country: " + geo.country.name + " Loc: " + geo.location + " TZ: " + geo.timezone)
        except:
            print ("exception..." + k)
            pass
    geo1 = reader.country('142.93.25.225')
    print("Count: " + str(v) + " IP Address: " + geo1.traits.ip_address + " Continent: " + geo1.continent.code + " Country: " + str(geo1.country.name) + " Loc: ")

is MaxMind.GeoIP2 thread-safe?

Dear all,

I was wondering whether python API for MaxMind GeoIP2 is thread safe or not.
In particular, is it possible to share a reader object among several threads?

best regards,
Valerio

MaxMind Database returning questionable data that doesn't match online demo

We're using the GeoIP2 City database with the GeoIP2-python library. We're making about 1,000 lookups per second against the database, spread out over thirty servers. Each server has its own copy of the database, so that's about 34 requests per second per GeoIP2 City database file. We're using prefork UWSGI processes, not actual multithreading. We open a geoip2.database.Reader in the parent process (this might be the problem) and use it in worker UWSGI processes.

Last night for about 30 minutes we got a surge of questionable data from our MaxMind database copies that doesn't make any sense. After that 30 minutes the problem went away and hasn't returned yet. I demonstrate this questionable data below in the following format: IP address / location according to MaxMind online demo / location according to our MaxMind database lookup on our server:

107.92.59.253 / "Houston, Texas, US" / "#, Aland Islands"
68.8.132.240 / "San Diego, California, US" / "<�, New Zealand"
66.87.83.113 / "Philadelphia, Pennsylvania, US" / "E, France"
66.115.83.131 / "Glasgow, Kentucky, US" / "x�, Pitcairn Islands"
23.31.73.65 / "Boulder, Colorado, US" / "���, Malaysia"
72.35.146.175 / "Bend, Oregon, US" / "(È#ÜK�®80, Cook Islands"
151.252.29.139 / "Sweden, Europe" / "ð�, Nauru"

What could be causing this? Is this a bug in the MaxMind database Python library? Or is it how we're using it?

Database Reader exceptions

I read the documentation here: https://github.com/maxmind/GeoIP2-python#database-reader-exceptions

If the database file does not exist or is not readable, the constructor will raise a FileNotFoundError.

But if the database is not readable (access denied, disk damaged, etc.), other exceptions will be thrown. It would be more correct to write like this:

If the database file does not exist or is not readable, the constructor will raise a FileNotFoundError.

Or even more:

If the database file does not exist or is not readable, the constructor will raise a FileNotFoundError.

And I think an example like this could be given here:

from geoip2.database import Reader
from geoip2.errors import AddressNotFoundError
from maxminddb.errors import InvalidDatabaseError

def get_iso_code(db_path, ip_address) -> str:
    reader = None

    try:
        reader = Reader(db_path, locales=['en'])
    except InvalidDatabaseError as err:
        print(f'GeoIP database {db_path} is invalid or there is an internal bug in the geoip2 reader: {err}')
    except FileNotFoundError:
        print(f'GeoIP database {db_path} not found')
    except PermissionError:
        print(f'You are not allowed to read GeoIP database {db_path}')
    except OSError:
        print(f'GeoIP database {db_path} is not readable')

    if reader is None:
        return ''

    try:
        response = reader.country(ip_address)
        return response.country.iso_code or ''
    except AddressNotFoundError:
        print(f'IP address {ip_address} is not in the database')
    except InvalidDatabaseError as err:
        print(f'GeoIP database {db_path} is invalid or there is an internal bug in the geoip2 reader: {err}')        
    except ValueError:
        print(f'IP address {ip_address} is invalid')

    return ''

print(get_iso_code('/path/to/GeoLite2-Country.mmdb', '8.8.8.8'))
# US
print(get_iso_code('/path/to/corrupted.mmdb', '8.8.8.8'))
# GeoIP database /path/to/corrupted.mmdb is invalid or there is an internal bug in the geoip2 reader: 
# Error looking up 8.8.8.8. The MaxMind DB file's search tree is corrupt
print(get_iso_code('/path/to/not_found', '8.8.8.8'))
# GeoIP database /path/to/not_found not found
print(get_iso_code('/path/to/GeoLite2-Country.mmdb', '127.0.0.1'))
# IP address 127.0.0.1 is not in the database
print(get_iso_code('/path/to/GeoLite2-Country.mmdb', '999.0.0.1'))
# IP address 999.0.0.1 is invalid

ERROR: Could not find a version that satisfies the requirement pyasn (from versions: 1.5.0b4, 1.5.0b6, 1.5.0b7, 1.6.0b1, 1.6.1)

Hi, I am trying to install GeoIP2 and I am getting the following error:

pip install maxminddb

WARNING: Discarding https://files.pythonhosted.org/packages/39/5b/08fd0790c2f97cc9148aa611e8df28cc11fe157e279740c992597a983ec3/pyasn-1.6.      -1_e45wy4'1.tar.gz#sha256=e942b54916363ec7b8b70eaeaecd0eb4e41ef1bcf4a2397b29a6d715fccdf925 (from https://pypi.org/simple/pyasn/). Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
ERROR: Could not find a version that satisfies the requirement pyasn (from versions: 1.5.0b4, 1.5.0b6, 1.5.0b7, 1.6.0b1, 1.6.1)          1.tar.
ERROR: No matching distribution found for pyasn       

As far as I see pyasn is compatible with Python 3.9, I am not clear where the problems come from.

Context:
Windows 10
Python 3.9.7

What am I doing wrong?

opening DB and reloading automatically when .mmdb updated

Hi, not sure if that request should be considered an issue, or just some enhancement in the documentation instead. For "legacy" maxmind DB, there was a way to specify some "options", like GeoIP.GEOIP_CHECK_CACHE to force verifying that if underlying DB was changed, it would reopen it automatically , instead of having a relaunch the .py script that has the pointer to the DB.

I don't see option being described in doc, so was wondering what would be the option. help(geoip2.database) seems to have some params, like :
MODE_AUTO = 0
MODE_FD = 16
MODE_FILE = 4
MODE_MEMORY = 8
MODE_MMAP = 2
MODE_MMAP_EXT = 1

But I don't know which one would do what I'd need : some py process running in background, and detecting if .mmdb file was changed and would reload it (like GeoIP.GEOIP_CHECK_CACHE was doing for legacy geoip)

Painfully slow, Stuck.

Maxmind GeoIP2 Database is painfully slow. Stuck for 15 minutes then it finally gets the information. Is there a way around this?

Nonexistent database raises FileNotFoundError

Can you please take a look at this:

Reproducer

  1. make sure that nonexistent.mmdb does not exist
  2. call geoip2.database.Reader('nonexistent.mmdb')

Actual result

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/tmp/venv/lib/python3.5/site-packages/geoip2/database.py", line 82, in __init__
    self._db_reader = maxminddb.open_database(filename, mode)
  File "/tmp/venv/lib/python3.5/site-packages/maxminddb/__init__.py", line 32, in open_database
    return maxminddb.extension.Reader(database)
FileNotFoundError: No such file or directory: 'nonexistent.mmdb'

Expected result

A ValueError or a change in the documentation.

Versions

geoip2 2.4.2
maxminddb 1.2.3
libmaxminddb-dev 1.0.4-2.1
libmaxminddb0 1.0.4-2.1
Python 3.5.2
Ubuntu 16.04.1

Async support

Hi!

Would be awesome to have asyncio support 😄

Cheers!

No module named 'geoip2.database'

I wanted to try out this library, followed the install instructions as follows:

pip install geoip2
-> successfully installed

python
>>> import geoip2.webservice
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/root/geoip2.py", line 1, in <module>
    import geoip2.database
ModuleNotFoundError: No module named 'geoip2.database'; 'geoip2' is not a package

Using python 3.6.5.
I tried pip3, pip3.6, python, python3, python3.6, but neither combo worked. I can successfully import other modules like urllib3, requests, etc.

root@545ec389e230:/usr/local/lib/python3.6/site-packages/geoip2# ls -lahi
total 76K
 48930118 drwxr-sr-x. 3 root staff  170 Jun 14 14:37 .
 41536224 drwxr-sr-x. 1 root staff 4.0K Jun 14 14:37 ..
101268987 -rw-r--r--. 1 root staff  199 Jun 14 14:37 __init__.py
 48930127 drwxr-sr-x. 2 root staff  247 Jun 14 14:37 __pycache__
101268988 -rw-r--r--. 1 root staff  458 Jun 14 14:37 compat.py
101268989 -rw-r--r--. 1 root staff 7.5K Jun 14 14:37 database.py
101268990 -rw-r--r--. 1 root staff 1.2K Jun 14 14:37 errors.py
101268991 -rw-r--r--. 1 root staff  410 Jun 14 14:37 mixins.py
101269462 -rw-r--r--. 1 root staff  14K Jun 14 14:37 models.py
101271072 -rw-r--r--. 1 root staff  18K Jun 14 14:37 records.py
101271073 -rw-r--r--. 1 root staff 9.0K Jun 14 14:37 webservice.py

Add geoip2 to conda or conda-forge

It will be very helpful to be able to install geoip2 through the standard conda or conda-forge, just like the legacy geoip. Currently, geoip2 is only available through activisiongamescience channel, but conda-forge is a much more popular one.

Get county from city database

How can we get county from the city database reader? We can get country, city, state and other entities but not the county.
Example:

ip = '74.221.14.63'
result = reader.city(ip)
state = result.subdivisions.most_specific.name
city = result.city.name
print(state) # Tennessee
print(city) # Millington

The county would be Shelby for the above entities
County finder: http://www.statsamerica.org/CityCountyFinder

Installation Fails (Ubuntu 13.10)

Yo guys,

I'm trying to get this library installed on my laptop so I can play around with the maxmind api services and database services.

I've installed the 'geoip2[db]' package from PyPI, installed the libmaxmind C libraries manually, but when I try to import the geoip2.database package I get the following traceback:

Python 2.7.5+ (default, Sep 19 2013, 13:48:49) 
[GCC 4.8.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import geoip2.database
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/rdegges/.virtualenvs/test/local/lib/python2.7/site-packages/geoip2/database.py", line 8, in <module>
    import maxminddb
ImportError: libmaxminddb.so.0: cannot open shared object file: No such file or directory
>>> 

Any suggestions?

database reader not working on python3

I got this stack trace from python3

Traceback (most recent call last):
  File "report.py", line 40, in <module>
    parse_file(fname)
  File "report.py", line 31, in parse_file
    country = geoip.country(ipaddr)
  File "/usr/local/lib/python3.3/dist-packages/geoip2/database.py", line 60, in country
    return self._model_for(geoip2.models.Country, ip_address)
AttributeError: 'module' object has no attribute 'models'

The solution I found was to open /usr/local/lib/python3.3/dist-packages/geoip2/database.py and add the line import geoip2.models.

I assume this is a python3 specific problem, but I didn't check if it works on python2.

Proxy kwarg doesn't seem to exist

Running:

import geoip2.webservice geo_client = geoip2.webservice.Client( account_id=xxxx, license_key='XXX', locales=['en'], proxy=os.environ['PROXY_VAR'])

returns:

TypeError: __init__() got an unexpected keyword argument 'proxy'

Despite the fact it's listed as a param here: https://github.com/maxmind/GeoIP2-python/blob/main/geoip2/webservice.py#L410

I also notice the param is non-existent in the documentation: https://geoip2.readthedocs.io/en/latest/#geoip2.webservice.Client

Is there any particular reason why?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.