ahupp / python-magic Goto Github PK

Please note that on Windows 7 64bit the dependent DLLs need to be placed in C:\Windows\SysWOW64 (not C:\Windows\System32).

When I do two instances of Magic

In [1]: import magic
In [2]: m1 = magic.Magic()
In [3]: m2 = magic.Magic()
In [4]: m1.from_file('./settings.pyc')
Out[4]: 'python 2.7 byte-compiled'
In [5]: m2.from_file('./settings.pyc')
Out[5]: 'python 2.7 byte-compiled'

but when I use Magic with custom magic file

In [1]: import magic
In [2]: m1 = magic.Magic(magic_file='my.magic')
In [3]: m2 = magic.Magic(magic_file='my.magic')
Segmentation fault

It crashes.

Using those code can only get mime image/jpeg without charset.

import magic

#1
magic.from_file('test.jpg', mime=True)

#2
magic.from_buffer(open('test.jpg').read(1024), mime=True)

#3
magic.Magic(mime=True, mime_encoding=True).from_file('test.jpg')

But command file --mime test.jpg will print test.jpg: image/jpeg; charset=binary. How to get the charset?

Dear,
I am a fedora packager and i am interested to package it.

The problem that is

• python-magic package name is already taken
• magic namespace is already taken too

Could you set another name like python-libmagic and libmagic as namespace?

libname = ctypes.util.find_library('magic') or ctypes.util.find_library('magic1')

should add
ctypes.util.find_library('msys-magic-1.dll')

I get the above error on Windows 8.1 x64 using either Git Bash, Cygwin or the regular Command Line.

A similar issue has been posted as #12, but the solution there was to install libmagic. I have the package properly installed:

> where file
c:\cygwin64\bin\file.exe

In terminal file --mime-type 1.mp3, and the result is "audio/mpeg".

But with python-magic, it's this:

>>> magic.from_file('1.mp3', mime=True)
>>> application/octet-stream

Why the result is not the same?

FYI, homebrew seems to have renamed the file-formula package.

$ brew --version
0.9.4
$ brew install file-formula
Error: No available formula for file-formula

Based on this URL you now have to brew libmagic instead. Note that I did not have to create the symbolic as that article suggest towards the end.

$ brew install libmagic
==> Downloading ftp://ftp.astron.com/pub/file/file-5.14.tar.gz
######################################################################## 100.0%
######################################################################## 100.0%==> Downloading patches
######################################################################## 100.0%
==> Patching
patching file src/magic.h.in
patch unexpectedly ends in middle of line
Hunk #1 succeeded at 64 with fuzz 1.
==> ./configure --prefix=/usr/local/Cellar/libmagic/5.14 --enable-fsect-man5
==> make install
  /usr/local/Cellar/libmagic/5.14: 12 files, 2.7M, built in 18 seconds
$

The documentation on the homepage (README.md) can probably be updated to reflect this change.

Really not sure what to make of this. Just tried the most basic example:

In [4]: magic.from_file("A1fa.csv")
32815952 A1fa.csv
---------------------------------------------------------------------------
ArgumentError                             Traceback (most recent call last)
<ipython-input-4-3cc8d143e6c2> in <module>()
----> 1 magic.from_file("A1fa.csv")

/usr/lib/python3.3/site-packages/python_magic-0.4.3-py3.3.egg/magic.py in from_file(filename, mime)
     99 def from_file(filename, mime=False):
    100     m = _get_magic_type(mime)
--> 101     return m.from_file(filename)
    102 
    103 def from_buffer(buffer, mime=False):

/usr/lib/python3.3/site-packages/python_magic-0.4.3-py3.3.egg/magic.py in from_file(self, filename)
     68 
     69         print(self.cookie, filename)
---> 70         return magic_file(self.cookie, filename)
     71 
     72     def __del__(self):

ArgumentError: argument 2: <class 'TypeError'>: wrong type

I have installed locally python-magic with virtualenv on my mac os x but in python repl, I get
AttributeError: 'module' object has no attribute 'open'.
Is this method deprecated?
Note that i imported successfully magic module in my repl.

Thanks in advance

python -c 'import magic; print magic.MIME'
Traceback (most recent call last):
File "", line 1, in
AttributeError: 'module' object has no attribute 'MIME'

test on my macos 10.8 x86_64

i try install both
brew install libmagic
pip install filemagic
pip install python-maigc

Linux's file command reports:

[moonstone Desktop] ‽ file dino.png
dino.png: PNG image data, 492 x 639, 8-bit colormap, non-interlaced

However, magic is reporting the mime type as "text/plain"

The image:

I have installed the latest Python-magic version on a Windows 7 64 bits virtual machine using the pip command.

I have also put the following files to my C:\Windows\System32 folder: magic1.dll, regex2.dll and zlib1.dll.

But when i try to import the magic lib, i have the following error:

Python 2.7.3 (default, Apr 10 2012, 23:24:47) [MSC v.1500 64 bit (AMD64)] on win32
Type "copyright", "credits" or "license()" for more information.
>>> import magic

Traceback (most recent call last):
  File "<pyshell#0>", line 1, in <module>
    import magic
  File "C:\Python27\lib\site-packages\magic.py", line 115, in <module>
    libmagic = ctypes.CDLL(dll)
  File "C:\Python27\lib\ctypes\__init__.py", line 365, in __init__
    self._handle = _dlopen(self._name, mode)
WindowsError: [Error 193] %1 is not a valid Win32 application

Any idea ?

When I want to know the mime type of big file, show in my console this error... what can i do to solve this problem?

thanks...

The LICENSE file and the README states the project is under MIT license but the setup.py says PSF license. Which one is true?

This is the same as #9
BUT I cannot encode to utf-8 as I do not know which encoding was used to create that filename. The file was created on some linux and is extracted on windows.

A test file is in this archive: https://github.com/pombredanne/python-magic/blob/issue_42/testdata/a.zip

When extracted on windows I can os.walk or os.listdir(u'some dir') (using a unicode string for the input dir) where this file has been extracted and open the file in Python alright. ie os.path.exists is True and the file can be opened.

Yet magic.from_file fails when passing the filename whether as a string or unicode

The point being that you cannot rely on my file system encoding to convert the file name, because it was created elsewhere, and the encoding is arbitrary, eventually unknown.

The best explanation of the problem is here IMHO: http://web.archive.org/web/20081219193532/http://boodebr.org/main/python/all-about-python-and-unicode#UNI_FILENAMES

NB: FWIW, I cannot get run the file command line to work on that path either... so it could be a problem with the file code itself, not your ctypes wrapper

[wodim@avalon ~]$ file /etc/magic
/etc/magic: magic binary file for file(1) cmd (version 11) (little endian)
[wodim@avalon ~]$ python2
Python 2.7.5 (default, Sep  6 2013, 09:55:21)
[GCC 4.8.1 20130725 (prerelease)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import magic
>>> magic.Magic(magic_file='/etc/magic')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.7/site-packages/magic.py", line 57, in __init__
    magic_load(self.cookie, magic_file)
  File "/usr/lib/python2.7/site-packages/magic.py", line 217, in magic_load
    return _magic_load(cookie, coerce_filename(filename))
  File "/usr/lib/python2.7/site-packages/magic.py", line 168, in errorcheck
    raise MagicException(err)
magic.MagicException: no magic files loaded

Maybe there's something I don't understand in here. It happened after updating to file 5.15.

Python 2.7.4 (default, Apr 19 2013, 18:32:33)
[GCC 4.7.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
import magic
magic.Magic
class magic.Magic at 0xb71a62fc
magic.Magic()
magic.Magic instance at 0xb719c3cc
magic.open
Traceback (most recent call last):
File "", line 1, in
AttributeError: 'module' object has no attribute 'open'

k@ubuntu:~$ python -c 'import magic; print magic.MIME'
Traceback (most recent call last):
File "", line 1, in
AttributeError: 'module' object has no attribute 'MIME'

I'm the Gentoo maintainer for eyeD3 python module:
http://eyed3.nicfit.net/
It seems that the new 0.7.4 version uses this python-magic module which is in fact a direct conflict with the python-magic version that has been a stable API for years, file-5.17's magic.py:
http://www.darwinsys.com/file/
so now some packages can no longer co-exist on the same system:
https://bugs.gentoo.org/show_bug.cgi?id=488270

While this project is still young, can you please rename the library, to, for example, magic2 as to imply this is the continuity of the prev. library if you are in fact sure the magic.py from the file package that is installed on every nix system is obsolete/no longer maintained, which is suspect it's not
I don't remember seeing any bugs about it, only use:
https://github.com/pazz/alot (alot mail client uses file's magic.py)
http://code.google.com/p/binwalk/ (tool identifying files embedded inside firmware images also uses file's magic)
Sabayon Linux distributions *package manager equo uses file's magic.py:
http://www.sabayon.org

Please :)

hello

ive installed python magic on a winxp sp3 32bit. ive tried testing it with a few sample code lines featured in README:
import magic
magic.from_file("bot.exe")
magic.from_buffer(open("bot.exe").read(1024))
magic.from_file("bot.exe", mime=True)

when im running the script im getting the following exception:
Traceback (most recent call last):
File "magic test.py", line 2, in
magic.from_file("bot.exe")
File "build\bdist.win32\egg\magic.py", line 99, in from_file
File "build\bdist.win32\egg\magic.py", line 96, in _get_magic_type
File "build\bdist.win32\egg\magic.py", line 89, in _get_magic
File "build\bdist.win32\egg\magic.py", line 51, in init
File "build\bdist.win32\egg\magic.py", line 138, in errorcheck
magic.MagicException: could not find any magic files!

must be something in the module!
any ideas?

Hi,

I have problem with file name containing nonASCII chars. I don't know, if the problem is in libmagic or binding this library to python.

Python 2.7.1 (r271:86832, Apr  3 2011, 19:21:03) 
[GCC 4.4.5] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import magic
>>> mime = magic.Magic(mime=True)
>>> mime.from_file(u'žlutý.pdf')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "magic.py", line 69, in from_file
    return magic_file(self.cookie, filename)
ctypes.ArgumentError: argument 2: <type 'exceptions.UnicodeEncodeError'>: 'ascii' codec can't encode character u'\u017e' in position 0: ordinal not in range(128)
>>>

Thanks
Tomas

There is already an issue for this, but before packages are merged, it would be nice to at least make it clear, that these two package are currently unrelated and one is not yet updated version of another. This would prevent confusions, like the one, which aroused in this Stack Overflow question.

I have a file named:

  'aktuelle_Dokumente.jsp?docTyp=ST&wp=15&dokNum=8.+Schulrechts\xe4\xae\xa4erungsgesetz&searchDru=suchen'

When I try to read this with magic's from_file method, I get the following exception:

  Traceback (most recent call last):
    File "repo-audit.py", line 133, in <module>
      auditor.run()
    File "repo-audit.py", line 27, in run
      e = AuditEntry(fullpath, self.logfile, self.mime_magic)
    File "repo-audit.py", line 61, in __init__
      self.mimetype = self.file_type()
    File "repo-audit.py", line 119, in file_type
      return self.mime_magic.from_file(self.path)
    File "/.../venv/lib/python2.7/site-packages/magic.py", line 70, in from_file
      return magic_file(self.cookie, filename)
    File "/.../venv/lib/python2.7/site-packages/magic.py", line 170, in magic_file
      return _magic_file(cookie, coerce_filename(filename))
    File "/.../venv/lib/python2.7/site-packages/magic.py", line 146, in coerce_filename
      return filename.encode(sys.getfilesystemencoding())
  UnicodeDecodeError: 'ascii' codec can't decode byte 0xe4 in position 130: ordinal not in range(128)

I am on MacOS X 10.8.2 with python 2.7.2.

Hi,

I use python-magic in eyeD3 and am quite happy with it, thanks!

On gentoo, at least, there is a package called sys-apps/file that provides libmagic.so and lays down a magic.py in site-packages. This collides with python-magic's file.. Which was first? I'm thinking of patching python-magic at install time to have a different module name. Or.. is python-magic going to replace what is in the 'file' pkg someday?

hello
ive downloaded and installed python-magic on windows (win 8) and placed the
regex2.dll zlib.dll magic1.dll on the windows32 folder. now when im checking the module via the help function in the python interactive shell this is the error i get
help> magic
problem in magic - <type 'exceptions.WindowsError'>: [Error 193] %1 is not a val
id Win32 application

any ideas?

Is this the same library that is on PyPi (http://pypi.python.org/pypi/python-magic/)? If so, I think that it needs to be updated to remove the ctype dependency b/c I am getting the ctypes error when doing a: pip install python-magic on ubuntu 10.10.

I have some files that are causing a MagicException to be thrown with no details. This is on Ubuntu 12.04 LTS, with all packages up to date, and using the latest version of python-magic installed with pip.

Stack trace is below. I have stepped through the code, and it appears that it's dying somewhere in the ctypes error handler:

Traceback (most recent call last):
  File "./test_magic.py", line 35, in <module>
    mime_type = magic.from_file(filename, mime=True)
  File "/usr/local/lib/python2.7/dist-packages/magic.py", line 121, in from_file
    return m.from_file(filename)
  File "/usr/local/lib/python2.7/dist-packages/magic.py", line 78, in from_file
    return magic_file(self.cookie, filename)
  File "/usr/local/lib/python2.7/dist-packages/magic.py", line 208, in magic_file
    return _magic_file(cookie, coerce_filename(filename))
  File "/usr/local/lib/python2.7/dist-packages/magic.py", line 169, in errorcheck_null
    raise MagicException(err)
magic.MagicException: None

I used the Python debugger, and in the errorcheck_null() method, the "result" parameter is "None", the "func" argument has a dict with "name==magic_file", and the "args" argument is a tuple with the numeric value 22135440 and the filename of the file I'm checking.

Since this is Ubuntu 12.04 LTS, the Python version is 2.7.3 (2.7.3-0ubuntu3.4) and the libmagic1 version is 5.09-2. I have been unable to reproduce this issue on Ubuntu 13.04 or Ubuntu 13.10, so I think it's something specific to either Python 2.7.3 or the version of libmagic1. Unfortunately I don't have the luxury of upgrading the Ubuntu version, since this is for a commercial production use.

What does the numeric value 22135440 mean? I realize that this may be a bug with either Python or libmagic1, but I'm not sure what the issue is to report it.

For what it's worth, if I use "/usr/bin/file" on the file, it works fine, and returns:

Composite Document File V2 Document, Little Endian, Os: Windows, Version 5.1, Code page: 936, Name of Creating Application: Microsoft Excel, Create Time/Date: Fri Sep 15 01:00:00 2006, Last Saved Time/Date: Mon Jan 17 19:59:34 2011, Security: 0

Hello,

I get the following error using the last python-magic.

magic.MagicException: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linkederror reading (Invalid argument)

It is possible to extend the ability of the file command on a per user biases by adding magic definitions to a .magic file in a users home directory.

Currently I have a .magic file in the home directory of a user named sendfax. The file looks like to following:

#------------------------------------------------------------------------------
# Detection for MS OOXML Documents

0   belong  0x504B0304
>4  belong  0x14000600  MS OOXML Document

When the file command is run from the command line on an MS docx file I get the following output

# su sendfax
$ file test.docx 
test.docx: MS OOXML Document
$ whoami
sendfax

If I do something similar in python using the magic module I get an entirely different response.

>>> from magic import Magic
>>> m = Magic()
>>> print m.from_file('test.docx')
Zip archive data, at least v2.0 to extract
>>> import getpass
>>> print getpass.getuser()
sendfax

for use current magic

import magic
path="path/to/file"
mime = magic.open(magic.MIME_TYPE) # you can choose other flag see magic.py
mime.load()
mime.file(path)

You can add new magic file! With load method, like:

mime.load(file="/path/to/magicfile")

Recently, I've been having problems getting python-magic to work on Windows. I have libmagic installed, along with all the necessary DLLs (including magic1.dll) but I kept getting Windows 126 errors(module not found). I tried specifying the DLL in the PATH or passing it into the constructor with no luck.

In the end I decided to rename magic1.dll to magic.dll and it started working.
After looking at the source it appears that python-magic tries to load either magic.dll or magic1.dll, however if magic.dll is not found, Windows directly throws an 126 error and doesn't look for magic1.dll. It could be worth considering another, more reliable way of finding the proper DLL.

I am using python-magic in an application where I spawn off a working thread that is given a python-magic object that is already loaded. Within the thread I use the from_buffer method to get the magic info from a previously loaded file. At first i was having minor response corruption issues where the first few bytes of the response were corrupted every so often (not consistently). I fixed this by reloading the magic object ever 20 uses. This project requires an extremely fast response time so reloading magic every single time would not work. As you can see below (after my code snippet) it looks like somewhere between magic and the libmagic something is getting freed twice which is making glibc very unhappy.

As an added data point, I typically run four of these programs in parallel on the same machine. When one of them dies via this error, the remaining instances also die. This is odd since they arent forked but run separately from the commandline. I'm using Ubuntu 10.04 LTS

<--- code snippet --->

def RefreshMagicObject(self):
    self.m_dwMagicUseCount = 0
    if self.m_oMagic != None:
        try:
            del self.m_oMagic
        except:
            pass

    self.m_oMagic = magic.Magic(magic_file="/db/magic")
    if self.m_oMagic == None:
        return False
    return True

def GetFileTypeString(self, carrFilestream):
    if self.m_dwMagicUseCount > 20:
        if self.RefreshMagicObject() == False:
            return "$$MAGICERROR$$"
    # increment the use count
    self.m_dwMagicUseCount += 1

    if self.m_oMagic == None: return None
    strMagic = self.m_oMagic.from_buffer(carrFilestream)

<---- error on console ---->

*** glibc detected *** python2.7: double free or corruption (fasttop): 0x00007f121198e9b0 ***
======= Backtrace: =========
/lib/libc.so.6[0x7f1224e27928]
/lib/libc.so.6(cfree+0x76)[0x7f1224e29a36]
/usr/lib/libmagic.so.1(file_vprintf+0x51)[0x7f1220ebdab1]
/usr/lib/libmagic.so.1(file_printf+0x88)[0x7f1220ebdb88]
/usr/lib/libmagic.so.1[0x7f1220eb632c]
/usr/lib/libmagic.so.1(file_softmagic+0x22b)[0x7f1220eb69fb]
/usr/lib/libmagic.so.1(file_buffer+0x18b)[0x7f1220ebdd1b]
/usr/lib/libmagic.so.1(magic_buffer+0x55)[0x7f1220eb0385]
/usr/local/lib/python2.7/site-packages/ctypes-1.0.2-py2.7-linux-x86_64.egg/_ctypes.so(ffi_call_unix64+0x4c)[0x7f12216ee758]
/usr/local/lib/python2.7/site-packages/ctypes-1.0.2-py2.7-linux-x86_64.egg/_ctypes.so(ffi_call+0x19a)[0x7f12216edeba]
/usr/local/lib/python2.7/site-packages/ctypes-1.0.2-py2.7-linux-x86_64.egg/_ctypes.so(_CallProc+0x322)[0x7f12216e8ba2]
/usr/local/lib/python2.7/site-packages/ctypes-1.0.2-py2.7-linux-x86_64.egg/_ctypes.so[0x7f12216e221f]
python2.7(PyObject_Call+0x5d)[0x41bc4d]
python2.7(PyEval_EvalFrameEx+0x3477)[0x4a9077]
python2.7(PyEval_EvalFrameEx+0x6d5b)[0x4ac95b]
python2.7(PyEval_EvalFrameEx+0x6d5b)[0x4ac95b]
python2.7(PyEval_EvalFrameEx+0x6d5b)[0x4ac95b]
python2.7(PyEval_EvalFrameEx+0x6d5b)[0x4ac95b]
python2.7(PyEval_EvalFrameEx+0x6d5b)[0x4ac95b]
python2.7(PyEval_EvalCodeEx+0x809)[0x4ad5c9]
python2.7[0x5104cb]
python2.7(PyObject_Call+0x5d)[0x41bc4d]
python2.7(PyEval_EvalFrameEx+0x1846)[0x4a7446]
python2.7(PyEval_EvalFrameEx+0x6d5b)[0x4ac95b]
python2.7(PyEval_EvalFrameEx+0x6d5b)[0x4ac95b]
python2.7(PyEval_EvalCodeEx+0x809)[0x4ad5c9]
python2.7[0x5103ce]
python2.7(PyObject_Call+0x5d)[0x41bc4d]
python2.7[0x423738]
python2.7(PyObject_Call+0x5d)[0x41bc4d]
python2.7(PyEval_CallObjectWithKeywords+0x56)[0x4a4a96]
python2.7[0x4e4fa2]
/lib/libpthread.so.0[0x7f1225797fc7]
/lib/libc.so.6(clone+0x6d)[0x7f1224e8359d]
======= Memory map: ========
00400000-0056f000 r-xp 00000000 08:05 56555 /usr/local/bin/python2.7
0076f000-007ab000 rw-p 0016f000 08:05 56555 /usr/local/bin/python2.7
007ab000-007ba000 rw-p 007ab000 00:00 0
01ef2000-357d9000 rw-p 01ef2000 00:00 0 [heap]
402c8000-402c9000 ---p 402c8000 00:00 0
402c9000-40ac9000 rwxp 402c9000 00:00 0
40ac9000-40aca000 ---p 40ac9000 00:00 0
40aca000-412ca000 rwxp 40aca000 00:00 0
41a94000-41a95000 ---p 41a94000 00:00 0
41a95000-42295000 rwxp 41a95000 00:00 0
42295000-42296000 ---p 42295000 00:00 0
42296000-42a96000 rwxp 42296000 00:00 0
42a96000-42a97000 ---p 42a96000 00:00 0
42a97000-43297000 rwxp 42a97000 00:00 0
43297000-43298000 ---p 43297000 00:00 0
43298000-43a98000 rwxp 43298000 00:00 0
43a98000-43a99000 ---p 43a98000 00:00 0
43a99000-44299000 rwxp 43a99000 00:00 0
44299000-4429a000 ---p 44299000 00:00 0
4429a000-44a9a000 rwxp 4429a000 00:00 0
44a9a000-44a9b000 ---p 44a9a000 00:00 0
44a9b000-4529b000 rwxp 44a9b000 00:00 0
4529b000-4529c000 ---p 4529b000 00:00 0
4529c000-45a9c000 rwxp 4529c000 00:00 0
45a9c000-45a9d000 ---p 45a9c000 00:00 0
45a9d000-4629d000 rwxp 45a9d000 00:00 0
4629d000-4629e000 ---p 4629d000 00:00 0
4629e000-46a9e000 rwxp 4629e000 00:00 0
46a9e000-46a9f000 ---p 46a9e000 00:00 0
46a9f000-4729f000 rwxp 46a9f000 00:00 0
4729f000-472a0000 ---p 4729f000 00:00 0
472a0000-47aa0000 rwxp 472a0000 00:00 0
47aa0000-47aa1000 ---p 47aa0000 00:00 0
47aa1000-482a1000 rwxp 47aa1000 00:00 0
482a1000-482a2000 ---p 482a1000 00:00 0
482a2000-48aa2000 rwxp 482a2000 00:00 0
48aa2000-48aa3000 ---p 48aa2000 00:00 0
48aa3000-492a3000 rwxp 48aa3000 00:00 0
492a3000-492a4000 ---p 492a3000 00:00 0
492a4000-49aa4000 rwxp 492a4000 00:00 0
49aa4000-49aa5000 ---p 49aa4000 00:00 0
49aa5000-4a2a5000 rwxp 49aa5000 00:00 0
4a2a5000-4a2a6000 ---p 4a2a5000 00:00 0
4a2a6000-4aaa6000 rwxp 4a2a6000 00:00 0
4aaa6000-4aaa7000 ---p 4aaa6000 00:00 0
4aaa7000-4b2a7000 rwxp 4aaa7000 00:00 0
4b2a7000-4b2a8000 ---p 4b2a7000 00:00 0
4b2a8000-4baa8000 rwxp 4b2a8000 00:00 0
4baa8000-4baa9000 ---p 4baa8000 00:00 0
4baa9000-4c2a9000 rwxp 4baa9000 00:00 0
4c2a9000-4c2aa000 ---p 4c2a9000 00:00 0
4c2aa000-4caaa000 rwxp 4c2aa000 00:00 0
4caaa000-4caab000 ---p 4caaa000 00:00 0
4caab000-4d2ab000 rwxp 4caab000 00:00 0
4d2ab000-4d2ac000 ---p 4d2ab000 00:00 0
4d2ac000-4daac000 rwxp 4d2ac000 00:00 0
4daac000-4daad000 ---p 4daac000 00:00 0
4daad000-4e2ad000 rwxp 4daad000 00:00 0
4e2ad000-4e2ae000 ---p 4e2ad000 00:00 0
4e2ae000-4eaae000 rwxp 4e2ae000 00:00 0
4eaae000-4eaaf000 ---p 4eaae000 00:00 0
4eaaf000-4f2af000 rwxp 4eaaf000 00:00 0
4f2af000-4f2b0000 ---p 4f2af000 00:00 0
4f2b0000-4fab0000 rwxp 4f2b0000 00:00 0
4fab0000-4fab1000 ---p 4fab0000 00:00 0
4fab1000-502b1000 rwxp 4fab1000 00:00 0
502b1000-502b2000 ---p 502b1000 00:00 0
502b2000-50ab2000 rwxp 502b2000 00:00 0
50ab2000-50ab3000 ---p 50ab2000 00:00 0
50ab3000-512b3000 rwxp 50ab3000 00:00 0
512b3000-512b4000 ---p 512b3000 00:00 0
512b4000-51ab4000 rwxp 512b4000 00:00 0
51ab4000-51ab5000 ---p 51ab4000 00:00 0
51ab5000-522b5000 rwxp 51ab5000 00:00 0
522b5000-522b6000 ---p 522b5000 00:00 0
522b6000-52ab6000 rwxp 522b6000 00:00 0
52ab6000-52ab7000 ---p 52ab6000 00:00 0
52ab7000-532b7000 rwxp 52ab7000 00:00 0
532b7000-532b8000 ---p 532b7000 00:00 0
532b8000-53ab8000 rwxp 532b8000 00:00 0
53ab8000-53ab9000 ---p 53ab8000 00:00 0
53ab9000-542b9000 rwxp 53ab9000 00:00 0
542b9000-542ba000 ---p 542b9000 00:00 0
542ba000-54aba000 rwxp 542ba000 00:00 0
54aba000-54abb000 ---p 54aba000 00:00 0
54abb000-552bb000 rwxp 54abb000 00:00 0
552bb000-552bc000 ---p 552bb000 00:00 0
552bc000-55abc000 rwxp 552bc000 00:00 0
55abc000-55abd000 ---p 55abc000 00:00 0
55abd000-562bd000 rwxp 55abd000 00:00 0
562bd000-562be000 ---p 562bd000 00:00 0
562be000-56abe000 rwxp 562be000 00:00 0
56abe000-56abf000 ---p 56abe000 00:00 0
56abf000-572bf000 rwxp 56abf000 00:00 0
572bf000-572c0000 ---p 572bf000 00:00 0
572c0000-57ac0000 rwxp 572c0000 00:00 0
57ac0000-57ac1000 ---p 57ac0000 00:00 0
57ac1000-582c1000 rwxp 57ac1000 00:00 0
582c1000-582c2000 ---p 582c1000 00:00 0
582c2000-58ac2000 rwxp 582c2000 00:00 0
58ac2000-58ac3000 ---p 58ac2000 00:00 0
58ac3000-592c3000 rwxp 58ac3000 00:00 0
592c3000-592c4000 ---p 592c3000 00:00 0
592c4000-59ac4000 rwxp 592c4000 00:00 0
59ac4000-59ac5000 ---p 59ac4000 00:00 0
59ac5000-5a2c5000 rwxp 59ac5000 00:00 0
5a2c5000-5a2c6000 ---p 5a2c5000 00:00 0
5a2c6000-5aac6000 rwxp 5a2c6000 00:00 0
5aac6000-5aac7000 ---p 5aac6000 00:00 0
5aac7000-5b2c7000 rwxp 5aac7000 00:00 0
5b2c7000-5b2c8000 ---p 5b2c7000 00:00 0
5b2c8000-5bac8000 rwxp 5b2c8000 00:00 0
5bac8000-5bac9000 ---p 5bac8000 00:00 0
5bac9000-5c2c9000 rwxp 5bac9000 00:00 0
5c2c9000-5c2ca000 ---p 5c2c9000 00:00 0
5c2ca000-5caca000 rwxp 5c2ca000 00:00 0
5caca000-5cacb000 ---p 5caca000 00:00 0
5cacb000-5d2cb000 rwxp 5cacb000 00:00 0
5d2cb000-5d2cc000 ---p 5d2cb000 00:00 0
5d2cc000-5dacc000 rwxp 5d2cc000 00:00 0
5dacc000-5dacd000 ---p 5dacc000 00:00 0
5dacd000-5e2cd000 rwxp 5dacd000 00:00 0
5e2cd000-5e2ce000 ---p 5e2cd000 00:00 0
5e2ce000-5eace000 rwxp 5e2ce000 00:00 0
5eace000-5eacf000 ---p 5eace000 00:00 0
5eacf000-5f2cf000 rwxp 5eacf000 00:00 0
5f2cf000-5f2d0000 ---p 5f2cf000 00:00 0
5f2d0000-5fad0000 rwxp 5f2d0000 00:00 0
5fad0000-5fad1000 ---p 5fad0000 00:00 0
5fad1000-602d1000 rwxp 5fad1000 00:00 0
602d1000-602d2000 ---p 602d1000 00:00 0
602d2000-60ad2000 rwxp 602d2000 00:00 0
60ad2000-60ad3000 ---p 60ad2000 00:00 0
60ad3000-612d3000 rwxp 60ad3000 00:00 0
612d3000-612d4000 ---p 612d3000 00:00 0
612d4000-61ad4000 rwxp 612d4000 00:00 0
61ad4000-61ad5000 ---p 61ad4000 00:00 0
61ad5000-622d5000 rwxp 61ad5000 00:00 0
622d5000-622d6000 ---p 622d5000 00:00 0
622d6000-62ad6000 rwxp 622d6000 00:00 0
62ad6000-62ad7000 ---p 62ad6000 00:00 0
62ad7000-632d7000 rwxp 62ad7000 00:00 0
632d7000-632d8000 ---p 632d7000 00:00 0
632d8000-63ad8000 rwxp 632d8000 00:00 0
63ad8000-63ad9000 ---p 63ad8000 00:00 0
63ad9000-642d9000 rwxp 63ad9000 00:00 0
642d9000-642da000 ---p 642d9000 00:00 0
642da000-64ada000 rwxp 642da000 00:00 0
64ada000-64adb000 ---p 64ada000 00:00 0
64adb000-652db000 rwxp 64adb000 00:00 0
652db000-652dc000 ---p 652db000 00:00 0
652dc000-65adc000 rwxp 652dc000 00:00 0
65adc000-65add000 ---p 65adc000 00:00 0
65add000-662dd000 rwxp 65add000 00:00 0
662dd000-662de000 ---p 662dd000 00:00 0
662de000-66ade000 rwxp 662de000 00:00 0
66ade000-66adf000 ---p 66ade000 00:00 0
66adf000-672df000 rwxp 66adf000 00:00 0
672df000-672e0000 ---p 672df000 00:00 0
672e0000-67ae0000 rwxp 672e0000 00:00 0
67ae0000-67ae1000 ---p 67ae0000 00:00 0
67ae1000-682e1000 rwxp 67ae1000 00:00 0
682e1000-682e2000 ---p 682e1000 00:00 0
682e2000-68ae2000 rwxp 682e2000 00:00 0
68ae2000-68ae3000 ---p 68ae2000 00:00 0
68ae3000-692e3000 rwxp 68ae3000 00:00 0
692e3000-692e4000 ---p 692e3000 00:00 0
692e4000-69ae4000 rwxp 692e4000 00:00 0
69ae4000-69ae5000 ---p 69ae4000 00:00 0
69ae5000-6a2e5000 rwxp 69ae5000 00:00 0
6a2e5000-6a2e6000 ---p 6a2e5000 00:00 0
6a2e6000-6aae6000 rwxp 6a2e6000 00:00 0
6aae6000-6aae7000 ---p 6aae6000 00:00 0
6aae7000-6b2e7000 rwxp 6aae7000 00:00 0
6b2e7000-6b2e8000 ---p 6b2e7000 00:00 0
6b2e8000-6bae8000 rwxp 6b2e8000 00:00 0
6bae8000-6bae9000 ---p 6bae8000 00:00 0
6bae9000-6c2e9000 rwxp 6bae9000 00:00 0
6c2e9000-6c2ea000 ---p 6c2e9000 00:00 0
6c2ea000-6caea000 rwxp 6c2ea000 00:00 0
6caea000-6caeb000 ---p 6caea000 00:00 0
6caeb000-6d2eb000 rwxp 6caeb000 00:00 0
6d2eb000-6d2ec000 ---p 6d2eb000 00:00 0
6d2ec000-6daec000 rwxp 6d2ec000 00:00 0
6daec000-6daed000 ---p 6daec000 00:00 0
6daed000-6e2ed000 rwxp 6daed000 00:00 0
6e2ed000-6e2ee000 ---p 6e2ed000 00:00 0
6e2ee000-6eaee000 rwxp 6e2ee000 00:00 0
6eaee000-6eaef000 ---p 6eaee000 00:00 0
6eaef000-6f2ef000 rwxp 6eaef000 00:00 0
6f2ef000-6f2f0000 ---p 6f2ef000 00:00 0
6f2f0000-6faf0000 rwxp 6f2f0000 00:00 0
6faf0000-6faf1000 ---p 6faf0000 00:00 0
6faf1000-702f1000 rwxp 6faf1000 00:00 0
702f1000-702f2000 ---p 702f1000 00:00 0
702f2000-70af2000 rwxp 702f2000 00:00 0
70af2000-70af3000 ---p 70af2000 00:00 0
70af3000-712f3000 rwxp 70af3000 00:00 0
712f3000-712f4000 ---p 712f3000 00:00 0
712f4000-71af4000 rwxp 712f4000 00:00 0
71af4000-71af5000 ---p 71af4000 00:00 0
71af5000-722f5000 rwxp 71af5000 00:00 0
722f5000-722f6000 ---p 722f5000 00:00 0
722f6000-72af6000 rwxp 722f6000 00:00 0
72af6000-72af7000 ---p 72af6000 00:00 0
72af7000-732f7000 rwxp 72af7000 00:00 0
732f7000-732f8000 ---p 732f7000 00:00 0
732f8000-73af8000 rwxp 732f8000 00:00 0
73af8000-73af9000 ---p 73af8000 00:00 0
73af9000-742f9000 rwxp 73af9000 00:00 0
742f9000-742fa000 ---p 742f9000 00:00 0
742fa000-74afa000 rwxp 742fa000 00:00 0
74afa000-74afb000 ---p 74afa000 00:00 0
74afb000-752fb000 rwxp 74afb000 00:00 0
752fb000-752fc000 ---p 752fb000 00:00 0
752fc000-75afc000 rwxp 752fc000 00:00 0
75afc000-75afd000 ---p 75afc000 00:00 0
75afd000-762fd000 rwxp 75afd000 00:00 0
762fd000-762fe000 ---p 762f

The title says it all, at line 158 the sys module is being re-imported.

Python-magic is sometimes providing incorrect values for HTML and CSS files (among others). Original downstream issue and code to reproduce: s3tools/s3cmd#198 (comment)

Does this look like an upstream (libmagic) bug, possibly w/ Debian/Ubuntu packaging?

Hi,

Thanks for creating this package. I'm getting this error "Can't find magic files" After reading the docs, you said the path should be passed to the constructor.

My question is, are the three dll's extentensions that I put in C:\Windows\System32 should be placed in the magic folder?

and should the path be passed to this constructor:

    def __init__(self, mime=False, magic_file=None, mime_encoding=False,
                     keep_going=False):

Kindly reply back.

I'm running python-magic==0.4.6 on a large dataset with mime=True, and one .html file triggered the following MagicException:

  File "/.../lib/python2.7/site-packages/magic.py", line 66, in from_buffer
    return magic_buffer(self.cookie, buf)
  File "/.../lib/python2.7/site-packages/magic.py", line 215, in magic_buffer
    return _magic_buffer(cookie, buf, len(buf))
  File "/.../lib/python2.7/site-packages/magic.py", line 168, in errorcheck_null
    raise MagicException(err)
magic.MagicException: line 60: regex error 16 for `^\s*except.*:', (invalid argument to regex routine)

Mimetypes are defined as normal unicode strings as far as I can see, but the library returns stuff like b'image/jpeg' in py3k.

We would be really helped with a release of python-magic that includes the ctypes-dependency-fix. (a75cf0a). Is there any plan to release on a short term? Anythoni I could help with in this regard?

install from latest source, it doesn't works on Mac OS X 10.6.8

$ python --version
Python 2.6.7


$ port info python26
python26 @2.6.7 (lang)
Variants:             ucs4, (-)universal

Description:          Python is an interpreted, interactive, object-oriented programming language.
Homepage:             http://www.python.org/

Library Dependencies: gettext, zlib, openssl, sqlite3, db46, ncurses, gdbm, bzip2, readline
Runtime Dependencies: python_select
Platforms:            darwin
License:              PSF
Maintainers:          [email protected], [email protected]

$ cd [proj source]
$ sudo python setup.py install 

$ python test.py 
Traceback (most recent call last):
  File "test.py", line 7, in <module>
    from magic import Magic, MagicException
  File "/Users/lee/Downloads/py_apps/python-magic/magic.py", line 128, in <module>
    raise ImportError('failed to find libmagic.  Check your installation')
ImportError: failed to find libmagic.  Check your installation
Lees-Box:python-magic lee$ python -c 'import magic'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "magic.py", line 128, in <module>
    raise ImportError('failed to find libmagic.  Check your installation')
ImportError: failed to find libmagic.  Check your installation

Instead of hardcoding a string with a specific version of libmagic installed throw Homebrew, u can use the following snippet:

import glob
glob.glob('/usr/local/Cellar/libmagic/*/lib/libmagic.dylib')
['/usr/local/Cellar/libmagic/5.11/lib/libmagic.dylib']

so the code could be more or less:

if not libmagic or not libmagic._name:
import sys
import glob
platform_to_lib = {
'darwin': [
'/usr/local/lib/libmagic.dylib',
'/opt/local/lib/libmagic.dylib'
] + glob.glob('/usr/local/Cellar/libmagic/*/lib/libmagic.dylib'),
'win32': [
'magic1.dll'
]
}
for dll in platform_to_lib.get(sys.platform, []):
try:
libmagic = ctypes.CDLL(dll)
break # <------------------------Break with the first hit
except OSError:
pass

We are using magic.py within yadt-config-rpm-maker.
Unfortunately magic.py does not contain a license header nor could I find a LICENSE file. After having a look at the PSF page we got the impression that we would have to include your license in the file too:

The above copyright notice and this permission notice shall be included
in all copies or substantial portions of the Software.

It would be great if you could add the license into the file, so we can add it to the file too...
Thanks in advance.

• Some unclosed files
• The date test doesn’t work in my time zone (UTC+2)
• Two file types are not recognized correctly

test.py:16: ResourceWarning: unclosed file <_io.BufferedReader name='testdata/magic.pyc'>
  value = m.from_buffer(open(filename, 'rb').read())
test.py:16: ResourceWarning: unclosed file <_io.BufferedReader name='testdata/test.gz'>
  value = m.from_buffer(open(filename, 'rb').read())
F.Ftest.py:16: ResourceWarning: unclosed file <_io.BufferedReader name='testdata/text-iso8859-1.txt'>
  value = m.from_buffer(open(filename, 'rb').read())
test.py:16: ResourceWarning: unclosed file <_io.BufferedReader name='testdata/text.txt'>
  value = m.from_buffer(open(filename, 'rb').read())
.F
======================================================================
FAIL: test_descriptions (__main__.MagicTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test.py", line 43, in test_descriptions
    'text.txt': 'ASCII text',
  File "test.py", line 18, in assert_values
    self.assertEqual(value, expected_value_bytes)
AssertionError: b'gzip compressed data, was "test", from Unix, last modified: Sun Jun 29 03:32:52 2008' != b'gzip compressed data, was "test", from Unix, last modified: Sun Jun 29 01:32:52 2008'

======================================================================
FAIL: test_keep_going (__main__.MagicTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test.py", line 71, in test_keep_going
    'application/octet-stream'.encode('utf-8'))
AssertionError: b'image/jpeg' != b'application/octet-stream'

======================================================================
FAIL: test_mime_types (__main__.MagicTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test.py", line 31, in test_mime_types
    b'\xce\xbb': 'text/plain',
  File "test.py", line 18, in assert_values
    self.assertEqual(value, expected_value_bytes)
AssertionError: b'binary' != b'application/octet-stream'

----------------------------------------------------------------------
Ran 5 tests in 0.027s

FAILED (failures=3)

Hi,

Getting this error when i try the
m = magic.Magic(mime=True).

What must be the cause?

Thanks in advance

in setup.py there is wrong keyword to set requirement to ctypes. it should be install_requires
http://peak.telecommunity.com/DevCenter/setuptools
This buildout profile is showing the issue:

[buildout]
parts = eggs
[eggs]
recipe = zc.recipe.egg
eggs = python-magic

Here is the fix:

diff --git a/setup.py b/setup.py
index 5d9d9e7..ea7c30c 100644
--- a/setup.py
+++ b/setup.py
@@ -8,7 +8,7 @@ setup(name='python-magic',
url="http://github.com/ahupp/python-magic",
version='0.3.1',
py_modules=['magic'],

•  requires=['ctypes'],
  

•  install_requires=['ctypes'],
  

  long_description="""This module uses ctypes to access the libmagic file type
  identification library. It makes use of the local magic database and
  supports both textual and MIME-type output.

When I run the following test program:

#!/usr/bin/python3

import magic

magic.from_file("/home/user/.bashrc")

It fails with:

Traceback (most recent call last):
  File "moo", line 2, in <module>
    magic.from_file("/home/user/.bashrc")
  File "/home/user/.local/lib64/python3.2/site-packages/magic.py", line 100, in from_file
    return m.from_file(filename)
  File "/home/user/.local/lib64/python3.2/site-packages/magic.py", line 69, in from_file
    return magic_file(self.cookie, filename)
ctypes.ArgumentError: argument 2: <class 'TypeError'>: wrong type

It looks like you need to add binary to string conversion and vice versa, which is probably easy to do.

Theoretically, this would return a list of all available file-types that libmagic can identify.

It's a feature request bug, and I hate those in my projects... so don't crucify me. It would be super-useful for my use-case though.

Hi,
I added feature to support mime_encoding recognition.
Feel free to include this commit into your branch.

My fork is not claimed to exists too long.
http://github.com/NicolasDelaby/python-magic

Regards,
Nicolas

Some people may not use MacPorts to install libmagic, but may instead use HomeBrew as an example.

magic.py looks in /opt/local/lib... whereas HomeBrew is in /usr/local/lib... Maybe make this configurable? And, how about linux?

Hi,

setup.py declare ctypes as dependency.
Ctypes is already included on standard library on version higher that python 2.5.
Furthermore Ctypes package on pypi is not installable with pip but only with easy install (due to the fact that ctypes is not needed with python 2.5 or higher).
This cause problems on package installation depending on python-magic using pip.
A possible solution could be patch setup.py like this:

try:
import ctypes
setup(name='python-magic',
......
install_requires=[],
......

except:
setup(name='python-magic',
......
install_requires=['ctypes'],
.........

Regards

Luca