mjdorma / yara-ctypes Goto Github PK

View Code? Open in Web Editor NEW

32.0 32.0 15.0 1.84 MB

A Python ctypes package for libyara

Home Page: http://plusvic.github.io/yara/

License: MIT License

Shell 3.96% Python 96.04%

yara-ctypes's People

Contributors

Stargazers

Watchers

Forkers

liamjm aliles emats fidelissetuid0 pombredanne s55-labs xdancho pombreda mjlock soumy caidongyun

yara-ctypes's Issues

SyntaxError: ("name 'scanner' is local and global",)

When installing yara 1.7.5 with pip, it seems to install correctly, but displays a SyntaxError.

Installing collected packages: yara
  Running setup.py install for yara
    SyntaxError: ("name 'scanner' is local and global",)


    Installing yara-ctypes script to /home/nick/msvirtualenv/bin
Successfully installed yara
Cleaning up...

[Errno 1] Operation not permitted: '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/libyara.so'

setup.py install returns this error

[Errno 1] Operation not permitted: '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/libyara.so'

Python3 unicode rule decoding error (with suggested patch)

Was getting an error such this:

Traceback (most recent call last):
  File "Oik.py", line 86, in checkOik
    match = rules.match(filename)
  File "c:\Python33\lib\site-packages\yara\rules.py", line 352, in match
    return self.match_path(filepath, **match_kwargs)
  File "c:\Python33\lib\site-packages\yara\rules.py", line 276, in match_path
    callback=callback)
  File "c:\Python33\lib\site-packages\yara\rules.py", line 186, in match
    raise YaraCallbackError(msg)
yara.libyara_wrapper.YaraCallbackError: Error in callback handler:
Traceback (most recent call last):
  File "c:\Python33\lib\site-packages\yara\rules.py", line 82, in _callback
    match = self._process_rule(rule)
  File "c:\Python33\lib\site-packages\yara\rules.py", line 134, in _process_rule
    match.contents.length))
  File "c:\Python33\lib\site-packages\yara\libyara_wrapper.py", line 352, in frombyte
    return str(s, 'utf-8')
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x89 in position 1: invalid start byte

with a rule such this (truncated from real rule):

rule ManyHoursLater 
{
    strings:
        $manyhourslater = { 48 89 5c  }

    condition:
        $manyhourslater
}

The fix is line 352 in libyara_wrapper.py which should be this:

return str(s.decode(encoding='utf-8',errors='ignore'))

also likely wise to fix line 343 in libyara_wrapper.py with as well:

return s.encode('utf-8',errors='ignore')

Tested and worked there after..

Callback verification improvement

In rules.py line 153 the test for callback function is incomplete.
if type(callback) is not types.FunctionType:

This will only test if the callback is of type function and will fail if you use a method.
Using a method as callback have been tested and it seems to run nicely.

The following change is proposed:

if type(callback) is not types.FunctionType and type(callback) is not types.MethodType:

distribute versions

When I install the yara package into a virtualenv on my Ubuntu 12.04 system, it complains that the distribute version is too old (it is 0.6.24 and yara wants >=0.6.25).

This is what happens when I try to install yara without first upgrading distribute

$ pip install yara
Downloading/unpacking yara
  Downloading yara-1.7.5.tar.gz (392Kb): 392Kb downloaded
  Running setup.py egg_info for package yara
    The required version of distribute (>=0.6.25) is not available,
    and can't be installed while this script is running. Please
    install a more recent version first, using
    'easy_install -U distribute'.

    (Currently using distribute 0.6.24 (/home/nick/msvirtualenv/lib/python2.7/site-packages/distribute-0.6.24-py2.7.egg))
    Complete output from command python setup.py egg_info:
    The required version of distribute (>=0.6.25) is not available,

and can't be installed while this script is running. Please

install a more recent version first, using

'easy_install -U distribute'.



(Currently using distribute 0.6.24 (/home/nick/msvirtualenv/lib/python2.7/site-packages/distribute-0.6.24-py2.7.egg))

----------------------------------------
Command python setup.py egg_info failed with error code 2 in /home/nick/msvirtualenv/build/yara
Storing complete log in /home/nick/.pip/pip.log

Ok, so, I have to upgrade distribute prior to installing the yara package.
pip install -U distribute

Then I can install the yara package.
pip install yara

What I don't understand is when I install yara, after upgrading distribute, why does the yara setup also download and build the distribute 0.6.25 egg?

If distribute >= 0.6.25 is a hard requirement to install the yara package, why can't the yara installer just use the version of distribute that I already have?

If for some reason the yara installer cannot use the distribute version that I already have, even if it is newer than 0.6.25, why is there a requirement for my system to have that newer version of distribute prior to installing the yara package? Especially if yara is going to download its own distribute version anyway.

I'm referring to what I see in the output here during a successful yara installation:

Downloading/unpacking yara (from -r requirements.txt (line 10))
  Downloading yara-1.7.5.tar.gz (392Kb): 392Kb downloaded
  Running setup.py egg_info for package yara
    Downloading http://pypi.python.org/packages/source/d/distribute/distribute-0.6.25.tar.gz
    Extracting in /tmp/tmpgS34Y_
    Now working in /tmp/tmpgS34Y_/distribute-0.6.25
    Building a Distribute egg in /home/nick/msvirtualenv/build/yara
    /home/nick/msvirtualenv/build/yara/distribute-0.6.25-py2.7.egg

As a separate, but related question, why does it require distribute of a specific version? With the recent re-integration of distribute and setuptools, can it also use setuptools?

Thanks,
Nick

easy_install sandbox failure

When installing yara with easy_install a Sandbox error occurs. The obvious answer is use pip. But I'm including yara as a dependency in my setup.py for a PyPI package publish of workbench (https://github.com/SuperCowPowers/workbench). So because that process uses easy_install (and I'm not aware of a way to get around that.. did lots of googling :), then I'm stuck. Here's the dump from easy_install...

% easy_install yara
Searching for yara
Reading https://pypi.python.org/simple/yara/
Reading http://code.google.com/p/yara-project/
Best match: yara 1.7.7
Downloading https://pypi.python.org/packages/source/y/yara/yara-1.7.7.tar.gz#md5=87f615af427bab78adf26c74099b8690
Processing yara-1.7.7.tar.gz
Writing /var/folders/3g/174tmxyd58b18l8s2bx2p5h80000gn/T/easy_install-uo2ihj/yara-1.7.7/setup.cfg
Running yara-1.7.7/setup.py -q bdist_egg --dist-dir       /var/folders/3g/174tmxyd58b18l8s2bx2p5h80000gn/T/easy_install-uo2ihj/yara-1.7.7/egg-dist-tmp-r9lBB4
error: Setup script exited with error: SandboxViolation:     open('/Users/briford/myPVE/test/bin/../lib/libyara.so', 'wb') {}

The package setup script has attempted to modify files on your system
that are not within the EasyInstall build area, and has been aborted.

This package cannot be safely installed by EasyInstall, and may not
support alternate installation locations even if you run its setup
script by hand.  Please inform the package's author and the EasyInstall
maintainers to find out if a fix or workaround is available.

preprocessor.py unexpected behavior with .yara rulefile extensions

Discovered an odd behavior in this yara python package today. Its common for yara rule files to have the extension .yar or .yara, however the preprocessor here seems to error out on rulefiles with the .yara extension.

Version: 1.7.7

Component: preprocessor.py

Expected Behavior: preprocess yara rules files with extension .yar or .yara

>>> rules = yara.compile("/opt/fsf/fsf-server/yara/rules.yara")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.7/site-packages/yara/rules.py", line 464, in compile
    rules = Rules(**kwargs)
  File "/usr/lib/python2.7/site-packages/yara/rules.py", line 226, in __init__
    string = preprocess(path, defines, include_path)
  File "/usr/lib/python2.7/site-packages/yara/preprocessor.py", line 69, in preprocess
    _process(infile, output, copy.copy(defines), include_paths, paths)
  File "/usr/lib/python2.7/site-packages/yara/preprocessor.py", line 128, in _process
    raise PreprocessError(msg, defines)
yara.preprocessor.PreprocessError: /opt/fsf/fsf-server/yara/rules.yara:1:Invalid file extension '.yara'.Can only include .yar

OS X libyara.so is missing yr_free_matches patch

The libyara.so for OS X seems to be missing the yr_free_matches patch. I'll get onto building the latest.

Error on centos:"undefined symbol: lookup_rule"

I'm running yara 3.2.0 on CentOS:

[server]$ yara -v
yara 3.2.0

I'm also using python 2.7

When I try to do anything with yara-ctypes I get the following error:

[server]$ yara-ctypes -h
Traceback (most recent call last):
File "/usr/local/bin/yara-ctypes", line 9, in
load_entry_point('yara==1.7.7', 'console_scripts', 'yara-ctypes')()
File "build/bdist.linux-x86_64/egg/pkg_resources.py", line 339, in load_entry_point
File "build/bdist.linux-x86_64/egg/pkg_resources.py", line 2470, in load_entry_point
File "build/bdist.linux-x86_64/egg/pkg_resources.py", line 2184, in load
File "/usr/local/lib/python2.7/site-packages/yara-1.7.7-py2.7.egg/yara/init.py", line 7, in
from yara.rules import compile
File "/usr/local/lib/python2.7/site-packages/yara-1.7.7-py2.7.egg/yara/rules.py", line 17, in
from yara.libyara_wrapper import *
File "/usr/local/lib/python2.7/site-packages/yara-1.7.7-py2.7.egg/yara/libyara_wrapper.py", line 362, in
libyaradll.lookup_rule.restype = POINTER(RULE)
File "/usr/local/lib/python2.7/ctypes/init.py", line 378, in getattr
func = self.getitem(name)
File "/usr/local/lib/python2.7/ctypes/init.py", line 383, in getitem
func = self._FuncPtr((name_or_ordinal, self))
AttributeError: /usr/local/lib/libyara.so: undefined symbol: lookup_rule

Syntax error: unexpected '*'

When using the latest update, I am receiving the following error message:

Failed to load rules with the following error(s):
syntax error, unexpected '*', expecting CONDITION
unterminated string

You could blacklist the erroneous rules using:
--blacklist=

This message does not appear when using the the revision before the latest update. I have narrowed down the issue that is causing this in my .yar file. If you put comments inside the rule and have one of your strings have comment delimiters, the code is looking for the opening and closing comment delimiters. Below is my example rule file that is causing the issue:

rule testRule1
{
meta:
author="Test"
comment="Test 2"
/*
Some comments
/
strings:
$str79="Accept: */"

condition:
    all of them

}

How to solve this problem

Failed to import 'E:\Tools\python\DLLs\libyara.dll'

match_path doesn't work

(Pdb) yara_rules.match_path("/tmp/ls")
{}
(Pdb) yara_rules.match_path("/tmp/")
*** yara.libyara_wrapper.YaraMatchError: Could not open file '/tmp/'

Wheels are built with the library in the wrong place

I don't know off-hand why this is broken, but when yara is built as a wheel, the library gets placed at an incorrect path, which results in a broken installation.

I suspect that this ansible issue is describing the same problem, so the notes there about absolute vs. relative paths may be relevant.

With wheel (broken):

$ virtualenv with-wheel && with-wheel/bin/pip install --upgrade pip && with-wheel/bin/pip install wheel && with-wheel/bin/pip install yara && find with-wheel -name libyara.so
Running virtualenv with interpreter /usr/bin/python2
New python executable in with-wheel/bin/python2
Also creating executable in with-wheel/bin/python
Installing setuptools, pip...done.
Downloading/unpacking pip from https://pypi.python.org/packages/9c/32/004ce0852e0a127f07f358b715015763273799bd798956fa930814b60f39/pip-8.1.2-py2.py3-none-any.whl#md5=0570520434c5b600d89ec95393b2650b
  Downloading pip-8.1.2-py2.py3-none-any.whl (1.2MB): 1.2MB downloaded
Installing collected packages: pip
  Found existing installation: pip 1.5.6
    Uninstalling pip:
      Successfully uninstalled pip
Successfully installed pip
Cleaning up...
Collecting wheel
  Downloading wheel-0.29.0-py2.py3-none-any.whl (66kB)
    100% |████████████████████████████████| 71kB 859kB/s 
Installing collected packages: wheel
Successfully installed wheel-0.29.0
Collecting yara
  Using cached yara-1.7.7.tar.gz
Building wheels for collected packages: yara
  Running setup.py bdist_wheel for yara ... done
  Stored in directory: /home/myuser/.cache/pip/wheels/75/07/12/b7042cb4c8ec9b99f5e6a320fbabd92bbdf349dbe8d54f816a
Successfully built yara
Installing collected packages: yara
Successfully installed yara-1.7.7
with-wheel/lib/python2.7/site-packages/tmp/test/with-wheel/lib/libyara.so

The normal installation process works fine:

virtualenv without-wheel && without-wheel/bin/pip install --upgrade pip && without-wheel/bin/pip install yara && find without-wheel -name libyara.so
Running virtualenv with interpreter /usr/bin/python2
New python executable in without-wheel/bin/python2
Also creating executable in without-wheel/bin/python
Installing setuptools, pip...done.
Downloading/unpacking pip from https://pypi.python.org/packages/9c/32/004ce0852e0a127f07f358b715015763273799bd798956fa930814b60f39/pip-8.1.2-py2.py3-none-any.whl#md5=0570520434c5b600d89ec95393b2650b
  Downloading pip-8.1.2-py2.py3-none-any.whl (1.2MB): 1.2MB downloaded
Installing collected packages: pip
  Found existing installation: pip 1.5.6
    Uninstalling pip:
      Successfully uninstalled pip
Successfully installed pip
Cleaning up...
Collecting yara
  Using cached yara-1.7.7.tar.gz
Installing collected packages: yara
  Running setup.py install for yara ... done
Successfully installed yara-1.7.7
without-wheel/lib/libyara.so