grammatech / sel Goto Github PK

Programmatic modification and evaluation of software

Home Page: https://grammatech.github.io/sel/#Top

License: Other

Common Lisp 79.48% Makefile 0.18% C 11.19% Shell 0.73% C++ 0.75% Assembly 0.14% Coq 0.14% Emacs Lisp 0.08% Io 5.12% JavaScript 0.15% Dockerfile 0.15% Python 1.83% Java 0.01% TypeScript 0.03% NewLisp 0.01% Rust 0.01%

software-engineering source-code evolutionary-computation program-synthesis

sel's Introduction

SOFTWARE-EVOLUTION-LIBRARY

The SOFTWARE-EVOLUTION-LIBRARY enables the programmatic modification and evaluation of software.

A common interface abstracts over multiple types of software objects including source code abstract syntax trees (primarily using tree-sitter), compiled assembler code, and binary software objects. Software transformation, analysis, and evaluation techniques are implemented on top of this interface supporting automation of common software engineering tasks.

See the manual, available at https://grammatech.github.io/sel.

SEL has been used to support many projects including:

The Mnemosyne automated software development assistant. https://grammatech.gitlab.io/Mnemosyne/docs/
A GitHub application for automated merge conflict resolution. https://mergeresolver.github.io
Automated Customized Bug-Benchmark Generation Source Code Analysis and Manipulation (SCAM) 2019 (distinguished paper)
Evolving Exact Decompilation Workshop on Binary Analysis Research (BAR) 2018
MuSynth: Program Synthesis via Code Reuse and Code Manipulation International Symposium on Search Based Software Engineering 2017
Repairing COTS router firmware without access to source code or test suites Workshop on Genetic Improvement 2015 (best paper)
Post-compiler Software Optimization for Reducing Energy Architectural Support for Programming Languages and Operating Systems (ASPLOS) 2014
Automated Repair of Binary and Assembly Programs for Cooperating Embedded Devices Architectural Support for Programming Languages and Operating Systems (ASPLOS) 2013
Software Mutational Robustness Genetic Programming and Evolvable Machines 2013

To cite SEL please use the following reference:

@manual{sel2018manual,
  title        = {Software Evolution Library},
  author       = {Eric Schulte and Contributors},
  organization = {GrammaTech},
  address      = {eschulte@grammatech.com},
  month        = 1,
  year         = 2018,
  note         = {https://grammatech.github.io/sel/}
}

sel's People

Contributors

Stargazers

Watchers

sel's Issues

Two systems don't build today

software-evolution-library/run-new-clang-round-trip and
software-evolution-library/run-clang-diff-test fail for me.

Build log is here: http://report.quicklisp.org/2019-12-16/failure-report/sel.html

Support ARM / apple silicon chips

At least trying to import the python asts binding, in an apple M1/2/3 laptop results into the following.

Recent versions of py-tree-sitter work just fine on any platform.

In [1]: import asts
---------------------------------------------------------------------------
OSError                                   Traceback (most recent call last)
Cell In[1], line 1
----> 1 import asts

File .venv/lib/python3.12/site-packages/asts/__init__.py:1
----> 1 from .asts import *  # noqa: F401, F403
      2 from .types import *  # noqa: F401, F403

File .venv/lib/python3.12/site-packages/asts/asts.py:710
    705                 _interface._check_for_process_crash()
    707         return response
--> 710 _interface.start()
    711 atexit.register(_interface.stop)
    713 # Generated tree-sitter AST types and user-defined method specializations

File .venv/lib/python3.12/site-packages/asts/asts.py:543, in _interface.start()
    540     cmdline = [cmd]
    542 # Startup the interface subprocess.
--> 543 _interface._proc = subprocess.Popen(
    544     cmdline,
    545     stdin=subprocess.PIPE,
    546     stdout=subprocess.PIPE,
    547     stderr=subprocess.PIPE,
    548 )
    550 # If the interface was built using the deploy package for use
    551 # in a python wheel file, read standard error waiting for the
    552 # requisite tree-sitter libraries to be loaded and the launching
    553 # application notification to be given.
    554 if cmd != _interface._DEFAULT_CMD_NAME:

File python3.12/subprocess.py:1026, in Popen.__init__(self, args, bufsize, executable, stdin, stdout, stderr, preexec_fn, close_fds, shell, cwd, env, universal_newlines, startupinfo, creationflags, restore_signals, start_new_session, pass_fds, user, group, extra_groups, encoding, errors, text, umask, pipesize, process_group)
   1022         if self.text_mode:
   1023             self.stderr = io.TextIOWrapper(self.stderr,
   1024                     encoding=encoding, errors=errors)
-> 1026     self._execute_child(args, executable, preexec_fn, close_fds,
   1027                         pass_fds, cwd, env,
   1028                         startupinfo, creationflags, shell,
   1029                         p2cread, p2cwrite,
   1030                         c2pread, c2pwrite,
   1031                         errread, errwrite,
   1032                         restore_signals,
   1033                         gid, gids, uid, umask,
   1034                         start_new_session, process_group)
   1035 except:
   1036     # Cleanup if the child failed starting.
   1037     for f in filter(None, (self.stdin, self.stdout, self.stderr)):

File python3.12/subprocess.py:1955, in Popen._execute_child(self, args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, restore_signals, gid, gids, uid, umask, start_new_session, process_group)
   1953     err_msg = os.strerror(errno_num)
   1954 if err_filename is not None:
-> 1955     raise child_exception_type(errno_num, err_msg, err_filename)
   1956 else:
   1957     raise child_exception_type(errno_num, err_msg)

OSError: [Errno 8] Exec format error: '.venv/lib/python3.12/site-packages/asts/tree-sitter-interface'

Outdated Installation instruction

I follow the installation instruction in this [https://grammatech.github.io/sel/Installation.html#Installation(https://grammatech.github.io/sel/Installation.html#Installation) to install the SEL library.

However, the instruction stated that cl-arrows recommends downloading from: https://github.com/eschulte/cl-arrows but it seems that the SEL library expects the cl-arrowz that can be downloaded from https://github.com/eschulte/cl-arrowz, which is a replacement of the cl-arrows library.

There are two possible ways to resolve this issue:

Update the installation instruction to use https://github.com/eschulte/cl-arrowz
Update the dependency to be cl-arrows instead of cl-arrowz

Some systems failed to build for Quicklisp dist

Building with SBCL 2.2.7.28-02bc916fd / ASDF 3.3.5 for quicklisp dist creation.

Trying to build commit id 556deca

software-evolution-library/run-tree-sitter-py-generator fails to build with the following error:

Unhandled SB-PCL::NO-APPLICABLE-METHOD-ERROR in thread #<SB-THREAD:THREAD "main thread" RUNNING {10016D0003}>: There is no applicable method for the generic function #<COMMON-LISP:STANDARD-GENERIC-FUNCTION FSET:CONVERT (99)> when called with arguments (PYTHON-AST "from typing import Any, List

Full log here

`asts.ASTLanguage.Python`: invalid resulting indentation for class' docstring

Have "test.py" python file:

class TestIndent:
    """\\ 
    """

Parse it:

import asts

with open("test.py") as f:
    text = f.read()

ast = asts.AST.from_string(text, language=asts.ASTLanguage.Python)
with open("test.parsed.py", "w") as f:
    f.write(ast.source_text)

The resulting "test.parsed.py" will have invalid indentation:

class TestIndent:
"""\\ 
    """

qlfile repositories point to internal GrammaTech git

From the qlfile:

git cl-arrows [email protected]:research/cl-arrows.git
git elf [email protected]:research/elf.git
git testbot [email protected]:synthesis/testbot.git
git trace-db [email protected]:research/trace-db.git
git asdf-encodings [email protected]:synthesis/asdf-encodings.git

Are all of these open source? Can we move them to github, or point them to other forks instead?
Also, can we remove testbot?

Fork CFFI

To load CFFI (and thus SEL) on a Mac M1 the latest changes from the main CFFI branch are required. Unfortunately death doesn't want to merge those into their fork of CFFI used by SEL. They suggest (see death/cffi#2 (comment)) maintaining a fork of CFFI with these changes and any others needed by SEL. You're welcome to use the fork I've started for this purpose (https://github.com/eschulte/cffi) or maybe you'd rather host one under GrammaTech on GitHub.

Implicit dependency on sb-posix

https://github.com/GrammaTech/sel/blob/master/utility/utility.lisp#L656 references sb-posix, but sb-posix is not listed as a dependency in the system file. That means loading it into sbcl will fail unless sb-posix is coincidentally already loaded.

software-evolution-library/test does not build - macro/format issue?

I get this:

; 
; caught WARNING:
;   Constant NIL conflicts with its asserted type (OR (VECTOR CHARACTER) (VECTOR NIL) BASE-STRING FUNCTION).

Full context and log is here: http://report.quicklisp.org/2019-07-23/failure-report/sel.html#software-evolution-library_test

Unable to parse sqlite3.c

Unable to parse the single-file sqlite3.c "amalgamation." The file is attached sqlite3.c.txt. The result is a single error AST.

SEL/SW/C-CPP> (defparameter sqlite (from-file (make-instance 'c) "~/Projects/sqlite-bld/sqlite3.c"))
SQLITE
SEL/SW/C-CPP> (describe (@ sqlite 0))
#<C-ERROR 1217039 :TEXT "/*******************...">
  [standard-object]

Slots with :CLASS allocation:
  CHILD-SLOTS                    = ((CHILDREN . 0))
  CHILD-SLOT-SPECIFIERS          = (#<FUNCTIONAL-TREES::SLOT-SPECIFIER CHILDREN 0>)
Slots with :INSTANCE allocation:
  STORED-HASH                    = NIL
  OID                            = 1219463
  DESCENDANT-MAP                 = #<unbound slot>
  SERIAL-NUMBER                  = 1217039
  SIZE                           = #<unbound slot>
  ANNOTATIONS                    = NIL
  BEFORE-TEXT                    = ""
  TEXT                           = "/********************************************************************..
  AFTER-TEXT                     = ""
  BEFORE-ASTS                    = NIL
  AFTER-ASTS                     = NIL
  ORDERED-CHILDREN               = NIL
  INDENT-CHILDREN                = NIL
  INDENT-ADJUSTMENT              = NIL
  CHILDREN                       = NIL
; No value

Loading test/template.lisp fails on macro-expansion of a match clause

Specifically I get the following error in my slime REPL. Commenting out the test-match-derive test allows the file to load. This is with the versions of all tree-sitter libraries listed in tools/tree-sitter-install.sh.

Error Backtrace:

; file: /Users/eschulte/quicklisp/local-projects/sel/test/template.lisp
; in: DEFTEST TEST-MATCH-DERIVE
;     (TRIVIA.LEVEL2:EMATCH (SOFTWARE-EVOLUTION-LIBRARY/TEST/TEMPLATE::MAKE-DERIVE
;                            '("PartialEq" "PartialOrd"))
;       ((SOFTWARE-EVOLUTION-LIBRARY/SOFTWARE/TREE-SITTER:RUST "#[derive(@ARGS)]"
;                                                              :ARGS
;                                                              SOFTWARE-EVOLUTION-LIBRARY/TEST/TEMPLATE::ARGS)
;        (STEFIL:IS
;         (FSET::EVERY #'SOFTWARE-EVOLUTION-LIBRARY/SOFTWARE/PARSEABLE:SOURCE-TEXT=
;                      SOFTWARE-EVOLUTION-LIBRARY/TEST/TEMPLATE::ARGS
;                      '("PartialEq" "PartialOrd")))))
; --> TRIVIA.LEVEL2:MATCH2+ 
; ==>
;   (TRIVIA.LEVEL2:MATCH2*+ ((SOFTWARE-EVOLUTION-LIBRARY/TEST/TEMPLATE::MAKE-DERIVE
;                             '("PartialEq" "PartialOrd")))
;       (T)
;     (((SOFTWARE-EVOLUTION-LIBRARY/SOFTWARE/TREE-SITTER:RUST "#[derive(@ARGS)]"
;                                                             :ARGS
;                                                             SOFTWARE-EVOLUTION-LIBRARY/TEST/TEMPLATE::ARGS))
;      (STEFIL:IS
;       (FSET::EVERY #'SOFTWARE-EVOLUTION-LIBRARY/SOFTWARE/PARSEABLE:SOURCE-TEXT=
;                    SOFTWARE-EVOLUTION-LIBRARY/TEST/TEMPLATE::ARGS
;                    '("PartialEq" "PartialOrd"))))
;     ((#:OTHERWISE1674)
;      (ERROR 'TRIVIA.LEVEL2:MATCH-ERROR :PATTERN '((# #)) :VALUES
;             (LIST #:OTHERWISE1674))))
; 
; caught ERROR:
;   during macroexpansion of
;   (MATCH2*+ (#)
;       (T)
;     ...).
;   Use *BREAK-ON-SIGNALS* to intercept.
;   
;    Unknown &KEY argument: :NAME-GENERATOR

Some systems failed to build for Quicklisp dist

Building with SBCL 2.2.1 / ASDF 3.3.5 for quicklisp dist creation.

Trying to build commit id a8e0ef7

software-evolution-library/deploy-tree-sitter-interface fails to build with the following error:

; caught ERROR:
;   during macroexpansion of (ITER (FOR AST IN-TREE ...) (MATCH AST #)). Use *BREAK-ON-SIGNALS* to intercept.
;    Failed to deduce the way to access the slot C/CPP-PATH
...
Unhandled UIOP/LISP-BUILD:COMPILE-FILE-ERROR in thread #<SB-THREAD:THREAD "main thread" RUNNING {1001B98003}>: COMPILE-FILE-ERROR while compiling #<CL-SOURCE-FILE "software-evolution-library/software/cpp" "file-type">

software-evolution-library/run-dump-store fails to build with the following error:

; caught ERROR:
;   during macroexpansion of (ITER (FOR AST IN-TREE ...) (MATCH AST #)). Use *BREAK-ON-SIGNALS* to intercept.
;    Failed to deduce the way to access the slot C/CPP-PATH
...
Unhandled UIOP/LISP-BUILD:COMPILE-FILE-ERROR in thread #<SB-THREAD:THREAD "main thread" RUNNING {10418AC073}>: COMPILE-FILE-ERROR while compiling #<CL-SOURCE-FILE "software-evolution-library/software/cpp" "file-type">

software-evolution-library/run-rest-server fails to build with the following error:

; caught ERROR:
;   during macroexpansion of (ITER (FOR AST IN-TREE ...) (MATCH AST #)). Use *BREAK-ON-SIGNALS* to intercept.
;    Failed to deduce the way to access the slot C/CPP-PATH
...
Unhandled UIOP/LISP-BUILD:COMPILE-FILE-ERROR in thread #<SB-THREAD:THREAD "main thread" RUNNING {1001B98003}>: COMPILE-FILE-ERROR while compiling #<CL-SOURCE-FILE "software-evolution-library/software/cpp" "file-type">

software-evolution-library/run-test-parse fails to build with the following error:

; caught ERROR:
;   during macroexpansion of (ITER (FOR AST IN-TREE ...) (MATCH AST #)). Use *BREAK-ON-SIGNALS* to intercept.
;    Failed to deduce the way to access the slot C/CPP-PATH
...
Unhandled UIOP/LISP-BUILD:COMPILE-FILE-ERROR in thread #<SB-THREAD:THREAD "main thread" RUNNING {1001B98003}>: COMPILE-FILE-ERROR while compiling #<CL-SOURCE-FILE "software-evolution-library/software/cpp" "file-type">

software-evolution-library/run-tree-sitter-interface fails to build with the following error:

; caught ERROR:
;   during macroexpansion of (ITER (FOR AST IN-TREE ...) (MATCH AST #)). Use *BREAK-ON-SIGNALS* to intercept.
;    Failed to deduce the way to access the slot C/CPP-PATH
...
Unhandled UIOP/LISP-BUILD:COMPILE-FILE-ERROR in thread #<SB-THREAD:THREAD "main thread" RUNNING {1001B98003}>: COMPILE-FILE-ERROR while compiling #<CL-SOURCE-FILE "software-evolution-library/software/cpp" "file-type">

software-evolution-library/run-tree-sitter-py-generator fails to build with the following error:

; caught ERROR:
;   during macroexpansion of (ITER (FOR AST IN-TREE ...) (MATCH AST #)). Use *BREAK-ON-SIGNALS* to intercept.
;    Failed to deduce the way to access the slot C/CPP-PATH
...
Unhandled UIOP/LISP-BUILD:COMPILE-FILE-ERROR in thread #<SB-THREAD:THREAD "main thread" RUNNING {1001B98003}>: COMPILE-FILE-ERROR while compiling #<CL-SOURCE-FILE "software-evolution-library/software/cpp" "file-type">

Full log here

Some systems failed to build for Quicklisp dist

Building with SBCL 2.0.5 / ASDF 3.3.1 for quicklisp dist creation.

Trying to build commit id e7a6ae7

software-evolution-library/run-dump-store fails to build with the following error:

Unhandled ASDF/FIND-COMPONENT:MISSING-DEPENDENCY in thread #<SB-THREAD:THREAD "main thread" RUNNING {10008D8083}>: Component "cl-tree-sitter" not found, required by #<PACKAGE-INFERRED-SYSTEM "software-evolution-library/software/tree-sitter">

software-evolution-library/run-rest-server fails to build with the following error:

Unhandled ASDF/FIND-COMPONENT:MISSING-DEPENDENCY in thread #<SB-THREAD:THREAD "main thread" RUNNING {10008D8083}>: Component "cl-tree-sitter" not found, required by #<PACKAGE-INFERRED-SYSTEM "software-evolution-library/software/tree-sitter">

Full log here

Some systems failed to build for Quicklisp dist

Building with SBCL 2.4.6 / ASDF 3.3.5 for quicklisp dist creation.

Trying to build commit id 9e564f8

software-evolution-library/deploy-tree-sitter-interface fails to build with the following error:

Unhandled SB-PCL::FIND-METHOD-LENGTH-MISMATCH in thread #<SB-THREAD:THREAD tid=3249693 "main thread" RUNNING {1000DC8003}>: The generic function #<COMMON-LISP:STANDARD-GENERIC-FUNCTION ECLECTOR.PARSE-RESULT:MAKE-SKIPPED-INPUT-RESULT (1)> takes 5 required arguments; was asked to find a method with specializers (CLIENT T T T)

software-evolution-library/run-dump-store fails to build with the following error:

Unhandled SB-PCL::FIND-METHOD-LENGTH-MISMATCH in thread #<SB-THREAD:THREAD tid=3249681 "main thread" RUNNING {103FC70093}>: The generic function #<COMMON-LISP:STANDARD-GENERIC-FUNCTION ECLECTOR.PARSE-RESULT:MAKE-SKIPPED-INPUT-RESULT (1)> takes 5 required arguments; was asked to find a method with specializers (CLIENT T T T)

software-evolution-library/run-rest-server fails to build with the following error:

Unhandled SB-PCL::FIND-METHOD-LENGTH-MISMATCH in thread #<SB-THREAD:THREAD tid=3249685 "main thread" RUNNING {103FE08093}>: The generic function #<COMMON-LISP:STANDARD-GENERIC-FUNCTION ECLECTOR.PARSE-RESULT:MAKE-SKIPPED-INPUT-RESULT (1)> takes 5 required arguments; was asked to find a method with specializers (CLIENT T T T)

software-evolution-library/run-test-parse fails to build with the following error:

Unhandled SB-PCL::FIND-METHOD-LENGTH-MISMATCH in thread #<SB-THREAD:THREAD tid=3249689 "main thread" RUNNING {1000DC8003}>: The generic function #<COMMON-LISP:STANDARD-GENERIC-FUNCTION ECLECTOR.PARSE-RESULT:MAKE-SKIPPED-INPUT-RESULT (1)> takes 5 required arguments; was asked to find a method with specializers (CLIENT T T T)

software-evolution-library/run-tree-sitter-interface fails to build with the following error:

Unhandled SB-PCL::FIND-METHOD-LENGTH-MISMATCH in thread #<SB-THREAD:THREAD tid=3249701 "main thread" RUNNING {1000DC8003}>: The generic function #<COMMON-LISP:STANDARD-GENERIC-FUNCTION ECLECTOR.PARSE-RESULT:MAKE-SKIPPED-INPUT-RESULT (1)> takes 5 required arguments; was asked to find a method with specializers (CLIENT T T T)

software-evolution-library/run-tree-sitter-py-generator fails to build with the following error:

Unhandled SB-PCL::FIND-METHOD-LENGTH-MISMATCH in thread #<SB-THREAD:THREAD tid=3249697 "main thread" RUNNING {1000DC8003}>: The generic function #<COMMON-LISP:STANDARD-GENERIC-FUNCTION ECLECTOR.PARSE-RESULT:MAKE-SKIPPED-INPUT-RESULT (1)> takes 5 required arguments; was asked to find a method with specializers (CLIENT T T T)

Full log here

Some systems failed to build for Quicklisp dist

Building with SBCL 2.1.10.195-bca540cd6 / ASDF 3.3.5 for quicklisp dist creation.

Trying to build commit id e5b4a22

software-evolution-library/deploy-tree-sitter-interface fails to build with the following error:

; caught ERROR:
;   during macroexpansion of (MATCH2*+ (JSON) (T) ...). Use *BREAK-ON-SIGNALS* to intercept.
;    Symbol JSON-DOCUMENT is unbound in namespace PATTERN
...
Unhandled UIOP/LISP-BUILD:COMPILE-FILE-ERROR in thread #<SB-THREAD:THREAD "main thread" RUNNING {1001BFC103}>: COMPILE-FILE-ERROR while compiling #<CL-SOURCE-FILE "software-evolution-library/software/javascript-project" "file-type">

software-evolution-library/run-dump-store fails to build with the following error:

; caught ERROR:
;   during macroexpansion of (MATCH2*+ (JSON) (T) ...). Use *BREAK-ON-SIGNALS* to intercept.
;    Symbol JSON-DOCUMENT is unbound in namespace PATTERN
...
Unhandled UIOP/LISP-BUILD:COMPILE-FILE-ERROR in thread #<SB-THREAD:THREAD "main thread" RUNNING {10418EC4A3}>: COMPILE-FILE-ERROR while compiling #<CL-SOURCE-FILE "software-evolution-library/software/javascript-project" "file-type">

software-evolution-library/run-rest-server fails to build with the following error:

; caught ERROR:
;   during macroexpansion of (MATCH2*+ (JSON) (T) ...). Use *BREAK-ON-SIGNALS* to intercept.
;    Symbol JSON-DOCUMENT is unbound in namespace PATTERN
...
Unhandled UIOP/LISP-BUILD:COMPILE-FILE-ERROR in thread #<SB-THREAD:THREAD "main thread" RUNNING {1001BFC103}>: COMPILE-FILE-ERROR while compiling #<CL-SOURCE-FILE "software-evolution-library/software/javascript-project" "file-type">

software-evolution-library/run-test-parse fails to build with the following error:

; caught ERROR:
;   during macroexpansion of (MATCH2*+ (JSON) (T) ...). Use *BREAK-ON-SIGNALS* to intercept.
;    Symbol JSON-DOCUMENT is unbound in namespace PATTERN
...
Unhandled UIOP/LISP-BUILD:COMPILE-FILE-ERROR in thread #<SB-THREAD:THREAD "main thread" RUNNING {1001BFC103}>: COMPILE-FILE-ERROR while compiling #<CL-SOURCE-FILE "software-evolution-library/software/javascript-project" "file-type">

software-evolution-library/run-tree-sitter-interface fails to build with the following error:

; caught ERROR:
;   during macroexpansion of (MATCH2*+ (JSON) (T) ...). Use *BREAK-ON-SIGNALS* to intercept.
;    Symbol JSON-DOCUMENT is unbound in namespace PATTERN
...
Unhandled UIOP/LISP-BUILD:COMPILE-FILE-ERROR in thread #<SB-THREAD:THREAD "main thread" RUNNING {1001BFC103}>: COMPILE-FILE-ERROR while compiling #<CL-SOURCE-FILE "software-evolution-library/software/javascript-project" "file-type">

software-evolution-library/run-tree-sitter-py-generator fails to build with the following error:

; caught ERROR:
;   during macroexpansion of (MATCH2*+ (JSON) (T) ...). Use *BREAK-ON-SIGNALS* to intercept.
;    Symbol JSON-DOCUMENT is unbound in namespace PATTERN
...
Unhandled UIOP/LISP-BUILD:COMPILE-FILE-ERROR in thread #<SB-THREAD:THREAD "main thread" RUNNING {1001BFC103}>: COMPILE-FILE-ERROR while compiling #<CL-SOURCE-FILE "software-evolution-library/software/javascript-project" "file-type">

Full log here

a weird problem when performing transformation

Hi there,

It is a great project, which makes up the transformation ability that the original tree-sitter haven't had yet.

Here I would like to perform a transformation that removing the type descriptor of the cast expression.
For example, given the input snippet as follows (which has two nested cast expressions in the corresponding AST):

void foo(byte *param)
{
  sVar = (char *)xmalloc((int)param + 8);
}

, the expected output should be

void foo(byte *param)
{
  sVar = xmalloc(param + 8);
}

To this end, I write the code as follows:

import asts


code = \
"""
void foo(byte *param)
{
  sVar = (char *)xmalloc((int)param + 8);
}
"""


def remove_cast(ast: asts.AST):
    if isinstance(ast, asts.CCastExpression):
        print('TRANS:', ast.source_text, '->', ast.value.source_text)
        return ast.value


def main():
    root = asts.AST.from_string(code, asts.ASTLanguage.C, deepest=True)
    new_root = asts.AST.transform(root, remove_cast)
    print('New Code:\n', new_root.source_text)


if __name__ == '__main__':
    main()

But the output is

TRANS: (char *)xmalloc((int)param + 8) -> xmalloc((int)param + 8)
TRANS: (int)param -> param
New Code:
 param

, there is only an identifier in the generated code, which is unexpected.

After some trials, I found it can work well under the situation that a cast expression doesn't contain other cast expressions.
For example, if the input is

void foo(byte *param)
{
  sVar = (char *)xmalloc(param + 8);
}

the output is

void foo(byte *param)
{
  sVar = xmalloc(param + 8);
}

, which is expected.

Now I have no idea about how to deal with this problem... Can you help me please?

Thanks!

Some systems failed to build for Quicklisp dist

Building with SBCL 2.0.5 / ASDF 3.3.1 for quicklisp dist creation.

Trying to build commit id e56fcf5

software-evolution-library/deploy-tree-sitter-interface fails to build with the following error:

Unhandled SB-PCL:CLASS-NOT-FOUND-ERROR in thread #<SB-THREAD:THREAD "main thread" RUNNING {1000A18083}>: There is no class named SOFTWARE-EVOLUTION-LIBRARY/SOFTWARE/TREE-SITTER::C-FUNCTION-DEFINITION.

software-evolution-library/run-dump-store fails to build with the following error:

Unhandled SB-PCL:CLASS-NOT-FOUND-ERROR in thread #<SB-THREAD:THREAD "main thread" RUNNING {1000A18083}>: There is no class named SOFTWARE-EVOLUTION-LIBRARY/SOFTWARE/TREE-SITTER::C-FUNCTION-DEFINITION.

software-evolution-library/run-rest-server fails to build with the following error:

Unhandled SB-PCL:CLASS-NOT-FOUND-ERROR in thread #<SB-THREAD:THREAD "main thread" RUNNING {1000A18083}>: There is no class named SOFTWARE-EVOLUTION-LIBRARY/SOFTWARE/TREE-SITTER::C-FUNCTION-DEFINITION.

software-evolution-library/run-test-parse fails to build with the following error:

Unhandled SB-PCL:CLASS-NOT-FOUND-ERROR in thread #<SB-THREAD:THREAD "main thread" RUNNING {1000A18083}>: There is no class named SOFTWARE-EVOLUTION-LIBRARY/SOFTWARE/TREE-SITTER::C-FUNCTION-DEFINITION.

software-evolution-library/run-tree-sitter-interface fails to build with the following error:

Unhandled SB-PCL:CLASS-NOT-FOUND-ERROR in thread #<SB-THREAD:THREAD "main thread" RUNNING {1000A18083}>: There is no class named SOFTWARE-EVOLUTION-LIBRARY/SOFTWARE/TREE-SITTER::C-FUNCTION-DEFINITION.

Full log here

Request to add example code / information from Evolving Exact Decompilation (BED)

As someone in the decompilation community, I'm very interested in doing my own research and exploration into this field. I was hoping some more specific information could be shared here (or elsewhere) about how sel and any other tools were leveraged in your research project.

Particularly, I am interested in performing a similar experiment but with MIPS-compiled c programs. If it makes more sense for me to reach out to someone privately and discuss elsewhere, happy to do that as well. Thanks in advance for your consideration.

Error in run-rest-server and run-dump-store

I get this: Unhandled SB-INT:INVALID-ARRAY-INDEX-ERROR in thread #<SB-THREAD:THREAD "main thread" RUNNING {1000A18083}>: Invalid index 65 for (SIMPLE-VECTOR 1), should be a non-negative integer below 1.

Full log here: http://report.quicklisp.org/2020-05-30/failure-report/sel.html

software-evolution-library/test fails with "return for unknown block: REST-SERVER"

Log is here: http://report.quicklisp.org/2019-03-08/failure-report/sel.html#software-evolution-library_test

Does not build without a git checkout?

Log is here: http://report.quicklisp.org/2020-01-18/failure-report/sel.html#software-evolution-library

I get this: Git failed: (:ABSOLUTE "home" "quicklisp" "quicklisp-controller" "dist" "build-cache" "sel" "ccbcc6b0d57db2cbc2c5290543ecfbed79846f25" "sel-20200118-git") is not in a git repository.

System "gt/full" not found

SEL used to work fine but recently I cannot load it and I get this error: 'System "gt/full" not found'
This can be found in the quicklisp file: "https://git.grammatech.com/synthesis/gt.git" which is a repo which is not accessible [publicly].

Updates to eclector break some systems

If you use the latest eclector, the following systems no longer build:

software-evolution-library/run-new-clang-round-trip
software-evolution-library/run-dump-store
software-evolution-library/run-clang-diff-test

I get this:
USE-PACKAGE #<PACKAGE "ECLECTOR.PARSE-RESULT"> causes name-conflicts in #<PACKAGE "SOFTWARE-EVOLUTION-LIBRARY/SOFTWARE/LISP"> between the following symbols: ECLECTOR.PARSE-RESULT:READ-PRESERVING-WHITESPACE, COMMON-LISP:READ-PRESERVING-WHITESPACE

Python library: should depend on setuptools or otherwise replace `pkg_resources`

The python asts package depends internally on pkg_resources (which is deprecated).

If still used, at least the setuptools package should be declared as a dependency. Otherwise, trying to pip-install & import asts, results in:

ModuleNotFoundError: No module named 'pkg_resources'

Failed to load "tree-sitter"

Hi, I'm having some trouble trying to run the examples.

Specifically, when I try to load the 003-neutral.lisp example, the lisp interpreter throws No package named "SOFTWARE-EVOLUTION-LIBRARY/SOFTWARE/TREE-SITTER.
So I manually load (ql:quickload :software-evolution-library/software/tree-sitter), but the interpreter says

> Error: The value :LOCAL-NICKNAMES is not of the expected type (MEMBER :USE :MIX :REEXPORT :USE-REEXPORT :MIX-REEXPORT :IMPORT-FROM :SHADOWING-IMPORT-FROM :NICKNAMES :DOCUMENTATION :SHADOW :EXPORT :INTERN :UNINTERN :RECYCLE).
> While executing: ASDF/PACKAGE-INFERRED-SYSTEM::PACKAGE-DEPENDENCIES, in process listener(1).

Also, the same error occurs when loading "(ql:quickload:software-evolution-library/software/clang)".

I've been haunted for a long time, so I decided to ask you for help.

Thanks.
Bo.

grammatech / sel Goto Github PK

sel's Introduction

SOFTWARE-EVOLUTION-LIBRARY

sel's People

Contributors

Stargazers

Watchers

Forkers

sel's Issues

Recommend Projects

Recommend Topics

Recommend Org