Code Monkey home page Code Monkey logo

clj-holmes's Introduction

clj-holmes

A CLI SAST (Static application security testing) tool which was built with the intent of finding vulnerable Clojure code via rules that use a simple pattern language. Although finding vulnerabilities is its main purpose, clj-holmes can also be used to find any kind of code pattern.

Installation

Download the release for your OS (mac or linux), copy it to a directory in your $PATH and add executable permission to the binary.

Linux example

curl -L https://github.com/clj-holmes/clj-holmes/releases/latest/download/clj-holmes-ubuntu-latest -o /tmp/clj-holmes
sudo install -m 755 /tmp/clj-holmes /usr/local/bin/clj-holmes
rm /tmp/clj-holmes

Rules

All public rules can be found here. It is also possible to maintain your own set of rules.

clj-holmes currently supports the following rules sources:

GitHub

The GitHub wagon supports public and private repositories. In order to fetch rules from a private repository the environment GITHUB_TOKEN variable needs to be set.

To fetch a rule set clj-holmes expects a GitHub repository URL following the specification below:

git://username/project-name#branch-name

Fetching Rules

NAME:
 clj-holmes fetch-rules - Fetch rules from an external server

USAGE:
 clj-holmes fetch-rules [command options] [arguments...]

OPTIONS:
   -r, --repository S        git://clj-holmes/clj-holmes-rules#main  Repository to download rules
   -o, --output-directory S  /tmp/clj-holmes-rules/                  Directory to save rules
   -?, --help

In order to execute a scan it is necessary to fetch the rules first. This can be achieve with the following command.

clj-holmes fetch-rules

It's also possible to provide another source for a rule set by adding the -r or --repository parameter followed by the GitHub repository URL.

clj-holmes fetch-rules -r git://clj-holmes/clj-holmes-private-rules#main

Scanning a Project


NAME:
 clj-holmes scan - Performs a scan for a path

USAGE:
 clj-holmes scan [command options] [arguments...]

OPTIONS:
   -p, --scan-path S*                                                Path to scan
   -d, --rules-directory S              /tmp/clj-holmes-rules/       Directory to read rules
   -o, --output-file S                  clj_holmes_scan_results.txt  Output file
   -t, --output-type json|sarif|stdout  stdout                       Output type
   -T, --rule-tags S                                                 Only use rules with specified tags to perform the scan
   -S, --rule-severity S                                             Only use rules with specified severity to perform the scan
   -P, --rule-precision S                                            Only use rules with specified precision to perform the scan
   -i, --ignored-paths S                                             Regex for paths and files that shouldn't be scanned
   -f, --[no-]fail-on-result                                         Enable or disable fail if results were found (useful for CI/CD)
   -v, --[no-]verbose                                                Enable or disable scan process feedback.
   -?, --help

After fetching the rules, it is possible to execute a scan by providing the -p or --scan-path parameter followed by the path of the Clojure project to be scanned.

clj-holmes scan -p /tmp/clojure-project

Who uses it

Build

Steps necessary to build clj-holmes.

Dependencies

Install native image

gu install native-image

Download project dependencies

lein deps

Clean target directory

lein clean

Generate clj-holmes uberjar

lein uberjar

Generate clj-holmes native binary

lein native -H:Name=clj-holmes

clj-holmes's People

Contributors

ericdallo avatar mthbernardes avatar rancorzin avatar snyk-bot avatar svdo avatar yongyan-gh avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

clj-holmes's Issues

Provide build instructions

I'd like to build clj-watson on NixOS and maybe later add it to nixpkgs repository :)
So, maybe instructions on README or anywhere how to build clj-watson would help

Provide support to macOS

The current installation method only supports linuxes
When I try to execute clj-holmes in a mac I get the following error:

$ clj-holmes
-bash: /usr/local/bin/clj-holmes: cannot execute binary file

I believe this is caused because the binary is formated as an ELF

$ objdump -a $(which clj-holmes)

/usr/local/bin/clj-holmes:     file format elf64-x86-64
/usr/local/bin/clj-holmes

Support: can't run in docker

Hi there. I want to run clj-holmes in CircleCI. To do this I need to generate a docker image containing clj-holmes:

ARG VERSION=v1.4.2

FROM cimg/base:stable
RUN sudo apt-get update -y && sudo apt-get full-upgrade -y
RUN curl https://github.com/clj-holmes/clj-holmes/releases/download/${VERSION}/clj-holmes-ubuntu-latest -o clj-holmes
RUN chmod +x clj-holmes
RUN ./clj-holmes fetch-rules

If I follow these steps outside of Docker, it works fine, but here's what I get:

$ docker build .
Sending build context to Docker daemon  3.072kB
Step 1/6 : ARG VERSION=v1.4.2
Step 2/6 : FROM cimg/base:stable
 ---> c349986a7037
Step 3/6 : RUN sudo apt-get update -y && sudo apt-get full-upgrade -y
 ---> Using cache
 ---> 860a626be7fb
Step 4/6 : RUN curl https://github.com/clj-holmes/clj-holmes/releases/download/${VERSION}/clj-holmes-ubuntu-latest -o clj-holmes
 ---> Using cache
 ---> 9ba767b013e8
Step 5/6 : RUN chmod +x clj-holmes
 ---> Using cache
  ---> 6a66237ef2b7
Step 6/6 : RUN ./clj-holmes fetch-rules
 ---> Running in d29567fa7464
+ ./clj-holmes fetch-rules
./clj-holmes: line 1: Not: command not found
The command '/bin/bash -exo pipefail -c ./clj-holmes fetch-rules' returned a non-zero code: 127

If I run a shell in the image, I get the same error regardless of CLI args. I wonder if this could be related to cli-matic as it barely seems to reach your code? However I wanted to check if you had any ideas please before going upstream.

SARIF output issues

Hi,

We are working with the Github team on the SARIF ecosystem, looking for the ability of clj-holmes to produce SARIF reports and upload the reports to Github in a custom workflow, so customers can easily create a workflow which uses clj-holmes action and generate code scanning alert in Github security tab for each vulnerability found.

To achieve that, 3 steps need to be done:

  1. Native SARIF output support.
  2. Create official GitHub action.
  3. Create official Github starter workflow.

It's great that steps 1 and 2 are already done!
We are testing uploading SARIF report generate by the tool to Github using workflow like below:

      - name: clj-holmes (Clojure)
        uses: clj-holmes/clj-holmes-action@main
        with:
          output-type: 'sarif'
          output-file: 'results.sarif'
          fail-on-result: 'false'
      
      - name: Upload sarif to Github
        uses: github/codeql-action/upload-sarif@v1
        with:
          sarif_file: results.sarif

And the upload action failed due to the version property being 'null' in the SARIF report. Besides this we also found a couple of issues in the SARIF report and a couple of suggestions, can you please take a look?

  1. The run.tool.driver.version was set to 'null'
  2. The string values contain the escape character ''. This can be found in many properties like inofrmationUri/artifactLocation.Uri and in the text properties as well.
  3. As mentioned in this article https://github.com/microsoft/sarif-tutorials/blob/main/docs/Authoring-rule-metadata-and-result-messages.md#human-readable-identifier, the suggested rule.name should be concise and readable short string e.g. 'IdentifierIsMisspelled'. Current rule.name content is more like a short description of the rule. Consider adding proper rule names. If you don't have a suitable rule name for now you don't need to set it and use the content as rule.shortDescription.
  4. If you have web page for the rules, you can provide the link in rule.helpUri property, the various SARIF viewers are able to generate hyper-link for customers so they can easily navigate to the rule page for more detail information.
  5. You can use the latest SARIF schema https://www.schemastore.org/schemas/json/sarif-2.1.0-rtm.5.json as $schema value.
  6. Consider assigning opaque rule ids. Rule id typically consists of an abbreviation for the tool name followed by a numeric code, e.g., CS1001 SEC1004.
{
  "$schema": "https:\/\/raw.githubusercontent.com\/oasis-tcs\/sarif-spec\/master\/Documents\/CommitteeSpecifications\/2.1.0\/sarif-schema-2.1.0.json",
  "version": "2.1.0",
  "runs": [
    {
      "tool": {
        "driver": {
          "name": "clj-holmes",
          "informationUri": "https:\/\/github.com\/mthbernardes\/clj-holmes",
          "version": null,
          "rules": [
            {
              "id": "clojure-read-string",
              "name": "Usage of clojure.core\/read-string",
              "properties": {
                "precision": "medium",
                "tags": [
                  "security",
                  "vulnerability"
                ]
              },
              "fullDescription": {
                "text": "Usage of vulnerable function clojure.core\/read-string. Attackers can exploit vulnerable deserialization functions which could lead to a remote code execution."
              },
              "shortDescription": {
                "text": "Usage of vulnerable function clojure.core\/read-string. Attackers can exploit vulnerable deserialization functions which could lead to a remote code execution."
              },

I have attached a sample SARIF file generated using clj-holmes.
results.zip

Thank you very much!

Security Policy violation Branch Protection

Allstar has detected that this repository’s Branch Protection security policy is out of compliance. Status:
Dismiss stale reviews not configured for branch main

This issue will auto resolve when the policy is in compliance.

Issue created by Allstar. See https://github.com/ossf/allstar/ for more information. For questions specific to the repository, please contact the owner or maintainer.

Security Policy violation SECURITY.md

Allstar has detected that this repository’s SECURITY.md security policy is out of compliance. Status:
Security policy not enabled.
A SECURITY.md file can give users information about what constitutes a vulnerability and how to report one securely so that information about a bug is not publicly visible. Examples of secure reporting methods include using an issue tracker with private issue support, or encrypted email with a published key.

To fix this, add a SECURITY.md file that explains how to handle vulnerabilities found in your repository. Go to https://github.com/clj-holmes/clj-holmes/security/policy to enable.

For more information, see https://docs.github.com/en/code-security/getting-started/adding-a-security-policy-to-your-repository.

This issue will auto resolve when the policy is in compliance.

Issue created by Allstar. See https://github.com/ossf/allstar/ for more information. For questions specific to the repository, please contact the owner or maintainer.

Suggest another way to install clj-holmes

For a security product which suggests using sudo curl ... as a installation method is not a good thing.
This trains people to accept sudo curl ... as a installation method without thinking, and just copy paste it into their shell.

I see that what is suggested here is benign, but I'd suggest something more along the lines of

$ curl -L https://github.com/clj-holmes/clj-holmes/releases/latest/download/clj-holmes -o /tmp/clj-holmes
$ cp /tmp/clj-holmes /some/path/on/your/system/

Also, one might need to make it runnable, with chmod u+x ?

Bump check

Create a test to check if the project.clj version was changed if a *.clj file is modified/added/deleted.

java.lang.IllegalArgumentException: Value out of range for int

When we run holmes (1.4.3) against our project, at some point in the rules evaluation process, we receive a stacktrace but no further information that would allow us to debug the problem

We run

clj-holmes fetch-rules
echo "Validating rules..."
clj-holmes validate-rules
echo "Scanning << parameters.path >>"
clj-holmes scan --fail-on-result -p worker

We get:

Fetching rules from github
Done
Validating rules...
/tmp/clj-holmes-rules/clj-holmes-clj-holmes-rules-fbf5e2e/correctness/schema-require-typo.yml
Success!

/tmp/clj-holmes-rules/clj-holmes-clj-holmes-rules-fbf5e2e/security/clojure-read-string/clojure-read-string.yml
Success!

/tmp/clj-holmes-rules/clj-holmes-clj-holmes-rules-fbf5e2e/security/clojure-weak-ssl-context/weak-ssl-context.yml
Success!

/tmp/clj-holmes-rules/clj-holmes-clj-holmes-rules-fbf5e2e/security/deprecated-blowfish/deprecated-blowfish.yml
Success!

/tmp/clj-holmes-rules/clj-holmes-clj-holmes-rules-fbf5e2e/security/deprecated-desede.yml
Success!

/tmp/clj-holmes-rules/clj-holmes-clj-holmes-rules-fbf5e2e/security/ecb-mode-of-operation/ecb-mode-of-operation.yml
Success!

/tmp/clj-holmes-rules/clj-holmes-clj-holmes-rules-fbf5e2e/security/insecure-hostname-verifier.yml
Success!

/tmp/clj-holmes-rules/clj-holmes-clj-holmes-rules-fbf5e2e/security/weak-hash-function-md5.yml
Success!

/tmp/clj-holmes-rules/clj-holmes-clj-holmes-rules-fbf5e2e/security/weak-hash-function-sha1.yml
Success!

/tmp/clj-holmes-rules/clj-holmes-clj-holmes-rules-fbf5e2e/security/xxe-clojure-xml/xxe-clojure-xml.yml
Success!

Scanning worker
** ERROR: **
Exception: #error {
 :cause Value out of range for int: 2589545481
 :via
 [{:type java.util.concurrent.ExecutionException
   :message java.lang.IllegalArgumentException: Value out of range for int: 2589545481
   :at [java.util.concurrent.FutureTask report FutureTask.java 122]}
  {:type java.lang.IllegalArgumentException
   :message Value out of range for int: 2589545481
   :at [clojure.lang.RT intCast RT.java 1248]}]
 :trace
 [[clojure.lang.RT intCast RT.java 1248]
  [clojure.lang.RT intCast RT.java 1218]
  [progrock.core$interval_str invokeStatic core.clj 47]
  [progrock.core$render invokeStatic core.clj 92]
  [progrock.core$print invokeStatic core.clj 98]
  [progrock.core$print invoke core.clj 98]
  [progrock.core$print invokeStatic core.clj 102]
  [clj_holmes.logic.progress$add_watch_to_counter$fn__519 invoke progress.clj 10]
  [clojure.lang.ARef notifyWatches ARef.java 81]
  [clojure.lang.Atom swap Atom.java 41]
  [clojure.core$swap_BANG_ invokeStatic core.clj 2354]
  [clj_holmes.engine$check_rules_in_code_structure invokeStatic engine.clj 9]
  [clj_holmes.engine$scan_STAR_$fn__1238 invoke engine.clj 21]
  [clojure.core$pmap$fn__8467$fn__8468 invoke core.clj 7024]
  [clojure.core$binding_conveyor_fn$fn__5758 invoke core.clj 2032]
  [clojure.lang.AFn call AFn.java 18]
  [java.util.concurrent.FutureTask run FutureTask.java 264]
  [java.util.concurrent.ThreadPoolExecutor runWorker ThreadPoolExecutor.java 1128]
  [java.util.concurrent.ThreadPoolExecutor$Worker run ThreadPoolExecutor.java 628]
  [java.lang.Thread run Thread.java 829]
  [com.oracle.svm.core.thread.JavaThreads threadStartRoutine JavaThreads.java 596]
  [com.oracle.svm.core.posix.thread.PosixJavaThreads pthreadStartRoutine PosixJavaThreads.java 192]]

We are currently scanning our other projects successfully with the same setup, so believe this is related to the code being scanned rather than any environmental considerations. Adding the verbose flag does not return any more information about the error.

Code scanning part failing

** ERROR: **
Exception: #error {
:cause No method in multimethod 'fetch*' for dispatch value: :https
:via
[{:type java.lang.IllegalArgumentException
:message No method in multimethod 'fetch*' for dispatch value: :https
:at [clojure.lang.MultiFn getFn MultiFn.java 156]}]
:trace
[[clojure.lang.MultiFn getFn MultiFn.java 156]
[clojure.lang.MultiFn invoke MultiFn.java 233]
[clj_holmes.rules.manager$fetch invokeStatic manager.clj 16]
[clj_holmes.rules.manager$fetch invoke manager.clj 13]
[cli_matic.core$invoke_subcmd invokeStatic core.cljc 546]
[cli_matic.core$run_cmd_STAR_ invokeStatic core.cljc 578]
[cli_matic.core$run_cmd invokeStatic core.cljc 591]
[clj_holmes.main$_main invokeStatic main.clj 78]
[clj_holmes.main$_main doInvoke main.clj 78]
[clojure.lang.RestFn applyTo RestFn.java 137]
[clj_holmes.main main nil -1]]}

I can't provide you any code in this case, but do you know what can the problem be?

Bug: Value out of range for int: 3379143159

I ran this command

clj-holmes scan --rules-directory ~/d/pub/clj-holmes-rules/ --scan-path ./ --output-file ./clj-holmes-results.json --output-type sarif

and got this output ,


7.692307889461517/100     7% [===                                               ]  ETA: 4022789:30** ERROR: **
Exception: #error {
:cause Value out of range for int: 3379143159
:via
[{:type java.util.concurrent.ExecutionException
  :message java.lang.IllegalArgumentException: Value out of range for int: 3379143159
  :at [java.util.concurrent.FutureTask report FutureTask.java 122]}
 {:type java.lang.IllegalArgumentException
  :message Value out of range for int: 3379143159
  :at [clojure.lang.RT intCast RT.java 1248]}]
:trace
[[clojure.lang.RT intCast RT.java 1248]
 [clojure.lang.RT intCast RT.java 1218]
 [progrock.core$interval_str invokeStatic core.clj 47]
 [progrock.core$render invokeStatic core.clj 92]
 [progrock.core$print invokeStatic core.clj 98]
 [progrock.core$print invoke core.clj 98]
 [progrock.core$print invokeStatic core.clj 102]
 [clj_holmes.logic.progress$add_watch_to_counter$fn__519 invoke progress.clj 10]
 [clojure.lang.ARef notifyWatches ARef.java 81]
 [clojure.lang.Atom swap Atom.java 41]
 [clojure.core$swap_BANG_ invokeStatic core.clj 2354]
 [clj_holmes.engine$check_rules_in_code_structure invokeStatic engine.clj 9]
 [clj_holmes.engine$scan_STAR_$fn__1238 invoke engine.clj 21]
 [clojure.core$pmap$fn__8467$fn__8468 invoke core.clj 7024]
 [clojure.core$binding_conveyor_fn$fn__5758 invoke core.clj 2032]
 [clojure.lang.AFn call AFn.java 18]
 [java.util.concurrent.FutureTask run FutureTask.java 264]
 [java.util.concurrent.ThreadPoolExecutor runWorker ThreadPoolExecutor.java 1128]
 [java.util.concurrent.ThreadPoolExecutor$Worker run ThreadPoolExecutor.java 628]
 [java.lang.Thread run Thread.java 829]
 [com.oracle.svm.core.thread.JavaThreads threadStartRoutine JavaThreads.java 596]
 [com.oracle.svm.core.posix.thread.PosixJavaThreads pthreadStartRoutine PosixJavaThreads.java 192]]}
 

using https://github.com/clj-holmes/clj-holmes/releases/latest/download/clj-holmes-macos-latest as of today.
git clone https://github.com/clj-holmes/clj-holmes-rules as of today

Sorry I can't give you any source code.

cause Value out of range for int: 3562345207

Exception: #error {
:cause Value out of range for int: 3562345207
:via
[{:type java.util.concurrent.ExecutionException
:message java.lang.IllegalArgumentException: Value out of range for int: 3562345207
:at [java.util.concurrent.FutureTask report FutureTask.java 122]}
{:type java.lang.IllegalArgumentException
:message Value out of range for int: 3562345207
:at [clojure.lang.RT intCast RT.java 1248]}]
:trace
[[clojure.lang.RT intCast RT.java 1248]
[clojure.lang.RT intCast RT.java 1218]
[progrock.core$interval_str invokeStatic core.clj 47]
[progrock.core$render invokeStatic core.clj 92]
[progrock.core$print invokeStatic core.clj 98]
[progrock.core$print invoke core.clj 98]
[progrock.core$print invokeStatic core.clj 102]
[clj_holmes.logic.progress$add_watch_to_counter$fn__519 invoke progress.clj 10]
[clojure.lang.ARef notifyWatches ARef.java 81]
[clojure.lang.Atom swap Atom.java 41]
[clojure.core$swap_BANG_ invokeStatic core.clj 2354]
[clj_holmes.engine$check_rules_in_code_structure invokeStatic engine.clj 9]
[clj_holmes.engine$scan_STAR_$fn__1238 invoke engine.clj 21]
[clojure.core$pmap$fn__8467$fn__8468 invoke core.clj 7024]
[clojure.core$binding_conveyor_fn$fn__5758 invoke core.clj 2032]
[clojure.lang.AFn call AFn.java 18]
[java.util.concurrent.FutureTask run FutureTask.java 264]
[java.util.concurrent.ThreadPoolExecutor runWorker ThreadPoolExecutor.java 1128]
[java.util.concurrent.ThreadPoolExecutor$Worker run ThreadPoolExecutor.java 628]
[java.lang.Thread run Thread.java 829]
[com.oracle.svm.core.thread.JavaThreads threadStartRoutine JavaThreads.java 596]
[com.oracle.svm.core.posix.thread.PosixJavaThreads pthreadStartRoutine PosixJavaThreads.java 192]]}

Support case sensitive strings

Describe the bug
clj-holmes does not match case-sensitive strings for instance if you have this line of code,
(MessageDigest/getInstance "md5")
It'll not match it within the md5 rule it only matches if the "md5" is capitalized.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.