Add capability for rules to depend on other rules (that may have applied to different files) about applicationinspector HOT 8 CLOSED

@valinha

Glad to hear that workaround will work for you.

I'll raise your feature request for paired rules that apply to different files in the same scan with the team at our next meeting.

from applicationinspector.

Hello Valinha,

I believe this is currently possible to accomplish with the conditions field. After a pattern is matched, all conditions must all be met. Conditions can either be negative (if matched they cancel the match) or positive (they must match in order to validate the match).

https://github.com/microsoft/ApplicationInspector/wiki/3.3.-Condition-Object-Schema

from applicationinspector.

Here's an example adapted from the DevSkim ruleset

This example requires both cheese and toast to be in the same file.

{
    "name": "Require cheese for toast",
    "id": "ConditionExample",
    "description": "Found both cheese and toast in the same file.",
    "tags": [
        "Toast.Cheese"
    ],
    "severity": "moderate",
    "patterns": [
        {
            "pattern": "toast",
            "type": "string",
            "scopes": [
                "code"
            ]
        }
    ],
    "conditions" : [
        {
            "pattern" : 
            {
                "pattern": "cheese",
                "type": "string",
                "scopes": [
                    "code"
                ]
            },
            "negate_finding": false,
            "search_in": "same-file"
        }
    ],
    "must-match": [
        "cheese and toast"
    ],
    "must-not-match": [
        "cheese and wine"
    ]
}

This example allows requires toast with either butter or cheese

{
    "name": "Require cheese or butter for toast",
    "id": "ConditionExample",
    "description": "Found both cheese or butter and toast in the same file.",
    "tags": [
        "Toast.ButterOrCheese"
    ],
    "severity": "moderate",
    "patterns": [
        {
            "pattern": "butter",
            "type": "string",
            "scopes": [
                "code"
            ]
        },
        {
            "pattern": "cheese",
            "type": "string",
            "scopes": [
                "code"
            ]
        }
    ],
    "conditions" : [
        {
            "pattern" : 
            {
                "pattern": "toast",
                "type": "string",
                "scopes": [
                    "code"
                ]
            },
            "negate_finding": false,
            "search_in": "same-file"
        }
    ],
    "must-match": [
        "cheese and toast",
        "butter and toast",
        "cheese and butter and toast"
    ],
    "must-not-match": [
        "cheese and wine"
    ]
}

This example requires toast with both butter and cheese

{
    "name": "Require cheese or butter for toast",
    "id": "ConditionExample",
    "description": "Found both cheese or butter and toast in the same file.",
    "tags": [
        "Toast.ButterOrCheese"
    ],
    "severity": "moderate",
    "patterns": [
        {
            "pattern": "toast",
            "type": "string",
            "scopes": [
                "code"
            ]
        }
    ],
    "conditions" : [
        {
            "pattern" : 
            {
                "pattern": "cheese",
                "type": "string",
                "scopes": [
                    "code"
                ]
            },
            "negate_finding": false,
            "search_in": "same-file"
        },
        {
            "pattern" : 
            {
                "pattern": "butter",
                "type": "string",
                "scopes": [
                    "code"
                ]
            },
            "negate_finding": false,
            "search_in": "same-file"
        }
    ],
    "must-match": [

        "cheese and butter and toast"
    ],
    "must-not-match": [
        "cheese and wine",
        "cheese and toast",
        "butter and toast"
    ]
}

from applicationinspector.

Thanks for the answer, in my case it can happen that the matches are in different files, an example would be to detect this functionality:
https://www.baeldung.com/spring-boot-redis-cache

Using some dependencies in pom.xml files and in .java code make use of some configuration. The result would be to create a rule that detects to do the 3 things. It is not clear to me if conditions allows to make checks in different files or it is always referred to the file in which the pattern matched... I am also not clear if in this case I could make use of must-match to indicate that the 3 patterns must be fulfilled... that is why I commented if it could be allowed to configure the pattens to work as and instead of or.
The example rule would be this:

{
    "name": "Spring Cache Redis",
    "id": "0000",
    "description": "https://www.baeldung.com/spring-boot-redis-cache",
    "applies_to_file_regex": [
      "pom.xml",
      "*\\.java"
    ],
    "tags": [
      "Spring.Cache.Redis"
    ],
    "severity": "important",
    "patterns": [
      {
        "pattern": "<artifactId>spring-boot-starter-cache</artifactId>",
        "type": "string",
        "scopes": [
          "code"
        ],
        "modifiers": [
          "i"
        ],
        "confidence": "medium"
      },
      {
        "pattern": "<artifactId>spring-boot-starter-data-redis</artifactId>",
        "type": "string",
        "scopes": [
          "code"
        ],
        "modifiers": [
          "i"
        ],
        "confidence": "medium"
      },
      {
        "pattern": "RedisCacheConfiguration",
        "type": "string",
        "scopes": [
          "code"
        ],
        "modifiers": [
          "i"
        ],
        "confidence": "medium"
      }
    ]
  }

Could you tell me what would be the best way to indicate that the three pattens must be met for this example?
thanks

from applicationinspector.

Reposted comment to fix line breaks:

Ah I see. Application inspector does not support rules that span multiple files. Rules are checked against one file at a time and each rule must match a single file. This is likely not a feature we will pursue, as it would be a dramatic divergence from how app inspector currently works.

To implement this logical check using application inspector Id recommend two different rules, with two different tags - one for the xml component and one for the Java. And then check for the presence of both tags.

Estimating from the rule you have here it looks like you have two requirements for the xml file, so those should be a pattern and a condition in the same rule if so, and then a different rule with one pattern for your Java file. After running the report you can then check for the presence of both tags.

For querying xml documents, I can also recommend using the xpaths feature, which should help address missing matches from white space/formatting. See https://github.com/microsoft/ApplicationInspector/wiki/3.6-Structured-Data-Queries-(XPath,-JSONPath,-YamlPath) for guidance.

It’s also possible if you want to automate this to either write a bit of code that leverages the app inspector rules engine library that includes doing a check for the presence of the multiple tags you desire found across multiple findings, or more trivially to write a script that parses the sarif format output and does the same.

Finally, the must-match field is for testing with the verify command. When running the verify command against rules it will check that the must-match field is matched by the overall pattern including conditions and must-not-match is not matched. These fields are essentially self tests to ensure your rule is matching what you expect. It doesn’t have any functional purpose when the rule is run against the actual content to check using analyze.

from applicationinspector.

Thanks for the idea, I will do what you say about having a process that detects the presence of multiple related tags to result in a final tag.
One idea would be to use some infix in the tag name that indicates that it is a subtag and the number of different subtags that have to be satisfied to result in the desired tag (SubTag-[totalsubtags]).

Example:

Spring.Cache.Redis.SubTag-2.Dependencies
Spring.Cache.Redis.SubTag-2.Configuration

Being a tag with the presence of the infix SubTag-[totalsubtags] I can check if there are 2 different subtags in the result of the analysis and in that case I will give as a result the tag:

Spring.Data.Redis

Thanks

from applicationinspector.

Good news @valinha. I was able to think of a pretty simple implementation here and the team agreed this is a great idea.

I have a PR open (#533) that I plan to merge today early next week that will add this feature via a new depends_on_tags field in the rule.

How the depends_on_tags field works: After all matches are collected from all files, we check the depends_on_tags field for each match's rule against the total set of unique tags collected from the scan. If any tag listed in the depends_on_tags field is not present in the overall set of tags, that match will be removed from the set. After the matches are filtered, the metadata is recalculated to get a new correct set of unique tags in the scan. You can check the PR linked above for some example rules that use the new field.

After merge, this will release as a beta of 1.8, the beta is planned to just be a couple weeks. During that time I will take the opportunity of SemVer bump to make a few other clean up changes (like completing and merging #528), add documentation for this feature, and respond to any feedback about how this new feature works.

If you can give it a try after the new Nuget is released and let me know what you think that would be great.

from applicationinspector.

This feature is now available in the 1.8 beta releases.

from applicationinspector.