CWE-190: Integer Overflow or Wraparound

CodeQL identifies CWE-190 in this code:

// parsePositiveInt parses value as an int. It returns an error if value cannot
// be parsed or is negative.
func parsePositiveInt(value string) (int, error) {
	switch i64, err := strconv.ParseInt(value, 10, 64); {
	case err != nil:
		return 0, fmt.Errorf("unable to parse positive integer %q: %v", value, err)
	case i64 < 0:
		return 0, fmt.Errorf("unable to parse positive integer %q: negative value", value)
	case i64 > math.MaxInt:
		return 0, fmt.Errorf("unable to parse positive integer %q: overflow", value)
	default:
		return int(i64), nil
	}
}

reporting:

Incorrect conversion of a 64-bit integer from to a lower bit size type int without an upper bound check.

for the line

		return int(i64), nil

Clicking on "Show paths" in the GitHub Code Scanning Alert shows:

I believe this is a false positive because the switch cases do include an upper bound check, but it looks like these are not taken into account by CodeQL.

When I use codeql-go to regexpMatch url like "http://github.com". I write this ql file.

import go
import semmle.go.security.RequestForgery::RequestForgery

 
from CallExpr call, DataFlow::Expr arg,string reg 
 where call.getCalleeName().regexpMatch(".*print.*") and
 reg = "/((https|http)?:\/\/[a-zA-Z0-9\.\?\/%-_]*)/gm" and
 call.getArgument(0).getStringValue().regexpMatch(reg) and
 arg = call.getArgument(0)
select call, arg.getStringValue()

But I get a token recognition error ,because this var reg .
And I remove "\" to slove this error,but this regexp is not work.
So, how can I make this regexp string work?
My codeql version is codeql-cli v2.6.0.

Source-locations at least for non-go-modules projects (i.e., those which are extracted by moving the project into a fake GOPATH and then performing post-hoc path-rewriting to make the source-locations reflect the pre-move file locations) don't use the realpath of the source filename. This means that if the source root isn't its own realpath (e.g., it's a symlink pointing elsewhere, such as /tmp -> /private/tmp on OSX), codeql bqrs interpret will discard the results because the source-location (which hasn't been realpath'd) doesn't appear to fall within the source root (which has been realpath'd by codeql database create).

Code:

package main


import (
	"database/sql"
	"net/http"
	"fmt"
)


type RequestStruct struct {
	Id         int64  `db:"id"`
	Category   []string  `db:"category"`
}


func handler(db *sql.DB, req *http.Request) {

	RequestData := &RequestStruct{
		Id: 	  1,
		Category: req.URL.Query()["category"],
	}

	q := fmt.Sprintf("SELECT ITEM,PRICE FROM PRODUCT WHERE ITEM_CATEGORY='%s' ORDER BY PRICE",
		RequestData.Category)
	db.Query(q)

	q2 := fmt.Sprintf("SELECT ITEM,PRICE FROM PRODUCT WHERE ITEM_CATEGORY='%s' ORDER BY PRICE",
		req.URL.Query()["category"])
	db.Query(q2)
}

Expected result:

This query depends on  a user-provided value . 	test.go:26:11
This query depends on  a user-provided value . 	test.go:30:11

Actual result:

This query depends on  a user-provided value . 	test.go:30:11

Hi,

I can run a single rule without any problem by using Codeql CLI, but when I pass the codeql-go rule folder, I get the following error.

...
Starting evaluation of codeql-go/RedundantCode/SelfAssignment.ql.
Starting evaluation of codeql-go/RedundantCode/RedundantRecover.ql.
Starting evaluation of codeql-go/localReferences.ql.
Starting evaluation of codeql-go/AlertSuppression.ql.
Starting evaluation of codeql-go/Metrics/FLinesOfCode.ql.
Starting evaluation of codeql-go/Metrics/FLinesOfComment.ql.
Starting evaluation of codeql-go/printAst.ql.
Oops! A fatal internal error occurred.
com.semmle.util.exception.CatastrophicError: Program depends on template extensional selectedSourceFile, but no value was supplied for it
        at com.semmle.inmemory.caching.TemplateStore.<init>(TemplateStore.java:34)
        at com.semmle.inmemory.MemoryBackend.executeQuery(MemoryBackend.java:324)
        at com.semmle.cli2.ql.EvaluationCommon.lambda$executeQueryAsync$2(EvaluationCommon.java:181)
        at com.semmle.util.concurrent.FutureUtils.futureAutoclose(FutureUtils.java:185)
        at com.semmle.cli2.ql.EvaluationCommon.executeQueryAsync(EvaluationCommon.java:179)
        at com.semmle.cli2.execute.ExecuteQueriesCommand.lambda$executeSubcommand$3(ExecuteQueriesCommand.java:296)
        at com.semmle.cli2.ql.Paralleliser$1.lambda$next$0(Paralleliser.java:68)
        at com.semmle.util.concurrent.FutureUtils.supplyCompose(FutureUtils.java:217)
        at com.semmle.cli2.ql.Paralleliser.lambda$startMoreJobs$1(Paralleliser.java:98)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
        at java.base/java.lang.Thread.run(Unknown Source)

Could you please help me to solve the problem?

Thanks!

Given this scenario:

func someFunc(ptr uintptr) {
	type SomeStruct struct {
		Hello uintptr
	}

	evilStruct := &SomeStruct{}
	evilStruct.Hello = ptr
}

Is it possible to propagate taint from evilStruct to ptr, so that a flow config with source evilStruct and sink ptr could succeed?

I found a note of mine from a long time ago that mentions HTTP::HeaderRead, but I cannot find the model.

Was it ever modeled?

Should it be modeled?

A taint-tracking class like this:

	private class ReaderReset extends TaintTracking::FunctionModel, Method {
	  // signature: func (*Reader).Reset(r io.Reader)
	  ReaderReset() { this.(Method).hasQualifiedName("bufio", "Reader", "Reset") }

	  override predicate hasTaintFlow(FunctionInput inp, FunctionOutput outp) {
	    inp.isParameter(0) and outp.isReceiver()
	  }
	}

will not taint-track code like this:

func TaintStepTest_BufioReaderReset(r io.Reader) {
	// The flow is from `r` into `reader`.

	// Declare `reader` variable:
	var reader bufio.Reader

	// Call the method that **should** transfer the taint
	// from the parameter `r` to the receiver `reader`
	// (`reader` should now be tainted).
	reader.Reset(r)

	// Sink the tainted `reader`:
	sink(reader)
}

This problem was brought up before in #118 (comment)

The underlying issue was solved/patched before (I believe in 0546c52)

Thanks!

Hello.

I tried to create a CodeQL database for kubernetes-1.9.0 (which is an old version) through LGTM.com.

It was built successfully, but I found a warning in the Golang 'Extraction' log, as follows.

...
[build] Detected go version: go version go1.15 linux/amd64.
[build] Kubernetes requires go1.9.1 or greater.
[build] Please install go1.9.1 or later.
...

As far as I know, I can specify the version through lgtm.yml for languages such as Python, C#, Java.

However, in this document, I could not find any words to specify the version of Golang.

Is there any way to specify the Golang version when creating a CodeQL database?

My LGTM project is https://lgtm.com/projects/g/donghyunlee00/v2-kubernetes-1.9.0, exactly same code with kubernetes-1.9.0 release code

Best Regards.

"XORM is a Simple & Powerful ORM Framework for Go Programming Language"

Link: https://xorm.io/

If there is a need for this, I will open a pull request later.

How would I access the key used when reading an element of a map?

For example,

id := vars["id"]

If I traced the dataflow to vars how would I output "id"?

This line triggered a CodeQL warning in the cli/cli repo:

var placeholderRE = regexp.MustCompile(`\:(owner|repo|branch)\b`)

The warning says:

A regular expression used here contains a literal backspace \b; did you mean \b, a word boundary?

And in its "recommendation" section, it says:

Alternatively, you can use backtick-delimited raw string literals. For example, the \b in regexp.Compile(hello\bworld) matches a word boundary, not a backspace character, as within backticks \b is not an escape sequence.

Well, we are using backtick-delimited raw string literal in the "offending" code, and it should be trivial for CodeQL to detect that. I think that CodeQL is convinced that we've passed a double-quoted string literal as argument to regexp.MustCompile(), which could be indicative of a deeper, more serious problem in CodeQL for Go.

CWE-117: Improper Output Neutralization for Logs
CWE-117 is being reported by CodeQL in the following code:

func makeErrorForHTTPResponse(resp *http.Response) error {
	bodyBytes, err := ioutil.ReadAll(resp.Body)
	if err != nil {
		return err
	}
	url := resp.Request.URL.String()
	safeURL := strings.Replace(url, "\n", "", -1)
	safeURL = strings.Replace(safeURL, "\r", "", -1)
	return fmt.Errorf("%s %s returned HTTP %s; \n\n %#q", resp.Request.Method, safeURL, resp.Status, bodyBytes)
}

Despite this code being near identical to the provided "good" example

func handlerGood(req *http.Request) {
	username := req.URL.Query()["username"][0]
	escapedUsername := strings.Replace(username, "\n", "", -1)
	escapedUsername = strings.Replace(escapedUsername, "\r", "", -1)
	log.Printf("user %s logged in.\n", escapedUsername)
}

here.

Here is a screen shot of the output logs for further clarification

It appears that CodeQL completely ignores the above two functions performing the string replacement.

Assumption A is a struct with fields {birthday, name, id}, i got instanceA from unmarshal input data.
I want to TaintTracking the field "instanceA.birthday",
How do I write the ql to location the source node?

Conextual queries for find references and definitions are not returning any results for go in VS Code. Currently using 2ba9bbf in the latest release of VSCode extension for codeql and master of vscode-codeql-starter workspace.

To reproduce:

1. Install https://lgtm.com/projects/g/kubernetes/kubernetes/ into vscode as a go database
2. Open src/opt/src/cmd/verifydependencies/verifydependencies.go
3. On line 178, select collector right-click -> find references

BUG
Expected: References to that symbol are found on line 186 and elsewhere
Actual: no references

Similarly, select the collector symbol on line 186 and invoke Go to Definition. No results are found.

See also github/codeql#4351 for a similar problem in C#.

It would be nice to move to comment-based tests as seen in the callgraph test and its corresponding source.

@smowton I think it would be nice to have a framework for doing data-flow and taint tests, so while doing this for #285 so ideally that would be somewhat general.

There's probably no need to do this all at once; since this is mostly nice for review I think we can just do the transition as the tests change.

It'd be really nice to have a guide on how to use this repo correctly. Most of the CLIs bundled inside of it have little or no usage info. I think I managed to create a database correctly using go-autobuilder, but I'm really not sure. I had to set a couple environment variables (the purpose of which I could only guess). I imagine that such docs would help both random interested parties like me and new members of your team.

I was setting up everything to run codeql cli for golang and was able to create the database, but when running the command to analyze the database, an error was returned:

Command I run: codeql database analyze ./database go1.ql --format=csv --output=output.csv

Output of the terminal:

Running queries.
Compiling query plan for /Users//projects/<repository_name>/go1.ql.
ERROR: Could not resolve module go. There should probably be a qlpack.yml file declaring dependencies in /Users//projects/<repository_name> or an enclosing directory. (/Users//projects/<repository_name>/go1.ql:1,8-10)

The content of the go1.ql file is the following:

import go

select "hello world"

The steps I followed for installing codeql and codeql-go were the following:

• https://codeql.github.com/docs/codeql-cli/getting-started-with-the-codeql-cli/#checking-out-the-codeql-source-code-directly
• https://github.com/github/codeql-go#installation

Pages I have read to understand the problem that the terminal returned:

• https://codeql.github.com/docs/codeql-cli/about-ql-packs/
• https://codeql.github.com/docs/ql-language-reference/modules/#importing-modules
• https://codeql.github.com/docs/ql-language-reference/name-resolution/#name-resolution
• https://codeql.github.com/docs/ql-language-reference/ql-language-specification/#module-resolution

After all this research, I couldn't fix it and make this work in the terminal. Has anyone had the same problem and could solve the issue? Thanks in advance.

What currently happens

If a function returns only one result, then FunctionOutput.isResult() will work, and FunctionOutput.isResult(0) will not work.

If a function returns two or more results, then FunctionOutput.isResult(0) will work, and FunctionOutput.isResult() will not work.

What IMHO would be better

It would be a lot less error-prone and a lot more awesome to be able to write FunctionOutput.isResult(0) regardless of whether there is one result or more.

Why it would be an improvement over the current situation

It would be a lot harder to make errors (such as using FunctionOutput.isResult() when in reality the function returns more than one result) if FunctionOutput.isResult(0) and FunctionOutput.isResult() were equivalent regardless of the number of results returned by the function.

I believe this would make things simpler to the end-developer that will be writing codeql.

I create a database for my repo with gomod, but all the dependencies are skipped, which result that, when I run a dataflow query, it can't detect the tainted actions and nodes in the library.

Is there any mistake in my creation or query?

logs:
[2021-10-11 12:22:22] [build-stderr] 2021/10/11 12:22:22 Skipping dependency package xx/gopkg/jsonx

Hello codeql-go team,

I'm working on a new tool that will generate concrete implementations of codeql concepts, and I'm starting with UntrustedFlowSource.

Below are examples of what the generated codeql might look like. Please let me know what you think about it. Thanks!

/**
 * This is a doc.
 */

import go

private module SomeFramework {
  /**
   * This is a doc.
   */
  private class Source extends UntrustedFlowSource::Range {
    Source() {
      // Example block 1: the type is a function.
      // The source is either the result (one or more), or a parameter (one or more),
      // or a mix of them.
      (
        // Function: github.com/example/package.GetSomething
        exists(Function fn, DataFlow::CallNode call |
          fn.hasQualifiedName("github.com/example/package", "GetSomething")
        |
          // The source is the result:
          call = fn.getACall() and this = call.getResult()
        )
        or
        // Function: github.com/example/package.ParseSomething
        exists(Function fn, DataFlow::CallNode call |
          fn.hasQualifiedName("github.com/example/package", "ParseSomething")
        |
          // The source is the 0th parameter:
          call = fn.getACall() and this = FunctionOutput::parameter(0).getExitNode(call)
        )
      )
      or
      // Example block 2: the type is a struct.
      // The source can be a method call (results or parameters of a call),
      // or a field read.
      (
        // Struct: github.com/example/package.Context
        exists(string typeName | typeName = "Context" |
          // Method calls on `Context`:
          exists(DataFlow::MethodCallNode call, string methodName |
            call.getTarget().hasQualifiedName("github.com/example/package", typeName, methodName) and
            (
              methodName = "FullPath" and
              (
                // The source is the method call result #0:
                this = call.getResult(0)
                or
                // The source is method call parameter #0:
                this = FunctionOutput::parameter(0).getExitNode(call)
              )
              or
              // The source is any result of the call?
              methodName = "GetHeader"
            )
          )
          or
          // Field reads on `Context`:
          exists(DataFlow::Field fld, string fieldName |
            fld.hasQualifiedName("github.com/example/package", typeName, fieldName) and
            // The source is one of these fields reads:
            fieldName in ["Accepted", "Params"] and
            this = fld.getARead()
          )
        )
        or
        // Struct: github.com/example/package.SomeStruct
        exists(DataFlow::Field fld, string fieldName |
          fld.hasQualifiedName("github.com/example/package", "SomeStruct", fieldName) and
          // The source is one of these fields reads:
          fieldName in ["Hello", "World"] and
          this = fld.getARead()
        )
      )
      or
      // Example block 3: the type is some custom type.
      // The source is a method call on that type (results or parameters of a call).
      // Some type: github.com/example/package.Slice
      exists(string typeName | typeName = "Slice" |
        // Method calls:
        exists(DataFlow::MethodCallNode call, string methodName |
          call.getTarget().hasQualifiedName("github.com/example/package", typeName, methodName) and
          (
            methodName = "GetHeader" and
            // The source is the method call result #0:
            this = call.getResult(0)
            or
            methodName = "ParseHeader" and
            // The source is method call parameter #1:
            this = FunctionOutput::parameter(1).getExitNode(call)
          )
        )
      )
      or
      // Example block 4: the type is an interface.
      // The source is a method call on that interface (results or parameters of a call).
      // Interface: github.com/example/package.SomeInterface
      exists(string typeName | typeName = "SomeInterface" |
        // Method calls:
        exists(DataFlow::MethodCallNode call, string methodName |
          call.getTarget().implements("github.com/example/package", typeName, methodName) and
          (
            methodName = "GetSomething" and
            // The source is the method call result #0:
            this = call.getResult(0)
            or
            methodName = "UnmarshalSomething" and
            // The source is method call parameter #2:
            this = FunctionOutput::parameter(2).getExitNode(call)
          )
        )
      )
    }
  }
}

I think plain go build is equivalent to go build ., but our extractor doesn't extract anything in the case where no arguments are provided. To reproduce: provide a Makefile that runs go build without args and observe that nothing is extracted, even though go build loads .go files.

Running the GitHub CodeQL scanning actions on the following code reports what I believe is a false positive CWE-190 / CWE-681:

package main

import (
    "os"
    "strconv"
)

func main() {
    for _, arg := range os.Args[1:] {
        n, err := strconv.ParseInt(arg, 10, 0)
        if err == nil {
            var itf interface{} = n
            i32 := itf.(int32)
            println(i32)
        }
    }
}

On line 13 (i32 := itf.(int32)) it reports:

Incorrect conversion of an integer with architecture-dependent bit size from to a lower bit size type int32 without an upper bound check.

However this is not a type conversion, this is a type assertion. When itf holds a type other than int32, the above code panics. (Perhaps that should be a different error.) Neither the logically equivalent type switch nor the two-valued type assertion report this.

            var i32 int32
            switch x := itf.(type) {
            case int32:
                i32 = x
            default:
                panic("bad type")
            }
            println(i32)

            i32, ok := itf.(int32)
            println(i32, ok)

The code from which this was extracted is an implementation of an interface similar to sql.Scanner, hence the indirection through an empty interface.

The example for finding calls to fmt.Println seems not to work. Maybe things have changed since it was written. Is there some other way to find calls to a particular import?

I've put up a demonstration here.

https://github.com/dittyroma/codeql-go/tree/find-calls

ql/src/experimental/Test contains example queries. Println.ql is a copy of the example calltofunction.ql. Functions.ql drops the filter to find all function calls. Methods.ql tries to find all method calls. All 3 queries find nothing on this program.

package main

import (
	"fmt"
)

func main() {
	fmt.Println("line")
}

What I'd like to do is find file system functions, eg for os.Create() calls that create files.

from Function function, DataFlow::CallNode call
where
  function.hasQualifiedName("os", "Create") and
  call = function.getACall()
select call

This command reproduces on my system.

$ codeql test run "/home/user/codeql/sft/codeql-repo-go/ql/test/experimental/Test"

Executing 3 tests in 1 directories.
Extracting test database in /home/user/codeql/sft/codeql-repo-go/ql/test/experimental/Test.
Compiling queries in /home/user/codeql/sft/codeql-repo-go/ql/test/experimental/Test.
Executing tests in /home/user/codeql/sft/codeql-repo-go/ql/test/experimental/Test.
Cannot find Methods.expected file.
Query produced no results.
[1/3 eval 703ms] FAILED(RESULT) /home/user/codeql/sft/codeql-repo-go/ql/test/experimental/Test/Methods.qlref
Cannot find Println.expected file.
Query produced no results.
[2/3 eval 56ms] FAILED(RESULT) /home/user/codeql/sft/codeql-repo-go/ql/test/experimental/Test/Println.qlref
Cannot find Functions.expected file.
Query produced no results.
[3/3 eval 44ms] FAILED(RESULT) /home/user/codeql/sft/codeql-repo-go/ql/test/experimental/Test/Functions.qlref
0 tests passed; 3 tests failed:
  FAILED: /home/user/codeql/sft/codeql-repo-go/ql/test/experimental/Test/Methods.qlref
  FAILED: /home/user/codeql/sft/codeql-repo-go/ql/test/experimental/Test/Println.qlref
  FAILED: /home/user/codeql/sft/codeql-repo-go/ql/test/experimental/Test/Functions.qlref

Comparing

If this is indeed wrong, feel free to fix it as I currently have no time for fixing it.

1. List only signature/type for each model kind without specifying what/where:

/**
 * Module summary:
 * 
 * UntrustedFlowSource:
 *   FUNCS:
 *     func (*clevergo.tech/clevergo.Context).BasicAuth() (username string, password string, ok bool)
 *     func (*clevergo.tech/clevergo.Context).Decode(v interface{}) (err error)
 *     func (*clevergo.tech/clevergo.Context).DefaultQuery(key string, defaultVlue string) string
 *     func (*clevergo.tech/clevergo.Context).FormValue(key string) string
 *     func (*clevergo.tech/clevergo.Context).GetHeader(name string) string
 *     func (*clevergo.tech/clevergo.Context).PostFormValue(key string) string
 *     func (*clevergo.tech/clevergo.Context).QueryParam(key string) string
 *     func (*clevergo.tech/clevergo.Context).QueryParams() net/url.Values
 *     func (*clevergo.tech/clevergo.Context).QueryString() string
 *     func (clevergo.tech/clevergo.Decoder).Decode(req *net/http.Request, v interface{}) error
 *     func (clevergo.tech/clevergo.Params).String(name string) string
 *   STRUCTS:
 *     clevergo.tech/clevergo.Context
 *     clevergo.tech/clevergo.Param
 *   TYPES:
 *     clevergo.tech/clevergo.Params
 * 
 * ---
 * TaintTracking:
 *   FUNCS:
 *     func (*clevergo.tech/clevergo.Application).RouteURL(name string, args ...string) (*net/url.URL, error)
 *     func (*clevergo.tech/clevergo.Context).Context() context.Context
 *     func (clevergo.tech/clevergo.Decoder).Decode(req *net/http.Request, v interface{}) error
 *     func (clevergo.tech/clevergo.Params).String(name string) string
 *     func (clevergo.tech/clevergo.Renderer).Render(w io.Writer, name string, data interface{}, c *clevergo.tech/clevergo.Context) error
 *     func clevergo.tech/clevergo.CleanPath(p string) string
 * 
 * ---
 * HTTP::Redirect:
 *   FUNCS:
 *     func (*clevergo.tech/clevergo.Context).Redirect(code int, url string) error
 * 
 * ---
 * HTTP::ResponseBody:
 *   FUNCS:
 *     func (*clevergo.tech/clevergo.Context).Blob(code int, contentType string, bs []byte) (err error)
 *     func (*clevergo.tech/clevergo.Context).Emit(code int, contentType string, body string) (err error)
 *     func (*clevergo.tech/clevergo.Context).Error(code int, msg string) error
 *     func (*clevergo.tech/clevergo.Context).HTML(code int, html string) error
 *     func (*clevergo.tech/clevergo.Context).HTMLBlob(code int, bs []byte) error
 *     func (*clevergo.tech/clevergo.Context).JSON(code int, data interface{}) error
 *     func (*clevergo.tech/clevergo.Context).JSONBlob(code int, bs []byte) error
 *     func (*clevergo.tech/clevergo.Context).JSONP(code int, data interface{}) error
 *     func (*clevergo.tech/clevergo.Context).JSONPBlob(code int, bs []byte) error
 *     func (*clevergo.tech/clevergo.Context).JSONPCallback(code int, callback string, data interface{}) error
 *     func (*clevergo.tech/clevergo.Context).JSONPCallbackBlob(code int, callback string, bs []byte) (err error)
 *     func (*clevergo.tech/clevergo.Context).SetContentType(v string)
 *     func (*clevergo.tech/clevergo.Context).SetContentTypeHTML()
 *     func (*clevergo.tech/clevergo.Context).SetContentTypeJSON()
 *     func (*clevergo.tech/clevergo.Context).SetContentTypeText()
 *     func (*clevergo.tech/clevergo.Context).SetContentTypeXML()
 *     func (*clevergo.tech/clevergo.Context).String(code int, s string) error
 *     func (*clevergo.tech/clevergo.Context).StringBlob(code int, bs []byte) error
 *     func (*clevergo.tech/clevergo.Context).Stringf(code int, format string, a ...interface{}) error
 *     func (*clevergo.tech/clevergo.Context).Write(data []byte) (int, error)
 *     func (*clevergo.tech/clevergo.Context).WriteString(data string) (int, error)
 *     func (*clevergo.tech/clevergo.Context).XML(code int, data interface{}) error
 *     func (*clevergo.tech/clevergo.Context).XMLBlob(code int, bs []byte) error
 * 
 * ---
 * HTTP::HeaderWrite:
 *   FUNCS:
 *     func (*clevergo.tech/clevergo.Context).SetHeader(key string, value string)
 */

The scripts/install-reps.sh file is missing on the v1.27.0 tag. I'm guessing we never merged that file into the lgtm branch. At this point, since codeql-go is being merged into codeql, this may not need to be fixed, but raising it anyway to make sure we are aware of the missing file.

Originally reported in github/codeql#8763

Query some repo with this CodeQL:

import go

from ForStmt f
select f

Expected: for loops with range statement are available in output
Seen: no such loops found in code, only other types of for in output

This issue is about adding support to the CodeQL VS Code extension for viewing ASTs of Go files. See the parent issue github/vscode-codeql#492.

Given the following Go code:

package main

func source() interface{} {
	return nil
}

type context struct{}

//
func (this *context) Blob(_ int, ct string, body []byte) {}

const constContentType = "application/const"
const constApplication = "application/"

func main() {
	// case 1:
	{
		bodyByte839a := source().([]byte)
		var rece context
		// The argument is a string literal:
		rece.Blob(200, "application/string-literal", bodyByte839a)
	}
	// case 2:
	{
		bodyByte839b := source().([]byte)
		var rece context
		ct := "application/variable"
		// The argument is a variable:
		rece.Blob(200, ct, bodyByte839b)
	}
	// case 3:
	{
		bodyByte839b := source().([]byte)
		var rece context
		// The argument is a constant:
		rece.Blob(200, constContentType, bodyByte839b)
	}
	// case 4:
	{
		bodyByte839b := source().([]byte)
		var rece context
		// The argument is a concatenation of string literals:
		rece.Blob(200, "application/"+"concat", bodyByte839b)
	}
	// case 5:
	{
		bodyByte839b := source().([]byte)
		var rece context
		ct := "application/"
		// The argument is a concatenation of a variable + a string literal:
		rece.Blob(200, ct+"concat", bodyByte839b)
	}
	// case 6:
	{
		bodyByte839b := source().([]byte)
		var rece context
		// The argument is a concatenation of a constant + a string literal:
		rece.Blob(200, constApplication+"concat", bodyByte839b)
	}
	// case 7:
	{
		bodyByte839b := source().([]byte)
		var rece context
		ct := "variable-concat"
		// The argument is a concatenation of a constant + variable:
		rece.Blob(200, constApplication+ct, bodyByte839b)
	}
}

.. the following CodeQL query is not able to select all instances of the desired parameter's string value via getStringValue():

/**
 */

import go

from Method m, DataFlow::CallNode bodySetterCall
where m.hasQualifiedName(_, "context", "Blob") and bodySetterCall = m.getACall()
select bodySetterCall.getArgument(1).getStringValue(), bodySetterCall

👋🏻 Hi, folks! I'm here on behalf of @github/frameworks-services. We're researching how and why teams are using the zap module (go.uber.org/zap) to help us in our development planning and inform the Go guidance we provide to internal teams.

From my research, it appears @github/codeql-go is using zap in the following repository/repositories:

• github/codeql-go

Questions

• Which environment(s) are the above repositories used in? (dotcom, GHAE, GHES, and/or other)
  • Is your logging (with zap) configured & enabled in all the environment(s) the repositories are used in?
• Why are you using zap instead of some other module? (ie was it an arbitrary choice, or did you have specific reason(s)?)
• Are you familiar with the internal go-log module?
  • If yes, are you aware of any feature gaps between zap and go-log or other obstacles that would make it difficult to switch from zap to go-log?
• Do you use any structured logging modules other than zap in your team? (If yes, please list them)
• What pain points / friction do you have with logging modules?

Given this test code:

package main

func main() {
	typ := getAType()
	_ = typ

	ptr := getAPointer()
	_ = ptr
}

type Something struct {
}

func getAType() Something {
	return Something{}
}

func getAPointer() *Something {
	return &Something{}
}

... the following codeql query will only select instances of typ, and ignore ptr:

/**
 * @kind problem
 */

import go

class Something extends ValueEntity {
  Something() {this.getType().hasQualifiedName(_, "Something") }
}

select any(Something c).getAReference()

To write a class that would select all ValueEntity instances of a type regardless of whether it's the base type or a pointer to that type, I need to do this:

/**
 * @kind problem
 */

import go

class Something extends ValueEntity {
  Something() { getBaseType(this.getType()).hasQualifiedName(_, "Something") }
}

Type getBaseType(Type typ) {
  result = getBaseType*(typ.(PointerType).getBaseType())
  or
  result = typ
}

select any(Something c).getAReference()

Is this a feature, or is this a bug?

Thanks

If I have an HTTP client, and I do a HTTP request to a remote server, does the response data count as UntrustedFlowSource?

let's say A is a struct with fields {userid, username}, B is untrusted source, A.userid = B

I expect that A'll be tainted, but actually not..
I check the source code in the function 'localAdditionalTaintStep', seems it doesn't cover this action..

So if I want to let A be tainted by B, what can I do in my overrided function 'isAdditionalTaintStep'?

Thanks very much!!

Hi I get an error when trying to run go queries

Running queries.
ERROR: Referenced pack 'codeql/suite-helpers' not found. (C:\codeql-home\codeql-go\ql\src\qlpack.yml:1,1-1)
A fatal error occurred: Could not resolve library path for C:\codeql-home\codeql-go\ql\src

Transport.RoundTrip() in net/http will performs an HTTP request to a URL

import (
	"bytes"
	"fmt"
	"net/http"
)

func request2(url string) {
	req, _ := http.NewRequest("GET", url, bytes.NewBuffer(nil))
	tr := &http.Transport{}
	resp, _ := tr.RoundTrip(req)
	fmt.Println(resp.StatusCode)
}

In #260 and #264, the printAST query was created and fixed up. There are still a few smaller issues. I'm breaking them into a new issue since this shouldn't prevent merging and closing of previous issues. I'm not even sure if everything here should be fixed:

• Method declaration receivers are not in parameter list (or anywhere in the ast)

• The FunctionName node is child of the FuncDecl/MethodDecl, though it seems more appropriate for it to be a child of the FuncTypeExpr since its source location is fully contained in that node.

• There is a File ast node, which has no source locations and no children. For the purposes of the AST viewer, it's not very interesting. Would be nice if we could exclude this.

• There is no Package statement node, but the identifier of the package statement is a node. So in here:

  package json

  there is no node for package json, but there is one for json, which is a direct decedent of the root node. And this is causing ordering issues in the AST viewer.

It seems that the go-autobuilder binary is not included in this repo , so my db creation is failed due to the binary not found .

codeql-cli command : codeql database create my_repo --language go

Am I missing something ?

The CodeQL Go extractor and libraries are moving to a new home!

Summary

• Old location: The github/codeql-go repository
• New location: The github/codeql repository, specifically https://github.com/github/codeql/tree/main/go.

The github/codeql-go repository currently contains the open-source CodeQL queries, libraries, and extractor for analyzing codebases written in the Go programming language with GitHub's CodeQL code analysis tools. These contents will be moved to the github/codeql repository, to live alongside similar libraries for all other programming languages supported by CodeQL.

CodeQL support of the Go programming language will stay the same, except that the code and the development will continue in github/codeql.

Once the move completes, GitHub will stop using the github/codeql-go repository and replace its contents with a placeholder announcement.

When will this change take place?

This change is scheduled for 23 May 2022.

What do I need to change as a user of CodeQL or code scanning?

For users of non-default CodeQL query suites

• When this applies: If you’re using the query or queries configuration directives in a CodeQL query suite.
• What you need to do:
  • Please make sure to refer to individual CodeQL queries for Go by their canonical query ID (rather than an absolute path). The query ID can be found after @id in the metadata section at the top of the query file.
  • Alternatively, please use relative paths to queries and directories within the enclosing CodeQL pack codeql/go-queries, rather than absolute paths.
  • See the CodeQL CLI docs for more information on the query suite syntax.

Examples:

- query: path/to/checkout/of/codeql-go/ql/src/Security/CWE-079/ReflectedXss.ql

- queries: .
  from: codeql/go-queries
- include:
   id: go/reflected-xss

- query: Security/CWE-079/ReflectedXss.ql
from: codeql/go-queries

- queries: path/to/checkout/of/codeql-go/ql/src/path/to/subdirectory

- queries: path/to/subdirectory
  from: codeql/go-queries
  ...

- queries: path/to/subdirectory
  from: codeql-go
  ...

- queries: path/to/subdirectory
  from: codeql/go-queries
  ...

- qlpack: codeql-go
  ... 

- qlpack: codeql/go-queries
  ... 

For users of the queries field in a code scanning YAML configuration

• When this applies: If the queries field in your code scanning YAML configuration file directly references a query or directory of queries in the github/codeql-go repository.
• What you need to do: Please change these references to use the packs: field and the corresponding path within the codeql/go-queries query pack.
  • We do not recommend running queries from the CodeQL standard libraries at a specific GitHub ref, since this may not be compatible with the latest released version of CodeQL running on GitHub Actions.
• Example before:

  name: "My CodeQL config"
  queries:
    - uses: github/codeql-go/ql/src/experimental/CWE-327@<ref>

• Example after:

  name: "My CodeQL config"
  packs:
    - 'codeql/go-queries:experimental/CWE-327'

For users of the queries field in the github/codeql-action/init step of a GitHub Actions workflow

• When this applies: If the queries field in your workflow file directly references queries in the github/codeql-go repository.
• What you need to do: Please change these references to use the packs: field and the corresponding path within the codeql/go-queries query pack.
  • We do not recommend running queries from the CodeQL standard libraries at a specific GitHub ref, since this may not be compatible with the latest released version of CodeQL running on GitHub Actions.
• Example before:

  - name: Initialize CodeQL
    uses: github/codeql-action/init@v2
    with:
      languages: go
      queries: github/codeql-go/ql/src/experimental/CWE-327@<ref>

• Example after:

  - name: Initialize CodeQL
    uses: github/codeql-action/init@v2
    with:
      languages: go
      packs: 'codeql/go-queries:experimental/CWE-327'

For users of the github/vscode-codeql-starter repository

• When this applies: If you are using a local Git checkout of the github/vscode-codeql-starter repository or one of its forks, which contain the github/codeql and github/codeql-go repositories as submodules.
• What you need to do: Please ensure your checkout is kept up to date, and use git submodule update --remote to keep the submodules up to date.
• What will change: GitHub will move Go-related contents into the ql submodule and workspace folder, and delete the codeql-go submodule and workspace folder. Up-to-date checkouts and VS Code workspaces will continue to work.

For users of local Git checkouts of the github/codeql and github/codeql-go repositories

• When this applies: If you have local Git checkouts of the github/codeql and github/codeql-go repositories in sibling directories for use with the CodeQL CLI, as recommended in the CodeQL CLI documentation.
• What you need to do: Please ensure both checkouts are kept up to date. If you are working on local Git branches in your checkout of github/codeql-go, please update and rebase them on github/codeql instead.
• What will change: GitHub will move Go-related contents into the github/codeql repository, and leave only placeholder data in the codeql-go repository. Up-to-date checkouts will reflect this change. After this point you are free to delete the checkout of the github/codeql-go repository.

If none of the above apply to you

We expect no change to be required to your CodeQL or code scanning setup.

Where do I go for help?

Please open an issue in github/codeql or file an issue with GitHub support if you encounter any difficulties with this change, and we will be happy to help.

Option A:

      // Package: github.com/revel/[email protected]
      // Function: func ActionInvoker(c *Controller, _ []Filter)
      exists(Function fn, DataFlow::CallNode call |
        fn.hasQualifiedName("github.com/revel/revel", "ActionInvoker")
      |
        call = fn.getACall() and
        this = FunctionOutput::parameter(0).getExitNode(call)
      )
      or
      // Package: github.com/revel/[email protected]
      // Function: func AddControllerType(moduleSource *Module, controllerType reflect.Type, methods []*MethodType) (newControllerType *ControllerType)
      exists(Function fn, DataFlow::CallNode call |
        fn.hasQualifiedName("github.com/revel/revel", "AddControllerType")
      |
        call = fn.getACall() and
        this = FunctionOutput::parameter(1).getExitNode(call)
      )

Option B:

      // Package: github.com/revel/[email protected]
      exists(Function fn, DataFlow::CallNode call |
        // Function: func ActionInvoker(c *Controller, _ []Filter)
        fn.hasQualifiedName("github.com/revel/revel", "ActionInvoker") and
        this = FunctionOutput::parameter(0).getExitNode(call)
        or
        // Function: func AddControllerType(moduleSource *Module, controllerType reflect.Type, methods []*MethodType) (newControllerType *ControllerType)
        fn.hasQualifiedName("github.com/revel/revel", "AddControllerType") and
        this = FunctionOutput::parameter(1).getExitNode(call)
      |
        call = fn.getACall()
      )

The example only shows 2 functions, but imagine that there are 20. Which is better? Is there any difference of some kind other than less code repetition? Thanks

Project proposal: Taint-track the golang.org/x/* packages.

Here is a list of the top 100 golang.org/x/* packages sorted by how many times they were imported (taken from here)

A package is relevant if it has elements that can be put into a TaintTracking model. Not all packages from the below list will be "relevant" at the end.

I will use this ticket to track the progress.

Page 1

1. golang.org/x/net/context (docs)
   • Description: Package context defines the Context type, which carries deadlines, cancelation signals, and other request-scoped values across API boundaries and between processes.
   • Imported by: 39240
   • Comment: 🔴 No apparent relevant elements.
2. golang.org/x/oauth2 (docs)
   • Description: Package oauth2 provides support for making OAuth2 authorized and authenticated HTTP requests, as specified in RFC 6749.
   • Imported by: 10797
   • Comment: 🔴 No apparent relevant elements.
3. golang.org/x/sys/unix (docs)
   • Description: Package unix contains an interface to the low-level operating system primitives.
   • Imported by: 6584
   • Comment: 🔴 No apparent relevant elements.
4. golang.org/x/crypto/ssh (docs)
   • Description: Package ssh implements an SSH client and server.
   • Imported by: 4456
   • Comment: 🟠
5. golang.org/x/crypto/bcrypt (docs)
   • Description: Package bcrypt implements Provos and Mazières's bcrypt adaptive hashing algorithm.
   • Imported by: 4000
   • Comment: 🟠
6. golang.org/x/crypto/ssh/terminal (docs)
   • Description: Package terminal provides support functions for dealing with terminals, as commonly found on UNIX systems.
   • Imported by: 3615
   • Comment: 🔴
7. golang.org/x/sync/errgroup (docs)
   • Description: Package errgroup provides synchronization, error propagation, and Context cancelation for groups of goroutines working on subtasks of a common task.
   • Imported by: 3514
   • Comment: 🔴
8. golang.org/x/net/websocket (docs)
   • Description: Package websocket implements a client and server for the WebSocket protocol as specified in RFC 6455.
   • Imported by: 3192
   • Comment: 🟠
9. golang.org/x/crypto/sha3 (docs)
   • Description: Package sha3 implements the SHA-3 fixed-output-length hash functions and the SHAKE variable-output-length hash functions defined by FIPS-202.
   • Imported by: 3098
   • Comment: 🔴
10. golang.org/x/oauth2/google (docs)
    • Description: Package google provides support for making OAuth2 authorized and authenticated HTTP requests to Google APIs.
    • Imported by: 3083
    • Comment: 🔴

Page 2

1. golang.org/x/net/context/ctxhttp (docs)
   • Description: Package ctxhttp provides helper functions for performing context-aware HTTP requests.
   • Imported by: 2814
   • Comment: 🔴
2. golang.org/x/net/html (docs)
   • Description: Package html implements an HTML5-compliant tokenizer and parser.
   • Imported by: 2369
   • Comment: 🟢 :
3. golang.org/x/crypto/pbkdf2 (docs)
   • Description: Package pbkdf2 implements the key derivation function PBKDF2 as defined in RFC 2898 / PKCS #5 v2.0.
   • Imported by: 2345
   • Comment: 🔴
4. golang.org/x/text/transform (docs)
   • Description: Package transform provides reader and writer wrappers that transform the bytes passing through as well as various transformations.
   • Imported by: 2261
   • Comment: 🟢
5. golang.org/x/xerrors (docs)
   • Description: Package xerrors implements functions to manipulate errors.
   • Imported by: 2224
   • Comment: 🟢
6. golang.org/x/time/rate (docs)
   • Description: Package rate provides a rate limiter.
   • Imported by: 2165
   • Comment: 🔴
7. golang.org/x/net/http2 (docs)
   • Description: Package http2 implements the HTTP/2 protocol.
   • Imported by: 1780
   • Comment: 🟢
8. golang.org/x/crypto/ripemd160 (docs)
   • Description: Package ripemd160 implements the RIPEMD-160 hash algorithm.
   • Imported by: 1380
   • Comment: 🔴
9. golang.org/x/text/encoding (docs)
   • Description: Package encoding defines an interface for character encodings, such as Shift JIS and Windows 1252, that can convert to and from UTF-8.
   • Imported by: 1333
   • Comment: 🟢
10. golang.org/x/tools/imports (docs)
    • Description: Package imports implements a Go pretty-printer (like package "go/format") that also adds or removes import statements as necessary.
    • Imported by: 1157
    • Comment: 🔴

Page 3

1. golang.org/x/text/language (docs)
   • Description: Package language implements BCP 47 language tags and related functionality.
   • Imported by: 1129
   • Comment: 🔴
2. golang.org/x/crypto/scrypt (docs)
   • Description: Package scrypt implements the scrypt key derivation function as defined in Colin Percival's paper "Stronger Key Derivation via Sequential Memory-Hard Functions" (https://www.tarsnap.com/scrypt/scrypt.pdf).
   • Imported by: 999
   • Comment: 🔴
3. golang.org/x/crypto/ed25519 (docs)
   • Description: Package ed25519 implements the Ed25519 signature algorithm.
   • Imported by: 994
   • Comment: 🔴
4. golang.org/x/tools/go/analysis (docs)
   • Description: Package analysis defines the interface between a modular static analysis and an analysis driver program.
   • Imported by: 974
   • Comment: 🔴
5. golang.org/x/text/unicode/norm (docs)
   • Description: Package norm contains types and functions for normalizing Unicode strings.
   • Imported by: 927
   • Comment: 🟢
6. golang.org/x/tools/go/loader (docs)
   • Description: Package loader loads a complete Go program from source code, parsing and type-checking the initial packages plus their transitive closure of dependencies.
   • Imported by: 897
   • Comment: 🔴
7. golang.org/x/tools/go/packages (docs)
   • Description: Package packages loads Go packages for inspection and analysis.
   • Imported by: 894
   • Comment: 🔴
8. golang.org/x/image/font (docs)
   • Description: Package font defines an interface for font faces, for drawing text on an image.
   • Imported by: 795
   • Comment: 🔴
9. golang.org/x/crypto/acme/autocert (docs)
   • Description: Package autocert provides automatic access to certificates from Let's Encrypt and any other ACME-based CA.
   • Imported by: 789
   • Comment: 🔴
10. golang.org/x/text/encoding/charmap (docs)
    • Description: Package charmap provides simple character encodings such as IBM Code Page 437 and Windows 1252.
    • Imported by: 785
    • Comment: 🟢

Page 4

1. golang.org/x/net/html/charset (docs)
   • Description: Package charset provides common text encodings for HTML documents.
   • Imported by: 781
   • Comment: 🟢 :
2. golang.org/x/net/proxy (docs)
   • Description: Package proxy provides support for a variety of protocols to proxy network data.
   • Imported by: 773
   • Comment: 🔴 :
3. golang.org/x/text/encoding/simplifiedchinese (docs)
   • Description: Package simplifiedchinese provides Simplified Chinese encodings such as GBK.
   • Imported by: 755
   • Comment: 🔴
4. golang.org/x/net/publicsuffix (docs)
   • Description: Package publicsuffix provides a public suffix list based on data from https://publicsuffix.org/ A public suffix is one under which Internet users can directly register names.
   • Imported by: 745
   • Comment: 🟠 :
5. golang.org/x/image/math/fixed (docs)
   • Description: Package fixed implements fixed-point integer types.
   • Imported by: 743
   • Comment: 🔴 :
6. golang.org/x/tools/go/ast/astutil (docs)
   • Description: Package astutil contains common utilities for working with the Go AST.
   • Imported by: 740
   • Comment: 🔴 :
7. golang.org/x/crypto/nacl/secretbox (docs)
   • Description: Package secretbox encrypts and authenticates small messages.
   • Imported by: 724
   • Comment: 🟢 :
8. golang.org/x/sys/windows (docs)
   • Description: Package windows contains an interface to the low-level operating system primitives.
   • Imported by: 722
   • Comment: 🟠 :
9. golang.org/x/crypto/blake2b (docs)
   • Description: Package blake2b implements the BLAKE2b hash algorithm defined by RFC 7693 and the extendable output function (XOF) BLAKE2Xb.
   • Imported by: 720
   • Comment: 🔴 :
10. golang.org/x/net/html/atom (docs)
    • Description: Package atom provides integer codes (also known as atoms) for a fixed set of frequently occurring HTML strings: tag names and attribute keys such as "p" and "id".
    • Imported by: 697
    • Comment: 🔴 :

Page 5

1. golang.org/x/crypto/curve25519 (docs)
   • Description: Package curve25519 provides an implementation of the X25519 function, which performs scalar multiplication on the elliptic curve known as Curve25519.
   • Imported by: 687
   • Comment: 🔴 :
2. golang.org/x/net/ipv4 (docs)
   • Description: Package ipv4 implements IP-level socket options for the Internet Protocol version 4.
   • Imported by: 679
   • Comment: 🟠 :
3. golang.org/x/sys/cpu (docs)
   • Description: Package cpu implements processor feature detection for various CPU architectures.
   • Imported by: 673
   • Comment: 🔴 :
4. golang.org/x/oauth2/jwt (docs)
   • Description: Package jwt implements the OAuth 2.0 JSON Web Token flow, commonly known as "two-legged OAuth 2.0".
   • Imported by: 638
   • Comment: 🔴 :
5. golang.org/x/text/encoding/unicode (docs)
   • Description: Package unicode provides Unicode encodings such as UTF-16.
   • Imported by: 634
   • Comment: 🔴 :
6. golang.org/x/net/idna (docs)
   • Description: Package idna implements IDNA2008 using the compatibility processing defined by UTS (Unicode Technical Standard) #46, which defines a standard to deal with the transition from IDNA2003.
   • Imported by: 628
   • Comment: 🟢 :
7. golang.org/x/crypto/openpgp (docs)
   • Description: Package openpgp implements high level operations on OpenPGP messages.
   • Imported by: 607
   • Comment: 🟢 :
8. golang.org/x/net/trace (docs)
   • Description: Package trace implements tracing of requests and long-lived objects.
   • Imported by: 599
   • Comment: 🟠 :
9. golang.org/x/tools/go/ssa (docs)
   • Description: Package ssa defines a representation of the elements of Go programs (packages, types, functions, variables and constants) using a static single-assignment (SSA) form intermediate representation (IR) for the bodies of functions.
   • Imported by: 591
   • Comment: 🔴 :
10. golang.org/x/sync/semaphore (docs)
    • Description: Package semaphore provides a weighted semaphore implementation.
    • Imported by: 588
    • Comment: 🔴 :

Page 6

1. golang.org/x/crypto/ssh/agent (docs)
   • Description: agent implements the ssh-agent protocol, and provides both a client and a server.
   • Imported by: 574
   • Comment: 🔴 :
2. golang.org/x/crypto/hkdf (docs)
   • Description: Package hkdf implements the HMAC-based Extract-and-Expand Key Derivation Function (HKDF) as defined in RFC 5869.
   • Imported by: 566
   • Comment: 🔴 :
3. golang.org/x/tools/go/types/typeutil (docs)
   • Description: typeutil defines various utilities for types, such as Map, a mapping from types.Type to interface{} values.
   • Imported by: 563
   • Comment: 🔴
4. golang.org/x/crypto/chacha20poly1305 (docs)
   • Description: Package chacha20poly1305 implements the ChaCha20-Poly1305 AEAD and its extended nonce variant XChaCha20-Poly1305, as specified in RFC 8439 and draft-irtf-cfrg-xchacha-01.
   • Imported by: 525
   • Comment: 🔴
5. golang.org/x/net/http2/hpack (docs)
   • Description: hpack implements HPACK, a compression format for efficiently representing HTTP header fields in the context of HTTP/2.
   • Imported by: 495
   • Comment: 🟢
6. golang.org/x/tools/go/analysis/passes/inspect (docs)
   • Description: an Analyzer that provides an AST inspector (golang.org/x/tools/go/ast/inspect.Inspect) for the syntax trees of a package.
   • Imported by: 495
   • Comment: 🔴 :
7. golang.org/x/tools/go/ast/inspector (docs)
   • Description: inspector provides helper functions for traversal over the syntax trees of a package, including node filtering by type, and materialization of the traversal stack.
   • Imported by: 494
   • Comment: 🔴
8. golang.org/x/crypto/openpgp/errors (docs)
   • Description: errors contains common error types for the OpenPGP packages.
   • Imported by: 483
   • Comment: 🔴
9. golang.org/x/sync/syncmap (docs)
   • Description: Package syncmap provides a concurrent map implementation.
   • Imported by: 481
   • Comment: 🔴
10. golang.org/x/oauth2/clientcredentials (docs)
    • Description: Package clientcredentials implements the OAuth2.0 "client credentials" token flow, also known as the "two-legged OAuth 2.0".
    • Imported by: 474
    • Comment: 🔴

Page 7

1. golang.org/x/text/encoding/japanese (docs)
   • Description: Package japanese provides Japanese encodings such as EUC-JP and Shift JIS.
   • Imported by: 457
   • Comment: 🔴 :
2. golang.org/x/crypto/openpgp/armor (docs)
   • Description: Package armor implements OpenPGP ASCII Armor, see RFC 4880.
   • Imported by: 446
   • Comment: 🔴
3. golang.org/x/net/ipv6 (docs)
   • Description: Package ipv6 implements IP-level socket options for the Internet Protocol version 6.
   • Imported by: 420
   • Comment: 🟠
4. golang.org/x/net/netutil (docs)
   • Description: Package netutil provides network utility functions, complementing the more common ones in the net package.
   • Imported by: 416
   • Comment: 🟠
5. golang.org/x/crypto/openpgp/packet (docs)
   • Description: Package packet implements parsing and serialization of OpenPGP packets, as specified in RFC 4880.
   • Imported by: 408
   • Comment: 🟠
6. golang.org/x/text/encoding/traditionalchinese (docs)
   • Description: Package traditionalchinese provides Traditional Chinese encodings such as Big5.
   • Imported by: 389
   • Comment: 🔴
7. golang.org/x/crypto/nacl/box (docs)
   • Description: Package box authenticates and encrypts small messages using public-key cryptography.
   • Imported by: 381
   • Comment: 🔴
8. golang.org/x/text/encoding/korean (docs)
   • Description: Package korean provides Korean encodings such as EUC-KR.
   • Imported by: 381
   • Comment: 🔴
9. golang.org/x/crypto/blowfish (docs)
   • Description: Package blowfish implements Bruce Schneier's Blowfish encryption algorithm.
   • Imported by: 358
   • Comment: 🔴
10. golang.org/x/crypto/pkcs12 (docs)
    • Description: Package pkcs12 implements some of PKCS#12.
    • Imported by: 356
    • Comment: 🔴

Page 8

1. golang.org/x/crypto/salsa20/salsa (docs)
   • Description: Package salsa provides low-level access to functions in the Salsa family.
   • Imported by: 356
   • Comment: 🔴
2. golang.org/x/tools/go/buildutil (docs)
   • Description: Package buildutil provides utilities related to the go/build package in the standard library.
   • Imported by: 355
   • Comment: 🔴
3. golang.org/x/tools/godoc/vfs (docs)
   • Description: defines types for abstract file system access and provides an implementation accessing the file system of the underlying OS.Description:
   • Imported by: 350
   • Comment: 🟠
4. golang.org/x/image/bmp (docs)
   • Description: Package bmp implements a BMP image decoder and encoder.
   • Imported by: 279
   • Comment: 🔴
5. golang.org/x/sync/singleflight (docs)
   • Description: Package singleflight provides a duplicate function call suppression mechanism.
   • Imported by: 279
   • Comment: 🔴
6. golang.org/x/tools/go/ssa/ssautil (docs)
   • Description:
   • Imported by: 278
   • Comment: 🔴
7. golang.org/x/crypto/argon2 (docs)
   • Description: Package argon2 implements the key derivation function Argon2.
   • Imported by: 276
   • Comment: 🔴
8. golang.org/x/tools/go/callgraph (docs)
   • Description: Package callgraph defines the call graph and various algorithms and utilities to operate on it.
   • Imported by: 274
   • Comment: 🔴
9. golang.org/x/net/http/httpguts (docs)
   • Description: Package httpguts provides functions implementing various details of the HTTP specification.
   • Imported by: 273
   • Comment: 🟢 :
10. golang.org/x/crypto/ocsp (docs)
    • Description: Package ocsp parses OCSP responses as specified in RFC 2560.
    • Imported by: 272
    • Comment: 🟢 :

Page 9

1. golang.org/x/crypto/poly1305 (docs)
   • Description: Package poly1305 implements Poly1305 one-time message authentication code as specified in https://cr.yp.to/mac/poly1305-20050329.pdf.
   • Imported by: 258
   • Comment: 🔴
2. golang.org/x/oauth2/jws (docs)
   • Description: Package jws provides a partial implementation of JSON Web Signature encoding and decoding.
   • Imported by: 257
   • Comment: 🟠
3. golang.org/x/net/icmp (docs)
   • Description: Package icmp provides basic functions for the manipulation of messages used in the Internet Control Message Protocols, ICMPv4 and ICMPv6.
   • Imported by: 254
   • Comment: 🟠 :
4. golang.org/x/image/colornames (docs)
   • Description: Package colornames provides named colors as defined in the SVG 1.1 spec.
   • Imported by: 251
   • Comment: 🔴
5. golang.org/x/text/message (docs)
   • Description: Package message implements formatted I/O for localized strings with functions analogous to the fmt's print functions.
   • Imported by: 249
   • Comment: 🟢 :
6. golang.org/x/crypto/md4 (docs)
   • Description: Package md4 implements the MD4 hash algorithm as defined in RFC 1320.
   • Imported by: 247
   • Comment: 🔴
7. golang.org/x/image/draw (docs)
   • Description: Package draw provides image composition functions.
   • Imported by: 239
   • Comment: 🔴
8. golang.org/x/sys/windows/svc (docs)
   • Description: Package svc provides everything required to build Windows service.
   • Imported by: 223
   • Comment: 🔴
9. golang.org/x/text/runes (docs)
   • Description: Package runes provide transforms for UTF-8 encoded text.
   • Imported by: 216
   • Comment: 🟠 :
10. golang.org/x/net/bpf (docs)
    • Description: Package bpf implements marshaling and unmarshaling of programs for the Berkeley Packet Filter virtual machine, and provides a Go implementation of the virtual machine.
    • Imported by: 206
    • Comment: 🔴

Page 10

1. golang.org/x/crypto/cast5 (docs)
   • Description: Package cast5 implements CAST5, as defined in RFC 2144.
   • Imported by: 206
   • Comment: 🔴
2. golang.org/x/mobile/event/size (docs)
   • Description: Package size defines an event for the dimensions, physical resolution and orientation of the app's window.
   • Imported by: 201
   • Comment: 🔴
3. golang.org/x/text/encoding/htmlindex (docs)
   • Description: Package htmlindex maps character set encoding names to Encodings as recommended by the W3C for use in HTML 5.
   • Imported by: 198
   • Comment: 🔴
4. golang.org/x/image/tiff (docs)
   • Description: Package tiff implements a TIFF image decoder and encoder.
   • Imported by: 193
   • Comment: 🔴
5. golang.org/x/tools/go/analysis/singlechecker (docs)
   • Description: Package singlechecker defines the main function for an analysis driver with only a single analysis.
   • Imported by: 179
   • Comment: 🔴
6. golang.org/x/crypto/openpgp/s2k (docs)
   • Description: Package s2k implements the various OpenPGP string-to-key transforms as specified in RFC 4800 section 3.7.1.
   • Imported by: 179
   • Comment: 🔴
7. golang.org/x/net/webdav (docs)
   • Description: Package webdav provides a WebDAV server implementation.
   • Imported by: 178
   • Comment: 🔴
8. golang.org/x/crypto/acme (docs)
   • Description: Package acme provides an implementation of the Automatic Certificate Management Environment (ACME) spec.
   • Imported by: 176
   • Comment: 🔴
9. golang.org/x/oauth2/github (docs)
   • Description: Package github provides constants for using OAuth2 to access Github.
   • Imported by: 174
   • Comment: 🔴
10. golang.org/x/image/math/f64 (docs)
    • Description: Package f64 implements float64 vector and matrix types.
    • Imported by: 170
    • Comment: 🔴

Similar to #635.

The suggested mitigation for "Improper Output Neutralization for Logs" includes performing a string replacement of "\n" characters using strings.Replace.

The "%q" formatting directive (double-quoted string safely escaped with Go syntax) is also a form of escaping user input, in particular for plain-text logs, as it escapes non-printable characters (including "\n", "\r", "\t", etc).

However, code using log.Printf("%q", ...) is still considered unsafe by CodeQL scanning.

Here's an example run:
https://github.com/getsentry/sentry-go/pull/399/checks?check_run_id=4583764415

Would it be possible to teach CodeQL about "%q"? There are also variations, such as "%+q", "%#q, "%060q", % q, %[2]q, etc, with the complete parsing rules being non-trivial.

Perhaps use the fact that fmt.(*fmt).fmtQ is called before writing to the io.Writer where the log message is going.

When investigating a failure in one of our pipelines, I noticed I was unable to satisfy the CodeQL query for CWE-326 / InsufficientKeySize the way I'd expect to be able to. I believe this is because it doesn't take into account some combination of switch/if statements that restrict valid key sizes.

I've been able to reproduce this with the test suite (ql/test/query-tests/Security/CWE-326/InsufficientKeySize.go) embedded in the codeql-go repository as follows.

Presently our code looks (to CodeQL) like the simplified:

func foo6() {
    foo8("rsa", 1024) // BAD
}

func foo7() {
    foo8("rsa", 2048) // GOOD
}

func foo8(keyType string, keyBits int) {
    switch keyType {
    case "rsa":
        rsa.GenerateKey(rand.Reader, keyBits)
    default:
        return
    }
}

...except 1024/2048 key sizes are dynamically chosen by the API caller, and additional restrictions are placed on the inputs via other function calls...

I was hoping to solve this as follows:

func foo9() {
    foo11("rsa", 1024) // GOOD
}

func foo10() {
    foo11("rsa", 2048) // GOOD
}

func foo11(keyType string, keyBits int) {
    switch keyType {
    case "rsa":
        if keyBits < 2048 || keyBits > 8192 {
            return
        }

        rsa.GenerateKey(rand.Reader, keyBits)
    default:
        return
    }
}

Here, we explicitly guard, at rsa.GenerateKey call-site, that the key sizes are within a valid, acceptable range. Note that, if foo11 is called as foo11("rsa" 1024), the key will not be generated, due to the if statement's early return in the rsa case branch.

However, when running this test, I generate the following:

[cipherboy@xps15 codeql-go]$ codeql test run ql/test/query-tests/Security/CWE-326/ --search-path . --consistency-queries ql/test/consistency
Executing 2 tests in 1 directories.
Extracting test database in /home/cipherboy/GitHub/github/codeql-go/ql/test/query-tests/Security/CWE-326.
Compiling queries in /home/cipherboy/GitHub/github/codeql-go/ql/test/query-tests/Security/CWE-326.
Executing tests in /home/cipherboy/GitHub/github/codeql-go/ql/test/query-tests/Security/CWE-326.
--- expected
+++ actual
@@ -2,6 +2,10 @@
 | InsufficientKeySize.go:13:10:13:13 | 1024 : int | InsufficientKeySize.go:14:31:14:34 | size |
 | InsufficientKeySize.go:18:7:18:10 | 1024 : int | InsufficientKeySize.go:25:11:25:14 | definition of size : int |
 | InsufficientKeySize.go:25:11:25:14 | definition of size : int | InsufficientKeySize.go:26:31:26:34 | size |
+| InsufficientKeySize.go:30:14:30:17 | 1024 : int | InsufficientKeySize.go:37:27:37:33 | definition of keyBits : int |
+| InsufficientKeySize.go:37:27:37:33 | definition of keyBits : int | InsufficientKeySize.go:40:32:40:38 | keyBits |
+| InsufficientKeySize.go:47:15:47:18 | 1024 : int | InsufficientKeySize.go:54:28:54:34 | definition of keyBits : int |
+| InsufficientKeySize.go:54:28:54:34 | definition of keyBits : int | InsufficientKeySize.go:61:32:61:38 | keyBits |
 nodes
 | InsufficientKeySize.go:9:31:9:34 | 1024 | semmle.label | 1024 |
 | InsufficientKeySize.go:13:10:13:13 | 1024 : int | semmle.label | 1024 : int |
@@ -9,7 +13,15 @@
 | InsufficientKeySize.go:18:7:18:10 | 1024 : int | semmle.label | 1024 : int |
 | InsufficientKeySize.go:25:11:25:14 | definition of size : int | semmle.label | definition of size : int |
 | InsufficientKeySize.go:26:31:26:34 | size | semmle.label | size |
+| InsufficientKeySize.go:30:14:30:17 | 1024 : int | semmle.label | 1024 : int |
+| InsufficientKeySize.go:37:27:37:33 | definition of keyBits : int | semmle.label | definition of keyBits : int |
+| InsufficientKeySize.go:40:32:40:38 | keyBits | semmle.label | keyBits |
+| InsufficientKeySize.go:47:15:47:18 | 1024 : int | semmle.label | 1024 : int |
+| InsufficientKeySize.go:54:28:54:34 | definition of keyBits : int | semmle.label | definition of keyBits : int |
+| InsufficientKeySize.go:61:32:61:38 | keyBits | semmle.label | keyBits |
 #select
 | InsufficientKeySize.go:9:31:9:34 | 1024 | InsufficientKeySize.go:9:31:9:34 | 1024 | InsufficientKeySize.go:9:31:9:34 | 1024 | The size of this RSA key should be at least 2048 bits. |
 | InsufficientKeySize.go:14:31:14:34 | size | InsufficientKeySize.go:13:10:13:13 | 1024 : int | InsufficientKeySize.go:14:31:14:34 | size | The size of this RSA key should be at least 2048 bits. |
 | InsufficientKeySize.go:26:31:26:34 | size | InsufficientKeySize.go:18:7:18:10 | 1024 : int | InsufficientKeySize.go:26:31:26:34 | size | The size of this RSA key should be at least 2048 bits. |
+| InsufficientKeySize.go:40:32:40:38 | keyBits | InsufficientKeySize.go:30:14:30:17 | 1024 : int | InsufficientKeySize.go:40:32:40:38 | keyBits | The size of this RSA key should be at least 2048 bits. |
+| InsufficientKeySize.go:61:32:61:38 | keyBits | InsufficientKeySize.go:47:15:47:18 | 1024 : int | InsufficientKeySize.go:61:32:61:38 | keyBits | The size of this RSA key should be at least 2048 bits. |
[1/2 eval 1.7s] FAILED(RESULT) /home/cipherboy/GitHub/github/codeql-go/ql/test/query-tests/Security/CWE-326/InsufficientKeySize.qlref
[2/2 eval 7ms] PASSED /home/cipherboy/GitHub/github/codeql-go/ql/test/query-tests/Security/CWE-326/CONSISTENCY/UnexpectedFrontendErrors.ql
1 tests passed; 1 tests failed:
  FAILED: /home/cipherboy/GitHub/github/codeql-go/ql/test/query-tests/Security/CWE-326/InsufficientKeySize.qlref

My understanding is this last line states that from foo9 (on line 47 when both excerpts foo6...foo11 are added to the test suite), rsa.GenerateKey(..., 1024) can be reached, which is a false-positive.

I've tried various combinations of:

1.  if keyBits < 2048 {
       return
    }
   
    ... code ...

2.  if keyBits >= 2048 {
        ... code ...
    } else {
        return
    }

3.  if keyBits < 2048 {
       return
    } else {
        ... code ...
    }

and similar, but was unable to make the CodeQL false-positive go away. Note that lines 50...52 (foo10) are absent from the report, because they satisfy the original source condition. Let me know if you need any more information, thank you!

/cc @mickael-hc

A megaproject example is modelling a few dozens web frameworks or the standard library :)

For the following Go code:

regexp.MustCompile(`\Achezmoi\.io\z`)

CodeQL generates the warning "Missing regular expression anchor". However, the regular expression is anchored with the strong anchors \A and \z (which match the beginning and end of the text), not the weaker anchors ^ and $ (which match the start and end of a line anywhere in the text).

To fix this, the regular expression in isInterestingUnanchoredRegexpString needs to be updated to detect the anchors \A and \z here:

Let me know if you'd like a PR, but it's probably a quick fix for a current contributor.