Skip to content

Rule which detects aliasing of values in RangeStmt #443

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Apr 24, 2020
Merged

Rule which detects aliasing of values in RangeStmt #443

merged 1 commit into from
Apr 24, 2020

Conversation

caccavale
Copy link
Contributor

@caccavale caccavale commented Feb 26, 2020

fixes #438

@codecov-io
Copy link

codecov-io commented Feb 26, 2020

Codecov Report

Merging #443 into master will not change coverage.
The diff coverage is n/a.

Impacted file tree graph

@@           Coverage Diff           @@
##           master     #443   +/-   ##
=======================================
  Coverage   71.42%   71.42%           
=======================================
  Files           9        9           
  Lines         651      651           
=======================================
  Hits          465      465           
  Misses        164      164           
  Partials       22       22           

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8662624...ce87709. Read the comment docs.

Copy link
Member

@ccojocar ccojocar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this contribution! This is good work but I would rework a bit the rule to avoid traversing multiple times the tree. You can use a cache and reference back the already parsed elements. See the IntegerOverflowCheck rule. I am looking forward to seeing an improved implementation such that we can merge it. Thanks!


var issue *gosec.Issue
var fun func(n ast.Node) bool
fun = func(n ast.Node) bool {
Copy link
Member

@ccojocar ccojocar Feb 28, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that you can compact these two lines by just:

fun := func(n ast.Node) bool {
}

continue
}

ast.Inspect(element, fun)
Copy link
Member

@ccojocar ccojocar Feb 28, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We try to avoid Inspect in the rules, because will effectively traverse again the tree which is not very efficient. Maybe have a look at this rule which uses a cache to traverse only once the tree

func (i *integerOverflowCheck) Match(node ast.Node, ctx *gosec.Context) (*gosec.Issue, error) {
.

@caccavale
Copy link
Contributor Author

Totally makes sense -- I'm looking into refactoring this to be less big O.

@caccavale
Copy link
Contributor Author

caccavale commented Mar 5, 2020

@ccojocar , is there a way to tell when GoSec leaves an AST node? The issue I'm having is the following:

for _, item := range []string{"A", "B", "C"} {
    foo(item) // This is safe.
}
item := "D"
bar(&item) // Also safe but problematic.

For ease of discussion, let "bad identifier" be an identifier yielded by a range statement which should not be referenced (found as the expression in a unary op with the '&' operator).

I'm trying to collect all bad identifiers and verify they are not referened within the body of the range loop. With the go/ast interface, I push an empty list of bad ID's onto a stack whenever I enter a node-- I add the name of any rangeStmt yielded values onto the top most environment (list in the stack) if the node is a rangeStmt. I pop the stack whenever the visitor.Visit(nil), emulating scope. Anytime I see a unary expression with & I check whether it is referencing a bad ID. It's a similar dance to what you all do with the ignore list in this project. Here's an example where *implicitAliasing has the operative data structure:

func (r *implicitAliasing) Match(n ast.Node, c *gosec.Context) (*gosec.Issue, error) {
	if n == nil {
		r.pop()
		return nil, nil
	}

	r.push([]string{})

	switch node := n.(type) {
	case *ast.RangeStmt:
		if node.Value != nil && node.Tok.String() == ":=" {
			if ident, ok := node.Value.(*ast.Ident); ok {
				r.add(ident.Name)
			}
		}
	case *ast.UnaryExpr:
		if ident, ok := node.X.(*ast.Ident); ok && node.Op.String() == "&" {
			if r.has(ident.Name) {
				return gosec.NewIssue(c, n, r.ID(), r.What, r.Severity, r.Confidence), nil
			}
		}

	return nil, nil
}

From what I can tell, GoSec doesn't have a way for a rule to visit all the nil's during the AST walk. I'm not sure I can do a very good job emulating scoping rules without it.

An alternative would be to carry a list of the bad IDs and invalidate them when they are rebound (filter on assignment statements) but this seems like a less desirable solution since the implementation (based on the names last use) in no way represents the problem it is attempting to solve (scope). I'd also feel less confident that this would catch all the ways that a name could be recycled and it would also be strange that this is invalid Go code but theoretically could get flagged:

for _, item := range []string{"A", "B", "C"} {
    foo(item) // This is safe.
}
bar(&item) // Not valid.

I doubt it makes sense to start passing all the nil's to rules (though it wouldn't be the worse since theoretically the rules can filter them out) but I'm not sure what would be the desired way to advance.

@ccojocar
Copy link
Member

@caccavale I believe we can catch the issue inside of the RangeStmt when an item yielded by the range statement is referenced. Let's try to work out an example:

package main

import "fmt"

var vector []*string

func appendVector(s *string) {
	vector = append(vector, s)
}

func printVector() {
	for _, item := range vector {
		fmt.Printf("%s", *item)
	}
	fmt.Println()
}

func main() {
	for _, item := range []string{"A", "B", "C"} {
		fmt.Printf("%p\n", &item)
		appendVector(&item)
	}

	printVector()
}

I run ./gosecutil -tool ast main.go > ast.txt to generate the AST tree (see it attached). Let's look around the range statement:

   276  .  .  .  .  .  0: *ast.RangeStmt {
   277  .  .  .  .  .  .  For: main.go:19:2
   278  .  .  .  .  .  .  Key: *ast.Ident {
   279  .  .  .  .  .  .  .  NamePos: main.go:19:6
   280  .  .  .  .  .  .  .  Name: "_"
   281  .  .  .  .  .  .  .  Obj: *ast.Object {
   282  .  .  .  .  .  .  .  .  Kind: var
   283  .  .  .  .  .  .  .  .  Name: "_"
   284  .  .  .  .  .  .  .  .  Decl: *ast.AssignStmt {
   285  .  .  .  .  .  .  .  .  .  Lhs: []ast.Expr (len = 2) {
   286  .  .  .  .  .  .  .  .  .  .  0: *(obj @ 278)
   287  .  .  .  .  .  .  .  .  .  .  1: *ast.Ident {
   288  .  .  .  .  .  .  .  .  .  .  .  NamePos: main.go:19:9
   289  .  .  .  .  .  .  .  .  .  .  .  Name: "item"
   290  .  .  .  .  .  .  .  .  .  .  .  Obj: *ast.Object {
   291  .  .  .  .  .  .  .  .  .  .  .  .  Kind: var
   292  .  .  .  .  .  .  .  .  .  .  .  .  Name: "item"
   293  .  .  .  .  .  .  .  .  .  .  .  .  Decl: *(obj @ 284)
   294  .  .  .  .  .  .  .  .  .  .  .  }
   295  .  .  .  .  .  .  .  .  .  .  }
   296  .  .  .  .  .  .  .  .  .  }
   297  .  .  .  .  .  .  .  .  .  TokPos: main.go:19:14
   298  .  .  .  .  .  .  .  .  .  Tok: :=
   299  .  .  .  .  .  .  .  .  .  Rhs: []ast.Expr (len = 1) {
   300  .  .  .  .  .  .  .  .  .  .  0: *ast.UnaryExpr {
   301  .  .  .  .  .  .  .  .  .  .  .  OpPos: main.go:19:17
   302  .  .  .  .  .  .  .  .  .  .  .  Op: range
   303  .  .  .  .  .  .  .  .  .  .  .  X: *ast.CompositeLit {
   304  .  .  .  .  .  .  .  .  .  .  .  .  Type: *ast.ArrayType {
   305  .  .  .  .  .  .  .  .  .  .  .  .  .  Lbrack: main.go:19:23
   306  .  .  .  .  .  .  .  .  .  .  .  .  .  Elt: *ast.Ident {
   307  .  .  .  .  .  .  .  .  .  .  .  .  .  .  NamePos: main.go:19:25
   308  .  .  .  .  .  .  .  .  .  .  .  .  .  .  Name: "string"
   309  .  .  .  .  .  .  .  .  .  .  .  .  .  }
   310  .  .  .  .  .  .  .  .  .  .  .  .  }
   311  .  .  .  .  .  .  .  .  .  .  .  .  Lbrace: main.go:19:31
   312  .  .  .  .  .  .  .  .  .  .  .  .  Elts: []ast.Expr (len = 3) {
   313  .  .  .  .  .  .  .  .  .  .  .  .  .  0: *ast.BasicLit {
   314  .  .  .  .  .  .  .  .  .  .  .  .  .  .  ValuePos: main.go:19:32
   315  .  .  .  .  .  .  .  .  .  .  .  .  .  .  Kind: STRING
   316  .  .  .  .  .  .  .  .  .  .  .  .  .  .  Value: "\"A\""
   317  .  .  .  .  .  .  .  .  .  .  .  .  .  }
   318  .  .  .  .  .  .  .  .  .  .  .  .  .  1: *ast.BasicLit {
   319  .  .  .  .  .  .  .  .  .  .  .  .  .  .  ValuePos: main.go:19:37
   320  .  .  .  .  .  .  .  .  .  .  .  .  .  .  Kind: STRING
   321  .  .  .  .  .  .  .  .  .  .  .  .  .  .  Value: "\"B\""
   322  .  .  .  .  .  .  .  .  .  .  .  .  .  }
   323  .  .  .  .  .  .  .  .  .  .  .  .  .  2: *ast.BasicLit {
   324  .  .  .  .  .  .  .  .  .  .  .  .  .  .  ValuePos: main.go:19:42
   325  .  .  .  .  .  .  .  .  .  .  .  .  .  .  Kind: STRING
   326  .  .  .  .  .  .  .  .  .  .  .  .  .  .  Value: "\"C\""
   327  .  .  .  .  .  .  .  .  .  .  .  .  .  }
   328  .  .  .  .  .  .  .  .  .  .  .  .  }
   329  .  .  .  .  .  .  .  .  .  .  .  .  Rbrace: main.go:19:45
   330  .  .  .  .  .  .  .  .  .  .  .  .  Incomplete: false
   331  .  .  .  .  .  .  .  .  .  .  .  }
   332  .  .  .  .  .  .  .  .  .  .  }
   333  .  .  .  .  .  .  .  .  .  }
   334  .  .  .  .  .  .  .  .  }
   335  .  .  .  .  .  .  .  }
   336  .  .  .  .  .  .  }
   337  .  .  .  .  .  .  Value: *(obj @ 287)
   338  .  .  .  .  .  .  TokPos: main.go:19:14
   339  .  .  .  .  .  .  Tok: :=
   340  .  .  .  .  .  .  X: *(obj @ 303)
   341  .  .  .  .  .  .  Body: *ast.BlockStmt {
   342  .  .  .  .  .  .  .  Lbrace: main.go:19:47
   343  .  .  .  .  .  .  .  List: []ast.Stmt (len = 2) {
   344  .  .  .  .  .  .  .  .  0: *ast.ExprStmt {
   345  .  .  .  .  .  .  .  .  .  X: *ast.CallExpr {
   346  .  .  .  .  .  .  .  .  .  .  Fun: *ast.SelectorExpr {
   347  .  .  .  .  .  .  .  .  .  .  .  X: *ast.Ident {
   348  .  .  .  .  .  .  .  .  .  .  .  .  NamePos: main.go:20:3
   349  .  .  .  .  .  .  .  .  .  .  .  .  Name: "fmt"
   350  .  .  .  .  .  .  .  .  .  .  .  }
   351  .  .  .  .  .  .  .  .  .  .  .  Sel: *ast.Ident {
   352  .  .  .  .  .  .  .  .  .  .  .  .  NamePos: main.go:20:7
   353  .  .  .  .  .  .  .  .  .  .  .  .  Name: "Printf"
   354  .  .  .  .  .  .  .  .  .  .  .  }
   355  .  .  .  .  .  .  .  .  .  .  }
   356  .  .  .  .  .  .  .  .  .  .  Lparen: main.go:20:13
   357  .  .  .  .  .  .  .  .  .  .  Args: []ast.Expr (len = 2) {
   358  .  .  .  .  .  .  .  .  .  .  .  0: *ast.BasicLit {
   359  .  .  .  .  .  .  .  .  .  .  .  .  ValuePos: main.go:20:14
   360  .  .  .  .  .  .  .  .  .  .  .  .  Kind: STRING
   361  .  .  .  .  .  .  .  .  .  .  .  .  Value: "\"%p\\n\""
   362  .  .  .  .  .  .  .  .  .  .  .  }
   363  .  .  .  .  .  .  .  .  .  .  .  1: *ast.UnaryExpr {
   364  .  .  .  .  .  .  .  .  .  .  .  .  OpPos: main.go:20:22
   365  .  .  .  .  .  .  .  .  .  .  .  .  Op: &
   366  .  .  .  .  .  .  .  .  .  .  .  .  X: *ast.Ident {
   367  .  .  .  .  .  .  .  .  .  .  .  .  .  NamePos: main.go:20:23
   368  .  .  .  .  .  .  .  .  .  .  .  .  .  Name: "item"
   369  .  .  .  .  .  .  .  .  .  .  .  .  .  Obj: *(obj @ 290)
   370  .  .  .  .  .  .  .  .  .  .  .  .  }
   371  .  .  .  .  .  .  .  .  .  .  .  }
   372  .  .  .  .  .  .  .  .  .  .  }
   373  .  .  .  .  .  .  .  .  .  .  Ellipsis: -
   374  .  .  .  .  .  .  .  .  .  .  Rparen: main.go:20:27
   375  .  .  .  .  .  .  .  .  .  }
   376  .  .  .  .  .  .  .  .  }
   377  .  .  .  .  .  .  .  .  1: *ast.ExprStmt {
   378  .  .  .  .  .  .  .  .  .  X: *ast.CallExpr {
   379  .  .  .  .  .  .  .  .  .  .  Fun: *ast.Ident {
   380  .  .  .  .  .  .  .  .  .  .  .  NamePos: main.go:21:3
   381  .  .  .  .  .  .  .  .  .  .  .  Name: "appendVector"
   382  .  .  .  .  .  .  .  .  .  .  .  Obj: *(obj @ 59)
   383  .  .  .  .  .  .  .  .  .  .  }
   384  .  .  .  .  .  .  .  .  .  .  Lparen: main.go:21:15
   385  .  .  .  .  .  .  .  .  .  .  Args: []ast.Expr (len = 1) {
   386  .  .  .  .  .  .  .  .  .  .  .  0: *ast.UnaryExpr {
   387  .  .  .  .  .  .  .  .  .  .  .  .  OpPos: main.go:21:16
   388  .  .  .  .  .  .  .  .  .  .  .  .  Op: &
   389  .  .  .  .  .  .  .  .  .  .  .  .  X: *ast.Ident {
   390  .  .  .  .  .  .  .  .  .  .  .  .  .  NamePos: main.go:21:17
   391  .  .  .  .  .  .  .  .  .  .  .  .  .  Name: "item"
   392  .  .  .  .  .  .  .  .  .  .  .  .  .  Obj: *(obj @ 290)
   393  .  .  .  .  .  .  .  .  .  .  .  .  }
   394  .  .  .  .  .  .  .  .  .  .  .  }
   395  .  .  .  .  .  .  .  .  .  .  }
   396  .  .  .  .  .  .  .  .  .  .  Ellipsis: -
   397  .  .  .  .  .  .  .  .  .  .  Rparen: main.go:21:21
   398  .  .  .  .  .  .  .  .  .  }
   399  .  .  .  .  .  .  .  .  }
   400  .  .  .  .  .  .  .  }
   401  .  .  .  .  .  .  .  Rbrace: main.go:22:2
   402  .  .  .  .  .  .  }
   403  .  .  .  .  .  }

We can have something like this:

  • On RangeStmt you can lookup for the AssignStmt and then you can get the Obj address:
   290  .  .  .  .  .  .  .  .  .  .  .  Obj: *ast.Object {
   291  .  .  .  .  .  .  .  .  .  .  .  .  Kind: var
   292  .  .  .  .  .  .  .  .  .  .  .  .  Name: "item"
   293  .  .  .  .  .  .  .  .  .  .  .  .  Decl: *(obj @ 284)
   294  .  .  .  .  .  .  .  .  .  .  .  }
  • You can store in a map this address with the RangStmt address as value
  • You can then keep watching for UnaryExpr and get their Obj address if the Op is a reference such as &:
   363  .  .  .  .  .  .  .  .  .  .  .  1: *ast.UnaryExpr {
   364  .  .  .  .  .  .  .  .  .  .  .  .  OpPos: main.go:20:22
   365  .  .  .  .  .  .  .  .  .  .  .  .  Op: &
   366  .  .  .  .  .  .  .  .  .  .  .  .  X: *ast.Ident {
   367  .  .  .  .  .  .  .  .  .  .  .  .  .  NamePos: main.go:20:23
   368  .  .  .  .  .  .  .  .  .  .  .  .  .  Name: "item"
   369  .  .  .  .  .  .  .  .  .  .  .  .  .  Obj: *(obj @ 290)
   370  .  .  .  .  .  .  .  .  .  .  .  .  }
   371  .  .  .  .  .  .  .  .  .  .  .  }
  • Look up the object address in the range map, if the object is present, we can create a gosec issue.

I think we can optimise this a bit by storing also the position of the Rbrace of theBlockStmt of the Body from the RangeStmt in the map (see Rbrace):

	RangeStmt struct {
		For        token.Pos   // position of "for" keyword
		Key, Value Expr        // Key, Value may be nil
		TokPos     token.Pos   // position of Tok; invalid if Key == nil
		Tok        token.Token // ILLEGAL if Key == nil, ASSIGN, DEFINE
		X          Expr        // value to range over
		Body       *BlockStmt
	}
	// A BlockStmt node represents a braced statement list.
	BlockStmt struct {
		Lbrace token.Pos // position of "{"
		List   []Stmt
		Rbrace token.Pos // position of "}", if any (may be absent due to syntax error)
	}

On the first UnaryExpr out of this position, we can reset the map (see OpPos)

	UnaryExpr struct {
		OpPos token.Pos   // position of Op
		Op    token.Token // operator
		X     Expr        // operand
	}

From that point on, we don't check anymore UnaryExpr nodes as long as a new RangeStmt is not recorded in the map (e.g. map is empty).

ast.txt

@ccojocar
Copy link
Member

@caccavale Do you need any additional help to be able to move forward with this rule improvement? Thanks

@caccavale
Copy link
Contributor Author

Sorry for the delay! Hows it look?

Thanks for the detailed explanation/walkthrough of the AST. I didn't realize the strength in the .Obj references being intentionally aliased. Let me know how to proceed from here.

Copy link
Member

@ccojocar ccojocar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for following up with a more optimal implementation. it looks good! I am going to merge it like this and see if there are any complains about false positives.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add a rule to detect implicit aliasing issues with RangeStmt's
3 participants