Hey @jrasell!

I did want to note that while this init function avoids the string concatenation overhead, this will not avoid the map regrowth cost also seen. It would be nice if we could look into this too, even if the investigation determines avoiding this is not possible.

funcMap is called once per template execution. The map must be created or significantly modified on each execution because the functions it contains are closures. They depend on template specific things, which is especially true for any of the Nomad specific functions (like nomadServices).

Because these functions capture variables (i.brain, i.used, i.env,i.destination, etc.) from the specific funcMapInput structure, it's not possible to simply create one global function map containing these functions. Doing so would cause all template executions to incorrectly share or use stale state (brain, used/missing sets, environment variables, destination paths) from whichever execution happened to create the global map first.

I am also curious on your thoughts of how using a private global variable and init function might impact applications such as Nomad, which utilise consul-template to run and manage any number of templates on a single host via a single control application.

I think this will be a nice improvement for scenarios where templates are executed in masses. The most expensive part of the funcMap function was about iterating through those 210 functions, doing string concatenations and inserting every time. Now Nomad can initialise the sprig functions exactly once upon package init, when the process starts, and upon template executions some copying is needed for the reasons mentioned above.

I prepared a Go benchmark for this:

package main

import (
	"testing"

	"github.com/hashicorp/consul-template/template"
)

const benchmarkTemplateContent = `{{ sprig_uuidv4 }}`

func BenchmarkTemplateExecute(b *testing.B) {
	brain := template.NewBrain()
	tmplInput := &template.NewTemplateInput{
		Contents: benchmarkTemplateContent,
	}
	tmpl, err := template.NewTemplate(tmplInput)
	if err != nil {
		b.Fatalf("Setup: Failed to create template: %v", err)
	}
	execInput := &template.ExecuteInput{
		Brain: brain,
	}
	b.ReportAllocs()
	b.ResetTimer()
	for i := 0; i < b.N; i++ {
		_, err := tmpl.Execute(execInput)
		if err != nil {
			b.Fatalf("Execution failed during benchmark: %v", err)
		}
	}
}

Results with go test -bench=. -benchmem -count=5 on Apple M1 Pro:

Main branch:

cpu: Apple M1 Pro
BenchmarkTemplateExecute-8         12168             97964 ns/op          131559 B/op        322 allocs/op
BenchmarkTemplateExecute-8         12291             97795 ns/op          131560 B/op        322 allocs/op
BenchmarkTemplateExecute-8         12235             97701 ns/op          131559 B/op        322 allocs/op
BenchmarkTemplateExecute-8         12276             98948 ns/op          131559 B/op        322 allocs/op
BenchmarkTemplateExecute-8         12171             98465 ns/op          131559 B/op        322 allocs/op

Proposed fix branch:

cpu: Apple M1 Pro
BenchmarkTemplateExecute-8         13532             89330 ns/op          118319 B/op        109 allocs/op
BenchmarkTemplateExecute-8         13861             86195 ns/op          118318 B/op        109 allocs/op
BenchmarkTemplateExecute-8         13850             86305 ns/op          118319 B/op        109 allocs/op
BenchmarkTemplateExecute-8         13882             86529 ns/op          118319 B/op        109 allocs/op
BenchmarkTemplateExecute-8         14067             85905 ns/op          118319 B/op        109 allocs/op

That's roughly 10% improvement on execution time and memory usage, and whopping 66% reduction in memory allocations.

In the PR code at the moment, it would be possible to create the funcmap with a fixed/known size, as it's possible to know the size of the sprig map before creating this I believe. This would avoid unnecessary map growth work in this area.

Since the init function is ran exactly once I think this won't affect the performance that much, but it's good practice anyway. Pushed a fix, thanks.

Consul Enterprise related test is fixed in #2053 - but even with the fix it fails, possibly due to an outdated license in the repository secrets.

LGTM