Skip to content

Commit 12a55fa

Browse files
niamsterTakashi Kusumi
niamster
and
Takashi Kusumi
authored
allow flexible log parsing and formatting (#239)
* cmd: append NL to the template if it does not have one * cmd: add color functions to the template parser * cmd: add formatTsRFC3339Nano helper function * cmd: add an option to get template from file * cmd: add tryParseJSON that does not return an error so one can use it as {{ with $msg := .Message | tryParseJSON }} {{ else }} {{ .Message }} {{ end }} * update readme * Revert "cmd: append NL to the template if it does not have one" This reverts commit 1bef6be. * rename formatTsRFC3339Nano to toRFC3339Nano * tryParseJSON: ensure that `UseNumber` is used Co-authored-by: Takashi Kusumi <tkusumi@zlab.co.jp> * cmd: precise that --template-file overrides --template * readme: add example of using template using --template-file and example of advanced parsing JSON logs * add a few test for --template-file * add toUTC to examples * do not expose `toUTC` in examples Co-authored-by: Takashi Kusumi <tkusumi@zlab.co.jp> * readme: add new functions to the reference table --------- Co-authored-by: Takashi Kusumi <tkusumi@zlab.co.jp>
1 parent 24c8716 commit 12a55fa

File tree

6 files changed

+131
-8
lines changed

6 files changed

+131
-8
lines changed

‎README.md

Lines changed: 32 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -95,6 +95,7 @@ Supported Kubernetes resources are `pod`, `replicationcontroller`, `service`, `d
9595
`--since`, `-s` | `48h0m0s` | Return logs newer than a relative duration like 5s, 2m, or 3h.
9696
`--tail` | `-1` | The number of lines from the end of the logs to show. Defaults to -1, showing all logs.
9797
`--template` | | Template to use for log lines, leave empty to use --output flag.
98+
`--template-file`, `-T` | | Path to template to use for log lines, leave empty to use --output flag. It overrides --template option.
9899
`--timestamps`, `-t` | `false` | Print timestamps.
99100
`--timezone` | `Local` | Set timestamps to specific timezone.
100101
`--verbosity` | `0` | Number of the log level verbosity
@@ -133,13 +134,25 @@ will receive the following struct:
133134
The following functions are available within the template (besides the [builtin
134135
functions](https://golang.org/pkg/text/template/#hdr-Functions)):
135136

136-
| func | arguments | description |
137-
|-------------|-----------------------|-----------------------------------------------------------------|
138-
| `json` | `object` | Marshal the object and output it as a json text |
139-
| `color` | `color.Color, string` | Wrap the text in color (.ContainerColor and .PodColor provided) |
140-
| `parseJSON` | `string` | Parse string as JSON |
141-
| `extjson` | `string` | Parse the object as json and output colorized json |
142-
| `ppextjson` | `string` | Parse the object as json and output pretty-print colorized json |
137+
| func | arguments | description |
138+
|-----------------|-----------------------|-----------------------------------------------------------------------------------|
139+
| `json` | `object` | Marshal the object and output it as a json text |
140+
| `color` | `color.Color, string` | Wrap the text in color (.ContainerColor and .PodColor provided) |
141+
| `parseJSON` | `string` | Parse string as JSON |
142+
| `tryParseJSON` | `string` | Attemp to parse string as JSON, return nil on failure |
143+
| `extjson` | `string` | Parse the object as json and output colorized json |
144+
| `ppextjson` | `string` | Parse the object as json and output pretty-print colorized json |
145+
| `toRFC3339Nano` | `object` | Parse timestamp (string, int, json.Number) and output it using RFC3339Nano format |
146+
| `levelColor` | `string` | Print log level using appropriate color |
147+
| `colorBlack` | `string` | Print text using black color |
148+
| `colorRed` | `string` | Print text using red color |
149+
| `colorGreen` | `string` | Print text using green color |
150+
| `colorYellow` | `string` | Print text using yellow color |
151+
| `colorBlue` | `string` | Print text using blue color |
152+
| `colorMagenta` | `string` | Print text using magenta color |
153+
| `colorCyan` | `string` | Print text using cyan color |
154+
| `colorWhite` | `string` | Print text using white color |
155+
143156

144157
### Log level verbosity
145158

@@ -247,6 +260,18 @@ Output using a custom template with `parseJSON`:
247260
stern --template='{{.PodName}}/{{.ContainerName}} {{with $d := .Message | parseJSON}}[{{$d.level}}] {{$d.message}}{{end}}{{"\n"}}' backend
248261
```
249262

263+
Output using a custom template that tries to parse JSON or fallbacks to raw format:
264+
265+
```
266+
stern --template='{{.PodName}}/{{.ContainerName}} {{ with $msg := .Message | tryParseJSON }}[{{ colorGreen (toRFC3339Nano $msg.ts) }}] {{ levelColor $msg.level }} ({{ colorCyan $msg.caller }}) {{ $msg.msg }}{{ else }} {{ .Message }} {{ end }}{{"\n"}}' backend
267+
```
268+
269+
Load custom template from file:
270+
271+
```
272+
stern --template-file=~/.stern.tpl backend
273+
```
274+
250275
Trigger the interactive prompt to select an 'app.kubernetes.io/instance' label value:
251276

252277
```

‎cmd/cmd.go

Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,7 @@ import (
1919
"encoding/json"
2020
goflag "flag"
2121
"fmt"
22+
"os"
2223
"regexp"
2324
"strconv"
2425
"strings"
@@ -29,6 +30,7 @@ import (
2930

3031
"github.com/fatih/color"
3132
"github.com/pkg/errors"
33+
"github.com/spf13/cast"
3234
"github.com/spf13/cobra"
3335
"github.com/spf13/pflag"
3436
"github.com/stern/stern/stern"
@@ -62,6 +64,7 @@ type options struct {
6264
version bool
6365
completion string
6466
template string
67+
templateFile string
6568
output string
6669
prompt bool
6770
podQuery string
@@ -85,6 +88,7 @@ func NewOptions(streams genericclioptions.IOStreams) *options {
8588
since: 48 * time.Hour,
8689
tail: -1,
8790
template: "",
91+
templateFile: "",
8892
timestamps: false,
8993
timezone: "Local",
9094
prompt: false,
@@ -302,6 +306,7 @@ func (o *options) AddFlags(fs *pflag.FlagSet) {
302306
fs.DurationVarP(&o.since, "since", "s", o.since, "Return logs newer than a relative duration like 5s, 2m, or 3h.")
303307
fs.Int64Var(&o.tail, "tail", o.tail, "The number of lines from the end of the logs to show. Defaults to -1, showing all logs.")
304308
fs.StringVar(&o.template, "template", o.template, "Template to use for log lines, leave empty to use --output flag.")
309+
fs.StringVarP(&o.templateFile, "template-file", "T", o.templateFile, "Path to template to use for log lines, leave empty to use --output flag. It overrides --template option.")
305310
fs.BoolVarP(&o.timestamps, "timestamps", "t", o.timestamps, "Print timestamps.")
306311
fs.StringVar(&o.timezone, "timezone", o.timezone, "Set timestamps to specific timezone.")
307312
fs.BoolVar(&o.onlyLogLines, "only-log-lines", o.onlyLogLines, "Print only log lines")
@@ -311,6 +316,13 @@ func (o *options) AddFlags(fs *pflag.FlagSet) {
311316

312317
func (o *options) generateTemplate() (*template.Template, error) {
313318
t := o.template
319+
if o.templateFile != "" {
320+
data, err := os.ReadFile(o.templateFile)
321+
if err != nil {
322+
return nil, err
323+
}
324+
t = string(data)
325+
}
314326
if t == "" {
315327
switch o.output {
316328
case "default":
@@ -348,6 +360,15 @@ func (o *options) generateTemplate() (*template.Template, error) {
348360
}
349361
return string(b), nil
350362
},
363+
"tryParseJSON": func(text string) map[string]interface{} {
364+
decoder := json.NewDecoder(strings.NewReader(text))
365+
decoder.UseNumber()
366+
obj := make(map[string]interface{})
367+
if err := decoder.Decode(&obj); err != nil {
368+
return nil
369+
}
370+
return obj
371+
},
351372
"parseJSON": func(text string) (map[string]interface{}, error) {
352373
obj := make(map[string]interface{})
353374
if err := json.Unmarshal([]byte(text), &obj); err != nil {
@@ -365,9 +386,44 @@ func (o *options) generateTemplate() (*template.Template, error) {
365386
}
366387
return strings.TrimSuffix(string(b), "\n"), nil
367388
},
389+
"toRFC3339Nano": func(ts any) string {
390+
return cast.ToTime(ts).Format(time.RFC3339Nano)
391+
},
392+
"toUTC": func(ts any) time.Time {
393+
return cast.ToTime(ts).UTC()
394+
},
368395
"color": func(color color.Color, text string) string {
369396
return color.SprintFunc()(text)
370397
},
398+
"colorBlack": color.BlackString,
399+
"colorRed": color.RedString,
400+
"colorGreen": color.GreenString,
401+
"colorYellow": color.YellowString,
402+
"colorBlue": color.BlueString,
403+
"colorMagenta": color.MagentaString,
404+
"colorCyan": color.CyanString,
405+
"colorWhite": color.WhiteString,
406+
"levelColor": func(level string) string {
407+
var levelColor *color.Color
408+
switch strings.ToLower(level) {
409+
case "debug":
410+
levelColor = color.New(color.FgMagenta)
411+
case "info":
412+
levelColor = color.New(color.FgBlue)
413+
case "warn":
414+
levelColor = color.New(color.FgYellow)
415+
case "error":
416+
levelColor = color.New(color.FgRed)
417+
case "dpanic":
418+
levelColor = color.New(color.FgRed)
419+
case "panic":
420+
levelColor = color.New(color.FgRed)
421+
case "fatal":
422+
levelColor = color.New(color.FgCyan)
423+
default:
424+
}
425+
return levelColor.SprintFunc()(level)
426+
},
371427
}
372428
template, err := template.New("log").Funcs(funs).Parse(t)
373429
if err != nil {

‎cmd/cmd_test.go

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -296,6 +296,42 @@ func TestOptionsGenerateTemplate(t *testing.T) {
296296
"",
297297
true,
298298
},
299+
{
300+
"template-file",
301+
func() *options {
302+
o := NewOptions(streams)
303+
o.templateFile = "test.tpl"
304+
305+
return o
306+
}(),
307+
"template message",
308+
"pod1 container1 template message",
309+
false,
310+
},
311+
{
312+
"template-file-json-log-ts-float",
313+
func() *options {
314+
o := NewOptions(streams)
315+
o.templateFile = "test.tpl"
316+
317+
return o
318+
}(),
319+
`{"ts": 123, "level": "INFO", "msg": "template message"}`,
320+
"pod1 container1 [1970-01-01T00:02:03Z] INFO template message",
321+
false,
322+
},
323+
{
324+
"template-file-json-log-ts-str",
325+
func() *options {
326+
o := NewOptions(streams)
327+
o.templateFile = "test.tpl"
328+
329+
return o
330+
}(),
331+
`{"ts": "1970-01-01T01:02:03+01:00", "level": "INFO", "msg": "template message"}`,
332+
"pod1 container1 [1970-01-01T00:02:03Z] INFO template message",
333+
false,
334+
},
299335
}
300336

301337
for _, tt := range tests {

‎cmd/test.tpl

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
{{color .PodColor .PodName}} {{color .ContainerColor .ContainerName}} {{ with $msg := .Message | tryParseJSON }}[{{ colorGreen (toRFC3339Nano (toUTC $msg.ts)) }}] {{ levelColor $msg.level }} {{ $msg.msg }}{{ else }}{{ .Message }}{{ end }}

‎go.mod

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,7 @@ require (
66
github.com/AlecAivazis/survey/v2 v2.3.6
77
github.com/fatih/color v1.13.0
88
github.com/pkg/errors v0.9.1
9+
github.com/spf13/cast v1.5.0
910
github.com/spf13/cobra v1.6.1
1011
github.com/spf13/pflag v1.0.5
1112
golang.org/x/sync v0.1.0

‎go.sum

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -38,6 +38,7 @@ github.com/evanphx/json-patch v5.6.0+incompatible/go.mod h1:50XU6AFN0ol/bzJsmQLi
3838
github.com/fatih/color v1.13.0 h1:8LOYc1KYPPmyKMuN8QV2DNRWNbLo6LZ0iLs8+mlH53w=
3939
github.com/fatih/color v1.13.0/go.mod h1:kLAiJbzzSOZDVNGyDpeOxJ47H46qBXwg5ILebYFFOfk=
4040
github.com/flowstack/go-jsonschema v0.1.1/go.mod h1:yL7fNggx1o8rm9RlgXv7hTBWxdBM0rVwpMwimd3F3N0=
41+
github.com/frankban/quicktest v1.14.3 h1:FJKSZTDHjyhriyC81FLQ0LY93eSai0ZyR/ZIkd3ZUKE=
4142
github.com/ghodss/yaml v1.0.0/go.mod h1:4dBDuWmgqj2HViK6kFavaiC9ZROes6MMH2rRYeMEF04=
4243
github.com/go-errors/errors v1.4.2 h1:J6MZopCL4uSllY1OfXM374weqZFFItUbrImctkmUxIA=
4344
github.com/go-errors/errors v1.4.2/go.mod h1:sIVyrIiJhuEF+Pj9Ebtd6P/rEYROXFi3BopGUQ5a5Og=
@@ -109,8 +110,8 @@ github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51/go.mod h1:C
109110
github.com/kisielk/errcheck v1.5.0/go.mod h1:pFxgyoBC7bSaBwPgfKdkLd5X25qrDl4LWUI2bnpBCr8=
110111
github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck=
111112
github.com/kr/pretty v0.2.0/go.mod h1:ipq/a2n7PKx3OHsz4KJII5eveXtPO4qwEXGdVfWzfnI=
112-
github.com/kr/pretty v0.2.1 h1:Fmg33tUaq4/8ym9TJN1x7sLJnHVwhP33CNkpYV/7rwI=
113113
github.com/kr/pretty v0.2.1/go.mod h1:ipq/a2n7PKx3OHsz4KJII5eveXtPO4qwEXGdVfWzfnI=
114+
github.com/kr/pretty v0.3.0 h1:WgNl7dwNpEZ6jJ9k1snq4pZsg7DOEN8hP9Xw0Tsjwk0=
114115
github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
115116
github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=
116117
github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
@@ -151,9 +152,12 @@ github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZb
151152
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
152153
github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA=
153154
github.com/rogpeppe/fastuuid v1.2.0/go.mod h1:jVj6XXZzXRy/MSR5jhDC/2q6DgLz+nrA6LYCDYWNEvQ=
155+
github.com/rogpeppe/go-internal v1.6.1 h1:/FiVV8dS/e+YqF2JvO3yXRFbBLTIuSDkuC7aBOAvL+k=
154156
github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
155157
github.com/sergi/go-diff v1.1.0 h1:we8PVUC3FE2uYfodKH/nBHMSetSfHDR6scGdBi+erh0=
156158
github.com/spaolacci/murmur3 v0.0.0-20180118202830-f09979ecbc72/go.mod h1:JwIasOWyU6f++ZhiEuf87xNszmSA2myDM2Kzu9HwQUA=
159+
github.com/spf13/cast v1.5.0 h1:rj3WzYc11XZaIZMPKmwP96zkFEnnAmV8s6XbB2aY32w=
160+
github.com/spf13/cast v1.5.0/go.mod h1:SpXXQ5YoyJw6s3/6cMTQuxvgRl3PCJiyaX9p6b155UU=
157161
github.com/spf13/cobra v1.6.1 h1:o94oiPyS4KD1mPy2fmcYYHHfCxLqYjJOhGsCHFZtEzA=
158162
github.com/spf13/cobra v1.6.1/go.mod h1:IOw/AERYS7UzyrGinqmz6HLUo219MORXGxhbaJUqzrY=
159163
github.com/spf13/pflag v1.0.5 h1:iy+VFUOCP1a+8yFto/drg2CJ5u0yRoB7fZw3DKv/JXA=

0 commit comments

Comments
 (0)