Support matching JSON body with CEL expressions #1255

juho9000 · 2024-06-05T12:00:28Z

This PR implements parsing a JSON response from the body and checking the result using a CEL query. Looking for feedback on this.

Sample usage:

modules:
  http_2xx:
    prober: http
    http:
      fail_if_body_json_not_matches_cel: "body.foo.bar == 'baz'"

If response is {"foo": { "bar": "qux" } } the probe fails. fail_if_body_json_matches_cel is also implemented and should work as expected.

juho9000 · 2024-06-17T11:16:36Z

@roidelapluie @mem Ping :)

mem

A couple of minor comments.

CONFIGURATION.md

prober/http.go

mem · 2024-07-23T17:46:08Z

Thank you for this, this seems useful!

Question, I have no experience with CEL... is it possible to use it to match HTML, too? I see in the code that Eval takes a map[string]any, but I couldn't spot how that any is handled.

The reason I'm asking is because I'm trying to figure out if it's possible to extend this to more inputs, or if it needs to be restricted to JSON.

prober/http.go

juho9000 · 2024-08-01T13:05:22Z

Thank you for this, this seems useful!

Question, I have no experience with CEL... is it possible to use it to match HTML, too? I see in the code that Eval takes a map[string]any, but I couldn't spot how that any is handled.

The reason I'm asking is because I'm trying to figure out if it's possible to extend this to more inputs, or if it needs to be restricted to JSON.

I atleast do not know how CEL could be used to match HTML in any sane way. You might want to check this out https://github.com/google/cel-spec/blob/master/doc/langdef.md#json-data-conversion

Thanks for your comments and suggestions so far. Really hoping we can get this merged. I think that having the ability to validate JSON responses would be an awesome addition to blackbox exporter. :)

zetaab · 2024-08-07T09:18:55Z

@mem kindly ping :) we are interested about this feature and would like to see it as well

zetaab · 2024-09-03T17:01:09Z

@mem @roidelapluie any comments to this PR?

juho9000 · 2024-10-01T14:41:33Z

@electron0zero @mem I haven't heard from any of the maintainers in a while. I fixed or clarified everything that was mentioned in the review comments. The initial message I got was that this seems useful, but it seems that it's forgotten now. Any update? I'd love to get this merged in order to avoid maintaining my own fork. :)

zetaab · 2024-12-09T09:47:59Z

ping @electron0zero @mem could you give some opinion about this?

juho9000 · 2025-01-23T13:15:56Z

@mem I'm willing to donate you a sixpack in exchange for a review at this point. Please either review this or close it if it's not going to make its way into blackbox exporter.

juho9000 · 2025-02-26T12:34:59Z

Signed-off-by: Juho Majasaari <juho.majasaari@iki.fi>

.idea/.gitignore

Signed-off-by: Juho Majasaari <juho.majasaari@iki.fi>

dgl

I started reviewing if JSON types other than objects would work, in the end I wrote some tests and made them work.

(This will probably need gofmt before committing, I should just have pushed a commit...)

config/config.go

prober/http_test.go

Co-authored-by: David Leadbeater <dgl@dgl.cx> Signed-off-by: Juho Majasaari <juho.majasaari@iki.fi>

dgl

LGTM, after comment resolved. Thanks! I'll leave final review to a maintainer though.

example.yml

Signed-off-by: Juho Majasaari <juho.majasaari@iki.fi>

electron0zero · 2025-03-13T18:19:42Z

thank you for your contribution , and patience with the review process 🙇

* [FEATURE] Support matching JSON body with CEL expressions #1255 * [BUGFIX] Fix condition when local dns lookup should happen #1272 * [BUGFIX] Stop scrape logger spam #1381 Signed-off-by: SuperQ <superq@gmail.com>

juho9000 force-pushed the feature/cel-json-body branch from 45e8b9c to 2f2d0fb Compare June 5, 2024 12:25

mem reviewed Jul 23, 2024

View reviewed changes

CONFIGURATION.md Outdated Show resolved Hide resolved

CONFIGURATION.md Outdated Show resolved Hide resolved

prober/http.go Outdated Show resolved Hide resolved

mem reviewed Jul 23, 2024

View reviewed changes

prober/http.go Outdated Show resolved Hide resolved

juho9000 force-pushed the feature/cel-json-body branch 2 times, most recently from 0c9b9a0 to f9a0036 Compare August 1, 2024 12:56

juho9000 requested a review from mem August 1, 2024 13:06

juho9000 force-pushed the feature/cel-json-body branch from f9a0036 to b9b903b Compare August 8, 2024 10:13

juho9000 force-pushed the feature/cel-json-body branch 3 times, most recently from fbb6791 to 9131539 Compare August 29, 2024 08:36

juho9000 force-pushed the feature/cel-json-body branch 2 times, most recently from 3101080 to 2b1939d Compare September 3, 2024 07:40

juho9000 force-pushed the feature/cel-json-body branch 3 times, most recently from a07544b to 704e04a Compare December 17, 2024 11:33

juho9000 force-pushed the feature/cel-json-body branch from 704e04a to 730f1ac Compare January 7, 2025 08:41

juho9000 force-pushed the feature/cel-json-body branch 2 times, most recently from 46b9342 to d6df03c Compare January 23, 2025 13:21

juho9000 force-pushed the feature/cel-json-body branch from d6df03c to 41ed81a Compare February 26, 2025 12:32

electron0zero requested review from SuperQ and electron0zero February 26, 2025 12:51

juho9000 added 3 commits March 3, 2025 12:10

Support matching JSON body with CEL expressions

be40dc1

Signed-off-by: Juho Majasaari <juho.majasaari@iki.fi>

Use ContextEval with Cel to ensure timeouts are respected

9937d54

Signed-off-by: Juho Majasaari <juho.majasaari@iki.fi>

Log Cel expressions on failures

4dd6b52

Signed-off-by: Juho Majasaari <juho.majasaari@iki.fi>

juho9000 force-pushed the feature/cel-json-body branch from 41ed81a to e60ce42 Compare March 3, 2025 10:11

juho9000 requested review from electron0zero and dgl March 4, 2025 07:40

juho9000 force-pushed the feature/cel-json-body branch from e60ce42 to 8bb2ecb Compare March 4, 2025 08:50

dgl reviewed Mar 5, 2025

View reviewed changes

.idea/.gitignore Outdated Show resolved Hide resolved

juho9000 added 6 commits March 5, 2025 14:39

bump cel-go version

513e36f

Signed-off-by: Juho Majasaari <juho.majasaari@iki.fi>

improve docs and add example on cel usage

bb07db2

Signed-off-by: Juho Majasaari <juho.majasaari@iki.fi>

improve logging on json unmarshalling failure

c2575a6

Signed-off-by: Juho Majasaari <juho.majasaari@iki.fi>

add cel failure metric only if cel options are used

5b47ac7

Signed-off-by: Juho Majasaari <juho.majasaari@iki.fi>

use consistent naming, add tests

76e06a5

Signed-off-by: Juho Majasaari <juho.majasaari@iki.fi>

add tests

ce74f98

Signed-off-by: Juho Majasaari <juho.majasaari@iki.fi>

juho9000 force-pushed the feature/cel-json-body branch from 8bb2ecb to ce74f98 Compare March 5, 2025 12:39

juho9000 requested a review from dgl March 6, 2025 09:55

dgl reviewed Mar 7, 2025

View reviewed changes

config/config.go Outdated Show resolved Hide resolved

prober/http_test.go Outdated Show resolved Hide resolved

prober/http_test.go Show resolved Hide resolved

prober/http_test.go Outdated Show resolved Hide resolved

prober/http_test.go Show resolved Hide resolved

support other json types besides just objects

edca4bb

Co-authored-by: David Leadbeater <dgl@dgl.cx> Signed-off-by: Juho Majasaari <juho.majasaari@iki.fi>

juho9000 force-pushed the feature/cel-json-body branch from 6f0ca3a to edca4bb Compare March 7, 2025 10:10

juho9000 requested a review from dgl March 7, 2025 10:20

dgl approved these changes Mar 7, 2025

View reviewed changes

example.yml Outdated Show resolved Hide resolved

fixup! improve docs and add example on cel usage

fa5615a

Signed-off-by: Juho Majasaari <juho.majasaari@iki.fi>

electron0zero approved these changes Mar 13, 2025

View reviewed changes

electron0zero merged commit f2078be into prometheus:master Mar 13, 2025
5 checks passed

SuperQ mentioned this pull request Jun 26, 2025

Release v0.27.0 #1431

Merged

electron0zero mentioned this pull request Jul 24, 2025

Add support for JSONPath matchers #1100

Open

pschiffe mentioned this pull request Jul 30, 2025

Update blackbox exporter grafana/alloy#4083

Open

Support matching JSON body with CEL expressions #1255

Support matching JSON body with CEL expressions #1255

Uh oh!

Conversation

juho9000 commented Jun 5, 2024

Uh oh!

juho9000 commented Jun 17, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mem left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mem commented Jul 23, 2024

Uh oh!

Uh oh!

juho9000 commented Aug 1, 2024

Uh oh!

zetaab commented Aug 7, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

zetaab commented Sep 3, 2024

Uh oh!

juho9000 commented Oct 1, 2024

Uh oh!

zetaab commented Dec 9, 2024

Uh oh!

juho9000 commented Jan 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

juho9000 commented Feb 26, 2025

Uh oh!

Uh oh!

dgl left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

dgl left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

electron0zero commented Mar 13, 2025

Uh oh!

Uh oh!

Uh oh!

juho9000 commented Jun 17, 2024 •

edited

Loading

zetaab commented Aug 7, 2024 •

edited

Loading

juho9000 commented Jan 23, 2025 •

edited

Loading