-
Notifications
You must be signed in to change notification settings - Fork 2.7k
WIP: Add webhook #1642
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
WIP: Add webhook #1642
Conversation
controllers/api/server.go
Outdated
@@ -71,6 +71,8 @@ func (as *Server) registerRoutes() { | |||
router.HandleFunc("/import/group", as.ImportGroup) | |||
router.HandleFunc("/import/email", as.ImportEmail) | |||
router.HandleFunc("/import/site", as.ImportSite) | |||
router.HandleFunc("/webhooks/", mid.Use(as.Webhooks, mid.RequirePermission(models.PermissionModifySystem))) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You might want to add goimports
to your IDE settings to run automatically on save. I've found this helps me from having to go back later and reformat all the files.
controllers/api/server.go
Outdated
@@ -71,6 +71,8 @@ func (as *Server) registerRoutes() { | |||
router.HandleFunc("/import/group", as.ImportGroup) | |||
router.HandleFunc("/import/email", as.ImportEmail) | |||
router.HandleFunc("/import/site", as.ImportSite) | |||
router.HandleFunc("/webhooks/", mid.Use(as.Webhooks, mid.RequirePermission(models.PermissionModifySystem))) | |||
router.HandleFunc("/webhooks/{id:[0-9]+}/ping", mid.Use(as.PingWebhook, mid.RequirePermission(models.PermissionModifySystem))) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's change this to validate
instead of ping
. This will match the pattern that we're doing with the IMAP additions here
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok. But using verbs in the URL in REST API is discouraged. It's recommended to use nouns. It'd be "/validation"
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'd prefer to leave it validate
. I understand that REST API semantics prefer for everything in the URL to be a resource, but to me validate
reads cleaner in that we're validating a resource, not referring to a new validation.
I appreciate the suggestion, though! Keep up the great work.
controllers/api/webhook.go
Outdated
func (as *Server) Webhooks(w http.ResponseWriter, r *http.Request) { | ||
switch { | ||
case r.Method == "GET": | ||
wh_s, err := models.GetWebhooks() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Minor nit - I try not to use _
's in variable names where possible. In this case, whs
would be fine.
controllers/route.go
Outdated
@@ -106,6 +106,8 @@ func (as *AdminServer) registerRoutes() { | |||
router.HandleFunc("/sending_profiles", mid.Use(as.SendingProfiles, mid.RequireLogin)) | |||
router.HandleFunc("/settings", mid.Use(as.Settings, mid.RequireLogin)) | |||
router.HandleFunc("/users", mid.Use(as.UserManagement, mid.RequirePermission(models.PermissionModifySystem), mid.RequireLogin)) | |||
router.HandleFunc("/webhooks", mid.Use(as.Webhooks, mid.RequirePermission(models.PermissionModifySystem), mid.RequireLogin)) | |||
router.HandleFunc("/webhooks/{id:[0-9]+}/ping", mid.Use(as.PingWebhook, mid.RequirePermission(models.PermissionModifySystem), mid.RequireLogin)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You likely won't need this. Pinging a webhook could be API-only and a button on the webhooks page.
go.mod
Outdated
@@ -0,0 +1,28 @@ | |||
module github.com/gophish/gophish |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please remove the go.sum
and go.mod
files before the final review. I'll add dependency management at some point, but that should be separate from the work here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok
models/webhook.go
Outdated
type Webhook struct { | ||
Id int64 `json:"id"` | ||
Title string `json:"url"` | ||
Url string `json:"url"` |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You'll want to include a Secret
here as well to handle signing.
models/webhook.go
Outdated
|
||
// GetUsers returns the users registered in Gophish | ||
func GetWebhooks(uid int64) ([]Webhook, error) { | ||
wh_s := []Webhook{} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Same minor nit as above - whs
would work fine here.
webhook/webhook.go
Outdated
//TODO | ||
|
||
|
||
func (whook *Webhook) Send(server string, secret []byte, data interface{}) error { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Super minor nit - I'd rename this to wh
instead of whook
. Keeps it cleaner and matches what you have in the models
package above.
webhook/webhook.go
Outdated
|
||
} | ||
|
||
func sign(data interface{}) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'd make sign
be a function off Webhook
:
func (wh *Webhook) sign(data interface{}) (string, error) {}
Or something similar.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hey @GildedHonour,
I know this is a WIP, just wanted to pass along feedback as I had it. Keep up the great work!
Nice work and good luck, @GildedHonour! |
@jordan-wright for indentation we use tabs in the project, not spaces, right? |
templates/webhooks.html
Outdated
|
||
<label class="control-label" for="url">Url:</label> | ||
<div class="form-group"> | ||
<input type="text" class="form-control" placeholder="Url" id="url" required /> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we change this placeholder to http://example.com/webhook
or something to indicate what the field should look like?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
fixed
"id" integer primary key autoincrement, | ||
"title" varchar(255), | ||
"url" varchar(1000), | ||
"secret" varchar(255) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You're missing the is_active
column in this migration.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not, you're looking at the old code.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also, why are you using an integer field for is_active
? We use boolean fields in migrations now and this seems like another good use case for that.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
is_active is boolean now
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I see. There're 2 migrations, in one of them it's missing
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is the latest code right? Like, the code right here that we're commenting on is for the SQLite migrations, and it doesn't have is_active
. MySQL does, but SQLite doesn't.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
SQLite does not have a separate Boolean storage class. Instead, Boolean values are stored as integers 0 (false) and 1 (true).
https://www.sqlite.org/datatype3.html
Sqlite doesn't support boolean, why are we using them in all the migrations for Sqlite?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It doesn't have a separate concept for booleans, but does support a boolean
data type- it just represents them internally as integers:
sqlite> .schema webhooks
CREATE TABLE IF NOT EXISTS "webhooks" (
"id" integer primary key autoincrement,
"title" varchar(255),
"url" varchar(1000),
"secret" varchar(255),
"is_active" boolean
);
sqlite> select * from webhooks;
1|Test Webhook|http://127.0.0.1:9999/webhook|mysecret|0
I use them both for clearer database structure, since it's clear that this is a true/false kind of field rather than an integer, as well as to try and avoid any possible weird issues with the underlying ORM, gorm.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fixed
Other fixes? |
Doing a bit more testing, I think the Instead, I think the user should be able to enable/disable the webhook as needed. This could be a checkbox in the modal popup that sends the value in the JSON payload. Then, the validation doesn't need to set the value - only send the request and give back the result. I have plans to do more testing tomorrow evening or so, but I think we're in the home stretch here! Maybe just a few very minor tweaks, then we'll be ready to merge. |
Okay. Fixed |
@jordan-wright what's left to fix? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hey there!
I've done another pass through the code itself and left some comments. We're definitely in the final stretch of things. Once we have these taken care of, I'll do a final pass using the webhooks locally and, if things are in good shape, we'll merge this in.
controllers/api/webhook.go
Outdated
return | ||
} | ||
err = webhook.Send(webhook.EndPoint{URL: wh.URL, Secret: wh.Secret}, "") | ||
if err == nil { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This can be cleaned up a bit by flipping the conditionals and removing the else
:
if err != nil {
JSONResponse(w, models.Response{Success: false, Message: err.Error()}, http.StatusBadRequest)
return
}
JSONResponse(w, wh, http.StatusOK)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How will flipping the conditionals clean up the code? We expect the condition without an error first. And that's what I'm checking first.
And there have to exist 2 conditions here.
err = webhook.Send(webhook.EndPoint{URL: wh.URL, Secret: wh.Secret}, "")
if err == nil {
JSONResponse(w, wh, http.StatusOK)
} else {
JSONResponse(w, models.Response{Success: false, Message: err.Error()}, http.StatusBadRequest)
}
How do you say we could do this?
err = webhook.Send(webhook.EndPoint{URL: wh.URL, Secret: wh.Secret}, "")
if err != nil {
JSONResponse(w, models.Response{Success: false, Message: err.Error()}, http.StatusBadRequest)
}
Return nothing in case of "no error"?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
My recommendation is really just encouraging the pattern we use everywhere else in the code, which is also what Go itself recommends. That is, a pattern like:
if err != nil {
handle the error
return
}
continue execution
The reason this is encouraged is that it keeps the nesting to a minimum so we don't wind up with things like:
if err == nil {
big block of code that happens if things go as planned
if err == nil {
another big block of code
etc, etc
} else {
handle the error
}
} else {
handle the error
}
So in my proposal we're handling the error first, then if the error was nil, you'll find the JSONResponse(w, wh, http.StatusOK)
at the end of the suggested code- this would be the default thing to run if no errors occurred. That's the way all of the other HTTP handlers work- this is the only one that has a different pattern when it doesn't need to.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Agree. Fixed
models/campaign.go
Outdated
@@ -157,6 +158,21 @@ func (c *Campaign) UpdateStatus(s string) error { | |||
func (c *Campaign) AddEvent(e *Event) error { | |||
e.CampaignId = c.Id | |||
e.Time = time.Now().UTC() | |||
|
|||
whs, err := GetActiveWebhooks() | |||
if err != nil { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
First, it seems like this doesn't work? You only run the body of the function if err != nil
, so if there was an error.
It appears that this is the correct way to structure this:
if err == nil {
for _, wh := range whs {
whEndPoints = append(whEndPoints, webhook.EndPoint{
URL: wh.URL,
Secret: wh.Secret,
})
}
webhook.SendAll(whEndPoints, e
} else {
log.Errorf("error getting active webhooks: %v", err)
}
Also notice the updated error message to give more context on what the error was. And in this case, it appears we're making a design decision to fail-open if the webhooks encounter errors. I'm ok with this for now.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
fixed
models/webhook.go
Outdated
return err | ||
} | ||
|
||
func validate(wh *Webhook) error { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
On the other models validate
is a function off the struct, and I think we should keep that pattern here. This would make it:
func (wh *Webhook) Validate() error {...}
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
fixed
models/webhook.go
Outdated
|
||
func validate(wh *Webhook) error { | ||
if wh.URL == "" { | ||
return errors.New("URL can't be empty") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please move these errors to exported variables at the top of the file (see the other models/
files for examples, such as models/campaign.go
)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
fixed
models/webhook.go
Outdated
return errors.New("URL can't be empty") | ||
} | ||
if wh.Name == "" { | ||
return errors.New("Name can't be empty") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please move these errors to exported variables at the top of the file (see the other models/
files for examples, such as models/campaign.go
)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
fixed
static/js/src/app/webhooks.js
Outdated
} | ||
Swal.fire({ | ||
title: "Are you sure?", | ||
text: `This will delete the webhook '${wh.name}'`, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We'll want to escape this to prevent xss. You can use escapeHtml
for this.
static/js/src/app/webhooks.js
Outdated
api.webhookId.ping(whId) | ||
.success(function(wh) { | ||
btn.disabled = false; | ||
successFlash(`Ping of "${wh.name}" webhook succeeded.`); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We'll want to escape this to prevent xss. You can use escapeHtml
for this.
static/js/src/app/webhooks.js
Outdated
if (!wh) { | ||
return | ||
} | ||
errorFlash(`Ping of "${wh.name}" webhook failed.`) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If we're returning an error from the webhook ping, we should display it to the user here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
fixed
templates/webhooks.html
Outdated
|
||
<label class="control-label" for="secret">Is active:</label> | ||
<div class="form-group"> | ||
<input type="checkbox" class="" id="is_active" value="true" /> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We have a way to make pretty checkboxes that I'd encourage you to use. You can find an example here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
fixed
@jordan-wright what's left to fix? |
I’ll let you know when I have a chance to review it. Should be soon.
… On Dec 12, 2019, at 11:20 PM, Alex Maslakov ***@***.***> wrote:
@jordan-wright what's left to fix?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.
|
I think we're at a point where I'm comfortable merging this in. There's some super minor tweaks left to do, but I can handle those. Thanks for your patience and all your hard work! I don't have an ETA on the next release, but it will probably be in the next few weeks or so. I'm hoping that I can get #1612 and my work on password policy improvements merged, then I'll cut the new release. Thanks again! |
No description provided.