-
Notifications
You must be signed in to change notification settings - Fork 8.1k
Mixer filter fixes #4718
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Mixer filter fixes #4718
Conversation
ostromart
commented
Apr 3, 2018
- Import some v1 mixer filter code and rewrite in terms of expected v2 envoy format (proto rather than JSON).
- API change to OnOutboundListener and OnInboundListener. Rather than bundling the filters in a struct, return the filters as a plugin slice. This is more convenient for appending.
- Various fixes.
case model.ProtocolTCP, model.ProtocolMongo: | ||
// TODO | ||
// Look at virtual service specs, and identity destinations, | ||
// call buildOutboundNetworkFilters.. and then construct TCPListener | ||
//buildTCPListener(buildOutboundNetworkFilters(clusterName, addresses, servicePort), | ||
// listenAddress, uint32(servicePort.Port), servicePort.Protocol) | ||
} | ||
|
||
for _, p := range registry.NewPlugins(registry.MixerPlugin) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So I would not do this.. Essentially, we should have no knowledge of the semantics of the plugin inside this code.
Since we are passing the node to all plugins, the should essentially skip nodes if it does not pertain to them.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Agreed, this is not good. Is it just a matter of checking if node is Router or Ingress?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yep.. thats all. Actually, its just Router.. (ingress is old code).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
Codecov Report
@@ Coverage Diff @@
## master #4718 +/- ##
========================================
Coverage ? 75%
========================================
Files ? 298
Lines ? 24792
Branches ? 0
========================================
Hits ? 18401
Misses ? 5614
Partials ? 777
Continue to review full report at Codecov.
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good to me.
We'll refine the plugin in separate PRs - the interface is still not ideal.
|
||
// buildTCPMixerFilterConfig builds a TCP filter config for inbound requests. | ||
func buildTCPMixerFilterConfig(mesh *meshconfig.MeshConfig, role model.Proxy, instance *model.ServiceInstance) *mccpb.TcpClientConfig { | ||
attrs := v1.StandardNodeAttributes(v1.AttrDestinationPrefix, role.IPAddress, role.ID, nil) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you please copy the code here? I know its duplication, but we are removing v1alpha3 enmasse from v1. So I don't want these things to get lost.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
i did it but i still don't understand the rationale. if you copy the code over you still have identical v1 code, just now it's harder to tell it's identical. you still have the same dependency it's just now masked.
only if you actually make any changes to the code (however small) does it make sense to actually copy imo.
// Can be used to add additional filters. | ||
OnInboundListener(in *CallbackListenerInputParams, mutable *CallbackListenerMutableObjects) error | ||
OnInboundListener(in *CallbackListenerInputParams, listener *xdsapi.Listener) ([]listener.Filter, []*http_conn.HttpFilter, error) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So with this new interface, we might need to call OnOutboundListener twice, once with nil for listerner to append filters (which caller also need to make sure to append them), and again to modify the listener itself?
(same for other plugin function)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
please take another look - per our discussion i've made changes to the way the listener is built. now, everything except filters is created and passed to plugins. a final step is to marshal the filters (needed because http filters are in serialized format that cannot easily be appended to).
@ostromart PR needs rebase |
/retest |
@@ -298,9 +315,11 @@ func (configgen *ConfigGeneratorImpl) buildSidecarOutboundListeners(env model.En | |||
port: servicePort.Port, | |||
protocol: servicePort.Protocol, | |||
} | |||
newListener := buildListener(listenerOpts) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this is incorrect. You need the final listenAddress before you step into buildListener
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
basically move after switch
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done - thanks.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Still reading through, but here's the initial pass.
|
||
newListener := buildListener(opts) | ||
|
||
params := &plugin.CallbackListenerInputParams{ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Move these into the loop to guard against modifications by a plugin please.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
} | ||
for _, p := range configgen.Plugins { | ||
if err := p.OnOutboundListener(params, mutable); err != nil { | ||
log.Error(err.Error()) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
log.Warn
; log.Error
is for irrecoverable errors that force Pilot to stop serving.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
var opts buildListenerOpts | ||
var listenerType plugin.ListenerType | ||
var networkFilters []listener.Filter | ||
var hTTPFilters []*http_conn.HttpFilter |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we need these vars? Just declare them inline in the CallbackListenerMutableObjects below.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yes, we do. they are passed by ptr and shared with marshalFilters.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah, I missed that; it's fine. Also, small nit: httpFilters
, not hTTPFilters
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
i hate this but https://github.com/golang/go/wiki/CodeReviewComments#initialisms. http is high on the watch list. that necessarily leads to local vars looking strange like this. we can relax this rule for pilot i suppose (it's already a mix) but that's the default for go.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What you posted agrees with what I said:
Words in names that are initialisms or acronyms (e.g. "URL" or "NATO") have a consistent case. For example, "URL" should appear as "URL" or "url" (as in "urlPony", or "URLPony"), never as "Url". As an example: ServeHTTP not ServeHttp. For identifiers with multiple initialized "words", use for example "xmlHTTPRequest" or "XMLHTTPRequest".
It should be httpFilters
, not hTTPFilters
as it is in the PR. This is because initialisms should have a consistent case and visibility requires that the first letter be lower case, therefore the entire initialism should be lower case.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
hmm, i wonder if this changed at some point. i could swear i've had to do these silly caps to make some lint check pass. anyway, done.
|
||
// Filters are serialized one time into an opaque struct once we have the complete list. | ||
if err := marshalFilters(newListener, opts, networkFilters, hTTPFilters); err != nil { | ||
log.Error(err.Error()) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
here and elsewhere, log.Warn
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
@@ -290,6 +303,9 @@ func (configgen *ConfigGeneratorImpl) buildSidecarOutboundListeners(env model.En | |||
listenAddress := WildcardAddress | |||
var addresses []string | |||
var listenerMapKey string | |||
var networkFilters []listener.Filter |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Here and elsewhere, try to avoid pre-declaring variables. E.g. for the filters, we only need them once to create the params; just declare them there.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
see above
@@ -30,12 +34,14 @@ import ( | |||
// Plugin is a mixer plugin. | |||
type Plugin struct{} | |||
|
|||
// NewPlugin returns an ptr to an initialized mixer.Plugin. | |||
func NewPlugin() *Plugin { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please update to return the interface (plugin.Plugin), and get rid of the pointer receivers. The empty struct should be passed by value.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ptr->struct: done, although for the record i disagree with this.
what is the reason for returning the interface here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
btw. i assume you're changing Plugin name so i've left this alone.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The reason is that there's no need to leak implementation details. Code should not depend on authentication.Plugin
, code should depend on plugin.Plugin
. It's to keep the code that depends on this less brittle, and more clearly shows the semantics of the situation (we're implementing a plugin.Plugin
; that's what is important, and the specific implementation is not important). I'd actually go further and make (authentication.
)Plugin
a private type.
W.r.t. pointer vs value, there's a couple of reasons. The empty struct is a true singleton in golang and the compiler can do more to optimize calls with it than with a pointer (which it can't prove much about). Further, you should always prefer value receivers to pointer receivers because a method with a value receiver can always be invoked on both a value and a pointer of the correct type, while a pointer method can only ever be invoked on a pointer of the correct type. See the relevant section of Effective Go.
Is there some reason you think this needs to be a pointer?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
One key reason that values beat pointers is that the compiler knows the a function called on a value type can't mutate that value for the caller, so it can generate better/faster code compared to the same function operating on a pointer.
For the same reason, updating the Plugin
interface to accept a value of CallbackListenerInputParams
rather than a pointer to it, while keeping CallbackListenerMutableObjects
a pointer, more clearly expresses the semantics of the usage of those parameters.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
thanks for the detailed response.
the argument for returning a concrete type vs. interface is it allows the caller to decide which they want. for example in unit tests inside the package you may want to test private methods - that's not possible unless you have the concrete type. i think leaking should not be a big concern: that is controlled through public vs. private methods. if the implementer really wants a public method outside the interface spec let it be so (although i have a hard time imagining an argument to support that). it's probably unnecessary to use a heavy hand like returning interface to prevent folks from extending beyond the interface if they insist, it can be pushed back in code review.
the reasons for ptrs vs. struct are: one, although right now the struct is empty, that may not remain so going forward, in fact i think it's quite likely to grow given that we are encouraging plugin authors to move more complexity out to plugins. that means that in the future we may end up passing sizeable structs around everywhere without even realizing when it happened. with ptr, the cost may be bigger than zero but it's very small and will always remain fixed going forward. i think that's the main reason why for any struct type one usually sees interfaces implemented for a ptr.
the second reason is actually the contrary of your point above: i think it's better to leave the door open to let methods mutate the receiver. i'm not sure that the plugin model necessarily implies an immutable receiver - i can think of a couple of cases where this might not be so, collecting stats and locking. when in doubt i'd leave struct types as ptr implementations by default unless there's a good reason to do otherwise.
having said that, i think this is really splitting hairs in this case which is why i'm totally ok to make these not ptrs (and interface vs. concrete if you want). your arguments are totally valid - it's a choice and as you're the guy who's responsible for pilot codebase i'm ok with whatever you decide in the end. realistically we are in full control here and the design doesn't have to be be agonized over - it's not a big deal to change these things going forward.
regarding CallbackListenerInputParams - i agree in principle with your reasoning. i think the reason to keep it as a ptr is that generally we want to pass structs by ptr everywhere by default for efficiency (i know the official word is use non-ptr as a first choice but just try doing a code review where you pass a struct by value with the same set of folks). actually that's the reason i changed the signature to return the slices in the original version of this PR rather than having a "mutable" parameter. because that convention leaves you free to pass by ptr everywhere for efficiency with the implicit assumption that all input params are always read only - all mutations are through return values or by methods on a mutable type. but that version got voted down, so here we are. i'd actually prefer to return to the version where mutations are returned rather than done in place.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
to paraphrase Martin's response to your question:
Is there some reason you think this needs to be a pointer?
Because we love C and come from C land ;)
plugins := make([]plugin.Plugin, 0) | ||
// NewDefaultPlugins returns a slice of default Plugins. | ||
func NewDefaultPlugins() []plugin.Plugin { | ||
var plugins []plugin.Plugin |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just return a literal:
func NewPlugins() []plugin.Plugin {
return []plugin.Plugin{
authn.NewPlugin(),
mixer.NewPlugin(),
}
}
Further, no need to name this NewDefaultPlugins
; either DefaultPlugins
or NewPlugins
, (best, IMO) just Plugins
. They're inherently the default plugins, since there's no other set of plugins.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
name change and return literal: done (there was previously a NewPlugins that did something else).
regarding NewPlugins->Plugins, it's more conventional to name constructors NewType (i've unsuccessfully argued in that past that MakeType is often more appropriate). it seems to be the de facto naming choice for better or worse.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That's fine; actually I don't like New*
, it's a java-ism that was imported into our codebase by people new to go. But, since that is the defacto standard in the codebase, I think consistency is more important than "correctness" (for whatever correctness is in this fairly subjective case).
} | ||
|
||
// addStandardNodeAttributes add standard node attributes with the given prefix | ||
func addStandardNodeAttributes(attr map[string]*mpb.Attributes_AttributeValue, prefix string, IPAddress string, ID string, labels map[string]string) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
is this function used? Is it the same as v1.AddStandardNodeAttributes?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
it's the same. that was exactly my point earlier in the review where @rshriram requested this to be copied in. imo there's no point copying if the code is identical.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
/lgtm
/lgtm |
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: ZackButcher The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |