Skip to content

Conversation

ostromart
Copy link
Contributor

  1. Import some v1 mixer filter code and rewrite in terms of expected v2 envoy format (proto rather than JSON).
  2. API change to OnOutboundListener and OnInboundListener. Rather than bundling the filters in a struct, return the filters as a plugin slice. This is more convenient for appending.
  3. Various fixes.

case model.ProtocolTCP, model.ProtocolMongo:
// TODO
// Look at virtual service specs, and identity destinations,
// call buildOutboundNetworkFilters.. and then construct TCPListener
//buildTCPListener(buildOutboundNetworkFilters(clusterName, addresses, servicePort),
// listenAddress, uint32(servicePort.Port), servicePort.Protocol)
}

for _, p := range registry.NewPlugins(registry.MixerPlugin) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So I would not do this.. Essentially, we should have no knowledge of the semantics of the plugin inside this code.
Since we are passing the node to all plugins, the should essentially skip nodes if it does not pertain to them.

Copy link
Contributor Author

@ostromart ostromart Apr 3, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, this is not good. Is it just a matter of checking if node is Router or Ingress?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep.. thats all. Actually, its just Router.. (ingress is old code).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@codecov
Copy link

codecov bot commented Apr 3, 2018

Codecov Report

❗ No coverage uploaded for pull request base (master@e00422a). Click here to learn what that means.
The diff coverage is 74%.

Impacted file tree graph

@@           Coverage Diff            @@
##             master   #4718   +/-   ##
========================================
  Coverage          ?     75%           
========================================
  Files             ?     298           
  Lines             ?   24792           
  Branches          ?       0           
========================================
  Hits              ?   18401           
  Misses            ?    5614           
  Partials          ?     777
Impacted Files Coverage Δ
pilot/pkg/config/aggregate/config.go 0% <ø> (ø)
...ilot/pkg/networking/plugin/authn/authentication.go 58% <0%> (ø)
pilot/pkg/proxy/envoy/v1/mixer.go 89% <100%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e00422a...138220d. Read the comment docs.

Copy link
Contributor

@costinm costinm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me.
We'll refine the plugin in separate PRs - the interface is still not ideal.


// buildTCPMixerFilterConfig builds a TCP filter config for inbound requests.
func buildTCPMixerFilterConfig(mesh *meshconfig.MeshConfig, role model.Proxy, instance *model.ServiceInstance) *mccpb.TcpClientConfig {
attrs := v1.StandardNodeAttributes(v1.AttrDestinationPrefix, role.IPAddress, role.ID, nil)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you please copy the code here? I know its duplication, but we are removing v1alpha3 enmasse from v1. So I don't want these things to get lost.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i did it but i still don't understand the rationale. if you copy the code over you still have identical v1 code, just now it's harder to tell it's identical. you still have the same dependency it's just now masked.
only if you actually make any changes to the code (however small) does it make sense to actually copy imo.

// Can be used to add additional filters.
OnInboundListener(in *CallbackListenerInputParams, mutable *CallbackListenerMutableObjects) error
OnInboundListener(in *CallbackListenerInputParams, listener *xdsapi.Listener) ([]listener.Filter, []*http_conn.HttpFilter, error)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So with this new interface, we might need to call OnOutboundListener twice, once with nil for listerner to append filters (which caller also need to make sure to append them), and again to modify the listener itself?

(same for other plugin function)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please take another look - per our discussion i've made changes to the way the listener is built. now, everything except filters is created and passed to plugins. a final step is to marshal the filters (needed because http filters are in serialized format that cannot easily be appended to).

@istio-merge-robot
Copy link

@ostromart PR needs rebase

@istio-merge-robot istio-merge-robot added the needs-rebase Indicates a PR needs to be rebased before being merged label Apr 4, 2018
@istio-testing istio-testing removed the needs-rebase Indicates a PR needs to be rebased before being merged label Apr 5, 2018
@ostromart
Copy link
Contributor Author

/retest

@@ -298,9 +315,11 @@ func (configgen *ConfigGeneratorImpl) buildSidecarOutboundListeners(env model.En
port: servicePort.Port,
protocol: servicePort.Protocol,
}
newListener := buildListener(listenerOpts)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is incorrect. You need the final listenAddress before you step into buildListener

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

basically move after switch

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done - thanks.

Copy link
Contributor

@ZackButcher ZackButcher left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still reading through, but here's the initial pass.


newListener := buildListener(opts)

params := &plugin.CallbackListenerInputParams{
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Move these into the loop to guard against modifications by a plugin please.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

}
for _, p := range configgen.Plugins {
if err := p.OnOutboundListener(params, mutable); err != nil {
log.Error(err.Error())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

log.Warn; log.Error is for irrecoverable errors that force Pilot to stop serving.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

var opts buildListenerOpts
var listenerType plugin.ListenerType
var networkFilters []listener.Filter
var hTTPFilters []*http_conn.HttpFilter
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need these vars? Just declare them inline in the CallbackListenerMutableObjects below.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, we do. they are passed by ptr and shared with marshalFilters.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, I missed that; it's fine. Also, small nit: httpFilters, not hTTPFilters

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i hate this but https://github.com/golang/go/wiki/CodeReviewComments#initialisms. http is high on the watch list. that necessarily leads to local vars looking strange like this. we can relax this rule for pilot i suppose (it's already a mix) but that's the default for go.

Copy link
Contributor

@ZackButcher ZackButcher Apr 6, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What you posted agrees with what I said:

Words in names that are initialisms or acronyms (e.g. "URL" or "NATO") have a consistent case. For example, "URL" should appear as "URL" or "url" (as in "urlPony", or "URLPony"), never as "Url". As an example: ServeHTTP not ServeHttp. For identifiers with multiple initialized "words", use for example "xmlHTTPRequest" or "XMLHTTPRequest".

It should be httpFilters, not hTTPFilters as it is in the PR. This is because initialisms should have a consistent case and visibility requires that the first letter be lower case, therefore the entire initialism should be lower case.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm, i wonder if this changed at some point. i could swear i've had to do these silly caps to make some lint check pass. anyway, done.


// Filters are serialized one time into an opaque struct once we have the complete list.
if err := marshalFilters(newListener, opts, networkFilters, hTTPFilters); err != nil {
log.Error(err.Error())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

here and elsewhere, log.Warn

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@@ -290,6 +303,9 @@ func (configgen *ConfigGeneratorImpl) buildSidecarOutboundListeners(env model.En
listenAddress := WildcardAddress
var addresses []string
var listenerMapKey string
var networkFilters []listener.Filter
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here and elsewhere, try to avoid pre-declaring variables. E.g. for the filters, we only need them once to create the params; just declare them there.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

see above

@@ -30,12 +34,14 @@ import (
// Plugin is a mixer plugin.
type Plugin struct{}

// NewPlugin returns an ptr to an initialized mixer.Plugin.
func NewPlugin() *Plugin {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please update to return the interface (plugin.Plugin), and get rid of the pointer receivers. The empty struct should be passed by value.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ptr->struct: done, although for the record i disagree with this.
what is the reason for returning the interface here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

btw. i assume you're changing Plugin name so i've left this alone.

Copy link
Contributor

@ZackButcher ZackButcher Apr 5, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reason is that there's no need to leak implementation details. Code should not depend on authentication.Plugin, code should depend on plugin.Plugin. It's to keep the code that depends on this less brittle, and more clearly shows the semantics of the situation (we're implementing a plugin.Plugin; that's what is important, and the specific implementation is not important). I'd actually go further and make (authentication.)Plugin a private type.

W.r.t. pointer vs value, there's a couple of reasons. The empty struct is a true singleton in golang and the compiler can do more to optimize calls with it than with a pointer (which it can't prove much about). Further, you should always prefer value receivers to pointer receivers because a method with a value receiver can always be invoked on both a value and a pointer of the correct type, while a pointer method can only ever be invoked on a pointer of the correct type. See the relevant section of Effective Go.

Is there some reason you think this needs to be a pointer?

Copy link
Contributor

@ZackButcher ZackButcher Apr 5, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One key reason that values beat pointers is that the compiler knows the a function called on a value type can't mutate that value for the caller, so it can generate better/faster code compared to the same function operating on a pointer.

For the same reason, updating the Plugin interface to accept a value of CallbackListenerInputParams rather than a pointer to it, while keeping CallbackListenerMutableObjects a pointer, more clearly expresses the semantics of the usage of those parameters.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for the detailed response.
the argument for returning a concrete type vs. interface is it allows the caller to decide which they want. for example in unit tests inside the package you may want to test private methods - that's not possible unless you have the concrete type. i think leaking should not be a big concern: that is controlled through public vs. private methods. if the implementer really wants a public method outside the interface spec let it be so (although i have a hard time imagining an argument to support that). it's probably unnecessary to use a heavy hand like returning interface to prevent folks from extending beyond the interface if they insist, it can be pushed back in code review.
the reasons for ptrs vs. struct are: one, although right now the struct is empty, that may not remain so going forward, in fact i think it's quite likely to grow given that we are encouraging plugin authors to move more complexity out to plugins. that means that in the future we may end up passing sizeable structs around everywhere without even realizing when it happened. with ptr, the cost may be bigger than zero but it's very small and will always remain fixed going forward. i think that's the main reason why for any struct type one usually sees interfaces implemented for a ptr.
the second reason is actually the contrary of your point above: i think it's better to leave the door open to let methods mutate the receiver. i'm not sure that the plugin model necessarily implies an immutable receiver - i can think of a couple of cases where this might not be so, collecting stats and locking. when in doubt i'd leave struct types as ptr implementations by default unless there's a good reason to do otherwise.
having said that, i think this is really splitting hairs in this case which is why i'm totally ok to make these not ptrs (and interface vs. concrete if you want). your arguments are totally valid - it's a choice and as you're the guy who's responsible for pilot codebase i'm ok with whatever you decide in the end. realistically we are in full control here and the design doesn't have to be be agonized over - it's not a big deal to change these things going forward.
regarding CallbackListenerInputParams - i agree in principle with your reasoning. i think the reason to keep it as a ptr is that generally we want to pass structs by ptr everywhere by default for efficiency (i know the official word is use non-ptr as a first choice but just try doing a code review where you pass a struct by value with the same set of folks). actually that's the reason i changed the signature to return the slices in the original version of this PR rather than having a "mutable" parameter. because that convention leaves you free to pass by ptr everywhere for efficiency with the implicit assumption that all input params are always read only - all mutations are through return values or by methods on a mutable type. but that version got voted down, so here we are. i'd actually prefer to return to the version where mutations are returned rather than done in place.

Copy link
Member

@rshriram rshriram Apr 6, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

to paraphrase Martin's response to your question:

Is there some reason you think this needs to be a pointer?

Because we love C and come from C land ;)

plugins := make([]plugin.Plugin, 0)
// NewDefaultPlugins returns a slice of default Plugins.
func NewDefaultPlugins() []plugin.Plugin {
var plugins []plugin.Plugin
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just return a literal:

func NewPlugins() []plugin.Plugin {
    return []plugin.Plugin{
        authn.NewPlugin(),
        mixer.NewPlugin(),
    }
}

Further, no need to name this NewDefaultPlugins; either DefaultPlugins or NewPlugins, (best, IMO) just Plugins. They're inherently the default plugins, since there's no other set of plugins.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

name change and return literal: done (there was previously a NewPlugins that did something else).
regarding NewPlugins->Plugins, it's more conventional to name constructors NewType (i've unsuccessfully argued in that past that MakeType is often more appropriate). it seems to be the de facto naming choice for better or worse.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's fine; actually I don't like New*, it's a java-ism that was imported into our codebase by people new to go. But, since that is the defacto standard in the codebase, I think consistency is more important than "correctness" (for whatever correctness is in this fairly subjective case).

}

// addStandardNodeAttributes add standard node attributes with the given prefix
func addStandardNodeAttributes(attr map[string]*mpb.Attributes_AttributeValue, prefix string, IPAddress string, ID string, labels map[string]string) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this function used? Is it the same as v1.AddStandardNodeAttributes?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's the same. that was exactly my point earlier in the review where @rshriram requested this to be copied in. imo there's no point copying if the code is identical.

Copy link
Contributor

@ZackButcher ZackButcher left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@ZackButcher
Copy link
Contributor

/lgtm

@istio-testing
Copy link
Collaborator

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ZackButcher

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@rshriram rshriram merged commit a585096 into istio:master Apr 6, 2018
@ostromart ostromart deleted the dev branch April 6, 2018 21:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

9 participants