Streaming support in InternalInvokeRequest / InternalInvokeResponse
#5648
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
ItalyPaleAle
commented
Dec 16, 2022
| limitations under the License. | ||
| */ | ||
|
|
||
| package v1 |
There was a problem hiding this comment.
Reviewers: this is a new base class for InvokeMethodRequest/InvokeMethodResponse.
Its job is to maintain a buffer where the entire request or response body is kept in memory if needed. Because streams can only be consumed once, if Resiliency is enabled we need to be able to replay requests/responses, which requires being able to keep the data in-memory too (while we send it to the consumer at the same time).
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
09c1a15 to
7e74dbc
Compare
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
Codecov Report
@@ Coverage Diff @@
## master #5648 +/- ##
==========================================
- Coverage 65.06% 64.76% -0.31%
==========================================
Files 144 149 +5
Lines 15370 15684 +314
==========================================
+ Hits 10001 10157 +156
- Misses 4660 4804 +144
- Partials 709 723 +14
Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here. |
…; branch 'master' of https://github.com/dapr/dapr into streaming-take-3
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
|
/ok-to-test |
Dapr E2E testCommit ref: 03ced34 ✅ Build succeeded for linux/amd64
✅ Infrastructure deployed
✅ Build succeeded for linux/arm64
✅ Build succeeded for windows/amd64
|
Dapr E2E testCommit ref: 03ced34 ✅ Build succeeded for linux/amd64
✅ Infrastructure deployed
✅ Build succeeded for linux/arm64
✅ Build succeeded for windows/amd64
|
|
@ItalyPaleAle if possible, can you test this PR commit with quickstarts and Java SDK, to make sure we have additional tests? |
|
Alright, but I won't be able to do that until I'm back to work next Tuesday. |
|
The Java SDK is passing the validation: https://github.com/ItalyPaleAle/dapr-java-sdk/actions/runs/3789407593 @yaron2 @daixiang0 any thoughts on this PR? Ready to be merged? |
daixiang0
approved these changes
Jan 3, 2023
DeepanshuA
approved these changes
Jan 3, 2023
This was referenced Jan 3, 2023
|
FYI - something about this PR breaks certification tests in contrib if dapr/dapr with this specific commit is imported. cc @ItalyPaleAle |
3 tasks
Merged
|
Fix here: #5707 |
mcandeia
added a commit
to mcandeia/dapr
that referenced
this pull request
Jan 11, 2023
Signed-off-by: Marcos Candeia <marrcooos@gmail.com> Signed-off-by: Marcos Candeia <marrcooos@gmail.com> Co-authored-by: Artur Souza <asouza.pro@gmail.com> Bump github.com/fasthttp/router from 1.4.13 to 1.4.14 (dapr#5666) Bumps [github.com/fasthttp/router](https://github.com/fasthttp/router) from 1.4.13 to 1.4.14. - [Release notes](https://github.com/fasthttp/router/releases) - [Commits](fasthttp/router@v1.4.13...v1.4.14) --- updated-dependencies: - dependency-name: github.com/fasthttp/router dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Artur Souza <asouza.pro@gmail.com> Streaming support in `InternalInvokeRequest` / `InternalInvokeResponse` (dapr#5648) * WIP - Updated pkg/messaging to make InvokeMethodRequest and InvokeMethodResponse replayable - Updated protos Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * WIP: custom io.MultiReader with io.Closer Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * WIP Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * 💄 Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * More WIP Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * Use a pool for buffers in replayableRequest too Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * Enabling replays where necessary Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * 💄 Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * Fixes in code and tests Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * More fixes Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * Fixed the remaining unit tests Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * Not yer time for CallLocalStream Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * These protos are unused for now Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * More currently-unused code Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * Various fixes Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * Update protoc version Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * Update protos Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * Updated version here too Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * Remove unused proto import Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * Mini tweaks Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * Fixes & other improvements-tests should now pass Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * More unit tests Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * Fixes a possible panic Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * Misc fixes Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * Changes to actors and to allow the test app to run in self-hosted Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * Fixes Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * More fixes Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * Some fixes for race conditions in unit tests Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * DRY Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * Update protoc version Extracted from dapr#5648 Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * Changed per review feedback Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * Added unit test Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * Changed per review feedback Co-authored-by: Mukundan Sundararajan <65565396+mukundansundar@users.noreply.github.com> Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * Misc tweaks Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> Co-authored-by: Mukundan Sundararajan <65565396+mukundansundar@users.noreply.github.com> Co-authored-by: Dapr Bot <56698301+dapr-bot@users.noreply.github.com> Fixed: replay buffer not resized (dapr#5697) * Fixed: replay buffer not resized Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * Add unit test Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> Bump github.com/prometheus/common from 0.37.0 to 0.39.0 (dapr#5694) Bumps [github.com/prometheus/common](https://github.com/prometheus/common) from 0.37.0 to 0.39.0. - [Release notes](https://github.com/prometheus/common/releases) - [Commits](prometheus/common@v0.37.0...v0.39.0) --- updated-dependencies: - dependency-name: github.com/prometheus/common dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Artur Souza <asouza.pro@gmail.com> Fix call to fetch subscriptions over gRPC but not implemented. (dapr#5652) * Fix call to fetch subscriptions over gRPC but not implemented. Signed-off-by: Artur Souza <artursouza.ms@outlook.com> * Guarantee non-null subscriptions when not implemented on gRPC. Signed-off-by: Artur Souza <artursouza.ms@outlook.com> Signed-off-by: Artur Souza <artursouza.ms@outlook.com> Co-authored-by: Artur Souza <artursouza.ms@outlook.com> Co-authored-by: Mukundan Sundararajan <65565396+mukundansundar@users.noreply.github.com> Co-authored-by: Alessandro (Ale) Segala <43508+ItalyPaleAle@users.noreply.github.com> Add resiliency to bulk publish API (dapr#5646) remove unsupported k8s version, update kind action (dapr#5649) Signed-off-by: Mukundan Sundararajan <65565396+mukundansundar@users.noreply.github.com> Signed-off-by: Mukundan Sundararajan <65565396+mukundansundar@users.noreply.github.com> Allow passing Dapr trust bundle flags via Helm charts (dapr#5470) * allow to pass sentry issuer related flags into charts for each components Signed-off-by: Marco <bardelli.marco@gmail.com> * Update according to feedback into dapr#5470 Signed-off-by: Marco <bardelli.marco@gmail.com> * quote added flags Signed-off-by: Marco <bardelli.marco@gmail.com> * improve explanation in README and remove too generic not strictly needed args Signed-off-by: Marco Bardelli <bardelli.marco@gmail.com> Signed-off-by: Marco <bardelli.marco@gmail.com> Signed-off-by: Marco Bardelli <bardelli.marco@gmail.com> Co-authored-by: Alessandro (Ale) Segala <43508+ItalyPaleAle@users.noreply.github.com> Co-authored-by: Artur Souza <asouza.pro@gmail.com> Fixes metric grouping for CPU usage graphs. (dapr#5525) "Total CPU Usage" graph is displaying `container_cpu_usage_seconds_total` but it is not grouping it by application and is instead using the `pod` field as its discriminating field. Each application is re-deployed daily and receives a new pod-id. As this metric graph uses the `pod` id in its legend, the same application ends up in represented as a series of disconnected metrics. This PR fixes the metric to grouping distinct "pods" under the same by "application id" using some metric-math. This in turn will allow us to observe how a given application behaves over time. Fixes dapr#5524 Signed-off-by: Tiago Alves Macambira <tmacam@burocrata.org> Signed-off-by: Tiago Alves Macambira <tmacam@burocrata.org> Co-authored-by: Loong Dai <long.dai@intel.com> Co-authored-by: Artur Souza <asouza.pro@gmail.com> Allow enabling preview features at build-time (dapr#5677) * Allow enabling preview features at build-time Added the `ENABLED_FEATURES` env var to the Makefile to define a (comma-separated) list of features that are always enabled, regardless of what's in the Configuration spec. The `Resiliency` feature was added to the list of always-enabled features for now (replacing the previous "hack" to have it always enabled - see dapr#5523). Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * Fixed unit tests Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * Features in unit tests Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> Co-authored-by: Dapr Bot <56698301+dapr-bot@users.noreply.github.com> Co-authored-by: Artur Souza <asouza.pro@gmail.com> Fix profile port Performance tests for pubsub http (dapr#5683) * Add pubsub perf test for multiples message size and delayed requests Signed-off-by: Marcos Candeia <marrcooos@gmail.com> * Add pubsub test on dapr_test mk Signed-off-by: Marcos Candeia <marrcooos@gmail.com> * Support appID parameter Signed-off-by: Marcos Candeia <marrcooos@gmail.com> * Preallocate VUs Signed-off-by: Marcos Candeia <marrcooos@gmail.com> * se in memory broker for pubsub http tests perf Signed-off-by: Marcos Candeia <marrcooos@gmail.com> * Set start time and use shared array Signed-off-by: Marcos Candeia <marrcooos@gmail.com> * Fix line formating Signed-off-by: Marcos Candeia <marrcooos@gmail.com> Signed-off-by: Marcos Candeia <marrcooos@gmail.com> Using realistic thresholds for actor type stress test (dapr#5710) Signed-off-by: Marcos Candeia <marrcooos@gmail.com> Signed-off-by: Marcos Candeia <marrcooos@gmail.com> return error on duplicated entry IDs in gRPC bulk publish (dapr#5672) Add metrics labels regex rules (dapr#5716) * add metrics labels regex rules Signed-off-by: yaron2 <schneider.yaron@live.com> * linter Signed-off-by: yaron2 <schneider.yaron@live.com> * update header to correct year Signed-off-by: yaron2 <schneider.yaron@live.com> * linter Signed-off-by: yaron2 <schneider.yaron@live.com> Signed-off-by: yaron2 <schneider.yaron@live.com> Deprecation notice for gRPC service invocation API (dapr#5324) * Deprecation notice for gRPC service invocation API Signed-off-by: sunzhaochang <zhchsun1992@gmail.com> * Add deprecation notices automatically when generating release notes Signed-off-by: sunzhaochang <zhchsun1992@gmail.com> * Update api.go Signed-off-by: Yaron Schneider <schneider.yaron@live.com> * Update api.go Signed-off-by: Yaron Schneider <schneider.yaron@live.com> Signed-off-by: sunzhaochang <zhchsun1992@gmail.com> Signed-off-by: Yaron Schneider <schneider.yaron@live.com> Co-authored-by: Yaron Schneider <schneider.yaron@live.com> Resiliency Support for Bulk Subscribe (dapr#5603) * Add filter for resiliency policy Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com> * Delete unrequired Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com> * Add Resiliency Support via Accumulator and misc refactorings Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com> * Fix linting Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com> * Incorporate review comments Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com> * Fix filter in bulkpub_resiliency Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com> * Add cap assertions Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com> * Add locks Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com> * Add locks Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com> * Incorporate review comments Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com> * contenttype correction Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com> Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com> Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> Co-authored-by: Mukundan Sundararajan <65565396+mukundansundar@users.noreply.github.com> Co-authored-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> fix flaky test (dapr#5723) Signed-off-by: yaron2 <schneider.yaron@live.com> Signed-off-by: yaron2 <schneider.yaron@live.com> fix flaky test sync issue (dapr#5728) Signed-off-by: yaron2 <schneider.yaron@live.com> Signed-off-by: yaron2 <schneider.yaron@live.com> Misc refactorings and fixes to shutdown sequence (dapr#5729) This PR contains misc refactorings extracted from the "firewall" branch, including some fixes to the shutdown sequence. Two user-facing changes: 1. Fixed: cannot stop daprd if it's waiting for the app to come online (often happens if the app crashed while using the Dapr CLI) 2. Can force shutdown (aborting any graceful shutdown sequence) by sending a second SIGTERM/SIGINT. Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> Fix pluggable component withblock usage on tests (dapr#5724) * Fix pluggable component withblock usage on tests Signed-off-by: Marcos Candeia <marrcooos@gmail.com> * Add grpc server listener Signed-off-by: Marcos Candeia <marrcooos@gmail.com> Signed-off-by: Marcos Candeia <marrcooos@gmail.com> Make Resiliency stable (dapr#5732) * Make Resiliency stable Remove the "Resiliency" feature flag and all the code paths where we biforcated based on whether Resiliency was enabled or not Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * Remove feature flag from Makefile Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> Initialize metrics prior to loading resiliency Resiliency was being loaded before the actual metric views/fields were being init. This caused the Resiliency init metric to be lost. This commit moves the init up a bit to go before Resiliency. dapr#5711 Signed-off-by: halspang <halspang@microsoft.com>
mcandeia
pushed a commit
to mcandeia/dapr
that referenced
this pull request
Jan 11, 2023
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> Co-authored-by: Artur Souza <artursouza.ms@outlook.com> Initiate Dapr shutdown after expiry of grace period - Issue 5481 (dapr#5562) * Kick off shutdown after expiry of grace period Signed-off-by: Akhila Chetlapalle <akhila@Akhilas-MacBook-Pro.local> * update shutdown with grace period. Add tests for pubsub actors and bindings Signed-off-by: Akhila Chetlapalle <akhila@Akhilas-MacBook-Pro.local> * fix linting and windows incompatibility Signed-off-by: Akhila Chetlapalle <akhila@Akhilas-MacBook-Pro.local> * fixed tests on windows. SIGTERM sending fails on windows. So invoke shutdown for windows tests Signed-off-by: Akhila Chetlapalle <akhila@Akhilas-MacBook-Pro.local> * review comments incorporated Signed-off-by: Akhila Chetlapalle <akhila@Akhilas-MacBook-Pro.local> * review comments incorporated Signed-off-by: Akhila Chetlapalle <akhila@Akhilas-MacBook-Pro.local> * removed comment Signed-off-by: Akhila Chetlapalle <akhila@Akhilas-MacBook-Pro.local> * update branch and add pubsub order check Signed-off-by: Akhila Chetlapalle <akhila@Akhilas-MacBook-Pro.local> * Fixed trace initiation and shutdown. Updated trace Registration interface to return Tracer Signed-off-by: Akhila Chetlapalle <akhila@Akhilas-MacBook-Pro.local> * reverting timeout pushed in test and moving trace shutdown to seperate method, moving trace shutdown to after api shutdown and removing go routine Signed-off-by: Akhila Chetlapalle <akhila@Akhilas-MacBook-Pro.local> * re-trigger pipeline Signed-off-by: Akhila Chetlapalle <akhila@Akhilas-MacBook-Pro.local> Signed-off-by: Akhila Chetlapalle <akhila@Akhilas-MacBook-Pro.local> Co-authored-by: Akhila Chetlapalle <akhila@Akhilas-MacBook-Pro.local> Co-authored-by: Yaron Schneider <schneider.yaron@live.com> Co-authored-by: Alessandro (Ale) Segala <43508+ItalyPaleAle@users.noreply.github.com> Co-authored-by: Loong Dai <long.dai@intel.com> Co-authored-by: Mukundan Sundararajan <65565396+mukundansundar@users.noreply.github.com> Co-authored-by: Artur Souza <artursouza.ms@outlook.com> Misc refactorings extracted from dapr#5170 (dapr#5609) Changes to the resiliency.NewRunner (dapr#5645) Remove dapr local replacement for pluggable apps (dapr#5642) * Remove dapr local replacement for pluggable apps Signed-off-by: Marcos Candeia <marrcooos@gmail.com> * Pin v0.0.8 on k6 operator Signed-off-by: Marcos Candeia <marrcooos@gmail.com> Signed-off-by: Marcos Candeia <marrcooos@gmail.com> Set actor stress tests thresholds based on previous run (dapr#5657) Signed-off-by: Marcos Candeia <marrcooos@gmail.com> Signed-off-by: Marcos Candeia <marrcooos@gmail.com> Fix ping method invoked before Init method for pluggable components (dapr#5659) Signed-off-by: Marcos Candeia <marrcooos@gmail.com> Signed-off-by: Marcos Candeia <marrcooos@gmail.com> Co-authored-by: Artur Souza <artursouza.ms@outlook.com> feature: add context to lock&pubsub API (dapr#5640) * feature: add context to lock&pubsub API Signed-off-by: seachen <seachen@tencent.com> * feature: add context to lock&pubsub API Signed-off-by: seachen <seachen@tencent.com> * Updated pinned components-contrib Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * upgrade sigs.k8s.io/controller-runtime to v0.14.1 Signed-off-by: seachen <seachen@tencent.com> * fixed golangci-lint Signed-off-by: seachen <seachen@tencent.com> Signed-off-by: seachen <seachen@tencent.com> Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> Co-authored-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> Update protoc version (dapr#5663) Extracted from dapr#5648 Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> Misc test fixes (dapr#5664) * Misc test fixes 1. Fixes some (not all) race conditions in tests for pkg/runtime 2. Improvements to test platform and the actorfeatures test to make testing locally (outside of K8s) easier 3. Some more logging in E2E test apps Extracted from dapr#5648 Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * Tailscale needs a bit more resources or it can crash with OOM Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * Revert change to test per review feedback. However, this re-introduces a race condition (test fails `go test -race`) that will need to be fixed Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> Co-authored-by: Mukundan Sundararajan <65565396+mukundansundar@users.noreply.github.com> optimize bulkpub resp processing (dapr#5498) * refactor code: bulk pub res from component only contains failed entries Signed-off-by: Mukundan Sundararajan <65565396+mukundansundar@users.noreply.github.com> * fixing dependency. fixing unit test. Signed-off-by: Mukundan Sundararajan <65565396+mukundansundar@users.noreply.github.com> * fix error response in gRPC bulk publish API Signed-off-by: Mukundan Sundararajan <65565396+mukundansundar@users.noreply.github.com> * fix pluggable comps go.mod Signed-off-by: Mukundan Sundararajan <65565396+mukundansundar@users.noreply.github.com> * change to point to correct contrib commit Signed-off-by: Mukundan Sundararajan <65565396+mukundansundar@users.noreply.github.com> * fix dependency for components-contrib Signed-off-by: Mukundan Sundararajan <65565396+mukundansundar@users.noreply.github.com> * update contrib to latest commit Signed-off-by: Mukundan Sundararajan <65565396+mukundansundar@users.noreply.github.com> * address review comments. Signed-off-by: Mukundan Sundararajan <65565396+mukundansundar@users.noreply.github.com> * remove new line Signed-off-by: Mukundan Sundararajan <65565396+mukundansundar@users.noreply.github.com> Signed-off-by: Mukundan Sundararajan <65565396+mukundansundar@users.noreply.github.com> Register new Cloudflare KV state store and Queues binding (dapr#5632) * Register new Cloudflare KV state store and Queues binding See dapr/components-contrib#2363 for the new components Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * Naming: workerskv Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * Mod tidy Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> Co-authored-by: Yaron Schneider <schneider.yaron@live.com> fix error message typo (dapr#5681) Signed-off-by: yaron2 <schneider.yaron@live.com> Signed-off-by: yaron2 <schneider.yaron@live.com> Replace `go.uber.org/atomic` and `github.com/pkg/errors` with standard library packages (dapr#5678) * Replace `go.uber.org/atomic` and `github.com/pkg/errors` with standard library packages The packages are now forbidden by a linter rule Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * Fixed E2E tests failing Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> Co-authored-by: Yaron Schneider <schneider.yaron@live.com> Do not start Dapr Watchdog runnable unless it's enabled (dapr#5689) Currently, the Dapr Watchdog runnable is added to the manager whether the watchdog is enabled or not. This forces the Dapr Operator service to request leadership election in all cases, even if the disable-leader-election flag is set. Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> More realistic metric for actor id stress tests (dapr#5687) Signed-off-by: Marcos Candeia <marrcooos@gmail.com> Signed-off-by: Marcos Candeia <marrcooos@gmail.com> Co-authored-by: Artur Souza <asouza.pro@gmail.com> Bump github.com/fasthttp/router from 1.4.13 to 1.4.14 (dapr#5666) Bumps [github.com/fasthttp/router](https://github.com/fasthttp/router) from 1.4.13 to 1.4.14. - [Release notes](https://github.com/fasthttp/router/releases) - [Commits](fasthttp/router@v1.4.13...v1.4.14) --- updated-dependencies: - dependency-name: github.com/fasthttp/router dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Artur Souza <asouza.pro@gmail.com> Streaming support in `InternalInvokeRequest` / `InternalInvokeResponse` (dapr#5648) * WIP - Updated pkg/messaging to make InvokeMethodRequest and InvokeMethodResponse replayable - Updated protos Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * WIP: custom io.MultiReader with io.Closer Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * WIP Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * 💄 Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * More WIP Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * Use a pool for buffers in replayableRequest too Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * Enabling replays where necessary Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * 💄 Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * Fixes in code and tests Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * More fixes Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * Fixed the remaining unit tests Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * Not yer time for CallLocalStream Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * These protos are unused for now Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * More currently-unused code Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * Various fixes Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * Update protoc version Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * Update protos Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * Updated version here too Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * Remove unused proto import Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * Mini tweaks Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * Fixes & other improvements-tests should now pass Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * More unit tests Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * Fixes a possible panic Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * Misc fixes Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * Changes to actors and to allow the test app to run in self-hosted Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * Fixes Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * More fixes Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * Some fixes for race conditions in unit tests Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * DRY Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * Update protoc version Extracted from dapr#5648 Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * Changed per review feedback Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * Added unit test Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * Changed per review feedback Co-authored-by: Mukundan Sundararajan <65565396+mukundansundar@users.noreply.github.com> Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * Misc tweaks Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> Co-authored-by: Mukundan Sundararajan <65565396+mukundansundar@users.noreply.github.com> Co-authored-by: Dapr Bot <56698301+dapr-bot@users.noreply.github.com> Fixed: replay buffer not resized (dapr#5697) * Fixed: replay buffer not resized Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * Add unit test Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> Bump github.com/prometheus/common from 0.37.0 to 0.39.0 (dapr#5694) Bumps [github.com/prometheus/common](https://github.com/prometheus/common) from 0.37.0 to 0.39.0. - [Release notes](https://github.com/prometheus/common/releases) - [Commits](prometheus/common@v0.37.0...v0.39.0) --- updated-dependencies: - dependency-name: github.com/prometheus/common dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Artur Souza <asouza.pro@gmail.com> Fix call to fetch subscriptions over gRPC but not implemented. (dapr#5652) * Fix call to fetch subscriptions over gRPC but not implemented. Signed-off-by: Artur Souza <artursouza.ms@outlook.com> * Guarantee non-null subscriptions when not implemented on gRPC. Signed-off-by: Artur Souza <artursouza.ms@outlook.com> Signed-off-by: Artur Souza <artursouza.ms@outlook.com> Co-authored-by: Artur Souza <artursouza.ms@outlook.com> Co-authored-by: Mukundan Sundararajan <65565396+mukundansundar@users.noreply.github.com> Co-authored-by: Alessandro (Ale) Segala <43508+ItalyPaleAle@users.noreply.github.com> Add resiliency to bulk publish API (dapr#5646) remove unsupported k8s version, update kind action (dapr#5649) Signed-off-by: Mukundan Sundararajan <65565396+mukundansundar@users.noreply.github.com> Signed-off-by: Mukundan Sundararajan <65565396+mukundansundar@users.noreply.github.com> Allow passing Dapr trust bundle flags via Helm charts (dapr#5470) * allow to pass sentry issuer related flags into charts for each components Signed-off-by: Marco <bardelli.marco@gmail.com> * Update according to feedback into dapr#5470 Signed-off-by: Marco <bardelli.marco@gmail.com> * quote added flags Signed-off-by: Marco <bardelli.marco@gmail.com> * improve explanation in README and remove too generic not strictly needed args Signed-off-by: Marco Bardelli <bardelli.marco@gmail.com> Signed-off-by: Marco <bardelli.marco@gmail.com> Signed-off-by: Marco Bardelli <bardelli.marco@gmail.com> Co-authored-by: Alessandro (Ale) Segala <43508+ItalyPaleAle@users.noreply.github.com> Co-authored-by: Artur Souza <asouza.pro@gmail.com> Fixes metric grouping for CPU usage graphs. (dapr#5525) "Total CPU Usage" graph is displaying `container_cpu_usage_seconds_total` but it is not grouping it by application and is instead using the `pod` field as its discriminating field. Each application is re-deployed daily and receives a new pod-id. As this metric graph uses the `pod` id in its legend, the same application ends up in represented as a series of disconnected metrics. This PR fixes the metric to grouping distinct "pods" under the same by "application id" using some metric-math. This in turn will allow us to observe how a given application behaves over time. Fixes dapr#5524 Signed-off-by: Tiago Alves Macambira <tmacam@burocrata.org> Signed-off-by: Tiago Alves Macambira <tmacam@burocrata.org> Co-authored-by: Loong Dai <long.dai@intel.com> Co-authored-by: Artur Souza <asouza.pro@gmail.com> Allow enabling preview features at build-time (dapr#5677) * Allow enabling preview features at build-time Added the `ENABLED_FEATURES` env var to the Makefile to define a (comma-separated) list of features that are always enabled, regardless of what's in the Configuration spec. The `Resiliency` feature was added to the list of always-enabled features for now (replacing the previous "hack" to have it always enabled - see dapr#5523). Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * Fixed unit tests Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * Features in unit tests Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> Co-authored-by: Dapr Bot <56698301+dapr-bot@users.noreply.github.com> Co-authored-by: Artur Souza <asouza.pro@gmail.com> Fix profile port Performance tests for pubsub http (dapr#5683) * Add pubsub perf test for multiples message size and delayed requests Signed-off-by: Marcos Candeia <marrcooos@gmail.com> * Add pubsub test on dapr_test mk Signed-off-by: Marcos Candeia <marrcooos@gmail.com> * Support appID parameter Signed-off-by: Marcos Candeia <marrcooos@gmail.com> * Preallocate VUs Signed-off-by: Marcos Candeia <marrcooos@gmail.com> * se in memory broker for pubsub http tests perf Signed-off-by: Marcos Candeia <marrcooos@gmail.com> * Set start time and use shared array Signed-off-by: Marcos Candeia <marrcooos@gmail.com> * Fix line formating Signed-off-by: Marcos Candeia <marrcooos@gmail.com> Signed-off-by: Marcos Candeia <marrcooos@gmail.com> Using realistic thresholds for actor type stress test (dapr#5710) Signed-off-by: Marcos Candeia <marrcooos@gmail.com> Signed-off-by: Marcos Candeia <marrcooos@gmail.com> return error on duplicated entry IDs in gRPC bulk publish (dapr#5672) Add metrics labels regex rules (dapr#5716) * add metrics labels regex rules Signed-off-by: yaron2 <schneider.yaron@live.com> * linter Signed-off-by: yaron2 <schneider.yaron@live.com> * update header to correct year Signed-off-by: yaron2 <schneider.yaron@live.com> * linter Signed-off-by: yaron2 <schneider.yaron@live.com> Signed-off-by: yaron2 <schneider.yaron@live.com> Deprecation notice for gRPC service invocation API (dapr#5324) * Deprecation notice for gRPC service invocation API Signed-off-by: sunzhaochang <zhchsun1992@gmail.com> * Add deprecation notices automatically when generating release notes Signed-off-by: sunzhaochang <zhchsun1992@gmail.com> * Update api.go Signed-off-by: Yaron Schneider <schneider.yaron@live.com> * Update api.go Signed-off-by: Yaron Schneider <schneider.yaron@live.com> Signed-off-by: sunzhaochang <zhchsun1992@gmail.com> Signed-off-by: Yaron Schneider <schneider.yaron@live.com> Co-authored-by: Yaron Schneider <schneider.yaron@live.com> Resiliency Support for Bulk Subscribe (dapr#5603) * Add filter for resiliency policy Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com> * Delete unrequired Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com> * Add Resiliency Support via Accumulator and misc refactorings Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com> * Fix linting Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com> * Incorporate review comments Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com> * Fix filter in bulkpub_resiliency Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com> * Add cap assertions Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com> * Add locks Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com> * Add locks Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com> * Incorporate review comments Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com> * contenttype correction Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com> Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com> Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> Co-authored-by: Mukundan Sundararajan <65565396+mukundansundar@users.noreply.github.com> Co-authored-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> fix flaky test (dapr#5723) Signed-off-by: yaron2 <schneider.yaron@live.com> Signed-off-by: yaron2 <schneider.yaron@live.com> fix flaky test sync issue (dapr#5728) Signed-off-by: yaron2 <schneider.yaron@live.com> Signed-off-by: yaron2 <schneider.yaron@live.com> Misc refactorings and fixes to shutdown sequence (dapr#5729) This PR contains misc refactorings extracted from the "firewall" branch, including some fixes to the shutdown sequence. Two user-facing changes: 1. Fixed: cannot stop daprd if it's waiting for the app to come online (often happens if the app crashed while using the Dapr CLI) 2. Can force shutdown (aborting any graceful shutdown sequence) by sending a second SIGTERM/SIGINT. Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> Fix pluggable component withblock usage on tests (dapr#5724) * Fix pluggable component withblock usage on tests Signed-off-by: Marcos Candeia <marrcooos@gmail.com> * Add grpc server listener Signed-off-by: Marcos Candeia <marrcooos@gmail.com> Signed-off-by: Marcos Candeia <marrcooos@gmail.com> Make Resiliency stable (dapr#5732) * Make Resiliency stable Remove the "Resiliency" feature flag and all the code paths where we biforcated based on whether Resiliency was enabled or not Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * Remove feature flag from Makefile Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> Initialize metrics prior to loading resiliency Resiliency was being loaded before the actual metric views/fields were being init. This caused the Resiliency init metric to be lost. This commit moves the init up a bit to go before Resiliency. dapr#5711 Signed-off-by: halspang <halspang@microsoft.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This PR is yet another byproduct / spinoff from #5170 (Streaming support for service invocation). As discussed in the last comment in that PR, the work was split into multiple parts to make it more manageable.
This PR contains part 2 of the 4 parts listed there: it changes
InternalInvokeRequest/InternalInvokeResponse(pkg/messaging/v1) to handle data as streams rather than buffering the entire message in memory every time.This should have no user-facing changes, as it just changes the internal data structures.
Issue reference
#3103 (partially fixed by this PR)
#4866
Checklist
Please make sure you've completed the relevant tasks for this PR, out of the following list: