Skip to content

Conversation

lynnagara
Copy link
Member

@lynnagara lynnagara commented Nov 4, 2022

The dlq part was the only bit that didn't pass the type checks in the consumer_test file. Once that is removed in #3343 we no longer need to omit this file from typechecking.

Currently the multistorage consumer completely ignores the policy
defined on the storage itself and just applies it's own policy
which produces the message to a topic. This is wrong and also
different to how the main consumer works where the policy defined on
the storage is the one that is used. It was also applied on a different
part of the pipeline (only wraps the collector step). This also means
the collector implementation was inconsistent in the two consumer variants.

When this hack was first introduced, it's likely there were practical
reasons we needed this at Sentry, but since this particular code path
(multistorage consumer with dlq) is never used today, those obviously
don't exist anymore.

This is a step on the path to folding the multistorage consumer
into the main consumer and making them behave in the same way.
The dlq part was the only one that didn't pass the type checks.
Once that is removed in #3343
we no longer need to omit this file from typechecking.
@lynnagara lynnagara requested a review from a team as a code owner November 4, 2022 20:12
Base automatically changed from multistorage-dlq to master November 7, 2022 18:35
@codecov-commenter
Copy link

Codecov Report

Base: 92.36% // Head: 21.80% // Decreases project coverage by -70.55% ⚠️

Coverage data is based on head (9945532) compared to base (03d0250).
Patch coverage: 0.00% of modified lines in pull request are covered.

Additional details and impacted files
@@             Coverage Diff             @@
##           master    #3344       +/-   ##
===========================================
- Coverage   92.36%   21.80%   -70.56%     
===========================================
  Files         702      664       -38     
  Lines       32310    31119     -1191     
===========================================
- Hits        29843     6787    -23056     
- Misses       2467    24332    +21865     
Impacted Files Coverage Δ
snuba/cli/__init__.py 0.00% <0.00%> (-86.67%) ⬇️
snuba/cli/devserver.py 0.00% <ø> (-13.96%) ⬇️
snuba/cli/multistorage_consumer.py 0.00% <0.00%> (-43.00%) ⬇️
snuba/consumers/consumer.py 0.00% <0.00%> (-84.40%) ⬇️
snuba/core/initialize.py 0.00% <0.00%> (ø)
snuba/datasets/factory.py 0.00% <0.00%> (-94.60%) ⬇️
test_initialization/test_initialize.py 0.00% <0.00%> (ø)
tests/test_consumer.py 0.00% <ø> (-100.00%) ⬇️
tests/base.py 0.00% <0.00%> (-100.00%) ⬇️
tests/helpers.py 0.00% <0.00%> (-100.00%) ⬇️
... and 638 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

@lynnagara lynnagara merged commit b29f306 into master Nov 7, 2022
@lynnagara lynnagara deleted the typecheck-consumer-test branch November 7, 2022 20:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants