Skip to content

Conversation

squahtx
Copy link
Contributor

@squahtx squahtx commented Nov 28, 2022

Depending on timings, the "Existing members see new members' presence"
test in TestMembersLocal may see the expected join and presence in an
initial or incremental sync. Both types of sync elicit different
behaviour in homeserver implementations, such as Synapse. Split the test
into initial and incremental sync variants to eliminate the race
condition.

In addition, fix the initial sync case by explicitly setting Bob's
presence to online. The spec does not guarantee that Bob will have a
non-offline presence after merely joining a room, nor does the spec
guarantee that offline presences will show up in initial syncs.

Fixes matrix-org/synapse#13199.

Sean Quah added 2 commits November 28, 2022 13:48
Split "Existing members see new members' presence" `TestMembersLocal`
test into initial and incremental sync variants.
Explicitly set Bob's presence to online, since the spec does not
guarantee that Bob will have a non-offline presence after joining a
room, nor does the spec guarantee that offline presences will show up
in initial syncs.
@squahtx squahtx requested review from a team as code owners November 28, 2022 13:53
Copy link
Contributor

@DMRobertson DMRobertson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Many thanks for digging into this Sean, LGTM

@squahtx squahtx requested a review from DMRobertson November 28, 2022 17:52
@squahtx
Copy link
Contributor Author

squahtx commented Nov 28, 2022

I had to move Bob's join out of the test case, since it was getting queued after the two test cases that relied upon it.

@DMRobertson
Copy link
Contributor

I had to move Bob's join out of the test case, since it was getting queued after the two test cases that relied upon it.

To check: was that just an oversight in porting the test to complement?

@squahtx
Copy link
Contributor Author

squahtx commented Nov 28, 2022

I had to move Bob's join out of the test case, since it was getting queued after the two test cases that relied upon it.

To check: was that just an oversight in porting the test to complement?

Looks like it. In the sytest, Bob joins the room as part of the setup.
https://github.com/matrix-org/sytest/blob/225593e864be317c6efd2823a03a43f56294eebf/tests/30rooms/02members-local.pl#L21-L28

When I split the last test case into two, it changed the pattern in which the tests run and made it a problem.

@squahtx
Copy link
Contributor Author

squahtx commented Nov 28, 2022

Thanks for all the fast reviews today!

@squahtx
Copy link
Contributor Author

squahtx commented Nov 28, 2022

The tests in this PR aren't run on Dendrite, and the Dendrite CI failures seen are also happening on main.

@squahtx squahtx merged commit ab6cf55 into main Nov 28, 2022
@squahtx squahtx deleted the squah/fix_test_members_local_flake branch November 28, 2022 18:02
@ShadowJonathan
Copy link
Contributor

This was already being addressed in #516, I would appreciate it if that PR was given attention instead of concurrent changes being merged at the same time.

@ShadowJonathan
Copy link
Contributor

ShadowJonathan commented Dec 10, 2022

This PR is also faulty, it uses BlueprintOneToOneRoom, which was already determined in #516 to provide faulty results, as Bob and Alice would already see eachother's presence changes through that pre-existing one-on-one room.

As this was already pre-determined in that PR, that apparently was not looked for, I will also be fixing this issue in that PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Complement Existing members see new members' presence is flaky
3 participants