Skip to content

Conversation

Aaronontheweb
Copy link
Member

Modified Akka.Actor benchmarks to get more accurate reading
Added dispatcher-specific specs for Akka.Remote
close #1547 - fixed benchmarks that depend on TestTransport
close #1556 - memory leak caused by not clearing TestTransport logs

{
public static Config ThreadPoolDispatcherConfig => ConfigurationFactory.ParseString(@"
akka.remote.default-remote-dispatcher {
type = Dispatcher
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might need someone else to verify this, but this is the correct way to specify the ThreadPoolDispatcher in HOCON, no?

@Aaronontheweb Aaronontheweb changed the title Modified Akka.Remote.Transport benchmarks to get more accurate readings [WIP] Modified Akka.Remote.Transport benchmarks to get more accurate readings Dec 16, 2015
@Aaronontheweb
Copy link
Member Author

Putting the WIP tag on this, as it was able to immediately reproduce the deadlock that I thought I fixed on the TestTransport.

And the errors with the ProtocolStateActor.Disassociate, which are extremely annoying, are happening because of a race in the spec itself (calling shutdown on both systems at the same time, which puts the Disassociate message at the head of the queue in front of the Disassociated message generated by disassociating from the TestTransport.)

@Aaronontheweb
Copy link
Member Author

Going to continue tinkering with this, but I could use some help from @akkadotnet/contributors on resolving #1547 as part of this PR. It could be any one of the following:

  1. Bug with Ask
  2. TPL issue (wrong TaskContinuationOption or something like that)
  3. Bug with Akka.Remote initial messages used to open a connection (the priority buffer and other fun stuff inside Akka.Remote)
  4. Bug with just the TestTransport's association code.

The bug occurs here inside the spec:

 [PerfSetup]
        public void Setup(BenchmarkContext context)
        {
            _remoteMessageThroughput = context.GetCounter(RemoteMessageCounterName);
            System1 = ActorSystem.Create("SystemA" + Counter.Next(), CreateActorSystemConfig("SystemA" + Counter.Current, "127.0.0.1", 0));
            _echo = System1.ActorOf(Props.Create(() => new EchoActor()), "echo");

            System2 = ActorSystem.Create("SystemB" + Counter.Next(), CreateActorSystemConfig("SystemB" + Counter.Current, "127.0.0.1", 0));
            _receiver =
                System2.ActorOf(
                    Props.Create(() => new BenchmarkActor(_remoteMessageThroughput, RemoteMessageCount, _resetEvent)),
                    "benchmark");

            var system1Address = RARP.For(System1).Provider.Transport.DefaultAddress;
            var system2Address = RARP.For(System2).Provider.Transport.DefaultAddress;

            var system1EchoActorPath = new RootActorPath(system1Address) / "user" / "echo";
            var system2RemoteActorPath = new RootActorPath(system2Address) / "user" / "benchmark";
            _remoteReceiver =
                System1.ActorSelection(system2RemoteActorPath).Ask<ActorIdentity>(new Identify(null), TimeSpan.FromSeconds(2)).Result.Subject;
            _remoteEcho =
                System2.ActorSelection(system1EchoActorPath)
                    .Ask<ActorIdentity>(new Identify(null), TimeSpan.FromSeconds(2))
                    .Result.Subject;
        }

The Ask operation in either case randomly times out and I'd like to fix that. Should never time out on an in-memory connection in under 2 seconds.

@Aaronontheweb
Copy link
Member Author

I think I've figured out #1547 - going to run the perf tests multiple times on the build server so we can be confident that it's resolved now.

@Aaronontheweb Aaronontheweb changed the title [WIP] Modified Akka.Remote.Transport benchmarks to get more accurate readings [Ready] Modified Akka.Remote.Transport benchmarks to get more accurate readings Dec 17, 2015
@Aaronontheweb Aaronontheweb added ready and removed WIP labels Dec 17, 2015
@Aaronontheweb
Copy link
Member Author

Ready to be merged.


System2.ActorSelection(system1EchoActorPath).Ask<string>("foo").ContinueWith(tr =>
{
Console.WriteLine("Received foo");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Whats the purpose of this line? are there some stdout capture going on?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, good catch - added it when I was debugging. I'll remove it.

var remoteTask1 = System1.ActorSelection(system2RemoteActorPath).Ask<ActorIdentity>(new Identify(null));
remoteTask1.Wait();
_remoteReceiver = remoteTask1.Result.Subject;
_remoteEcho =
System2.ActorSelection(system1EchoActorPath)
.Ask<ActorIdentity>(new Identify(null), TimeSpan.FromSeconds(2))
.Result.Subject;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

System2.ActorSelection(system1EchoActorPath).ResolveOne().Result.Subject would be more idiomatic.
Might not matter in the test but I've seen the above pop up a few times before.

Modified Akka.Actor benchmarks to get more accurate reading
Added dispatcher-specific specs for Akka.Remote
close akkadotnet#1547 - fixed benchmarks that depend on `TestTransport`
close akkadotnet#1556 - memory leak caused by not clearing `TestTransport` logs
fixed shutdown spam errors
added association stress test
Aaronontheweb added a commit that referenced this pull request Dec 17, 2015
[Ready] Modified Akka.Remote.Transport benchmarks to get more accurate readings
@Aaronontheweb Aaronontheweb merged commit 9d875f0 into akkadotnet:dev Dec 17, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Akka.Remote memory leak Race conditions within Akka.Remote.Transport.TestTransport
2 participants