So, whilst this seems like a worthwhile "fix" from a code point of view it raised a query for me..

The parallel:drop and parallel:create rake tasks don't appear to look at the passthrough argument to see if -n or -m are populated. They seem to rely on their count: argument. Assuming I haven't missed anything, that means all the processor count * multiplier logic that parallel:spec runs to work out num_processes effectively needs to be manually performed before calling drop and create, so that the calculated value can be put in the count: argument for them? (which seems counter intuitive to me so I'm assuming I've missed something really obvious?)

rake tasts don't take arguments, so that can't work ... or do you mean the test-options rake argument ?

yeah would be best to make options[:multiply_processes] fall back to an env var, so it can be used for create+drop etc too

lmk if you want this released or want to do another PR for that first

lmk if you want this released or want to do another PR for that first

Looks like you've merged it so I'll say yes please do haha..

What I meant above re arguments is:

  ParallelTests::Tasks.for_each_database do |name|
    task_name = 'drop'
    task_name += ":#{name}" if name
    desc "Drop test#{" #{name}" if name} database via db:#{task_name} --> parallel:#{task_name}[num_cpus]"
    task task_name.to_sym, :count do |_, args|     <--- the :count argument here comes in the args hash as args[:count]
      ParallelTests::Tasks.run_in_parallel(
        [
          $0,
          "db:#{task_name}",
          "RAILS_ENV=#{ParallelTests::Tasks.rails_env}",
          "DISABLE_DATABASE_ENVIRONMENT_CHECK=1"
        ],
        args
      )
    end
  end

Whereas the main spec run accepts :count plus a few others - including :pass_through

  ['test', 'spec', 'features', 'features-spinach'].each do |type|
    desc "Run #{type} in parallel with parallel:#{type}[num_cpus]"
    task type, [:count, :pattern, :options, :pass_through] do |_t, args|
      ParallelTests::Tasks.check_for_pending_migrations
      ParallelTests::Tasks.load_lib
      command = ParallelTests::Tasks.build_run_command(type, args)

      abort unless system(*command) # allow to chain tasks e.g. rake parallel:spec parallel:features
    end
  end

The problem as I see it is the -n processes and -m multiplier can be included in the :pass_through argument, but they don't get fed into the drop/create tasks... So - at least in my testing - I'm ending up dropping Parallel.processor_count databases, and creating Parallel.processor_count databases.. and then I try to run -n processes... and if -n val > Parallel.processor_count then rspec crashes trying to connect to databases that don't exist..

@grosser - my first thought is to split out the options parse and num_processes calculation logic into something that could be called from each - but that might be just based on how we use it (ie we throw the options hash at all of the tasks):

  args.with_defaults(
    pattern: Dir.glob('spec/*/').tap { _1.delete('spec/api/') }.join('|'),
    pass_through: '-m 0.25 -i spec/services/isolated_spec.rb'
  )
  Rake::Task['parallel:drop'].execute(args)
  Rake::Task['parallel:setup'].execute(args)
  Rake::Task['parallel:spec'].execute(args)

I'm neck deep in go-live work at the moment so it'll be a next month thing if I can get it happening. That said, just musing on using an environment variable like you mentioned above my gut feel is it's awkward. ie. I'm going to be copying the "num_processes" code out of the Gem just so I can run it in myself in my rake task to compute the processes and define the variable. In my mind it would be a lot cleaner if drop and setup accepted pass_through and were smart enough to use the options that were relevant to them.

Let me know what you think so I don't bother making a PR that isn't the path forward you want for the Gem.

• not a fan of adding pass_through, the rake interface is already ugly enough :) ... maybe as env PARALLEL_TEST_ARGS=X and then using the option parser on that and merging stuff in to have a universal solution
• env var for just that option seems simple to me PARALLEL_TESTS_* makes it pretty clear

for now released bugfix as 5.3.1