Description

Currently, the job_file_dir is set from the law config. I have a use case where multiple HTCondorWorkflows may be running simultaneously, in which case I noticed that job files were being overwritten by one another.

The workaround I found was to hack in to the htcondor_create_job_file_factory hook:

class MyHTCWorkflow(HTCondorWorkflow):

def htcondor_create_job_file_factory(self, **kwargs):
        # set the job file dir to proper location
        kwargs["dir"] = self.job_file_dir
        return super().htcondor_create_job_file_factory(**kwargs) 

So that I can set the job_file_dir in a task dependent way.

This solution works fine, but this seems like a common enough use case that it should be more obvious how / where this can be set. I propose making this a parameter of the HTCondorWorkflow that can default to reading from the config if left unset.

Can put together a PR if this is something that is desirable.