Skip to content

Handle Global configuration properties #134

@potiuk

Description

@potiuk

It is related to #39 but it's about making sure that global configuration properties are passed to all tasks inside generated dags.

Most actions in a workflow need several of the same settings (e.g., the JobTracker URI, NameNode URI, Hadoop scheduler queue, etc.). Users can avoid repeating all these settings and job configuration values in every action of the workflow by defining them once in the workflow’s section. The properties defined in the section are available to all actions of the workflow.

Seems that might be probably somehow mapped to default_args in most cases, but then we have to make sure we map all the configuration properties from #39 to some arg:

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestnice-to-haveNice to have - not needed for the initial implementation

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions