![]() If you review the source code, it won’t be immediately obvious that this parameter is required.īut, if you review the source code for BaseOperator the abstract base class for TriggerDagRunOperator, you will see that a task_id is required for initialization. execution_date: this is an optional date time objectĪ gotcha is that, in addition to those three parameters, it also requires a unique task_id as well.python_callable: an optional python method that receives the current context object and is also passed the dag_run object.Airflow documentation as of 1.10.10 states that this TriggerDagRunOperator requires the following parameters: This can be achieved through the DAG run operator TriggerDagRunOperator. Triggering a DAG can be accomplished from any other DAG so long as you have the other DAG that you want to trigger’s task ID. This is undesirable - as we would want the backfill date string being provided to dag_a to also be propagated through to the second DAG, dag_b. Instead, a new context is generated for dag_b, and, as a result, dag_b has a context which has the current date, 06-01. But, when the first DAG triggers the second DAG, dag_b, dag_b does not receive the same context. When the backfill DAG job is triggered in Airflow, dag_a receives a context which includes the date for the backfill job (in this case, the 04-13 date). The backfill date is going to be for 04-13-2020. Let’s say the current date is 06-01-2020. In this example, you might have one DAG and a second, let’s call them dag_a and dag_b. Why would you want to pass the context from the first DAG to the second? For example, let’s say you want to trigger a backfill or rerun a DAG for a prior date. Second, I demonstrate how to pass through the context from the first DAG to the second. First, I document how to trigger a DAG from within another Airflow DAG, without the trigger being attached to either DAG. The intent of this post is to demonstrate how to accomplish two tasks. External DAG triggers in Airflow May 31, 2020
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |