_tablename_ = sla_miss ¶ task_id ¶ dag_id ¶ execution_date ¶ email_sent ¶ timestamp ¶ description ¶ notification_sent ¶ _table_args_ ¶ _repr_ ( self ) ¶ class airflow.models. It is used to keep track of SLA failures over time and to avoid double Model that stores a history of the SLA that have been missed. ImportError ¶īases: _tablename_ = import_error ¶ id ¶ timestamp ¶ filename ¶ stacktrace ¶ class airflow.models. id ¶ pickle ¶ created_dttm ¶ pickle_hash ¶ _tablename_ = dag_pickle ¶ class airflow.models. The executors pick up the DagPickle id and read the dag definition from A pickle is a native python serialized object,Īnd in this case gets stored in the database for the duration of the job. Object represents a version of a DAG and becomes a source of truth forĪ BackfillJob execution. DagPickle ( dag ) ¶ĭags can originate from different places (user repos, master repo, …)Īnd also get executed in different places (different executors). “list of datasets” -> manually defined list of DataSetĬlass airflow.models. “list of task_ids” -> picks up outlets from the upstream task_ids If A -> B -> C and B does not have outlets but A does, these are provided as inlets. “auto” -> picks up any outlets from direct upstream tasks that have outlets defined, as such that Saves the lineage to XCom and if configured to do so sends it Whether the task exists in this bag Return typeĪ_dag.BaseDag airflow.models. dag_ids ¶ ReturnsĪ list of DAG IDs in this bag Return type BaseDagBag ¶īase object that both the SimpleDagBag and DagBag inherit. The pickle ID for this DAG, if it has one. Whether this DAG is paused or not Return type Maximum number of tasks that can run simultaneously from this DAG Return type The absolute path to the file that contains this DAG’s definition Return type _metaclass_ ¶ dag_id ¶ ReturnsĪ list of task IDs that are in this DAG Return type BaseDag ¶īase DAG object that both the SimpleDag and DAG inherit. Reschedule_date – The date when the task should be rescheduled class airflow.models. Raise when the task should be re-scheduled at a later time. AirflowRescheduleException ( reschedule_date ) ¶īases: AirflowTaskTimeout ¶īases: exception airflow.models. AirflowSkipException ¶īases: exception airflow.models. AirflowException ¶Įach custom exception should be derived from this class status_code = 500 ¶ exception airflow.models. AirflowDagCycleException ¶īases: exception airflow.models.
Sync ( self ) ¶ end ( self ) ¶ start ( self ) ¶ execute_async ( self, key, command, queue=None, executor_config=None ) ¶ sync ( self ) ¶ end ( self ) ¶ exception airflow.models. start ( self ) ¶ execute_async ( self, key, command ) ¶ Parameters Implements LocalExecutor with limited parallelism using a task queue toĬoordinate work distribution. Sync ( self ) ¶ end ( self ) ¶ class _LimitedParallelism ( executor ) ¶ Key ( tuple ( dag_id, task_id, execution_date )) – the key to identify the TI Implements LocalExecutor with unlimited parallelism, starting one process class _UnlimitedParallelism ( executor ) ¶ Multiprocessing Python library and queues to parallelize the execution LocalExecutor executes tasks locally in parallel. LocalExecutor ¶īases: _executor.BaseExecutor
GetDefaultExecutor ( ) ¶Ĭreates a new instance of the configured executor if none exists and returns it class airflow.models.
0 kommentar(er)
