Airflow Xcom Exclusive

XCom is essential for building dynamic DAGs where downstream tasks depend on the output of upstream tasks.

def explicit_push(**context): context['ti'].xcom_push(key='my_key', value='my_value')

XComs are strictly tied to specific task instances and execution dates. airflow xcom exclusive

XCom (short for Cross-Communication) is a mechanism in Apache Airflow that allows tasks within a DAG to exchange small pieces of data. It acts as a message board where one task can push data (key-value pair) and another task can pull that data.

Use these strategies depending on your requirement: XCom is essential for building dynamic DAGs where

There are two main ways to push data:

Method 1: The Return Value (Best Practice) If a task returns a value, Airflow automatically pushes it to XCom with the key return_value. This is the cleanest method. def explicit_push(**context): context['ti']

def extract_data(**kwargs):
    # logic here
    file_path = "/tmp/data_2023.csv"
    return file_path  # This is automatically pushed to XCom
extract_task = PythonOperator(
    task_id='extract_data',
    python_callable=extract_data,
)

Method 2: Explicit Push You can explicitly push data using the xcom_push method inside the function. This is useful if you need to push multiple values.

def process_data(**kwargs):
    ti = kwargs['ti']
    ti.xcom_push(key='processed_file', value='/tmp/processed.csv')
    ti.xcom_push(key='record_count', value=500)

# Pushing XCom (implicitly via return)
def push_task(**context):
    return "some_value"