In some cases you want to end the Azure Data Factory (ADF) pipeline execution based on a logic in the pipeline itself. For example, when there’s no record coming from one of the inputs datasets then you need to fail quickly to either reduce cost or to avoid any logical errors.

The challenge is there’s no activity in ADF that cancels execution. In this case, the web activity comes handy

Let’s take a quick example, in the picture below, a simple logic pipeline that sets a variable to true and then based on the value of the variable, it has If Condition activity to cancel the pipeline execution when it’s true (of course it’s always true in this example, but you get the point) simple logic pipeline

Inside the true branch of the If Condition add Web activity (under general) Web Activity

Now what are we trying to do?

Since there’s no activity then we need to call the ADF REST API which is part of the Azure Resource Manager ( . The API we are interested in is the Cancel Pipeline Run API

The API format POST{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.DataFactory/factories/{factoryName}/pipelineruns/{runId}/cancel?api-version=2018-06-01 Luckily the {factoryName} and the {runId} can be obtained using ADF functions during runtime (with a catch) using pipeline().DataFactory and pipeline().RunId respectively

What’s the catch? The RunId doesn’t work when execute the pipeline under debug mode. You need to trigger the pipeline to get the correct RunId That makes the REST call like this

@concat('***subscriptionID***/resourceGroups/***resource group name***/providers/Microsoft.DataFactory/factories/',pipeline().DataFactory,'/pipelineruns/',pipeline().RunId,'/cancel?api-version=2018-06-01')

How about authentication? Correct, ARM REST API calling can be daunting because of the oauth authentication workflows. Fortunately the Web activity supports Managed (Service) Identity . In nutshell, every Data Factory instance has a service identity created in Azure AD to be used by this instance.

The settings page should look like this

Web Activity Settings

Note that the resource is the resource that you want to connect to which is the Azure Resource Manager (ARM) API ==>

If you run the pipeline this way, probably you will get access denied error because this managed identity doesn’t have permissions to stop the pipeline run for this data factory. To grant this permission, assign the managed identity the Data Factory contributor Role (least privileged) or Contributor role (more privileged)


The Managed Identity name is the same name as the data factory There’s optional parameter to this API to cancel recursively so if the pipeline calls another pipeline, the called pipeline will be cancelled as well

The json for the web activity is added in this code snippet.