Azure Data Factory check rowcount of copied records

azure-data-factory
jbazelmans picture jbazelmans · Jul 27, 2018 · Viewed 6.9k times · Source

I am designing a ADF pipeline that copies rows from a SQL table to a folder in Azure Data Lake. After that the rows in SQL should be deleted. But for this delete action takes place I want to know if the number rows that are copied are the same as the number of rows that were I selected in the beginning of the pipeline. Is there a way to get the rowcount fo the copy action and use this in another action (like a lookup)

Edit follow up question: Bo Xiao's answer is OK. BUt then I have a follow up question. After the copy-activity I put an If Condition with the following expression:

@activity('LookUpActivity').output.firstRow.RecordsRead ==      @{activity('copyActivity').output.rowsCopied

But then I get the error: @activity('LookUpActivity').output.firstRow.RecordsRead == @{activity('copyActivity').output.rowsCopied

Isn't it possible to compare output parameters of two activities to see if this is True?

extra edit: I just found an error in this piece of code. I forgot a "{" at the begin of the code. But then the code is still wrong. To compare two outputs from earlier activities the code must be:

@equals(activity('LookUpActivity').output.firstRow.RecordsRead,activity('copyActivity').output.rowsCopied)

Answer

Bo Xiao picture Bo Xiao · Jul 27, 2018

You can find copied rows in activity output

And you can use the output value like this: 

@activity('copyActivity').output.rowsCopied

Related questions

The client with object id does not have authorization to perform action 'Microsoft.DataFactory/datafactories/datapipelines/read' over scope

I was trying to invoke data factory pipeline from azure function programmatically. Its throwing following error. link: http://eatcodelive.com/2016/02/24/starting-an-azure-data-factory-pipeline-from-c-net/ AuthorizationFailed: The client 'XXXX-XXXXX-XXXX' with object id 'XXX829e05'XXXX-XXXXX' does not have authorization to perform action 'Microsoft.DataFactory/…

Read More
The subscription is not registered to use namespace 'Microsoft.DataFactory error

Going through this tutorial "Create a pipeline with Copy Activity using Visual Studio" and recieving this error when i hit publish. Error not mentioned anywhere on net and very little help/knowledge on azure generally on web.

Read More
Using Azure Data Factory to get data from a REST API

Is it possible to use Azure Data Factory to get data from a REST API and insert it to a Azure database table?

Read More