Hello! In a previous post, I wrote about how the runtime state of ADF triggers is ignored even if it is set in the ARM template. This means we have a couple of options when manging runtime state acorss different deployments of the same ADF. The first one is to use the UI of the ADF to set the trigger state. Even as someone who is quite vociferous against using UI’s to manage configuration, I think this is far from the worst idea.
Hello! So, continuing the saga of deploying the ARM templates of ADFs that are generated by the npm package made avilable by Microsoft, I came across something that I was not aware of, and honestly I really should have been. Although runtimestate can be included in an ARM Template to deploy a Trigger, the runtimestate on the deployed ADF will not be altered to match what is in the ARM template.
Hello! This series on ADF deployment management seems to be never ending and always evolving. In a previous post I shared some code that could be added to an Azure DevOps pipelines that would check for failed deployment messages. This was necessary as typically we’d not get a useful error back, and in some environments we did not have access to the Resource Group to check for deployment errors. Now that we have moved to making deployments using linked templates for ADF this code is no longer helpful, as it returns the same top error: each linked template has its own entry in the Deployment registry.
Hello! I’ve spent the day parallelising the entire PrePostDeploymentScript.Ver2.ps1, as well as merely the deleting deployments. I’ve also fixed a casing issue where the parameters file was not being found on the ubuntu-latest Azure DevOps Agent. This is the link for the Pull Request, and this is the link for the copy of the file in my forked repo. I have no idea if the changes will be accepted or not, but hopefully it will as it’s a good script it just needs a few tweaks to it.
Hello! The saga continues with deleting deployments for linked templates… come to find out that deleting linked templates serially can be extremely slow. So I updated my script to parallelise the deletion of the templates. Fortunately ForEach-Object has a parallel feature and can throttle the number of threads. It’s quite neat that each deplyoyment has all the info in it so the script block is able to reference the object for all it needs.
Hello! In a previous post, I wrote about some of the issues with the PowerShell script used to delete deployments. This is packaged as part of the npm package provided by Microsoft. We recently hit an issue with the number of resources in the ARM template generated by this npm package. Helpfully the package also creates the ARM templates as linkedtemplates. This was all fine, however the delete deployments section of this ill-fated PowerSHell script does not account for multiple deployments, which is exactly what happens when you use linked templates.
Hello! As part of the npm package provided by Microsoft, there is a PowerShell script called PrePostDeployment script. This script very helpfully sorts out the ADF prior to the ARM tempalte being deployed. This script however is not without its problems. Firstly when it attempts to stop triggers it does so by getting a list of all those that are not stopped. This line used to not include the -and $_.
Hello! We’ve been using the npm package to build the ADF source code. The npm package, which is officially from Microsoft, will generate an ARM tempalte, which can then be deployed. As we’re using Azure DevOps it makes sense to use the AzureResourceManagerTemplateDeployment@3 task to deploy the ARM Template. All very nice, but we have found that owing to the sheer number of objects being deployed we get a very unhelpful error message back when it fails:
Hello! ADF is one otf those Azure tools that seems to be getting updated constantly. And to keep up with the changes I like to watch the videos on Channel 9. And last Friday there was another video which spoke about being able to tag pipelines. It’s talked about in the first few minutes of this video. Great! But we have pipelines that are executed by other pipelines. Wouldn’t it be even more great if these annotations accepted dynamic content, like most of the rest of ADF does?
Hello! I’ve been working quite a bit lately with Azure Data Factory (V2) and Ive been deploying them using ARm Templates. Well technically, I’m using terraform. But terraform doesn’t support ADF and so it’s a case of using terraform to deploy an ARm Template. Anyway, I digress… I’ve included the triggers in the ARM Template to be deployed, yet after the first deployment of the triggers i got this error below -