ChatGPT as DevOps Engineer


ChatGPT has been on everyone’s lips for last two months. I personally think, that it is a great piece of technology, that will definitely has it place in the software ecosystem. To find out how good it really is, I made a test.

I am a DevOps guy by heart and I have read a lot of twitter discussions and blog posts about how the ChatGPT is going to replace us all, because we are just mere humans. However I had my doubts about it and as I wanted to know my enemy better, I asked some series of questions from it.

The Interview

After short but serious conversation to really measure my upcoming enemy, I asked ChatGPT to make me an Azure DevOps release pipeline for Blazor application that is deployed into Azure App Service.

YAML for Blazor WebAssembly app

And I have to say that it did a really good job. It used variables, comments and named the jobs really. It even added some description about what the pipeline is doing. I never do that.

At the end you get nice description about what the pipeline is doing

The Mistake

As I was trembling in my chair and sank deeper into the abyss of incompetence, I noticed one interesting thing in the YAML. The ChatGPT used AzureWebApp@1 task to deploy the app, but it didn’t provide the azureSubscription input which is mandatory. With this little boost of confidence I asked ChatGPT to refine the pipeline a little bit and to use App Service Slot swap (which is very popular app service feature) to deploy the app.

Refined pipeline which uses App Service slots

Ok it knows the slots too, damn it! Lets see what the final YAML looked like.

# Name of the pipeline

name: MyBlazorAppDeployment

# Trigger definition: Pipeline runs on every push to the main branch




- main

# Variables definition: Define variables that can be used throughout the pipeline


# Azure resources

azureSubscription: 'MyAzureSubscription'

resourceGroup: 'MyResourceGroup'

appServiceName: 'MyAppServiceName'

# Build and deploy settings

buildConfiguration: 'Release'

publishDir: '$(Build.ArtifactStagingDirectory)/wwwroot'

# Jobs definition: The set of steps to be executed in the pipeline


# Job named "build"

- job: Build


vmImage: 'windows-latest'


# Checkout the code from the repository

- checkout: self

# Build and publish the app

- task: DotNetCoreCLI@2

displayName: 'dotnet publish'


command: publish

publishWebProjects: false

projects: '**/*.csproj'

arguments: '-c $(buildConfiguration) -o $(publishDir)'

# Publish the built application to the pipeline

- task: PublishBuildArtifacts@1


artifactName: 'drop'

pathToPublish: '$(publishDir)'

publishLocation: 'Container'

# Job named "deploy"

- job: Deploy

dependsOn: Build


vmImage: 'windows-latest'


# Download the built artifacts from the previous job ("Build")

- download: current

artifact: drop

# Authenticate to Azure and set subscription context

- task: AzureCLI@2


azureSubscription: $(azureSubscription)

scriptType: 'pscore'

scriptLocation: 'inlineScript'

inlineScript: |

az account show

az account set --subscription $(azureSubscription)

# Create a new deployment slot with the name "staging"

- task: AzureWebAppDeployment@4

displayName: 'Create Deployment Slot'


azureSubscription: $(azureSubscription)

appType: 'webAppLinux'

WebAppName: $(appServiceName)

ResourceGroupName: $(resourceGroup)

SlotName: 'staging'

# Deploy the app to the "staging" slot

- task: AzureWebApp@1

displayName: 'Azure Web App Deploy (Staging)'


package: '$(System.ArtifactsDirectory)/drop'

appName: $(appServiceName)

slotName: 'staging'

resourceGroupName: $(resourceGroup)

# Swap the slots to make the "staging" slot the production slot

- task: AzureAppServiceManage@0

displayName: 'Swap Slots'


ConnectedServiceName: $(azureSubscription)

Action: 'SwapSlots'

ResourceGroupName: $(resourceGroup)

WebAppName: $(appServiceName)

SourceSlot: 'staging'

Ok that’s quite impressive. Let’s see how well it works in real life and put it into Azure DevOps.

ChatGPT tried to use task called AzureWebAppDeployment, which doesn’t exist.

As it turns out, it doesn’t. The ChatGPT made up a completely new task called AzureWebAppDeployment. There is no task in Azure DevOps with that name. There is a task called AzureRmWebAppDeployment, so I think it meant to use that (we all do mistakes). Even after changing that, tweaking handful of parameters and fixing typos I still couldn’t get it running. It was just so far off, that I would have done it maybe faster from scratch, than spent all this time fixing missing parameters and wrong file paths.

After few hours of fixing variables and typos I gave up.

Closing Speech

One big problem with ChatGPT is, that it looks really promising and credible. It can even made up its own tasks, that doesn’t exist and we believe it. However (and luckily for me), at this stage it cannot replace humans and I think that is not even it’s purpose. ChatGPT is a tool. Tool like IDE, Powershell or Google. It helps us to solve problems, but it doesn’t try to step into our shoes.

I will definitely keep eye on this project and see how it develops. It has great potential as a tool to support my DevOps career, but if it tries to take my place… it can try to.

Want more information about Azure? Gain access to our premium content by signing up to the ESPC community.

About the Author

Panu Oksala mainly writes about Azure, DevOps and C#. You can find him also on Twitter @panuoksala and LinkedIn panu.oksala.


Oksala, P., 2023, ChatGPT as DevOps Engineer,, Available at: [Accessed on 4 July 2023]

Share this on...

Rate this Post: