Problem
While using the Azure ARM plugin for cloud profile management of VMs for TeamCity agents - and we're currently at v0.14.0 of that plugin.
We're consistently getting failures on certain machines (4 of them) when TeamCity goes to create a VM in Azure based on the cloud profile:
---Failed to create deployment <id> in resource group <resource_id>: At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details. Disk <disk_id> already exists in resource group <resource_id>. Only CreateOption.Attach is supported. (ConflictingUserInput) (Conflict)---
Context
- Azure ARM plugin version 0.14.0
Solution
These IDs are generated based on the virtual machine name and are modified with suffixes directly in the deployment template.
It appears that the default VM removal strategy is currently being used. In this mode, TeamCity builds a resource tree and deletes resources one by one. Sometimes Azure may return a list of deployment operations where earlier operations have a later timestamp. This can lead to errors similar to the one observed.
Based on the deployment templates present in the logs, they all use deleteOption=Delete for the VM’s dependent resources. In such cases, it is recommended to enable the following flag at the project level (add it as a project parameter):
teamcity.cloud.azure.arm.agent.features.template.safeRemoval=true
With this flag enabled, TeamCity deletes only the virtual machine itself (and deployment template), while Azure automatically removes the associated resources (such as disks and network adapters).
Before enabling this flag, it should be verified that there are no deployment templates in the project where dependent resources are created without the deleteOption=Delete option. This mainly applies to custom user deployment templates. Internal deployment templates used by TeamCity already include these options. Otherwise, disks and network adapters may remain in Azure after the VM is removed.
After enabling the flag, it will be necessary to wait for new VMs to start, since their deployment templates will include a special tag indicating that simple VM removal is supported
Additional information
In some cases, this issue might not be resolved by the steps above. In that case, chances are that the issue is on Azure's side.
Try attaching the problematic disks to a test virtual machine and then removing them manually. One way to do this is:
create any test VM,
• in Azure Portal, navigate to VM → Disks → Attach existing disks,
• check whether the missing disks appear in the dropdown list.
If such a disk can be attached, the VM (along with all attached disks) can then be removed normally via the portal.
Alternatively, the same can be done via PowerShell as described here:
In this case, instead of retrieving disk information, the disk ID can be specified explicitly, for example:
/subscriptions/12345678-1234-1234-9b67-9fbe12fad665/resourceGroups/us5-tc-mgmt/providers/Microsoft.Compute/disks/az-deploy-1
Comments
0 comments
Please sign in to leave a comment.