Azure pool advanced options
acknowledged
B
Brass Wildcat
Presently, Nextflow offers advanced options for Azure pools that aren't accessible through the UI. This presents a valuable opportunity to enhance Azure support by incorporating these advanced options, providing administrators with seamless control over Azure compute environments directly from the UI.
Intended Outcome:
Administrators will be empowered to create Azure compute environments with advanced options, previously available only in Nextflow, directly through the UI. This eliminates the need for workarounds, streamlining the configuration process for Azure pools.
Configurations to be added:
- azure.batch.pools..publisher: High priority, to be implemented per pool.
- azure.batch.pools..offer: High priority, to be implemented per pool.
- azure.batch.pools..sku: High priority, to be implemented per pool.
- azure.batch.pools..schedulePolicy: High priority, to be implemented per pool.
- azure.batch.pools..privileged: High priority, to be implemented per pool.
- azure.batch.pools..runAs: High priority, to be implemented per pool.
- azure.batch.pools..scaleFormula: Low priority, to be implemented per pool; Can be specified on the NF config.
- azure.batch.pools..scaleInterval: Low priority, to be implemented per pool; Can be specified on the NF config.
Drew DiPalma
Merged in a post:
Azure Batch: Add option for TargetDedicatedNodes or TargetLowPriorityNodes or TargetSpotNodes when using forge.
V
Vanilla Galliform
There is no native support for Azure Batch low priority or spot machines. If you create a pool with Batch Forge, you must update the autoscaling formula afterwards to replace
TargetDedicatedNodes
with TargetLowPriorityNodes
which will change the target nodes to low priority spot machines. However, due to the limitations of Azure Batch having a pool hang and require manual deleting is a frequent occurrence. This then requires manually recreating the pool. Letting forge create and destroy the pools as required is much cleaner.
Adam Talbot
Related issue: https://seqera.canny.io/feature-requests/p/azure-pool-advanced-options
Rob Newman
acknowledged