Ability to use the tower-cli (`tw`) to run Nextflow jobs interactively
acknowledged
F
Flamingo pink Python
- As per a conversation with Phil Ewels on how to make the Nextflow Tower / Seqera Platform more amenable to pipeline development (as opposed to production pipeline runs)
There are two main blockers for using Platform as the location for pipeline development. One of them is the fact that you cannot run your current local copy of your Nextflow pipeline as-is, and are forced to
git commit
unfinished changes in order to be able to run them. A second issue is the constant context-switching needed to jump between your dev applications (terminal, editor) and the Platform itself (web browser) to see the state of any pipeline run that you may have submitted with tw
.An idea for handling the latter case is to simply make
tw run
an interactive command, the same way that nextflow run
currently is. So instead of the current usage of tw launch
that simply sends the job over to Platform to run, instead there should be a command such as tw run
that sends the job over to Platform but then interactively monitors and reports on the job in the exact same way that nextflow run
does. This could be really helpful to help make
tw
a more central part of the pipeline development workflow, removing the need to essentially stop all your dev work and context-switch over to the web browser every time you want to see if your changes to the pipeline break or not.Rob Newman
Flamingo pink Python - have you investigating using Data Studios for developing/troubleshooting pipelines in the platform using VSCode? Do you need a walkthrough?
F
Flamingo pink Python
Rob Newman I have not. I am not sure that anyone on our side is using Data Studios at all yet. We should schedule a demo of it at some point.
F
Flamingo pink Python
thanks for the feedbacks. I will be evalulating the
tw --wait
feature. In general, I was envisioning that ultimately tw
could be a drop-in replacement for nextflow run
itself in this kind of context. I think that would be cool.Rob Newman
acknowledged
Phil Ewels
As discussed,
tw
has a --wait
flag which tells it to keep running until the desired run state is reached. However it does not route the Nextflow CLI log output to the terminal, so isn't exactly as requested here.