Show Mean + StD Task Duration on Cluster Details Page

[evamaxfield]

Hello all!

First I want to say thank you for putting coiled together, it has made it so much easier to use cloud compute than before and really scale out ❤️

Second, I am wondering if it would be possible to add a new metric to the "Cluster Details" dashboard page. Specifically, I would love to have a "mean" and "std" duration per task.

My primary use case for this is to understand which parts of my pipeline are taking up lots of work and where they could possibly be reduced / where I should spend my time to optimize things. Don't get me wrong, I have a general understanding of what is taking longer and shorter amounts of time but putting a number on it would be great.

This is not a blocker to my work at all, just a nice to have.

[dchudz]

Hi Eva! Thanks for the praise, and the suggestion.

Until we have more of what you want in the Coiled dashboard, I suspect you'd find things in the dask scheduler dashboard (only available while the cluster is running) that are helpful.

I'm not an expert on what's there, but for example I do think you'll be able to find some profiling information.

[evamaxfield]

Hey! Yes I know about the Dask Dashboard and the profiling tool. Both very useful. However in the case I want to not have Bokeh installed (or can't maybe due to conflicting dep versions) then the Dask Dashboard isn't available and I would still love just a simple mean + std task duration.

Again not a priority at all, just think it would be a nice to have down the line