Error fitting model on Jupyter lab via HPC cluster

[li-xuyang28]

Hi developer/maintainer of SEACells,

Thanks for your effort in bringing out this tool! I am trying to compute SEACells with a decently large scRNAseq data (~230k cells after down-sampling) on an HPC cluster. The only way that I can get it to run is with the cpu sparse matrix option (can get up to 400GB mem, when computing the A matrix with use_sparse = F it complained that cannot allocate 396GiB memory, but won't be able to go any bigger than 400GB).

However, with sparse representation, it kept getting stuck at the first iteration, then the kernel would die (in Jupyter err log: AsyncIOLoopKernelRestarter: restarting kernel (1/5), keep random ports). I was wondering if you would have any suggestions on a work-around?

Thanks.