Skip to content

CUDA error while running "model.fit" #66

@NekoClaviS

Description

@NekoClaviS

I am facing some cupy cuda error while running the following code:

CUPY_DUMP_CUDA_SOURCE_ON_ERROR =1
CUPY_DISABLE_JITIFY_CACHE = 1
model.fit(min_iter=30, max_iter=35)
model.plot_convergence()

The above code runs for 1 iteration and then gives the following error:

6 errors detected in the compilation of "cupy_jitify_exercise".

RuntimeError                              Traceback (most recent call last)
Cell In[80], line 3
      1 CUPY_DUMP_CUDA_SOURCE_ON_ERROR =1
      2 CUPY_DISABLE_JITIFY_CACHE = 1
----> 3 model.fit(min_iter=30, max_iter=35)
      4 model.plot_convergence()

File ~/anaconda3/envs/scenvi/lib/python3.9/site-packages/SEACells/core.py:587, in SEACells.fit(self, max_iter, min_iter, initial_archetypes)
    585 if max_iter < min_iter:
    586     raise ValueError("The maximum number of iterations specified is lower than the minimum number of iterations specified.")
--> 587 self._fit(max_iter=max_iter, min_iter=min_iter, initial_archetypes=initial_archetypes, initial_assignments=None)

File ~/anaconda3/envs/scenvi/lib/python3.9/site-packages/SEACells/core.py:544, in SEACells._fit(self, max_iter, min_iter, initial_archetypes, initial_assignments)
    532 def _fit(self, max_iter: int = 50, min_iter:int=10, initial_archetypes=None, initial_assignments=None):
    533     """
    534     Compute archetypes and loadings given kernel matrix K. Iteratively updates A and B matrices until maximum
    535     number of iterations or convergence has been achieved.
   (...)
    542 
    543     """
--> 544     self.initialize(initial_archetypes=initial_archetypes, initial_assignments=initial_assignments)
    546     converged = False
    547     n_iter = 0
...
File cupy/cuda/jitify.pyx:192, in cupy.cuda.jitify._init_cupy_headers_from_scratch()

File cupy/cuda/jitify.pyx:264, in cupy.cuda.jitify.jitify()

RuntimeError: Runtime compilation failed

I have tried multiple things and have even tried rebuilding my anaconda venv, but unable to fix this, it would be really helpful if someone could help with the same.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions