Cache TASTy files content

[mbovel]

No description provided.

[bishabosha]

is an AbstractFile a good key for a weakhashmap? i.e. the same exact AbstractFile will be used over and over? I would think perhaps between runs the abstract file is GC'd possibly

[bishabosha]

I guess it depends on how dotty's zipclasspath does its own caching?

[mbovel]

I guess it depends on how dotty's zipclasspath does its own caching?

Yes exactly. This relies on classpaths being cached here:

scala3/compiler/src/dotty/tools/dotc/classpath/ZipAndJarFileLookupFactory.scala

Lines 19 to 36 in 46fcf2f

	/**
	* A trait providing an optional cache for classpath entries obtained from zip and jar files.
	* It allows us to e.g. reduce significantly memory used by PresentationCompilers in Scala IDE
	* when there are a lot of projects having a lot of common dependencies.
	*/
	sealed trait ZipAndJarFileLookupFactory {
	private val cache = new FileBasedCache[ClassPath]
	def create(zipFile: AbstractFile)(using Context): ClassPath =
	val release = Option(ctx.settings.javaOutputVersion.value).filter(_.nonEmpty)
	if (ctx.settings.YdisableFlatCpCaching.value || zipFile.file == null) createForZipFile(zipFile, release)
	else createUsingCache(zipFile, release)
	protected def createForZipFile(zipFile: AbstractFile, release: Option[String]): ClassPath
	private def createUsingCache(zipFile: AbstractFile, release: Option[String]): ClassPath =
	cache.getOrCreate(zipFile.file.toPath, () => createForZipFile(zipFile, release))
	}

Only in that case do we get stable AbstractFiles instances across runs.

I tried that for two reasons:

• Absolute path and modified time are not enough as keys; we can get files with different content but same absolute path and modified time. It happens when loading classes from the JRT at least (that's why [Experiment] Global file content cache #24630 fails).
• It's a good way to avoid memory leaks: we only retains content for AbstractFiles that are reachable. Otherwise the WeakMap will let the GC do its job, which is what we want! That's similar to Cache AbstractFile.toByteArray #24644, but with an external map instead of using a field. So we avoid caching too much with this solution, but the drawback of course is that we risk caching too little.