wip: sanitizers

[rmacnak-google]

Adds the ability to run tests on the standalone Dart VM under Address Sanitizer, Memory Sanitizer or Thread Sanitizer.

This is useful for finding issues when using foreign libraries through dart:ffi, such as use-after-free, use of initialized memory and data races.

This is also useful for pure Dart programs that might have data races through the use of shared fields.

Only available on 64-bit Linux.

https://clang.llvm.org/docs/AddressSanitizer.html
https://clang.llvm.org/docs/MemorySanitizer.html
https://clang.llvm.org/docs/ThreadSanitizer.html

[github-actions]

PR Health

Changelog Entry ❗

Package	Changed Files
package:test_api	pkgs/test_api/lib/src/backend/runtime.dart
package:test_core	pkgs/test_core/lib/src/runner/loader.dart pkgs/test_core/lib/src/runner/vm/platform.dart

Changes to files need to be accounted for in their respective changelogs.

This check can be disabled by tagging the PR with skip-changelog-check.

[natebosch]

Can you edit the first comment on the PR to add some details about this change and the motivation? We can use the first comment to decide the content of the final commit message when this lands.

[natebosch]

If the test is intentionally testing only that some code runs without exception we should make that explicit.

Suggested change

	test('uninitialized', () {
	var a = malloc(8);
	var b = malloc(8);
	memcmp(a, b, 8);
	free(b);
	free(a);
	});
	test('uninitialized', () {
	expect(() {
	var a = malloc(8);
	var b = malloc(8);
	memcmp(a, b, 8);
	free(b);
	free(a);
	}, returnsNormally);
	});

[rmacnak-google]

No, this code is testing MSAN flagging use of uninitialized memory.

[natebosch]

Similar to the above, we should match against returnsNormally to indicate blocks that we expect only to run without exceptions.

[natebosch]

is this unintentionally empty?

[natebosch]

We have had difficulty in the past using a direct exit call from the test runner, I don't think we should land another.

What are the scenarios where this would hang?

[rmacnak-google]

When the sanitizers exit the child, the parent was hanging. This is hacky, and probably the real issue is the consumer of the socket created between parent and child is failing to react to a disconnect, but I haven't found that code yet.

[natebosch]

Why do we need to change the start mode?

[rmacnak-google]

This is a better version of the removed stdout/stderr forwarding. In particular, when the parent has ttys instead of pipes, the child will have the same ttys and the sanitizer in the child will emit colored output. It also avoids issues with the parent exiting before all the output is forwarded.

[natebosch]

I am unsure about modeling this as a runtime when the only effect is has is to change compiler arguments. What do you think about modeling this as a different compiler?

[rmacnak-google]

Using the sanitizers involves changes to both the compiler and the runtime. It's more like running a different program than running the same program with a different strategy. In the SDK's test harness, we model the sanitizers as a separate dimension alongside architecture, compiler and runtime.