Auto merge of #147372 - jieyouxu:rust-analyzer-main-tests, r=Kobzol

Run main rust-analyzer tests in rust-lang/rust CI

Part of rust-lang/rust#147370.
MCP: rust-lang/compiler-team#923

This PR prepares `rust-analyzer` crates with `in-rust-tree` cargo featues where needed, and and updates bootstrap to run the main `rust-analyzer` tests in rust-lang/rust CI, not just the `proc-macro-srv` crate tests.

This supersedes the earlier attempt at rust-lang/rust#136779. I was honestly expecting more failures in this PR, but looking back at the previous attempt, that makes sense because we no longer run `i686-mingw` (32-bit windows-gnu) which had a _bunch_ of these failures. In the earlier attempt I also disabled the `i686-mingw`-related failures for `i686-msvc` since I didn't feel like digging into 32-bit msvc at the time. Try results from this PR shows that it's most likely limited to 32-bit windows-gnu specifically.

### `rust-analyzer` test remarks

- I actually had to _remove_ the `CARGO_WORKSPACE_DIR` `expect-test`-hack in order for `expect-test` to be able to find the test expectation HTML files (for `syntax_highlighting` tests in `ide`). When I added the hack, ironically, it made `expect-test` unable to find the expectation files. I think this was because previously the path was of the `proc-macro-srv` crate specifically, now we point to the root r-a workspace?
- The `cfg`-related differences on `aarch64-apple-darwin` might've been fixed? I can't tell, but we don't seem to be observing the differences now.
- I'm not sure why `config::{generate_config_documentation, generate_package_json_config}` no longer fails. Perhaps they were fixed to no longer try to write to source directory?

### Review remarks

- Commit 1 updates r-a crates that are involved in tests needing artifacts from `rustc_private` compiler crates to use the `in-rust-tree` cargo feature. I briefly tried to use a plain `--cfg=in_rust_tree`, but quickly realized it was very hacky, and needed invasive bootstrap changes. The cargo feature approach seems most "natural"/well-supported to both bootstrap and cargo.
- Commit 2 updates bootstrap to not only run the `proc-macro-srv` tests, but the whole r-a tests.
- Commit 3 restricts r-a main tests to non-32-bit targets we test in CI, since (1) r-a repo does not run tests against 32-bit platforms, and (2) there are some target pointer width sensitive hash differences causing tests to fail. Notably, this means that we also no longer run r-a `proc-macro-srv` tests against 32-bit targets, but we don't expect that crate to be have target pointer width differences. Discussed this in [#t-compiler/rust-analyzer > 32-bit tests?](https://rust-lang.zulipchat.com/#narrow/channel/185405-t-compiler.2Frust-analyzer/topic/32-bit.20tests.3F/with/563145736).

---

// try-job: aarch64-gnu
// try-job: aarch64-apple
// try-job: x86_64-mingw-1
// try-job: i686-msvc-1
// try-job: x86_64-msvc-1
// try-job: aarch64-msvc-1

Author: bors

crates/base-db/Cargo.toml

@@ -31,5 +31,9 @@ vfs.workspace = true
 span.workspace = true
 intern.workspace = true
 
+[features]
+default = []
+in-rust-tree = []
+
 [lints]
 workspace = true

crates/base-db/src/lib.rs

@@ -1,5 +1,10 @@
 //! base_db defines basic database traits. The concrete DB is defined by ide.
 
+#![cfg_attr(feature = "in-rust-tree", feature(rustc_private))]
+
+#[cfg(feature = "in-rust-tree")]
+extern crate rustc_driver as _;
+
 pub use salsa;
 pub use salsa_macros;
 

crates/cfg/Cargo.toml

@@ -33,5 +33,9 @@ syntax.workspace = true
 # tt is needed for testing
 cfg = { path = ".", default-features = false, features = ["tt"] }
 
+[features]
+default = []
+in-rust-tree = []
+
 [lints]
 workspace = true

crates/cfg/src/lib.rs

@@ -1,5 +1,10 @@
 //! cfg defines conditional compiling options, `cfg` attribute parser and evaluator
 
+#![cfg_attr(feature = "in-rust-tree", feature(rustc_private))]
+
+#[cfg(feature = "in-rust-tree")]
+extern crate rustc_driver as _;
+
 mod cfg_expr;
 mod dnf;
 #[cfg(test)]

crates/ide-completion/Cargo.toml

@@ -37,5 +37,9 @@ expect-test = "1.5.1"
 test-utils.workspace = true
 test-fixture.workspace = true
 
+[features]
+default = []
+in-rust-tree = []
+
 [lints]
 workspace = true

crates/ide-completion/src/lib.rs

@@ -3,6 +3,11 @@
 // It's useful to refer to code that is private in doc comments.
 #![allow(rustdoc::private_intra_doc_links)]
 
+#![cfg_attr(feature = "in-rust-tree", feature(rustc_private))]
+
+#[cfg(feature = "in-rust-tree")]
+extern crate rustc_driver as _;
+
 mod completions;
 mod config;
 mod context;

crates/ide-db/Cargo.toml

@@ -52,5 +52,9 @@ line-index.workspace = true
 [dev-dependencies]
 expect-test = "1.5.1"
 
+[features]
+default = []
+in-rust-tree = []
+
 [lints]
 workspace = true

crates/ide-db/src/lib.rs

@@ -2,6 +2,11 @@
 //!
 //! It is mainly a `HirDatabase` for semantic analysis, plus a `SymbolsDatabase`, for fuzzy search.
 
+#![cfg_attr(feature = "in-rust-tree", feature(rustc_private))]
+
+#[cfg(feature = "in-rust-tree")]
+extern crate rustc_driver as _;
+
 extern crate self as ide_db;
 
 mod apply_change;

crates/ide-diagnostics/Cargo.toml

@@ -34,5 +34,9 @@ expect-test = "1.5.1"
 test-utils.workspace = true
 test-fixture.workspace = true
 
+[features]
+default = []
+in-rust-tree = []
+
 [lints]
 workspace = true

crates/ide-diagnostics/src/lib.rs

@@ -23,6 +23,11 @@
 //! There are also a couple of ad-hoc diagnostics implemented directly here, we
 //! don't yet have a great pattern for how to do them properly.
 
+#![cfg_attr(feature = "in-rust-tree", feature(rustc_private))]
+
+#[cfg(feature = "in-rust-tree")]
+extern crate rustc_driver as _;
+
 mod handlers {
     pub(crate) mod await_outside_of_async;
     pub(crate) mod bad_rtn;