Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
85125f1
[P0] Verify the clang++ version used in the test #18
yuqisun May 22, 2025
8b3dfd9
Update main.yml
yuqisun May 23, 2025
09b8363
Merge branch 'coredac:main' into 18-p0-verify-the-clang-version-used-…
yuqisun May 23, 2025
98044c2
Test with apt install llvm instead of source code.
yuqisun May 23, 2025
7b872b6
Revert to install from source code, install from apt doesn't include …
yuqisun May 23, 2025
8a4e8b5
Upgrade to llvm-19.
yuqisun May 23, 2025
b4ab42a
Clone the latest llvm repo code only to speed up.
yuqisun May 23, 2025
e4e18e1
Specify branch for cloning LLVM repository in workflow
yuqisun May 23, 2025
9e43f7b
Add Dockerfile for automated build and test environment setup
yuqisun May 23, 2025
2bb49ff
Update LLVM checkout to specific commit cd70802 in Dockerfile and wor…
yuqisun May 24, 2025
5e0e577
Fix checkout order for LLVM setup in workflow
yuqisun May 24, 2025
24b88ce
Clone includes history commit tree.
yuqisun May 24, 2025
fd95c0e
Update LLVM checkout to full commit hash in Dockerfile and workflow
yuqisun May 24, 2025
61949a5
Shallow clone LLVM source code for faster checkout.
yuqisun May 24, 2025
db3a1af
Refine LLVM build instructions in README.md for clarity and stability
yuqisun May 24, 2025
accef66
Update Dockerfile and workflow to install additional dependencies and…
yuqisun May 26, 2025
6d3dd27
DCMAKE_EXPORT_COMPILE_COMMANDS=ON for VS Code users (to avoid warning…
yuqisun May 26, 2025
b934374
Merge remote-tracking branch 'origin/main' into 18-p0-verify-the-clan…
yuqisun May 26, 2025
389e3d1
Move docker path.
yuqisun May 26, 2025
6b56c40
Refactor LLVM build setup by removing clang installation and adjustin…
yuqisun May 28, 2025
2f06d19
Check clang version instead of installing it in the workflow
yuqisun May 28, 2025
6da365e
Update README and test documentation for clang version consistency
yuqisun May 28, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 11 additions & 2 deletions .github/workflows/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -34,13 +34,16 @@ jobs:
sudo apt-get install ccache
sudo apt-get install lld

# install clang for LLVM build
- name: check clang version
run: clang --version

# setup LLVM
- name: install a specific version of LLVM
working-directory: ${{github.workspace}}
run: |
git clone https://github.com/llvm/llvm-project.git
git clone --depth 1 --branch release/19.x https://github.com/llvm/llvm-project.git
cd llvm-project
git checkout cd70802
mkdir build && cd build
cmake -G Ninja ../llvm \
-DLLVM_ENABLE_PROJECTS="mlir" \
Expand All @@ -55,10 +58,16 @@ jobs:
-DLLVM_ENABLE_LLD=ON \
-DMLIR_INSTALL_AGGREGATE_OBJECTS=ON \
-DLLVM_ENABLE_RTTI=ON \
-DCMAKE_BUILD_WITH_INSTALL_RPATH=ON \
-DCMAKE_EXPORT_COMPILE_COMMANDS=ON \
-DCMAKE_C_COMPILER_LAUNCHER=ccache \
-DCMAKE_CXX_COMPILER_LAUNCHER=ccache

cmake --build . --target check-mlir
dpkg -l | grep llvm
sudo apt-get purge llvm-13
sudo apt-get purge llvm-14
sudo apt-get purge llvm-15

# setup mlir-cgra
- name: setup dataflow tool-chain
Expand Down
43 changes: 36 additions & 7 deletions README.md
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we also mention we prefer the clang is also in llvm-19 version?

I am wondering, when we build LLVM-19.x from source, it is compiled using the built-in clang, which may not be llvm-19, right? Then if user install clang in llvm-19, should we update this

// RUN: clang++ -S -emit-llvm -o %t-kernel.ll kernel.cpp
to distinguish the system clang and the user-installed clang?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about now: Compile both clang and mlir and add note at the beginning of test.mlir to recommend user to add llvm-19 clang to PATH.

-DLLVM_ENABLE_PROJECTS="clang;mlir" --- To compile clang in llvm-19 but take more time.

Original file line number Diff line number Diff line change
Expand Up @@ -2,18 +2,21 @@

Build LLVM & Neura
--------------------------------------------------------
- Clone llvm-project.

- Clone this repo.

- Clone llvm-project (we prefer llvm-19).
```sh
$ git clone --depth 1 --branch release/19.x https://github.com/llvm/llvm-project.git
```
- Build LLVM:
- Check out to commit `cd70802` (a stable version randomly picked, will sync to the latest version).
- Build:
> **Note:**
> `clang` is optional for building Neura itself, but installing and building `clang` together with `mlir` (by setting `-DLLVM_ENABLE_PROJECTS="clang;mlir"`) ensures that you have a `clang`/`clang++` matching the same LLVM/MLIR version (19).
> This is recommended if you want to use `clang++` for C++ to LLVM IR/MLIR workflows and to avoid version mismatch issues.

```sh
$ cd llvm-project
$ mkdir build && cd build
# May need install ccache and lld.
$ cmake -G Ninja ../llvm \
-DLLVM_ENABLE_PROJECTS="mlir" \
-DLLVM_ENABLE_PROJECTS="clang;mlir" \
-DLLVM_BUILD_EXAMPLES=OFF \
-DLLVM_TARGETS_TO_BUILD="Native" \
-DCMAKE_BUILD_TYPE=Release \
Expand All @@ -25,9 +28,14 @@ Build LLVM & Neura
-DLLVM_ENABLE_LLD=ON \
-DMLIR_INSTALL_AGGREGATE_OBJECTS=ON \
-DLLVM_ENABLE_RTTI=ON \
-DCMAKE_BUILD_WITH_INSTALL_RPATH=ON \
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is for?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added this because Ninja cannot auto fix RPATH:

CMake Error at cmake/modules/AddLLVM.cmake:1029 (add_executable):
The install of the llvm-tblgen target requires changing an RPATH from the
build tree, but this is not supported with the Ninja generator unless on an
ELF-based or XCOFF-based platform. The CMAKE_BUILD_WITH_INSTALL_RPATH
variable may be set to avoid this relinking step.
Call Stack (most recent call first):
cmake/modules/TableGen.cmake:181 (add_llvm_executable)
utils/TableGen/CMakeLists.txt:32 (add_tablegen)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's also have DCMAKE_EXPORT_COMPILE_COMMANDS=ON for VS Code users (to avoid warning in the IDE for include paths), I just set it up with copilot.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added.

-DCMAKE_EXPORT_COMPILE_COMMANDS=ON \
-DCMAKE_C_COMPILER_LAUNCHER=ccache \
-DCMAKE_CXX_COMPILER_LAUNCHER=ccache
$ cmake --build . --target check-mlir
$ ninja
$ ninja check-clang check-mlir
$ export PATH=/workspace/llvm-project/build/bin:$PATH
```

- Build Neura:
Expand All @@ -54,3 +62,24 @@ Build LLVM & Neura
$ /workspace/llvm-project/build/bin/llvm-lit *
```

Docker-based Environment Setup
--------------------------------------------------------
**Option 1: Pull Pre-built Image**

You can directly pull and use the pre-built Docker image:
```sh
$ docker pull cgra/neura:v1
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Image is pushing in progress:

The push refers to repository [docker.io/cgra/neura]
3b5d14681ffd: Pushing [==================================================>]  12.92GB
346f14bf17b9: Layer already exists

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Uploaded to cgra/neura:v1.

$ docker run --name myneura -it cgra/neura:v1
```

**Option 2: Build Image from Dockerfile**

Alternatively, you can build the Docker image yourself using the provided Dockerfile:
```sh
# In the root directory of this repo:
$ cd docker
$ docker build -t neura:v1 .
$ docker run --name myneura -it neura:v1
```

Both methods will provide a consistent environment with LLVM/MLIR(version 19) and Neura built and ready to use.
61 changes: 61 additions & 0 deletions docker/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
FROM ubuntu:22.04

SHELL ["/bin/bash", "-c"]

ENV DEBIAN_FRONTEND=noninteractive

RUN apt-get update \
&& apt install software-properties-common -y \
&& add-apt-repository ppa:deadsnakes/ppa -y \
&& apt update \
&& apt install python3.12 -y \
&& apt-get install ninja-build -y \
&& apt-get install ccache -y \
&& apt-get install lld -y \
&& apt-get install -y clang \
&& apt-get -y install build-essential cmake git libxml2-dev zlib1g-dev libncurses5-dev \
&& mkdir /cgra \
&& cd /cgra \
&& git clone https://github.com/yuqisun/dataflow.git \
&& cd dataflow \
&& until git clone --depth 1 --branch release/19.x https://github.com/llvm/llvm-project.git; do sleep 5; done \
&& cd llvm-project \
&& mkdir build && cd build \
&& cmake -G Ninja ../llvm \
-DLLVM_ENABLE_PROJECTS="clang;mlir" \
-DLLVM_BUILD_EXAMPLES=OFF \
-DLLVM_TARGETS_TO_BUILD="Native" \
-DCMAKE_BUILD_TYPE=Release \
-DLLVM_ENABLE_ASSERTIONS=ON \
-DCMAKE_C_COMPILER=clang \
-DCMAKE_CXX_COMPILER=clang++ \
-DCMAKE_CXX_STANDARD=17 \
-DCMAKE_CXX_FLAGS="-std=c++17 -frtti" \
-DLLVM_ENABLE_LLD=ON \
-DMLIR_INSTALL_AGGREGATE_OBJECTS=ON \
-DLLVM_ENABLE_RTTI=ON \
-DCMAKE_BUILD_WITH_INSTALL_RPATH=ON \
-DCMAKE_EXPORT_COMPILE_COMMANDS=ON \
-DCMAKE_C_COMPILER_LAUNCHER=ccache \
-DCMAKE_CXX_COMPILER_LAUNCHER=ccache \
# && cmake --build . --target check-mlir \
&& ninja \
&& ninja check-clang check-mlir \
&& dpkg -l | grep llvm \
&& cd /cgra/dataflow \
&& mkdir build && cd build \
&& cmake -G Ninja .. \
-DLLVM_DIR=/cgra/dataflow/llvm-project/build/lib/cmake/llvm \
-DMLIR_DIR=/cgra/dataflow/llvm-project/build/lib/cmake/mlir \
-DMLIR_SOURCE_DIR=/cgra/dataflow/llvm-project/mlir \
-DMLIR_BINARY_DIR=/cgra/dataflow/llvm-project/build \
-DCMAKE_CXX_FLAGS="-std=c++17" \
&& ninja \
&& cd /cgra/dataflow/test \
&& /cgra/dataflow/llvm-project/build/bin/llvm-lit * -v

ENV PATH=/cgra/dataflow/llvm-project/build/bin:$PATH

CMD ["tail", "-f", "/dev/null"]


6 changes: 6 additions & 0 deletions test/c2llvm2mlir/test.mlir
Original file line number Diff line number Diff line change
@@ -1,3 +1,9 @@
// Note:
// We recommend using the clang++ built from the same LLVM/MLIR 19.x source.
// Please add your build directory to PATH before running tests:
// `export PATH=/path/to/llvm-project/build/bin:$PATH`
// This ensures all clang++ invocations use the same version.

// Compiles the original kernel.
// RUN: clang++ kernel.cpp -o %t-kernel.out

Expand Down