From ff958fc4b2c606d70de8dbfe083bee98896c0072 Mon Sep 17 00:00:00 2001 From: Viktor Szakats Date: Fri, 2 Jan 2026 11:03:12 +0100 Subject: [PATCH 1/3] GHA/linux: switch scan-build jobs to cmake (for 2x perf) Somewhat unexpectedly, switching autotools jobs to identical (non-unity, non-debug, same options) cmake ones, makes them complete 2x faster. Most of it comes from cmake building shared libcurl only, while autotools was using defaults and building both, in two separate passes. Thers is about a minute (per job) of gain due to other reasons. Before: MultiSSL: 10m30: https://github.com/curl/curl/actions/runs/20656775456/job/59311070197 H3: 9m14s: https://github.com/curl/curl/actions/runs/20656775456/job/59311070204 After: MultiSSL: 4m52s: https://github.com/curl/curl/actions/runs/20658343323/job/59315501903 H3: 4m7s: https://github.com/curl/curl/actions/runs/20658343323/job/59315501918 H3: 5m4s: https://github.com/curl/curl/actions/runs/20659294959/job/59318215987 (autotools shared only, for comparison, not merged) Also: - drop building examples with scan-build in the second (shorter) job. This offers no extra coverage over the long job that has both GnuTLS and OpenSSL. Saving an extra ~30s. Closes #20159 --- .github/workflows/linux.yml | 29 +++++++++++++++-------------- 1 file changed, 15 insertions(+), 14 deletions(-) diff --git a/.github/workflows/linux.yml b/.github/workflows/linux.yml index 2fd1817f7b52..cc78f2bcc420 100644 --- a/.github/workflows/linux.yml +++ b/.github/workflows/linux.yml @@ -251,15 +251,15 @@ jobs: CC: clang configure-prefix: scan-build make-prefix: scan-build --status-bugs - PKG_CONFIG_PATH: /home/linuxbrew/.linuxbrew/opt/gsasl/lib/pkgconfig - LDFLAGS: -Wl,-rpath,/home/runner/wolfssl-opensslextra/lib -Wl,-rpath,/home/linuxbrew/.linuxbrew/opt/gsasl/lib - configure: >- - --with-wolfssl=/home/runner/wolfssl-opensslextra --with-openssl --with-rustls --with-mbedtls=/home/runner/mbedtls --with-gnutls --with-libgsasl - --with-librtmp --with-libssh2 --with-libidn2 - --enable-ech --with-gssapi --enable-ssls-export - --disable-debug --disable-unity --disable-typecheck + LDFLAGS: -Wl,-rpath,/home/runner/wolfssl-opensslextra/lib -Wl,-rpath,/home/runner/mbedtls/lib -Wl,-rpath,/home/runner/rustls/lib -Wl,-rpath,/home/linuxbrew/.linuxbrew/opt/gsasl/lib + PKG_CONFIG_PATH: /home/runner/wolfssl-opensslextra/lib/pkgconfig:/home/runner/mbedtls/lib/pkgconfig:/home/runner/rustls/lib/pkgconfig:/home/linuxbrew/.linuxbrew/opt/gsasl/lib/pkgconfig + generate: >- + -DCURL_USE_OPENSSL=ON -DCURL_USE_WOLFSSL=ON -DCURL_USE_GNUTLS=ON -DCURL_USE_MBEDTLS=ON -DCURL_USE_RUSTLS=ON + -DCURL_USE_GSASL=ON -DUSE_LIBRTMP=ON + -DUSE_ECH=ON -DCURL_USE_GSSAPI=ON -DUSE_SSLS_EXPORT=ON + -DCMAKE_UNITY_BUILD=OFF -DCURL_DISABLE_TYPECHECK=ON - - name: 'scan-build H3 c-ares' + - name: 'scan-build H3 c-ares !examples' install_packages: clang-tools clang libidn2-dev libssh-dev libnghttp2-dev install_steps: skipall install_steps_brew: openssl libngtcp2 libnghttp3 c-ares @@ -268,11 +268,11 @@ jobs: make-prefix: scan-build --status-bugs LDFLAGS: -Wl,-rpath,/home/linuxbrew/.linuxbrew/opt/openssl/lib -Wl,-rpath,/home/linuxbrew/.linuxbrew/opt/libngtcp2/lib -Wl,-rpath,/home/linuxbrew/.linuxbrew/opt/libnghttp3/lib -Wl,-rpath,/home/linuxbrew/.linuxbrew/opt/c-ares/lib PKG_CONFIG_PATH: /home/linuxbrew/.linuxbrew/opt/libngtcp2/lib/pkgconfig:/home/linuxbrew/.linuxbrew/opt/libnghttp3/lib/pkgconfig:/home/linuxbrew/.linuxbrew/opt/c-ares/lib/pkgconfig - configure: >- - --with-openssl=/home/linuxbrew/.linuxbrew/opt/openssl --with-ngtcp2 --with-nghttp3= - --with-libidn2 --with-libssh --enable-httpsrr --enable-ares - --disable-debug --disable-unity --disable-typecheck - --disable-verbose + generate: >- + -DCURL_USE_OPENSSL=ON -DOPENSSL_ROOT_DIR=/home/linuxbrew/.linuxbrew/opt/openssl -DUSE_NGTCP2=ON + -DCURL_USE_LIBSSH2=OFF -DCURL_USE_LIBSSH=ON -DUSE_HTTPSRR=ON -DENABLE_ARES=ON + -DCMAKE_UNITY_BUILD=OFF -DCURL_DISABLE_TYPECHECK=ON + -DCURL_DISABLE_VERBOSE_STRINGS=ON - name: 'address-sanitizer' install_packages: clang libssl-dev libssh-dev libidn2-dev libnghttp2-dev libubsan1 libasan8 libtsan2 @@ -689,6 +689,7 @@ jobs: [[ "${MATRIX_INSTALL_STEPS}" = *'awslc'* ]] && sudo apt-get -o Dpkg::Use-Pty=0 purge libssl-dev [ -n "${MATRIX_PKG_CONFIG_PATH}" ] && export PKG_CONFIG_PATH="${MATRIX_PKG_CONFIG_PATH}" if [ "${MATRIX_BUILD}" = 'cmake' ]; then + ${MATRIX_CONFIGURE_PREFIX} \ cmake -B bld -G Ninja \ -DCMAKE_INSTALL_PREFIX="$HOME"/curl-install \ -DCMAKE_C_COMPILER_TARGET="$(uname -m)-pc-linux-gnu" -DBUILD_STATIC_LIBS=ON \ @@ -818,7 +819,7 @@ jobs: ../.github/scripts/randcurl.pl 60 ../bld/src/curl - name: 'build examples' - if: ${{ !contains(matrix.build.install_packages, 'valgrind') && matrix.build.make-custom-target != 'tidy' }} + if: ${{ !contains(matrix.build.install_packages, 'valgrind') && matrix.build.make-custom-target != 'tidy' && !contains(matrix.build.name, '!examples') }} run: | if [ "${MATRIX_BUILD}" = 'cmake' ]; then ${MATRIX_MAKE_PREFIX} cmake --build bld --verbose --target curl-examples-build From 2cbcde90e7a5eb7144079dcacc21393322b1daf7 Mon Sep 17 00:00:00 2001 From: Viktor Szakats Date: Fri, 2 Jan 2026 03:13:32 +0100 Subject: [PATCH 2/3] GHA/linux: split valgrind jobs to job-pairs for parallelism, to finish in 10m To make CI turnaround time shorter, by cutting the longest running jobs. After this patch all jobs should finish around 10-11 minutes. Down from 15-16 minutes before this patch. Suggested-by: Stefan Eissing The fuzzing workflow is now the slowest (with a 7-minute startup time needed to build deps from source on each run), followed by macOS and Windows torture tests (both split in two now). Without fuzzing, it's under 10 minutes. Notes: - an extra cost with job-pairs is installing prereqs, configuring/building curl and tests twice. GitHub doesn't support making a matrix job a prereq for another workflow that may fix this: https://github.com/orgs/community/discussions/42335 This overhead is significant on Windows: 11m20 -> 9m20 + 8m40 - job-pairs are annoying to maintain and keep in sync. - splitting tests into halves is a manual process and needs to be revisited from time to time. Possibly something to automate with a runtests option, e.g. with `1 of 50%` and `2 of 50%`? Also: - split torture tests in two equal pieces, replacing the `FTP` + `!FTP` split used earlier. Related perf improvements from today: - make scan-build 2x fast: ff958fc4b2c606d70de8dbfe083bee98896c0072 #20159 - drop build-only Testi386 fuzz workflow (from curl), saving 9m per run: https://github.com/curl/curl-fuzzer/commit/0106023c1f2f0b9ff3aa86e19402b4a03e4f925c https://github.com/curl/curl-fuzzer/pull/236 Closes #20153 --- .github/workflows/linux.yml | 71 +++++++++++++++++++++++++++++------ .github/workflows/macos.yml | 8 ++-- .github/workflows/windows.yml | 3 +- 3 files changed, 65 insertions(+), 17 deletions(-) diff --git a/.github/workflows/linux.yml b/.github/workflows/linux.yml index cc78f2bcc420..44f078ce2409 100644 --- a/.github/workflows/linux.yml +++ b/.github/workflows/linux.yml @@ -81,9 +81,16 @@ jobs: install_steps: libressl pytest codeset-test configure: LDFLAGS=-Wl,-rpath,/home/runner/libressl/lib --with-openssl=/home/runner/libressl --with-gssapi --enable-debug - - name: 'libressl krb5 valgrind' + - name: 'libressl krb5 valgrind 1' install_packages: libnghttp2-dev libldap-dev libkrb5-dev valgrind install_steps: libressl + tflags: '--min=870 1 to 950' + generate: -DOPENSSL_ROOT_DIR=/home/runner/libressl -DCURL_USE_GSSAPI=ON -DENABLE_DEBUG=ON -DCURL_LIBCURL_VERSIONED_SYMBOLS=ON + + - name: 'libressl krb5 valgrind 2' + install_packages: libnghttp2-dev libldap-dev libkrb5-dev valgrind + install_steps: libressl + tflags: '--min=900 951 to 9999' generate: -DOPENSSL_ROOT_DIR=/home/runner/libressl -DCURL_USE_GSSAPI=ON -DENABLE_DEBUG=ON -DCURL_LIBCURL_VERSIONED_SYMBOLS=ON - name: 'libressl clang' @@ -95,14 +102,34 @@ jobs: install_steps: wolfssl-all configure: LDFLAGS=-Wl,-rpath,/home/runner/wolfssl-all/lib --with-wolfssl=/home/runner/wolfssl-all --enable-ech --enable-debug - - name: 'wolfssl-opensslextra valgrind' + - name: 'wolfssl-opensslextra valgrind 1' install_packages: valgrind install_steps: wolfssl-opensslextra + tflags: '--min=780 1 to 950' configure: LDFLAGS=-Wl,-rpath,/home/runner/wolfssl-opensslextra/lib --with-wolfssl=/home/runner/wolfssl-opensslextra --enable-ech --enable-debug - - name: 'mbedtls gss valgrind' + - name: 'wolfssl-opensslextra valgrind 2' + install_packages: valgrind + install_steps: wolfssl-opensslextra + tflags: '--min=800 951 to 9999' + configure: LDFLAGS=-Wl,-rpath,/home/runner/wolfssl-opensslextra/lib --with-wolfssl=/home/runner/wolfssl-opensslextra --enable-ech --enable-debug + + - name: 'mbedtls gss valgrind 1' install_packages: libnghttp2-dev libidn2-dev libldap-dev libgss-dev valgrind install_steps: mbedtls + tflags: '--min=830 1 to 950' + generate: >- + -DCURL_USE_MBEDTLS=ON -DENABLE_DEBUG=ON + -DMBEDTLS_INCLUDE_DIR=/home/runner/mbedtls/include + -DMBEDTLS_LIBRARY=/home/runner/mbedtls/lib/libmbedtls.a + -DMBEDX509_LIBRARY=/home/runner/mbedtls/lib/libmbedx509.a + -DMBEDCRYPTO_LIBRARY=/home/runner/mbedtls/lib/libmbedcrypto.a + -DCURL_USE_GSSAPI=ON + + - name: 'mbedtls gss valgrind 2' + install_packages: libnghttp2-dev libidn2-dev libldap-dev libgss-dev valgrind + install_steps: mbedtls + tflags: '--min=800 951 to 9999' generate: >- -DCURL_USE_MBEDTLS=ON -DENABLE_DEBUG=ON -DMBEDTLS_INCLUDE_DIR=/home/runner/mbedtls/include @@ -152,8 +179,14 @@ jobs: install_steps: pytest configure: --with-openssl --enable-debug --disable-unity - - name: 'openssl libssh2 sync-resolver valgrind' + - name: 'openssl libssh2 sync-resolver valgrind 1' + install_packages: libidn2-dev libssh2-1-dev libnghttp2-dev libldap-dev valgrind + tflags: '--min=920 1 to 950' + generate: -DENABLE_DEBUG=ON -DENABLE_THREADED_RESOLVER=OFF + + - name: 'openssl libssh2 sync-resolver valgrind 2' install_packages: libidn2-dev libssh2-1-dev libnghttp2-dev libldap-dev valgrind + tflags: '--min=910 951 to 9999' generate: -DENABLE_DEBUG=ON -DENABLE_THREADED_RESOLVER=OFF - name: 'openssl' @@ -166,9 +199,16 @@ jobs: configure: CFLAGS=-std=gnu89 --with-openssl --enable-debug --disable-verbose image: 'ubuntu-24.04-arm' - - name: 'openssl -O3 libssh valgrind' + - name: 'openssl -O3 libssh valgrind 1' + install_packages: libssh-dev valgrind + CFLAGS: -O3 + tflags: '--min=920 1 to 950' + generate: -DENABLE_DEBUG=ON -DCURL_USE_LIBSSH=ON -DCMAKE_UNITY_BUILD_BATCH_SIZE=50 + + - name: 'openssl -O3 libssh valgrind 2' install_packages: libssh-dev valgrind CFLAGS: -O3 + tflags: '--min=890 951 to 9999' generate: -DENABLE_DEBUG=ON -DCURL_USE_LIBSSH=ON -DCMAKE_UNITY_BUILD_BATCH_SIZE=50 - name: 'openssl clang krb5 openldap static' @@ -195,17 +235,17 @@ jobs: --without-libssh2 --without-libssh --disable-tftp --disable-ftp --disable-file --disable-smb - - name: 'openssl torture !FTP' + - name: 'openssl torture 1' install_packages: libnghttp2-dev libssh2-1-dev libc-ares-dev - generate: -DCURL_USE_OPENSSL=ON -DENABLE_DEBUG=ON -DENABLE_ARES=ON - tflags: '-t --shallow=25 !FTP --min=1450' + tflags: '-t --shallow=25 --min=920 1 to 950' torture: true + generate: -DCURL_USE_OPENSSL=ON -DENABLE_DEBUG=ON -DENABLE_ARES=ON - - name: 'openssl torture FTP' + - name: 'openssl torture 2' install_packages: libnghttp2-dev libssh2-1-dev libc-ares-dev - generate: -DCURL_USE_OPENSSL=ON -DENABLE_DEBUG=ON -DENABLE_ARES=ON - tflags: '-t --shallow=20 FTP --min=260' + tflags: '-t --shallow=20 --min=900 951 to 9999' torture: true + generate: -DCURL_USE_OPENSSL=ON -DENABLE_DEBUG=ON -DENABLE_ARES=ON - name: 'openssl i686' install_packages: gcc-14-i686-linux-gnu libssl-dev:i386 librtmp-dev:i386 libssh2-1-dev:i386 libidn2-dev:i386 libc-ares-dev:i386 zlib1g-dev:i386 @@ -319,9 +359,16 @@ jobs: configure: --enable-debug --disable-shared --disable-threaded-resolver --with-libssh --with-openssl tflags: '-n --test-duphandle' - - name: 'rustls valgrind' + - name: 'rustls valgrind 1' + install_packages: libnghttp2-dev libldap-dev valgrind + install_steps: rust rustls + tflags: '--min=780 1 to 950' + generate: -DCURL_USE_RUSTLS=ON -DUSE_ECH=ON -DENABLE_DEBUG=ON + + - name: 'rustls valgrind 2' install_packages: libnghttp2-dev libldap-dev valgrind install_steps: rust rustls + tflags: '--min=820 951 to 9999' generate: -DCURL_USE_RUSTLS=ON -DUSE_ECH=ON -DENABLE_DEBUG=ON - name: 'rustls' diff --git a/.github/workflows/macos.yml b/.github/workflows/macos.yml index 77bd81ea855f..be61c4d312f7 100644 --- a/.github/workflows/macos.yml +++ b/.github/workflows/macos.yml @@ -336,18 +336,18 @@ jobs: install: rustls-ffi generate: -DENABLE_DEBUG=ON -DCURL_USE_RUSTLS=ON -DUSE_ECH=ON -DCURL_DISABLE_LDAP=ON tflags: '--min=1650' - - name: 'OpenSSL torture !FTP' + - name: 'OpenSSL torture 1' compiler: clang install: libnghttp3 install_steps: torture generate: -DENABLE_DEBUG=ON -DBUILD_SHARED_LIBS=OFF -DENABLE_THREADED_RESOLVER=OFF -DOPENSSL_ROOT_DIR=/opt/homebrew/opt/openssl -DUSE_OPENSSL_QUIC=ON - tflags: '-t --shallow=25 !FTP --min=1450' - - name: 'OpenSSL torture FTP' + tflags: '-t --shallow=25 --min=920 1 to 950' + - name: 'OpenSSL torture 2' compiler: clang install: libnghttp3 install_steps: torture generate: -DENABLE_DEBUG=ON -DBUILD_SHARED_LIBS=OFF -DENABLE_THREADED_RESOLVER=OFF -DOPENSSL_ROOT_DIR=/opt/homebrew/opt/openssl -DUSE_OPENSSL_QUIC=ON - tflags: '-t --shallow=20 FTP --min=260' + tflags: '-t --shallow=20 --min=910 951 to 9999' exclude: # opt out jobs from combinations that have the compiler set manually - { compiler: llvm@18, build: { compiler: 'clang' } } diff --git a/.github/workflows/windows.yml b/.github/workflows/windows.yml index 700913e092f8..d7c2bcd8cea0 100644 --- a/.github/workflows/windows.yml +++ b/.github/workflows/windows.yml @@ -207,7 +207,8 @@ jobs: - { build: 'autotools', sys: 'mingw64' , env: 'x86_64' , tflags: '' , config: '--enable-debug --with-openssl --enable-windows-unicode --enable-ares --with-openssl-quic --enable-shared=no', install: 'mingw-w64-x86_64-c-ares mingw-w64-x86_64-openssl mingw-w64-x86_64-nghttp3 mingw-w64-x86_64-libssh2', name: 'c-ares U' } - { build: 'cmake' , sys: 'mingw64' , env: 'x86_64' , tflags: '--min=1650', config: '-DENABLE_DEBUG=ON -DBUILD_SHARED_LIBS=OFF -DCURL_USE_SCHANNEL=ON -DENABLE_UNICODE=ON -DENABLE_ARES=ON', install: 'mingw-w64-x86_64-c-ares mingw-w64-x86_64-libssh2', type: 'Debug', name: 'schannel c-ares U' } # MinGW torture - - { build: 'cmake' , sys: 'mingw64' , env: 'x86_64' , tflags: '-t --shallow=13 !FTP --min=1300', config: '-DENABLE_DEBUG=ON -DBUILD_SHARED_LIBS=OFF -DCURL_USE_SCHANNEL=ON -DENABLE_UNICODE=ON -DENABLE_ARES=ON', install: 'mingw-w64-x86_64-c-ares mingw-w64-x86_64-libssh2', type: 'Debug', name: 'schannel U torture' } + - { build: 'cmake' , sys: 'mingw64' , env: 'x86_64' , tflags: '-t --shallow=13 --min=700 1 to 950' , config: '-DENABLE_DEBUG=ON -DBUILD_SHARED_LIBS=OFF -DCURL_USE_SCHANNEL=ON -DENABLE_UNICODE=ON -DENABLE_ARES=ON', install: 'mingw-w64-x86_64-c-ares mingw-w64-x86_64-libssh2', type: 'Debug', name: 'schannel U torture 1' } + - { build: 'cmake' , sys: 'mingw64' , env: 'x86_64' , tflags: '-t --shallow=13 --min=700 951 to 9999', config: '-DENABLE_DEBUG=ON -DBUILD_SHARED_LIBS=OFF -DCURL_USE_SCHANNEL=ON -DENABLE_UNICODE=ON -DENABLE_ARES=ON', install: 'mingw-w64-x86_64-c-ares mingw-w64-x86_64-libssh2', type: 'Debug', name: 'schannel U torture 2' } # WARNING: libssh uses hard-coded world-writable paths (/etc/..., ~/.ssh/) to # read its configuration from, making it vulnerable to attacks on # Windows. Do not use this component till there is a fix for these. From 7e08d56c23bab0ae80ff9378d97fa854da80334b Mon Sep 17 00:00:00 2001 From: Viktor Szakats Date: Fri, 2 Jan 2026 16:23:17 +0100 Subject: [PATCH 3/3] GHA/linux: simplify configuring mbedTLS in two jobs Closes #20165 --- .github/workflows/linux.yml | 22 +++++++--------------- 1 file changed, 7 insertions(+), 15 deletions(-) diff --git a/.github/workflows/linux.yml b/.github/workflows/linux.yml index 44f078ce2409..21ae0b2c0b41 100644 --- a/.github/workflows/linux.yml +++ b/.github/workflows/linux.yml @@ -118,25 +118,17 @@ jobs: install_packages: libnghttp2-dev libidn2-dev libldap-dev libgss-dev valgrind install_steps: mbedtls tflags: '--min=830 1 to 950' - generate: >- - -DCURL_USE_MBEDTLS=ON -DENABLE_DEBUG=ON - -DMBEDTLS_INCLUDE_DIR=/home/runner/mbedtls/include - -DMBEDTLS_LIBRARY=/home/runner/mbedtls/lib/libmbedtls.a - -DMBEDX509_LIBRARY=/home/runner/mbedtls/lib/libmbedx509.a - -DMBEDCRYPTO_LIBRARY=/home/runner/mbedtls/lib/libmbedcrypto.a - -DCURL_USE_GSSAPI=ON + LDFLAGS: -Wl,-rpath,/home/runner/mbedtls/lib + PKG_CONFIG_PATH: /home/runner/mbedtls/lib/pkgconfig + generate: -DCURL_USE_MBEDTLS=ON -DENABLE_DEBUG=ON -DCURL_USE_GSSAPI=ON - name: 'mbedtls gss valgrind 2' install_packages: libnghttp2-dev libidn2-dev libldap-dev libgss-dev valgrind install_steps: mbedtls tflags: '--min=800 951 to 9999' - generate: >- - -DCURL_USE_MBEDTLS=ON -DENABLE_DEBUG=ON - -DMBEDTLS_INCLUDE_DIR=/home/runner/mbedtls/include - -DMBEDTLS_LIBRARY=/home/runner/mbedtls/lib/libmbedtls.a - -DMBEDX509_LIBRARY=/home/runner/mbedtls/lib/libmbedx509.a - -DMBEDCRYPTO_LIBRARY=/home/runner/mbedtls/lib/libmbedcrypto.a - -DCURL_USE_GSSAPI=ON + LDFLAGS: -Wl,-rpath,/home/runner/mbedtls/lib + PKG_CONFIG_PATH: /home/runner/mbedtls/lib/pkgconfig + generate: -DCURL_USE_MBEDTLS=ON -DENABLE_DEBUG=ON -DCURL_USE_GSSAPI=ON - name: 'mbedtls clang' install_packages: libssh-dev libnghttp2-dev libldap-dev clang @@ -277,8 +269,8 @@ jobs: install_steps: skipall mbedtls rustls wolfssl-opensslextra install_steps_brew: gsasl make-custom-target: tidy - PKG_CONFIG_PATH: /home/linuxbrew/.linuxbrew/opt/gsasl/lib/pkgconfig LDFLAGS: -Wl,-rpath,/home/runner/wolfssl-opensslextra/lib -Wl,-rpath,/home/linuxbrew/.linuxbrew/opt/gsasl/lib + PKG_CONFIG_PATH: /home/linuxbrew/.linuxbrew/opt/gsasl/lib/pkgconfig configure: >- --with-wolfssl=/home/runner/wolfssl-opensslextra --with-openssl --with-rustls --with-mbedtls=/home/runner/mbedtls --with-gnutls --with-libgsasl --with-librtmp --with-libssh2 --with-libidn2