This repository was archived by the owner on Dec 27, 2025. It is now read-only.
sync #2
Open
sync #2
+24,151
−3,513
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…truncated on a null terminator (#385) Similar to #379, a response returned by the request handler in `net.serve` would have its body truncated on the first null terminator. This PR repeats the fix in #379 to apply to http responses as well. Test ```luau --test-vm.luau local net = require("@lute/net") local self = {} function self.createServer(port: number, body: string) net.serve({ port = port, hostname = "localhost", reuseport = true, handler = function(request) return { status = 200, body = body, } end }) end return self ``` ```luau --test.luau local net = require("@lute/net") local vm = require("@lute/vm") local function test(body: string) local serverController = vm.create("./test-vm") serverController.createServer(8081, body) local response = net.request("http://localhost:8081") assert(body == response.body, `{#body} != {#response.body}`) print("good!") end test("Hello World") -- good! test("Hello\0World") -- 11 != 5 (before) good! (after) ```
Fixes #348 by copying `compile_commands.json` to `build/compile_commands.json` during the configure step.
#387 broke luthier when using lute via toolchain managers since the old version they use doesn't have `fs.copy`.
The bootstrap script didn't correctly generate the lute standard library headers that the `luthier` script did. This PR: - generates a minimal header and implementation for the lute standard libraries during the bootstrap phase - uses these headers to build a `bootstrapped-lute` binary - uses that `bootstrapped-lute` to run `luthier.luau` which builds the actual `lute` executable with the `std` library correctly generated. - accepts an `--install` argument to install this `lute` binary to `$HOME/.lute/bin`. There are some additional improvements that have been made: 1. General README fixes 2. This script can now be run multiple times in sequence, and it correctly deletes artifacts generated by the bootstrapping process each time (e.g. generated std headers, dependencies). 3. Uses git commands to `clone` only the particular revision of the `extern` dependencies we care about, not the entire history. --------- Co-authored-by: ariel <aweiss@hey.com>
…in lute (#393) Based on [frktest](https://github.com/itsfrank/frktest) This testing utility needs a bunch of assertions to be more useful, but this PR just scopes out what a simple, builtin testing api in Luau could look like. For some tests that look like: ``` local test = require("@std/test") local check = test.assert test.case("foo", function() check.eq(3, 3) check.eq(6, 1) end) test.case("bar", function() check.eq("a", "b") end) test.case("baz", function() check.neq("a", "b") end) test.suite("MySuite", function() test.case("subcase", function() check.eq(5 * 5, 3) end) end) test.run(); ``` this will render as: ``` ❯ ./build/xcode/debug/lute/cli/lute run examples/testing.luau ================================================== TEST RESULTS ================================================== Failed Tests (3): ❌ foo ./examples/testing.luau:5 eq: 6 ~= 1 ❌ bar ./examples/testing.luau:10 eq: a ~= b ❌ MySuite.subcase ./examples/testing.luau:20 eq: 25 ~= 3 -------------------------------------------------- Total: 4 Passed: 1 ✓ Failed: 3 ✗ ================================================== ``` Currently reports: - [x] Test suites and cases - [x] Line numbers - [x] Stack traces - [x] The expression that failed.
… where the typedefs are stored (#397) I ran into a small bug with this script where I had `~/.lute` on my device, but not the rest of the path. The setup script only checks if this directory exists and assumes that the rest of the path is there if `~/.lute` is there. This can cause errors with writing the type def files because the subpaths won't exist. This change also refactors the code that makes the subdirs by letting you pass the full path and then making the subdirectories as you go.
…uild system (#396) This PR is a bit of a grab bag of small fixes to both the build system and the bootstrap script. The important fix here is that we force `CURL_ZSTD` off so that we won't have linking failures if you _happen_ to have zstd present on your system for other reasons. Otherwise, we tweak the message a bit when printing the Lute version in CMake, and tweak the bootstrap script to echo the key commands it's running to make the whole output from it easier to interpret. We also change the default behavior for the bootstrap script to assume something vaguely POSIX-y without specialized needs, which makes the script work on additional Unix platforms like FreeBSD. Finally, we change `bootstrapped-lute` to `lute0` for the initial version of lute generated without the standard library embedded in it because the term `bootstrapped` here feels confusing as it reads like it's the _final_ version, rather than an intermediary. `lute0` is "the first build" and, god help us, if our process is ever multistaged for bootstrapping, it has an obvious extension to `lute1`, `lute2`, etc. This may be reasonable to do later for validation if only to make sure that `lute1` and `lute2` are the same executable, byte for byte.
This is needed to add in the codemod source code to lute commands, because it contains the character sequence `")`, which current early terminates the raw string literal.
In order to have `compile_commands.json` in place, we want to rerun `configure` before doing the clean build during bootstrapping. This should generally be a noop, besides copying the file.
…, `test`, or `check-format` jobs fail (#401) ### Problem Our CI aggregator job was unconditionally running and passing. This wasn't too much of an issue because contributors weren't merging their PRs until all checks passed, but PRs set to auto-merge were being merged automatically. Because of this, some PRs introduced bugs that got merged in: #386 and #396. ### Solution The aggregator job's now depends on the success of the `build`, `test`, and `check-format` jobs. If any job fails, the aggregator fails, which will block PRs from being merged. The bug in #396 was fixed in #400. To fix the bug introduced in #386, this PR also reverts that PR's changes to our GitHub Actions setup (using the bootstrap script to create a `lute` binary). We already have Foreman installed on these runners and there's no need to use bootstrapping here—in fact, it makes each CI job quite a bit slower since two fresh builds are needed in order to run anything.
…ed (#402) We use [concurrency groups](https://docs.github.com/en/actions/how-tos/write-workflows/choose-when-workflows-run/control-workflow-concurrency) to cancel in-progress CI runs when they're invalidated by a new commit.
…iguous (#405) Resolves #332. If navigating up a virtual filesystem yields `Ambiguous` during the alias discovery phase, the Luau.Require library currently fails silently. This is an issue, but Lute could also do better here. In the following filesystem, `main.luau`'s parent is _technically_ ambiguous, but because filesystems are trees, it's not actually ambiguous whether we should navigate to the file `folder.luau` or the directory `folder` (the latter is correct): - folder.luau - folder - main.luau Solution: in the case of parent operations in the VFS, ambiguity is overridden to be treated as a success. I'll also be putting up a separate PR to fix the silent error in the Luau.Require library.
This lets Luau subcommands see command line arguments Plus a clang-format change
…ing the task scheduling system (#404) Resolves #384. ### Problem The current `net.serve` logic is to schedule a continuation that runs every task scheduler step, which (1) explicitly calls `run()` on our `uWS::TemplatedApp<T>`, and (2) schedules itself to run the next frame. This would work if `run()` were actually `step()`, but uWS doesn't provide an easy way to step an app. ### Solution After some investigation, because uWS is configured to use libuv under the hood, we can patch in the libuv event loop that our task system is managing by calling `uWS::Loop::get(uv_default_loop())` before creating the app object. We get many things for free here (e.g. reference counting for keeping the server alive, managed by libuv), and it's simpler to reason about. This also means we don't need to explicitly "run" the app; it simply gets registered to run on the same event loop. To ensure this works, some minor changes were made to the task scheduler logic to ensure that we are consuming events from the libuv event loop if it is live and to ensure we aren't blocking on I/O.
Resolves #306. Enables discovering files to run by using require-by-string semantics. The bulk of the changes here enable creating a `ModulePath` object, which handles all the require-by-string semantics for us (with some minor changes made to `ModulePath`'s API to easily extract whether we're pointing at a file or directory). Other than this, I removed `#include <filesystem>` to remain consistent with how the rest of Lute handles these operations. I also changed up some function signatures: ```cpp // old (mutates filePath directly) bool checkValidPath(std::filesystem::path& filePath) // new (returns an optional with the new value) std::optional<std::string> getValidPath(std::string filePath) ``` If we are unable to find a file to run using require-by-string semantics, we still fallback to checking `./.lute/file.luau` and `./.lute/file.lua`. See test for expected behavior: `require_by_string_semantics_in_cli`.
…cally executed by our CI (#417) This implements something like what is described here: #410 , which allows us to write test cases in luau using the std/test library. We can build lute for all the different platforms we support and then run the lute tests as part of a separate pass. Test Failure documented here: https://github.com/luau-lang/lute/actions/runs/18421725097/job/52496749349?pr=417 Resolves #410.
…et_version.cmake` output (#419) Our release workflow depends on the output of `get_version.cmake` being the raw version. We added some extra text in #396 that is currently causing errors during the release tag generation step, so this PR reverts that change. Someone reached out about recent changes not being "released" yet, which was surprising because we're on a nightly release schedule—this was the root cause.
This PR comes with a small change to tools/luthier.luau to pick a different delimiter string.
tin makes net.request work with https on windows. should also work on linux according to the [docs](https://curl.se/libcurl/c/CURLOPT_SSL_OPTIONS.html)
### Problem
The issue can be replicated with a simple script that calls
`process.run` twice, like this:
```luau
local process = require("@lute/process")
process.run({"echo", "hello!"})
process.run({"echo", "goodbye!"})
```
Very rarely, some runs throw unexpected errors, crash, or corrupt Lute
output with random bytes. The problem is a subtle UAF bug in
`process.cpp`. Despite the small PR diff, this was a very tricky bug to
nail down. Had to go digging pretty deep into the libuv source to
understand what was going on.
Essentially, it's not correct to check if the `stdout` and `stderr`
pipes are active before closing them. Since these pipes are used as
read-only data streams, they're automatically made inactive by libuv
upon reaching EOF. In this situation, we don't need to call
`uv_read_stop` on these streams because they're already stopped
(although it is fine to do it; `uv_read_stop` is
[idempotent](https://docs.libuv.org/en/v1.x/stream.html#c.uv_read_stop)),
but we do still need to close the underlying pipe handle.
We're currently in a situation where calling `process.run` might
immediately close either the `stdout` or `stderr` streams (at EOF),
causing the `uv_is_active` check to fail, preventing us from calling
`uv_close` on the handle and from incrementing `pendingCloses`.
Failing to close these handles means that libuv retains data internally
associated with them, and by not incrementing `pendingCloses`
(essentially a refcount), we end up destroying the owning
`ProcessHandle` of these handles before libuv has been able to clean
them up internally.
Then, when we go to initialize new handles for the next `process.run`
call, we have a UAF manifest deep inside of libuv's circular doubly
linked queue code: adding a new handle to the data structure requires
adjusting pointers on the last-added handle, which happens to be a
handle created from the previous `process.run` call. At this point, it
is freed memory that was supposed to be removed from libuv's internals
before being freed (using `uv_close`).
### Solution
Check instead if the handles still need closing with `!uv_is_closing`
since `uv_is_active` is not relevant to whether `uv_close` needs to be
called in this case.
### Problem There was no std lib for `system` related functionalities. ### Solution We introduce a `std` library analogue for `system`, where we can now check the environment with boolean flags. Closes #167 To test (with huge help from @vrn-sn), we wanted to write a C++ file that compares the "os" captured by the C++ executable and compares to the `@std/system` OS. To write that (and other similar output comparison test cases), we exposed the `CliRuntimeFixture` and a `runCode()` function that was used by `require.test.cpp` before but we added a `capture()` function based on the `Luau/ReplFixture` class. Then, we changed some function signatures to ensure the program args (`argc` and `argv`) are not statically passed to avoid some tests failing. ### Example: ```lua local system = require("@std/system") if system.linux then print("Running on Linux") end ``` See `examples/system_environment_check.luau` for example use case
continuation of #416 (comment) to avoid duplicating a `static` function each time the header is included --------- Co-authored-by: Varun Saini <61795485+vrn-sn@users.noreply.github.com>
Some filesystem operations didn't clean up and free all their data. This PR attempts to fix this. Some general observations: - `uv_fs_req_cleanup` has to be called on all requests (sync/async). The documentation doesn't really mention this in much detail. For some operations, this might be a noop, but that's not part of the API and libuv is free to change this afaik. - When handling errors, the request wasn't always free'd. - The data associated with requests wasn't always free'd. - Some code was unreachable due to a Lua error thrown before. These errors seem hard to detect statically. Maybe there could be some RAII type for typed requests?
This should now provide exit as an option when invoking process. + provide better in editor typechecking.
These prints are a little distracting, so we're commenting them out. They're left in place and can be uncommented locally since they're useful for debugging.
Cleans up methods in the visitor library so that they match the AST types they correspond to
…668) Move all setup for the various CLI-facing virtual filesystems into their own files: `requiresetup.{h,cpp}`. Our `climain.cpp` file was getting a bit confusing to parse (and would have gotten worse after merging #657). Ran clang-format, and also made some trivial changes to clean up some other files in CLI.
This now matches the default behavior of Luau's CLI tools. We can still use `setLuauFlag` to disable any problematic flags.
Resolves #652 and #653 Added all of the tests for JSON numbers from https://github.com/nst/JSONTestSuite, and fixed all of the bugs related to numbers
Added all of the tests for JSON arrays from https://github.com/nst/JSONTestSuite, and fixed all of the bugs related to arrays
Added all of the remaining non-string tests for JSON from https://github.com/nst/JSONTestSuite, and 3 lines of style-ish changes
…kage awareness (#657) Implement a simple layer that bridges the gap between Loom's generated `loom.lock` file and Lute's `Package::RequireVfs` (introduced in #339). Running a file with `lute pkgrun <file>` searches for a lockfile called `loom.lock` before executing `<file>`. We use regex matching to parse out the set of dependencies from the lockfile, which are expected to be installed in a `Packages` folder in the same directory as `loom.lock`. Using this, we construct a `Package::RequireVfs`, which automatically injects aliases to dependencies when executing user code. Because of limitations with the current `loom.lock` format, there are still several improvements we need to make. I've left a few TODO comments related to this. Long term, regex matching is also not a great way to parse `loom.lock`, but I didn't want to embed a C++ TOML parser or deal with the complexities of embedding `@batteries/toml` and calling into it from C++. Instead, we can probably use Luau syntax for `loom.lock` since Lute can easily parse Luau tables. We could potentially base the format on [`.config.luau` files](https://rfcs.luau.org/config-luauconfig.html).
This is needed for the package manager. Very barebones for now, and we will still need to expose the other parts of `@lute/net` under `@std/net`.
I encountered some `.gitignore` parsing bugs when I was trying to run `lute lint` in the lint directory. I've: - Ported over a fix that went into `luau-lang/codemod` but for some reason didn't make it into `lute transform` when I initially ported it - Added some additional fixups needed in `.gitignore` parsing - Removed some redundant rules from our actual `.gitignore` - Added tests for `cli/lib/files` which is built on the ignore parsing. the ignore stuff doesn't have a very nice api surface, so i opted to test it using the `files` stuff instead - Use system-specific path separators so it works on Windows (I would love to eventually port this stuff to just run on path objects so the OS stuff is abstracted away)
) Added all of the tests for JSON strings from https://github.com/nst/JSONTestSuite, and fixes all of the bugs related to strings
…ile system operations (open, close, read, write) (#661) This PR ensures that `fs.(open, close, read, write)` are non-blocking - that is, other `Luau` threads that could continue to run, will not blocked by long i/o. This fixes #642, whereby a very long read, e.g. to /dev/random could cause the main Lute Runtime thread to block synchronously (which would prevent stepping the uv event loop as well). I've tested this locally and it works as expected, but I'm not sure how to write a test for this in a cross platform way. This PR also: - Adds a base `UVRequest` type - responsible for interfacing with the lute runtime to resume the original luau thread. During destruction, will invoke the uv_*_cleanup function. - Makes File Handles a lightuserdata. - Rewrites `stdfs.writefiletostring` and `stdfs.readstringfromfile` in terms of the low level `lute` primitives that don't block anymore. - Removes `readasync`, `writefiletostring` and `readstringfromfile` from the `lute` api surface.
…sertions + make runtime error detection less brittle (#651) This PR attempts to refactor the test library to be slightly more maintainable. While trying to bring #519 up to date, I experienced a lot of brittleness with our test libraries ability to detect when a test failed due to runtime errors. The test runner also produces debug information in an adhoc manner multiple times in different places, which really sucks. This PR ports over the [frktest](https://github.com/itsfrank/frktest/blob/main/src/assert_impl.luau) implementation of assertions. Rather than having `assertions` immediately yield control flow via `error`, all assertions internally return a typed `failure?`. When this failure is emitted, we extract all of the debug information needed for tests, exactly once at the point of the assertion 'failure'. This means that the test runner no longer has to know how to produce debug information (woo separation of concerns!), and the actual assertion implementation is responsible for this. Another freebie is that we only have to produce stacktraces in exactly one place - I've gone ahead and added that, along with updating the runtime error tests. ## Future Work - The stacktraces currently produces are not very good - this is because `lute run` compiles the script with all optimizations on, and the inling negatively impacts the quality of stacktraces. I discussed an option with @vrn-sn to lower the optimization level for .test.luau files. - The current implementation of `asserts` behaves like `REQUIRE` - control flow is immediately stopped at this point. Since all assertion implementations are wrapped by a function that determines how to handle the control flow, we can now also provide an implementation of `CHECK`, where you can fail a test and resume control flow.
…pe (#676) 1. Expose `net.serve` in `@std/net` as a wrapper for `@lute/net`'s version. 2. Add a `Server` return type to this function that seemed to have been missed when it was first added. 3. Update the `net.test.luau` unit test to create a local server and request from it instead of hitting an arbitrary URL (this was causing some flakiness). 4. Add types to our example file `examples/serve_html.luau` so that it passes a type check.
This allows users to write lint rules outside the lute repo while still getting lint-specific type definitions by using `require(@lint/types)`. There were also some bugs in `luthier` because it was calling functions which were deleted from `@lute/fs`. This was uncaught in CI because our CI jobs currently call `luthier` using a pinned, previous version of `lute`. We should add CI jobs which use `bootstrap.sh` to build `lute0` to call `luthier`.
We make use of the existing Luau-syntax configuration file extraction API to parse our Luau-syntax lock file (`loom.lock.luau`). The temporary regex hack has been deleted.
@vrn-sn pointed out that the times are quite bad for bootstrapping in CI, but we also want to retain the benefits of confirming that it is still working on a per-PR basis. The compromise position here is that the marginal difference between macOS without bootstrap on GH Actions runners and Linux with bootstrap on GH Actions runners is quite small (8 minutes vs 9 minutes). We'll remove the macOS bootstrap since that one is prone to timing out, but the Linux bootstrap is only marginally slower than our existing CI and gets us most of the benefit of knowing if `luthier` broke somehow.
…found (#682) However, if we're outputting a json, doesn't report any error code. Also adds lute lint to GC
There were some bug reports from @Vighnesh-V and @wmccrthy about deserialized objects returning `nil` when correctly accessed as a table, and that led to us discovering there were issues with the `deserializationString` logic I wrote in #674 even though the test cases were passing. I essentially rewrote the logic to be more similar to the original implementation of it and just added invalid case checks to make it RFC compliant, so this new version now works with actual use cases (see the test case)
Printing line numbers in `prettydiff` provides a lot of utility/clarity in properly visualizing differences b/w files. It also helps distinguish the diff output from other arbitrary text in the command line. Here is an example on a 100 line diff, to showcase the alignment of printed side-headers: <img width="351" height="1313" alt="Screenshot 2025-12-08 at 3 22 31 PM" src="https://github.com/user-attachments/assets/79780be7-0298-4f61-9483-f2bf233ba152" />
**Luau**: Updated from `0.702` to `0.703` **Release Notes:** https://github.com/luau-lang/luau/releases/tag/0.703 --- *This PR was automatically created by the [Update Luau workflow](https://github.com/luau-lang/lute/actions/workflows/update-prs.yml)* --------- Co-authored-by: aatxe <744293+aatxe@users.noreply.github.com> Co-authored-by: Varun Saini <61795485+vrn-sn@users.noreply.github.com>
We use the new `to_alias_override` Luau.Require callback to ensure that the `@std` and `@lute` aliases always point at the embedded libraries, not ones on disk that the user may have set up using `.luaurc` or `.config.luau` files. @Vighnesh-V, this means that `@std/test` will always point at the embedded module now.
When invoking lute, it will attempt to discover a script named with the provided argument, or within a `.lute` directory. This change supports the same discovery mechanism, but additionally supports discovering scripts in hierarchical `.lute` directories. As a practical example, I provided a luthier shorthand to demonstrate how this mechanism can be used. Instead of `lute tools/luthier.luau ...args` we can now invoke `lute luthier ...args`, and this can now be invoked from within any nested directory of lute.
**Lute**: Updated from `0.1.0-nightly.20251205` to `0.1.0-nightly.20251220` **Release Notes:** https://github.com/luau-lang/lute/releases/tag/0.1.0-nightly.20251220 --- *This PR was automatically created by the [Update Luau workflow](https://github.com/luau-lang/lute/actions/workflows/update-prs.yml)* Co-authored-by: aatxe <744293+aatxe@users.noreply.github.com>
Thanks to recent changes to how require works, namely that `@std` requires will always resolve to the single builtin instance in the runtime (ignoring aliases), `lute test` can now handle transitive and batteries requires correctly without needing to intercept requires at all. This was the last blocker remaining before `lute test` could be used in CI. This PR: a) updates CI to call `lute test` b) removes the require interception that `lute test` did b) removes the explicit call to test.run() within the test files in our repo, since test discovery handles this I've left `test.run()` in for now, in case there is some desire to use the testing library without `lute test` (and the standard library tests in `std.test.luau` exercise this behaviour). I'm open to opinions about what to do with it, and can address those in a followup. Resolves issue #634
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
15 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.