Improves Homography code and implement drawCorners method

[kalwalt]

This pull request introduces several significant improvements and updates across the codebase, focusing on enhanced example usage, improved robustness in tracking, developer experience upgrades, and modernization of the build environment. The main changes include new and improved example HTML/CSS for demos, better error handling and validation in the tracking logic, and updates to dependencies and CI workflows to use more recent Node.js and Ubuntu versions.
Thus this PR try to implement the 2 point of issue #8.

Example and Demo Improvements:

• Added a new CSS file (nft-style.css) and updated example HTML files to provide a more modern, visually appealing, and functional demo experience, including overlay canvases for drawing tracking results and improved event handling for detected markers. [1] [2] [3] [4]

Robustness and Error Handling in Tracking:

• Improved the WebARKitCV class to validate image data before processing, skip invalid trackables, and provide a stopTracking method to cleanly cancel animation frames. [1] [2] [3] [4]
• Enhanced the WebARKitCVOrbWorker to validate ImageData buffer sizes before posting to the worker, improving debugging and preventing subtle errors. Also updated the worker's message handling for better state management. [1] [2] [3] [4] [5]

Developer Experience and Testing:

• Added a mock for the uuid module and introduced a new Jest ESM configuration for improved and more flexible testing. [1] [2]

Build and Dependency Updates:

• Updated various devDependencies and dependencies to more recent versions, including uuid, TypeScript, Jest, Babel, Webpack, and others. [1] [2]

CI and Workflow Modernization:

• Updated GitHub Actions workflows to use Node.js 22.13.x and Ubuntu 24.04 for both CI and build workflows, ensuring compatibility with newer environments. [1] [2]

Final note
Even it's not completely implemented the features of the main title, now the two examples can detect and track the images provided.

[kalwalt]

with the testing class WebARKItCOreCV2 i'am on the right path, at least we have a debug image in the example_image.html

[kalwalt]

The simple example example.html will not works because the imageData is not continuosly updated, until i will not improve that it worth to focus on other aspects.

[kalwalt]

with the commit 48bdfd6 the example_image example doesn't works, I need to understand what bug was introduced...

[kalwalt]

the image_example can detect the face pattern and we can see the descriptors and the three red, green, blu line onto the image, we get at the end this error:

Uncaught TypeError: Cannot read properties of undefined (reading '$$')
    at GI.oI [as toWireType] (999068ed-ca1d-4048-a091-a096c233a195:1:5141060)
    at Object.calcOpticalFlowPyrLK (eval at aI (999068ed-ca1d-4048-a091-a096c233a195:1:5142885), <anonymous>:7:26)
    at CI.A.<computed> [as calcOpticalFlowPyrLK] (999068ed-ca1d-4048-a091-a096c233a195:1:5138750)
    at i.estimateCameraPosition (999068ed-ca1d-4048-a091-a096c233a195:1:5180424)
    at i.track (999068ed-ca1d-4048-a091-a096c233a195:1:5180124)
    at R (999068ed-ca1d-4048-a091-a096c233a195:1:5185877)
    at D.onmessage (999068ed-ca1d-4048-a091-a096c233a195:1:5185740)Understand this errorAI
lockdown-install.js:1 SES_UNCAUGHT_EXCEPTION: null

but i think can be easily solved.
Instead the example with the webcam stream can not detect the pinball image. It seems that the Worker never receive the imagedata from the outside. I think there is some inconsistencies in the WebARKitCVCore code. When i will get back from holydays i will develop a video file example, that maybe will avoid these issues.

[kalwalt]

The problem with the webcam example is that in the track method:

webarkit-cv/src/core/WebARKitCoreCV2.ts

Lines 78 to 90 in b943eea

	track(msg: any) {
	console.log("Tracking...", msg);
	if (!msg.imagedata) {
	return;
	}
	console.log("msg-imagedata while Tracking...", msg.imagedata);
	const imageData = new ImageData(
	new Uint8ClampedArray(msg.imagedata),
	msg.vWidth,
	msg.vHeight,
	);
	return this.estimateCameraPosition({ id: 0, imageData: imageData });
	}

we are providing wrong width and height params (in this case 1280 720) but imagedata is filled witha canvas with 320 and 240 height... we should provide these values in the builder with setWidth and setHeight methods

webarkit-cv/examples/example.html

Lines 42 to 45 in b943eea

	webarkit.setWidth(oWidth)
	.setHeight(oHeight)
	.addTrackable("pinball", "./pinball.jpg")
	.loadTrackables()

i should move this block inside the camera initialization

webarkit-cv/examples/example.html

Line 49 in b943eea

cameraView.initialize(videoSettingData).then(() => {

[Copilot]

Pull Request Overview

This pull request enhances the WebARKitCV library by improving homography computation, adding corner visualization capabilities, and updating project dependencies. The changes primarily focus on refactoring the core computer vision logic to use a more modern approach with optical flow tracking, implementing methods to draw detected corners, and upgrading various development dependencies.

Key changes:

• Refactored WebARKitCoreCV to use optical flow tracking with solvePnP for improved pose estimation
• Added drawCorners method to CameraViewRenderer for visualizing detected marker corners
• Updated development dependencies including TypeScript, Babel, Jest, and build tools to newer versions

Reviewed Changes

Copilot reviewed 20 out of 53 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
src/core/WebARKitCoreCV.ts	Major refactoring to implement optical flow tracking, keypoint matching, and projection matrix computation
src/io/CameraViewRenderer.ts	Added corner drawing functionality and improved image caching mechanism
src/Workers/WebARKitCVWorkers.ts	Enhanced worker message handling to include corners and processed image data
src/Workers/Worker.ts	Updated message processing logic and improved error handling
src/WebARKitCV.ts	Added tracking lifecycle management with stopTracking method
package.json	Updated multiple dependencies to latest versions
examples/example.html	Implemented overlay canvas for visualizing tracking results
examples/example_image.html	Added corner drawing and overlay visualization functions
tests/CameraViewRenderer.test.js	Added tests for drawCorners method
tests-esm/*.ts	Added ESM-specific tests for builder, tracker, and trackable classes
jest.esm.config.cjs	New Jest configuration for ESM testing
tsconfig.jest.esm.json	New TypeScript configuration for ESM tests

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.