From 3a5f26bc7396d4067f2b8983666ba028b22b3d99 Mon Sep 17 00:00:00 2001 From: bluepal-yaswanth-peravali Date: Fri, 28 Nov 2025 21:50:50 +0530 Subject: [PATCH 1/8] added nodejs foxx support --- CHANGELOG.md | 138 ++++++++ Dockerfile.nodejs.template | 17 + baseimages/Dockerfile.node22base | 33 ++ baseimages/imagelist.txt | 1 + baseimages/scripts/entrypoint.sh | 38 ++- scripts/prepareproject-nodejs.sh | 126 ++++++++ src/main.rs | 323 ++++++++++++++++--- testprojects/itzpapalotl-node/LICENSE | 202 ++++++++++++ testprojects/itzpapalotl-node/README.md | 11 + testprojects/itzpapalotl-node/itzpapalotl.js | 118 +++++++ testprojects/itzpapalotl-node/manifest.json | 12 + testprojects/itzpapalotl-node/package.json | 11 + 12 files changed, 972 insertions(+), 58 deletions(-) create mode 100644 CHANGELOG.md create mode 100644 Dockerfile.nodejs.template create mode 100644 baseimages/Dockerfile.node22base create mode 100644 scripts/prepareproject-nodejs.sh create mode 100644 testprojects/itzpapalotl-node/LICENSE create mode 100644 testprojects/itzpapalotl-node/README.md create mode 100644 testprojects/itzpapalotl-node/itzpapalotl.js create mode 100644 testprojects/itzpapalotl-node/manifest.json create mode 100644 testprojects/itzpapalotl-node/package.json diff --git a/CHANGELOG.md b/CHANGELOG.md new file mode 100644 index 0000000..d1b16a6 --- /dev/null +++ b/CHANGELOG.md @@ -0,0 +1,138 @@ +# Changelog + +All notable changes to ServiceMaker will be documented in this file. + +The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), +and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). + +## [Unreleased] + +### Added + +#### Node.js/Foxx Service Support +- **Node.js Base Image**: Added `Dockerfile.node22base` for Node.js 22 base image with pre-installed ArangoDB packages + - Installs Node.js 22 from NodeSource + - Pre-installs `@arangodb/node-foxx`, `@arangodb/node-foxx-launcher`, and `@arangodb/arangodb` packages + - Creates base `node_modules` with checksums for dependency tracking + - Added to `baseimages/imagelist.txt` as `node22base` + +- **Node.js Dockerfile Template**: Created `Dockerfile.nodejs.template` for building Node.js/Foxx service images + - Supports wrapper structure for single service directories + - Configures working directory and user permissions + - Executes `prepareproject-nodejs.sh` for dependency management + +- **Dependency Management Script**: Added `scripts/prepareproject-nodejs.sh` + - Copies base `node_modules` from base image + - Installs project-specific dependencies from `package.json` + - Ensures `node-foxx` binary is always available with multiple safety checks + - Tracks new dependencies using SHA256 checksums + - Separates base packages from project packages + - Includes automatic recovery mechanism if base packages are removed during `npm install` + - Handles `package.json` copying to wrapper root for proper dependency installation + +- **Project Type Detection**: Extended `detect_project_type()` to support: + - `python`: Projects with `pyproject.toml` + - `foxx`: Multi-service projects with `package.json` and `services.json` + - `foxx-service`: Single service directory with `package.json` (creates wrapper structure) + - `nodejs`: Generic Node.js projects + +- **Wrapper Structure Generation**: Automatic wrapper creation for single service directories + - Creates `wrapper/` directory structure + - Copies service directory into `wrapper/{service-name}/` + - Generates `services.json` automatically with mount path configuration + - Copies `package.json` to wrapper root for dependency installation + +- **CLI Arguments**: + - `--mount-path`: Required for `foxx-service` type, specifies the mount path for the Foxx service + +- **Services JSON Generation**: Added `generate_services_json()` function + - Automatically generates `services.json` for single service directories + - Configures mount path and base path for Foxx services + +- **Package.json Support**: Added functions to read Node.js project metadata + - `read_name_from_package_json()`: Extracts project name from `package.json` + - `read_service_info_from_package_json()`: Extracts name and version for Helm charts + +- **Entrypoint Enhancement**: Updated `baseimages/scripts/entrypoint.sh` to support Node.js/Foxx services + - Detects service type based on project files + - Automatically runs `node-foxx` for Foxx services + - Falls back to generic Node.js execution for non-Foxx services + - Maintains backward compatibility with Python services + +- **Test Service**: Added `itzpapalotl-node` test service in `testprojects/` + - Example Node.js/Foxx service for testing ServiceMaker functionality + - Demonstrates wrapper structure generation and dependency management + +### Changed + +- **Main Application Logic**: Extended `src/main.rs` to support Node.js projects + - Added project type detection for Node.js/Foxx services + - Updated file copying logic to handle wrapper structure + - Modified Dockerfile generation to use appropriate template based on project type + - Updated Helm chart generation to support Node.js projects + - Added `prepareproject-nodejs.sh` to embedded scripts list + +- **Entrypoint Script**: Enhanced `baseimages/scripts/entrypoint.sh` to support Node.js/Foxx services + - Added service type detection based on project files + - Maintains backward compatibility with Python services + +- **Base Image List**: Updated `baseimages/imagelist.txt` to include `node22base` + +- **File Copying Logic**: Updated `copy_dir_recursive()` to skip `node_modules` directories + - Prevents copying local `node_modules` which should be installed in Docker build + +### Fixed + +- **Windows Compatibility**: Fixed Windows build issues in `src/main.rs` + - Added conditional compilation for Unix-specific file permissions (`#[cfg(unix)]`) + - Windows builds now skip `set_mode()` calls that are Unix-only + +### Technical Details + +- **Base Image Structure**: + - Base `node_modules` located at `/home/user/base_node_modules/node_modules` + - Checksums stored at `/home/user/sums_sha256` for dependency tracking + - Base packages: `@arangodb/node-foxx@^0.0.1-alpha.0`, `@arangodb/node-foxx-launcher@^0.0.1-alpha.0`, `@arangodb/arangodb@^0.0.1-alpha.0` + +- **Wrapper Structure**: + ``` + wrapper/ + ├── package.json # Copied from service for npm install + ├── services.json # Auto-generated with mount path + ├── node_modules/ # Installed dependencies (base + project) + └── {service-name}/ # Service directory + ├── package.json + └── ... + ``` + +- **Dependency Tracking**: + - Uses SHA256 checksums to identify new files vs. base files + - New project dependencies copied to `/project/node_modules/` for tracking + - Base packages remain in base image for efficiency + +## [0.9.2] - Previous Release + +### Existing Features +- Python service support with `pyproject.toml` +- Base image management for Python 3.13 +- Docker image building and pushing +- Helm chart generation +- Project tar.gz creation +- Virtual environment management with `uv` + +--- + +## Version History + +- **0.9.2**: Initial release with Python support +- **Unreleased**: Added Node.js/Foxx service support + +--- + +## Notes + +- All changes maintain backward compatibility with existing Python projects +- Node.js support is additive and does not affect Python service functionality +- Base images must be built separately using `baseimages/build.sh` +- Windows users should use WSL or Linux environment for building base images + diff --git a/Dockerfile.nodejs.template b/Dockerfile.nodejs.template new file mode 100644 index 0000000..a5b48ef --- /dev/null +++ b/Dockerfile.nodejs.template @@ -0,0 +1,17 @@ +FROM {BASE_IMAGE} + +USER root + +COPY ./scripts /scripts +COPY {PROJECT_DIR} /project/{PROJECT_DIR} +RUN chown -R user:user /project/{PROJECT_DIR} + +USER user +WORKDIR /project/{PROJECT_DIR} + +RUN /scripts/prepareproject-nodejs.sh + +EXPOSE {PORT} + +CMD ["node_modules/.bin/node-foxx"] + diff --git a/baseimages/Dockerfile.node22base b/baseimages/Dockerfile.node22base new file mode 100644 index 0000000..2a86e04 --- /dev/null +++ b/baseimages/Dockerfile.node22base @@ -0,0 +1,33 @@ +FROM debian:trixie-slim + +COPY ./scripts /scripts + +# FIX: Convert Windows CRLF → LF and make scripts executable +RUN apt-get update && apt-get install -y dos2unix \ + && dos2unix /scripts/*.sh \ + && chmod +x /scripts/*.sh + +# Now the script will run successfully +RUN /scripts/debinstall.sh + +# Install Node.js 22 +RUN curl -fsSL https://deb.nodesource.com/setup_22.x | bash - && \ + apt-get install -y nodejs && \ + apt-get clean + +WORKDIR /home/user +USER user + +# Create base node_modules with node-foxx dependencies +RUN mkdir -p base_node_modules && \ + cd base_node_modules && \ + npm init -y && \ + npm install @arangodb/node-foxx@^0.0.1-alpha.0 \ + @arangodb/node-foxx-launcher@^0.0.1-alpha.0 \ + @arangodb/arangodb@^0.0.1-alpha.0 + +# Create checksums for base node_modules +RUN find base_node_modules/node_modules -type f -print0 | \ + xargs -0 sha256sum > sums_sha256 + +CMD [ "/scripts/entrypoint.sh" ] diff --git a/baseimages/imagelist.txt b/baseimages/imagelist.txt index 5620215..a608fae 100644 --- a/baseimages/imagelist.txt +++ b/baseimages/imagelist.txt @@ -1,3 +1,4 @@ py13base py13cugraph py13torch +node22base diff --git a/baseimages/scripts/entrypoint.sh b/baseimages/scripts/entrypoint.sh index 9878098..c31c97e 100755 --- a/baseimages/scripts/entrypoint.sh +++ b/baseimages/scripts/entrypoint.sh @@ -14,16 +14,40 @@ if test -e project.tar.gz ; then tar xzvf project.tar.gz > /dev/null fi -# Run the entrypoint if configured: +# Detect service type and run accordingly if test -e entrypoint ; then ENTRYPOINT=$(cat entrypoint) echo Running project ... - . /home/user/.local/bin/env - . /home/user/the_venv/bin/activate - for p in /project/the_venv/lib/python*/site-packages ; do - export PYTHONPATH=$p - done - exec python $ENTRYPOINT + + # Check if it's a Node.js/Foxx service + if [ -f "package.json" ] && [ -f "services.json" ]; then + # Node.js/Foxx service + echo "Detected Node.js/Foxx service" + if [ -f "node_modules/.bin/node-foxx" ]; then + exec node_modules/.bin/node-foxx + else + echo "Error: node-foxx not found. Make sure node_modules are installed." + exit 1 + fi + elif [ -f "package.json" ]; then + # Generic Node.js service + echo "Detected Node.js service" + if [ -f "$ENTRYPOINT" ]; then + exec node "$ENTRYPOINT" + else + echo "Error: Entrypoint file not found: $ENTRYPOINT" + exit 1 + fi + else + # Python service (existing logic) + echo "Detected Python service" + . /home/user/.local/bin/env + . /home/user/the_venv/bin/activate + for p in /project/the_venv/lib/python*/site-packages ; do + export PYTHONPATH=$p + done + exec python $ENTRYPOINT + fi fi echo No entrypoint found, running bash instead... diff --git a/scripts/prepareproject-nodejs.sh b/scripts/prepareproject-nodejs.sh new file mode 100644 index 0000000..afd07eb --- /dev/null +++ b/scripts/prepareproject-nodejs.sh @@ -0,0 +1,126 @@ +#!/bin/bash +# This is to be run in the project directory to install dependencies. +# It copies base node_modules from the base image and installs additional +# project dependencies, tracking changes similar to the Python script. + +set -e + +# We're already in /project/{PROJECT_DIR} from the Dockerfile WORKDIR +PROJECT_DIR=$(pwd) + +# Copy base node_modules if they exist in base image +if [ -d "/home/user/base_node_modules/node_modules" ]; then + echo "Copying base node_modules..." + if [ -d "node_modules" ]; then + # Merge with existing node_modules if any + cp -r /home/user/base_node_modules/node_modules/* ./node_modules/ 2>/dev/null || true + else + cp -r /home/user/base_node_modules/node_modules ./node_modules + fi + + # Verify node-foxx binary exists after copy + if [ -f "node_modules/.bin/node-foxx" ]; then + echo "✓ Base node-foxx binary copied successfully" + else + echo "Warning: node-foxx binary not found after copying base node_modules" + fi + + # Track existing files + cd /home/user + if [ -f "sums_sha256" ]; then + sha256sum -c sums_sha256 || true + fi + cd "$PROJECT_DIR" +else + echo "Warning: Base node_modules not found at /home/user/base_node_modules/node_modules" +fi + +# Install additional project dependencies if package.json exists +if [ -f "package.json" ]; then + echo "Installing project dependencies..." + + # Always ensure base packages are installed first (they provide node-foxx binary) + # This ensures node-foxx is available even if npm install does a clean install + echo "Ensuring base node-foxx packages are installed..." + npm install --production --no-save \ + @arangodb/node-foxx@^0.0.1-alpha.0 \ + @arangodb/node-foxx-launcher@^0.0.1-alpha.0 \ + @arangodb/arangodb@^0.0.1-alpha.0 || { + echo "Warning: Failed to install base packages, continuing anyway..." + } + + # Verify node-foxx exists after installing base packages + if [ ! -f "node_modules/.bin/node-foxx" ]; then + echo "ERROR: node-foxx binary not found after installing base packages!" + echo "Listing node_modules/.bin contents:" + ls -la node_modules/.bin/ 2>/dev/null || echo "node_modules/.bin directory does not exist" + exit 1 + fi + echo "✓ Base node-foxx packages installed" + + # Install project dependencies (this should preserve base packages) + npm install --production --no-save + + # Final verification that node-foxx still exists + if [ -f "node_modules/.bin/node-foxx" ]; then + echo "✓ node-foxx binary exists after installing project dependencies" + else + echo "ERROR: node-foxx binary missing after npm install!" + echo "Reinstalling base packages..." + npm install --production --no-save \ + @arangodb/node-foxx@^0.0.1-alpha.0 \ + @arangodb/node-foxx-launcher@^0.0.1-alpha.0 \ + @arangodb/arangodb@^0.0.1-alpha.0 + + # Final check + if [ ! -f "node_modules/.bin/node-foxx" ]; then + echo "ERROR: Failed to install node-foxx binary!" + echo "Current directory: $(pwd)" + echo "Listing node_modules/.bin contents:" + ls -la node_modules/.bin/ 2>/dev/null || echo "node_modules/.bin directory does not exist" + echo "Listing node_modules contents:" + ls -la node_modules/ 2>/dev/null | head -20 + exit 1 + fi + echo "✓ node-foxx binary restored" + fi + + # Find all files in node_modules and create checksums + find node_modules -type f -print0 | xargs -0 sha256sum > /tmp/node_modules_sha256_new 2>/dev/null || true + + # Find new files (files in current node_modules that weren't in base) + if [ -f "/home/user/sums_sha256" ] && [ -f "/tmp/node_modules_sha256_new" ]; then + cat /tmp/node_modules_sha256_new /home/user/sums_sha256 | sort | uniq -c | grep "^ 1 " | awk '{ print $3 }' > /tmp/newfiles || true + else + # If no base checksums, all files are new + if [ -f "/tmp/node_modules_sha256_new" ]; then + awk '{ print $3 }' /tmp/node_modules_sha256_new > /tmp/newfiles || true + fi + fi + + # Move new files to /project/node_modules (similar to Python approach) + if [ -f "/tmp/newfiles" ]; then + mkdir -p /project/node_modules + while IFS= read -r filename; do + # Skip empty lines + [[ -z "$filename" ]] && continue + + # Check if source file exists and is relative to current dir + if [[ "$filename" == node_modules/* ]] && [ -f "$filename" ]; then + # Get the directory part of the filename + DIR=$(dirname "$filename") + + # Create the destination directory structure + mkdir -p "/project/$DIR" + + # Copy the file preserving the directory hierarchy + cp "$filename" "/project/$filename" + fi + done < "/tmp/newfiles" + fi + + rm -f /tmp/node_modules_sha256_new /tmp/newfiles +fi + +echo "Node.js project prepared successfully" + diff --git a/src/main.rs b/src/main.rs index 0ece9be..f094bc5 100644 --- a/src/main.rs +++ b/src/main.rs @@ -50,13 +50,17 @@ const SCRIPT_FILES: &[ScriptFile] = &[ path: "prepareproject.sh", content: include_str!("../scripts/prepareproject.sh"), }, + ScriptFile { + path: "prepareproject-nodejs.sh", + content: include_str!("../scripts/prepareproject-nodejs.sh"), + }, ScriptFile { path: "zipper.sh", content: include_str!("../scripts/zipper.sh"), }, ]; -/// A tool to wrap Python projects as Docker services +/// A tool to wrap Python and Node.js projects as Docker services #[derive(Parser, Debug)] #[command(author, version, about, long_about = None)] struct Args { @@ -64,7 +68,7 @@ struct Args { #[arg(long)] name: Option, - /// Path to the folder containing the Python project + /// Path to the folder containing the project #[arg(long)] project_home: Option, @@ -91,6 +95,10 @@ struct Args { /// Whether to create a tar.gz file with project files and virtual environment changes #[arg(long, default_value = "false")] make_tar_gz: bool, + + /// Mount path for the service (required for Foxx services, e.g., /itz) + #[arg(long)] + mount_path: Option, } fn main() -> Result<(), Box> { @@ -108,11 +116,78 @@ fn main() -> Result<(), Box> { return Err(format!("Project home does not exist: {}", project_home.display()).into()); } - // Try to get name from pyproject.toml if not provided on command line - if args.name.is_none() - && let Ok(name) = read_name_from_pyproject(project_home) - { - args.name = Some(name); + // Detect project type + let project_type = detect_project_type(project_home)?; + println!("Detected project type: {}", project_type); + + // Handle project type-specific configuration + match project_type.as_str() { + "python" => { + // Try to get name from pyproject.toml if not provided on command line + if args.name.is_none() + && let Ok(name) = read_name_from_pyproject(project_home) + { + args.name = Some(name); + } + + // Try to auto-detect entrypoint if exactly one .py file exists + if args.entrypoint.is_none() + && let Ok(Some(py_file)) = find_single_py_file(project_home) + { + args.entrypoint = Some(py_file); + } + + // Prompt for entrypoint if still not set + if args.entrypoint.is_none() { + args.entrypoint = Some(prompt("Python entrypoint script (e.g., main.py)")?); + } + } + "foxx" => { + // Multi-service structure (has services.json) + // Try to get name from package.json if not provided + if args.name.is_none() + && let Ok(name) = read_name_from_package_json(project_home) + { + args.name = Some(name); + } + + // Set default base image for Node.js if not specified + if args.base_image == "arangodb/py13base:latest" { + args.base_image = "arangodb/node22base:latest".to_string(); + } + } + "foxx-service" | "nodejs" => { + // Single service directory - will create wrapper structure + let service_name = project_home.file_name().unwrap().to_str().unwrap(); + + // Try to get name from package.json if not provided + if args.name.is_none() { + if let Ok(name) = read_name_from_package_json(project_home) { + args.name = Some(name); + } else { + args.name = Some(service_name.to_string()); + } + } + + // Set default base image for Node.js if not specified + if args.base_image == "arangodb/py13base:latest" { + args.base_image = "arangodb/node22base:latest".to_string(); + } + + // Prompt for mount path if not provided (for foxx-service) + if project_type == "foxx-service" && args.mount_path.is_none() { + let default_mount = format!("/{}", service_name.to_lowercase()); + let mount_input = prompt(&format!("Mount path (default: {})", default_mount))?; + args.mount_path = Some(if mount_input.is_empty() { + default_mount + } else { + mount_input + }); + } + } + _ => { + return Err(format!("Unsupported project type: {}", project_type).into()); + } } // Prompt for name if still not set @@ -120,7 +195,7 @@ fn main() -> Result<(), Box> { args.name = Some(prompt("Project name")?); } - let project_dir = project_home.file_name().unwrap().to_str().unwrap(); + let initial_project_dir = project_home.file_name().unwrap().to_str().unwrap(); if args.port.is_none() { let port_str = prompt("Exposed port number")?; @@ -131,32 +206,25 @@ fn main() -> Result<(), Box> { args.image_name = Some(prompt("Docker image name")?); } - // Try to auto-detect entrypoint if exactly one .py file exists - if args.entrypoint.is_none() - && let Ok(Some(py_file)) = find_single_py_file(project_home) - { - args.entrypoint = Some(py_file); - } - - // Prompt for entrypoint if still not set - if args.entrypoint.is_none() { - args.entrypoint = Some(prompt("Python entrypoint script (e.g., main.py)")?); - } - let name = args.name.as_ref().unwrap(); let project_home = args.project_home.as_ref().unwrap(); let image_name = args.image_name.as_ref().unwrap(); - let entrypoint = args.entrypoint.as_ref().unwrap(); let port = args.port.unwrap(); println!("\n=== Configuration ==="); println!("Project name: {}", name); + println!("Project type: {}", project_type); println!("Project home: {}", project_home.display()); - println!("Project directory name: {}", project_dir); + println!("Project directory name: {}", initial_project_dir); println!("Base image: {}", args.base_image); println!("Port: {}", port); println!("Image name: {}", image_name); - println!("Entrypoint: {}", entrypoint); + if let Some(ref entrypoint) = args.entrypoint { + println!("Entrypoint: {}", entrypoint); + } + if let Some(ref mount_path) = args.mount_path { + println!("Mount path: {}", mount_path); + } println!("Push: {}", args.push); println!("Make tar.gz: {}", args.make_tar_gz); println!("=====================\n"); @@ -174,28 +242,75 @@ fn main() -> Result<(), Box> { // Copy scripts to temp directory with executable permissions copy_scripts_to_temp(&temp_dir)?; - // Copy project directory to temp directory - let project_dest = temp_dir.join(project_dir); - println!( - "Copying project from {} to {}", - project_home.display(), - project_dest.display() - ); - copy_dir_recursive(project_home, &project_dest)?; - - // Extract Python version from base image name - let python_version = extract_python_version(&args.base_image); - - // Read and modify Dockerfile template - let dockerfile_template = include_str!("../Dockerfile.template"); - let modified_dockerfile = modify_dockerfile( - dockerfile_template, - &args.base_image, - project_dir, - entrypoint, - port, - &python_version, - ); + // Handle Node.js single service directory differently - create wrapper structure + let project_dir = if project_type == "foxx-service" { + let service_name = initial_project_dir; + + // Create wrapper directory + let wrapper_dir = temp_dir.join("wrapper"); + fs::create_dir_all(&wrapper_dir)?; + + // Copy service directory into wrapper + let service_dest = wrapper_dir.join(service_name); + println!( + "Creating wrapper structure: copying {} to wrapper/{}", + project_home.display(), + service_name + ); + copy_dir_recursive(project_home, &service_dest)?; + + // Copy package.json from service to wrapper root for dependency installation + let service_package_json = service_dest.join("package.json"); + if service_package_json.exists() { + let wrapper_package_json = wrapper_dir.join("package.json"); + fs::copy(&service_package_json, &wrapper_package_json)?; + println!("Copied package.json to wrapper root for dependency installation"); + } + + // Generate services.json with mount path + let mount_path = args + .mount_path + .as_deref() + .ok_or("Mount path is required for foxx-service")?; + let services_json_content = generate_services_json(service_name, mount_path); + let services_json_path = wrapper_dir.join("services.json"); + fs::write(&services_json_path, services_json_content)?; + println!("Generated services.json: {}", services_json_path.display()); + + "wrapper".to_string() + } else { + // Normal case - copy project directory as-is + let project_dest = temp_dir.join(initial_project_dir); + println!( + "Copying project from {} to {}", + project_home.display(), + project_dest.display() + ); + copy_dir_recursive(project_home, &project_dest)?; + initial_project_dir.to_string() + }; + + // Choose Dockerfile template and modify based on project type + let modified_dockerfile = match project_type.as_str() { + "python" => { + let python_version = extract_python_version(&args.base_image); + let entrypoint = args.entrypoint.as_ref().unwrap(); + let dockerfile_template = include_str!("../Dockerfile.template"); + modify_dockerfile_python( + dockerfile_template, + &args.base_image, + &project_dir, + entrypoint, + port, + &python_version, + ) + } + "foxx" | "foxx-service" | "nodejs" => { + let dockerfile_template = include_str!("../Dockerfile.nodejs.template"); + modify_dockerfile_nodejs(dockerfile_template, &args.base_image, &project_dir, port) + } + _ => return Err("Unsupported project type".into()), + }; // Write modified Dockerfile to temp directory let dockerfile_path = temp_dir.join("Dockerfile"); @@ -321,9 +436,21 @@ fn main() -> Result<(), Box> { // Generate Helm chart println!("\n=== Generating Helm Chart ==="); - let (service_name, version) = read_service_info_from_pyproject(project_home)?; - println!("Service name from pyproject.toml: {}", service_name); - println!("Version from pyproject.toml: {}", version); + let (service_name, version) = match project_type.as_str() { + "python" => { + let (name, ver) = read_service_info_from_pyproject(project_home)?; + println!("Service name from pyproject.toml: {}", name); + println!("Version from pyproject.toml: {}", ver); + (name, ver) + } + "foxx" | "foxx-service" | "nodejs" => { + let (name, ver) = read_service_info_from_package_json(project_home)?; + println!("Service name from package.json: {}", name); + println!("Version from package.json: {}", ver); + (name, ver) + } + _ => return Err("Unsupported project type for Helm chart generation".into()), + }; let chart_dir = temp_dir.join(&service_name); @@ -400,7 +527,7 @@ fn extract_python_version(base_image: &str) -> String { "3.13".to_string() } -fn modify_dockerfile( +fn modify_dockerfile_python( template: &str, base_image: &str, project_dir: &str, @@ -416,6 +543,71 @@ fn modify_dockerfile( .replace("{PYTHON_VERSION}", python_version) } +fn modify_dockerfile_nodejs( + template: &str, + base_image: &str, + project_dir: &str, + port: u16, +) -> String { + template + .replace("{BASE_IMAGE}", base_image) + .replace("{PROJECT_DIR}", project_dir) + .replace("{PORT}", &port.to_string()) +} + +fn detect_project_type(project_home: &Path) -> Result> { + let pyproject = project_home.join("pyproject.toml"); + let package_json = project_home.join("package.json"); + let services_json = project_home.join("services.json"); + + if pyproject.exists() { + Ok("python".to_string()) + } else if package_json.exists() && services_json.exists() { + Ok("foxx".to_string()) + } else if package_json.exists() { + // Single service directory - needs wrapper structure + Ok("foxx-service".to_string()) + } else { + Err(format!( + "Could not detect project type. Expected pyproject.toml (Python) or package.json (Node.js) in: {}", + project_home.display() + ) + .into()) + } +} + +fn generate_services_json(service_name: &str, mount_path: &str) -> String { + format!( + r#"[ + {{ + "mount": "{}", + "basePath": "{}" + }} +]"#, + mount_path, service_name + ) +} + +fn read_name_from_package_json(project_home: &Path) -> Result> { + let package_json_path = project_home.join("package.json"); + + if !package_json_path.exists() { + return Err(format!("package.json not found in: {}", project_home.display()).into()); + } + + let content = fs::read_to_string(&package_json_path)?; + let value: serde_json::Value = serde_json::from_str(&content)?; + + // Extract project name + let name = value + .get("name") + .and_then(|n| n.as_str()) + .ok_or("Missing 'name' in package.json")? + .to_string(); + + Ok(name) +} + fn copy_dir_recursive(src: &Path, dst: &Path) -> io::Result<()> { if !dst.exists() { fs::create_dir_all(dst)?; @@ -426,8 +618,8 @@ fn copy_dir_recursive(src: &Path, dst: &Path) -> io::Result<()> { let path = entry.path(); let file_name = entry.file_name(); - // Skip .venv directories - if file_name == ".venv" { + // Skip .venv directories (Python) and node_modules (Node.js) + if file_name == ".venv" || file_name == "node_modules" { continue; } @@ -521,6 +713,35 @@ fn read_service_info_from_pyproject( Ok((name, version)) } +fn read_service_info_from_package_json( + project_home: &Path, +) -> Result<(String, String), Box> { + let package_json_path = project_home.join("package.json"); + + if !package_json_path.exists() { + return Err(format!("package.json not found in: {}", project_home.display()).into()); + } + + let content = fs::read_to_string(&package_json_path)?; + let value: serde_json::Value = serde_json::from_str(&content)?; + + // Extract project name + let name = value + .get("name") + .and_then(|n| n.as_str()) + .ok_or("Missing 'name' in package.json")? + .to_string(); + + // Extract version (default to "1.0.0" if not present) + let version = value + .get("version") + .and_then(|v| v.as_str()) + .unwrap_or("1.0.0") + .to_string(); + + Ok((name, version)) +} + fn copy_scripts_to_temp(temp_dir: &Path) -> Result<(), Box> { let scripts_dir = temp_dir.join("scripts"); fs::create_dir_all(&scripts_dir)?; diff --git a/testprojects/itzpapalotl-node/LICENSE b/testprojects/itzpapalotl-node/LICENSE new file mode 100644 index 0000000..7a4a3ea --- /dev/null +++ b/testprojects/itzpapalotl-node/LICENSE @@ -0,0 +1,202 @@ + + Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + + END OF TERMS AND CONDITIONS + + APPENDIX: How to apply the Apache License to your work. + + To apply the Apache License to your work, attach the following + boilerplate notice, with the fields enclosed by brackets "[]" + replaced with your own identifying information. (Don't include + the brackets!) The text should be enclosed in the appropriate + comment syntax for the file format. We also recommend that a + file or class name and description of purpose be included on the + same "printed page" as the copyright notice for easier + identification within third-party archives. + + Copyright [yyyy] [name of copyright owner] + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. \ No newline at end of file diff --git a/testprojects/itzpapalotl-node/README.md b/testprojects/itzpapalotl-node/README.md new file mode 100644 index 0000000..4544e0f --- /dev/null +++ b/testprojects/itzpapalotl-node/README.md @@ -0,0 +1,11 @@ +itzpapalotl +=========== + +This ia a simple example application written for ArangoDB-Foxx. +It returns a random Aztec deity name. + +The application provides a simple REST API with the following methods: +* `GET /itzpapalotl/index`: shows an HTML overview page +* `GET /itzpapalotl/random`: returns a random deity name as JSON +* `GET /itzpapalotl/{deity}/summon`: summon a deity + diff --git a/testprojects/itzpapalotl-node/itzpapalotl.js b/testprojects/itzpapalotl-node/itzpapalotl.js new file mode 100644 index 0000000..7fc9847 --- /dev/null +++ b/testprojects/itzpapalotl-node/itzpapalotl.js @@ -0,0 +1,118 @@ +/*jslint indent: 2, nomen: true, maxlen: 100, white: true, plusplus: true, unparam: true */ +/*global require */ + +//////////////////////////////////////////////////////////////////////////////// +/// @brief An example Foxx-Application for ArangoDB +/// +/// @file +/// +/// DISCLAIMER +/// +/// Copyright 2010-2013 triagens GmbH, Cologne, Germany +/// +/// Licensed under the Apache License, Version 2.0 (the "License"); +/// you may not use this file except in compliance with the License. +/// You may obtain a copy of the License at +/// +/// http://www.apache.org/licenses/LICENSE-2.0 +/// +/// Unless required by applicable law or agreed to in writing, software +/// distributed under the License is distributed on an "AS IS" BASIS, +/// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +/// See the License for the specific language governing permissions and +/// limitations under the License. +/// +/// Copyright holder is triAGENS GmbH, Cologne, Germany +/// +/// @author Jan Steemann +/// @author Copyright 2011-2013, triAGENS GmbH, Cologne, Germany +//////////////////////////////////////////////////////////////////////////////// + + +"use strict"; + +const createRouter = require('@arangodb/node-foxx/router'); +const { context } = require('@arangodb/node-foxx/locals'); +const joi = require("joi"); +const router = createRouter(); + +context.use(router); + +// our app is about the following Aztec deities: +var deities = [ + "CentzonTotochtin", + "Chalchihuitlicue", + "Chantico", + "Chicomecoatl", + "Cihuacoatl", + "Cinteotl", + "Coatlicue", + "Coyolxauhqui", + "Huehuecoyotl", + "Huitzilopochtli", + "Ilamatecuhtli", + "Itzcoliuhqui", + "Itzpaplotl", + "Mayauel", + "Meztli", + "Mictlantecuhtli", + "Mixcoatl", + "Quetzalcoatl", + "Tezcatlipoca", + "Tialoc", + "Tlauixcalpantecuhtli", + "Tlazolteotl", + "Tonatiuh", + "Tozi", + "XipeTotec", + "Xochiquetzal", + "Xolotl", + "Yacatecuhtli" +]; + +// install index route (this is the default route mentioned in manifest.json) +// this route will create an HTML overview page +router.get('/index', function (req, res) { + res.set("content-type", "text/html"); + + var body = "

" + context.service.manifest.name + " (" + context.service.manifest.version + ")

"; + body += "

an example application demoing a few Foxx features

"; + + deities.forEach(function (deity) { + body += "summon " + deity + "
"; + }); + + body += "
pick a random Aztec deity"; + + res.body = body; +}) + .summary("prints an overview page"); + +// install route to return a random deity name in JSON +router.get('/random', function (req, res) { + var idx = Math.round(Math.random() * (deities.length - 1)); + res.json({ name: deities[idx] }); +}) + .summary("returns a random deity name"); + +// install deity-specific route for summoning +// deity name is passed as part of the URL +router.get('/:deity/summon', function (req, res) { + var deity = req.pathParams.deity; + + console.log("request to summon %s", deity); + + if (deities.indexOf(deity) === -1) { + // unknown deity + res.throw(404, "The requested deity could not be found"); + } + + console.log("summoning %s", deity); + + res.json({ name: deity, summoned: true }); +}) + .summary("summons the requested deity") + .pathParam("deity", + joi.string().required() + ); + diff --git a/testprojects/itzpapalotl-node/manifest.json b/testprojects/itzpapalotl-node/manifest.json new file mode 100644 index 0000000..2ec5de3 --- /dev/null +++ b/testprojects/itzpapalotl-node/manifest.json @@ -0,0 +1,12 @@ +{ + "name": "itzpapalotl", + "version": "1.2.0", + "author": "jsteemann", + "description": "random Aztec deity service", + "license": "Apache License, Version 2.0", + "main": "itzpapalotl.js", + "engines": { + "arangodb": "^3.0" + }, + "defaultDocument": "index" +} \ No newline at end of file diff --git a/testprojects/itzpapalotl-node/package.json b/testprojects/itzpapalotl-node/package.json new file mode 100644 index 0000000..e16dce3 --- /dev/null +++ b/testprojects/itzpapalotl-node/package.json @@ -0,0 +1,11 @@ +{ + "name": "demo-itzpapalotl", + "version": "1.0.0", + "description": "itzpapalotl ===========", + "main": "itzpapalotl.js", + "scripts": { + "test": "echo \"Error: no test specified\" && exit 1" + }, + "author": "", + "license": "ISC" +} From e53489f7a7a7cb57cd7904e3de171ed1e6b74649 Mon Sep 17 00:00:00 2001 From: bluepal-yaswanth-peravali Date: Fri, 28 Nov 2025 21:59:20 +0530 Subject: [PATCH 2/8] removed CRLF to LF conversion in node22base docker --- baseimages/Dockerfile.node22base | 7 ------- 1 file changed, 7 deletions(-) diff --git a/baseimages/Dockerfile.node22base b/baseimages/Dockerfile.node22base index 2a86e04..8fa42b2 100644 --- a/baseimages/Dockerfile.node22base +++ b/baseimages/Dockerfile.node22base @@ -1,13 +1,6 @@ FROM debian:trixie-slim COPY ./scripts /scripts - -# FIX: Convert Windows CRLF → LF and make scripts executable -RUN apt-get update && apt-get install -y dos2unix \ - && dos2unix /scripts/*.sh \ - && chmod +x /scripts/*.sh - -# Now the script will run successfully RUN /scripts/debinstall.sh # Install Node.js 22 From 2bf3e7fe0f58cacb2b102df075768cea1c7a549b Mon Sep 17 00:00:00 2001 From: bluepal-yaswanth-peravali Date: Tue, 2 Dec 2025 14:49:06 +0530 Subject: [PATCH 3/8] attended PR comments --- CHANGELOG.md | 43 ++---- Dockerfile.nodejs.template | 11 +- baseimages/Dockerfile.node22base | 43 ++++-- baseimages/scripts/entrypoint.sh | 14 +- docs/ARCHITECTURE.md | 31 ++++ scripts/prepareproject-nodejs.sh | 146 +++++-------------- src/main.rs | 130 ++++++++--------- testprojects/itzpapalotl-node/LICENSE | 202 -------------------------- 8 files changed, 192 insertions(+), 428 deletions(-) create mode 100644 docs/ARCHITECTURE.md delete mode 100644 testprojects/itzpapalotl-node/LICENSE diff --git a/CHANGELOG.md b/CHANGELOG.md index d1b16a6..417879c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -22,13 +22,12 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - Executes `prepareproject-nodejs.sh` for dependency management - **Dependency Management Script**: Added `scripts/prepareproject-nodejs.sh` - - Copies base `node_modules` from base image - - Installs project-specific dependencies from `package.json` - - Ensures `node-foxx` binary is always available with multiple safety checks - - Tracks new dependencies using SHA256 checksums - - Separates base packages from project packages - - Includes automatic recovery mechanism if base packages are removed during `npm install` - - Handles `package.json` copying to wrapper root for proper dependency installation + - Base `node_modules` at `/home/user/node_modules` is immutable and never copied + - Installs only missing or incompatible packages to project's `node_modules` + - Uses NODE_PATH for module resolution (project first, then base) + - npm automatically handles version conflicts (project version takes precedence) + - Verifies `node-foxx` binary accessibility from either location + - Keeps base image immutable for security scanning - **Project Type Detection**: Extended `detect_project_type()` to support: - `python`: Projects with `pyproject.toml` @@ -38,16 +37,15 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - **Wrapper Structure Generation**: Automatic wrapper creation for single service directories - Creates `wrapper/` directory structure - - Copies service directory into `wrapper/{service-name}/` - - Generates `services.json` automatically with mount path configuration - - Copies `package.json` to wrapper root for dependency installation + - Copies service directory directly to `/project/{service-name}/` + - Generates `services.json` automatically with mount path "/" in the service directory + - `package.json` and `services.json` are in the same directory where `node_modules` will be created - **CLI Arguments**: - - `--mount-path`: Required for `foxx-service` type, specifies the mount path for the Foxx service - **Services JSON Generation**: Added `generate_services_json()` function - Automatically generates `services.json` for single service directories - - Configures mount path and base path for Foxx services + - Configures mount path as "/" and base path for Foxx services - **Package.json Support**: Added functions to read Node.js project metadata - `read_name_from_package_json()`: Extracts project name from `package.json` @@ -89,26 +87,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Technical Details -- **Base Image Structure**: - - Base `node_modules` located at `/home/user/base_node_modules/node_modules` - - Checksums stored at `/home/user/sums_sha256` for dependency tracking - - Base packages: `@arangodb/node-foxx@^0.0.1-alpha.0`, `@arangodb/node-foxx-launcher@^0.0.1-alpha.0`, `@arangodb/arangodb@^0.0.1-alpha.0` - -- **Wrapper Structure**: - ``` - wrapper/ - ├── package.json # Copied from service for npm install - ├── services.json # Auto-generated with mount path - ├── node_modules/ # Installed dependencies (base + project) - └── {service-name}/ # Service directory - ├── package.json - └── ... - ``` - -- **Dependency Tracking**: - - Uses SHA256 checksums to identify new files vs. base files - - New project dependencies copied to `/project/node_modules/` for tracking - - Base packages remain in base image for efficiency +For detailed information about base image structure, service architecture, and module resolution, see [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md). ## [0.9.2] - Previous Release diff --git a/Dockerfile.nodejs.template b/Dockerfile.nodejs.template index a5b48ef..de3ec20 100644 --- a/Dockerfile.nodejs.template +++ b/Dockerfile.nodejs.template @@ -7,11 +7,18 @@ COPY {PROJECT_DIR} /project/{PROJECT_DIR} RUN chown -R user:user /project/{PROJECT_DIR} USER user -WORKDIR /project/{PROJECT_DIR} +WORKDIR /project/{WORKDIR} + +# Set NODE_PATH to resolve from project node_modules first, then base node_modules +# This allows npm to install only missing/incompatible packages in project directory +# while still accessing base packages from /home/user/node_modules +ENV NODE_PATH={NODE_PATH} RUN /scripts/prepareproject-nodejs.sh EXPOSE {PORT} -CMD ["node_modules/.bin/node-foxx"] +# Run node-foxx from current directory (wrapper or project root) +# Use base node-foxx binary (guaranteed to exist), NODE_PATH handles module resolution +CMD ["/home/user/node_modules/.bin/node-foxx"] diff --git a/baseimages/Dockerfile.node22base b/baseimages/Dockerfile.node22base index 8fa42b2..39d3dda 100644 --- a/baseimages/Dockerfile.node22base +++ b/baseimages/Dockerfile.node22base @@ -1,6 +1,13 @@ FROM debian:trixie-slim COPY ./scripts /scripts + +# FIX: Convert Windows CRLF → LF and make scripts executable +RUN apt-get update && apt-get install -y dos2unix \ + && dos2unix /scripts/*.sh \ + && chmod +x /scripts/*.sh + +# Now the script will run successfully RUN /scripts/debinstall.sh # Install Node.js 22 @@ -11,16 +18,34 @@ RUN curl -fsSL https://deb.nodesource.com/setup_22.x | bash - && \ WORKDIR /home/user USER user -# Create base node_modules with node-foxx dependencies -RUN mkdir -p base_node_modules && \ - cd base_node_modules && \ - npm init -y && \ - npm install @arangodb/node-foxx@^0.0.1-alpha.0 \ - @arangodb/node-foxx-launcher@^0.0.1-alpha.0 \ - @arangodb/arangodb@^0.0.1-alpha.0 +# Create base node_modules with standard packages (immutable, pre-scanned) +# This location is used by all projects and should never be modified +# Standard packages are selected to benefit most projects while keeping image size reasonable +RUN npm init -y && \ + npm install \ + # ArangoDB/Foxx core + @arangodb/node-foxx@^0.0.1-alpha.0 \ + @arangodb/node-foxx-launcher@^0.0.1-alpha.0 \ + @arangodb/arangodb@^0.0.1-alpha.0 \ + # Essential utilities + lodash \ + dayjs \ + uuid \ + dotenv \ + # HTTP clients + axios \ + # Validation + joi \ + # Logging + winston \ + # Async utilities + async \ + # Security + jsonwebtoken \ + bcrypt -# Create checksums for base node_modules -RUN find base_node_modules/node_modules -type f -print0 | \ +# Create checksums for base node_modules (for tracking, base remains immutable) +RUN find node_modules -type f -print0 | \ xargs -0 sha256sum > sums_sha256 CMD [ "/scripts/entrypoint.sh" ] diff --git a/baseimages/scripts/entrypoint.sh b/baseimages/scripts/entrypoint.sh index c31c97e..0c38dd2 100755 --- a/baseimages/scripts/entrypoint.sh +++ b/baseimages/scripts/entrypoint.sh @@ -19,25 +19,19 @@ if test -e entrypoint ; then ENTRYPOINT=$(cat entrypoint) echo Running project ... - # Check if it's a Node.js/Foxx service + # Check if it's a Node.js/Foxx service (requires both package.json and services.json) if [ -f "package.json" ] && [ -f "services.json" ]; then # Node.js/Foxx service echo "Detected Node.js/Foxx service" if [ -f "node_modules/.bin/node-foxx" ]; then exec node_modules/.bin/node-foxx + elif [ -f "/home/user/node_modules/.bin/node-foxx" ]; then + # Fallback to base node-foxx binary + exec /home/user/node_modules/.bin/node-foxx else echo "Error: node-foxx not found. Make sure node_modules are installed." exit 1 fi - elif [ -f "package.json" ]; then - # Generic Node.js service - echo "Detected Node.js service" - if [ -f "$ENTRYPOINT" ]; then - exec node "$ENTRYPOINT" - else - echo "Error: Entrypoint file not found: $ENTRYPOINT" - exit 1 - fi else # Python service (existing logic) echo "Detected Python service" diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md new file mode 100644 index 0000000..fbb14d7 --- /dev/null +++ b/docs/ARCHITECTURE.md @@ -0,0 +1,31 @@ +# Architecture + +This document describes the technical architecture and design decisions for ServiceMaker. + +## Node.js/Foxx Services + +### Base Image Structure + +- Base `node_modules` located at `/home/user/node_modules` (immutable, pre-scanned) +- Checksums stored at `/home/user/sums_sha256` for dependency tracking +- Base packages: `@arangodb/node-foxx@^0.0.1-alpha.0`, `@arangodb/node-foxx-launcher@^0.0.1-alpha.0`, `@arangodb/arangodb@^0.0.1-alpha.0` +- Base image is never modified - projects install only missing/incompatible packages to their own `node_modules` + +### Service Structure + +``` +/project/{service-name}/ +├── services.json # Auto-generated with mount path "/" +├── package.json # Service package.json +├── node_modules/ # Project-specific packages ONLY (missing/incompatible) +└── ... # Service code files +``` + +### Module Resolution + +- Base packages: `/home/user/node_modules` (immutable, pre-scanned) +- Project packages: `/project/{service-name}/node_modules` (mutable) +- NODE_PATH configured to resolve from project first, then base +- npm automatically installs only missing or incompatible packages +- Version conflicts handled automatically (project version takes precedence) + diff --git a/scripts/prepareproject-nodejs.sh b/scripts/prepareproject-nodejs.sh index afd07eb..24d497b 100644 --- a/scripts/prepareproject-nodejs.sh +++ b/scripts/prepareproject-nodejs.sh @@ -1,125 +1,59 @@ #!/bin/bash -# This is to be run in the project directory to install dependencies. -# It copies base node_modules from the base image and installs additional -# project dependencies, tracking changes similar to the Python script. +# This script installs only missing project dependencies to the project's node_modules. +# Base node_modules at /home/user/node_modules is immutable and never copied. +# npm automatically resolves from both locations via NODE_PATH. set -e -# We're already in /project/{PROJECT_DIR} from the Dockerfile WORKDIR +# We're in /project/{service-name} (WORKDIR) +# services.json is in the current directory +# package.json is in the current directory +# node_modules will be created in the current directory PROJECT_DIR=$(pwd) -# Copy base node_modules if they exist in base image -if [ -d "/home/user/base_node_modules/node_modules" ]; then - echo "Copying base node_modules..." - if [ -d "node_modules" ]; then - # Merge with existing node_modules if any - cp -r /home/user/base_node_modules/node_modules/* ./node_modules/ 2>/dev/null || true - else - cp -r /home/user/base_node_modules/node_modules ./node_modules - fi - - # Verify node-foxx binary exists after copy - if [ -f "node_modules/.bin/node-foxx" ]; then - echo "✓ Base node-foxx binary copied successfully" - else - echo "Warning: node-foxx binary not found after copying base node_modules" - fi - - # Track existing files - cd /home/user - if [ -f "sums_sha256" ]; then - sha256sum -c sums_sha256 || true - fi - cd "$PROJECT_DIR" +# Verify base node_modules exists (immutable, pre-scanned) +if [ ! -d "/home/user/node_modules" ]; then + echo "ERROR: Base node_modules not found at /home/user/node_modules" + exit 1 +fi + +echo "Base node_modules found at /home/user/node_modules (immutable)" + +# Verify base node-foxx binary exists +if [ -f "/home/user/node_modules/.bin/node-foxx" ]; then + echo "✓ Base node-foxx binary available" else - echo "Warning: Base node_modules not found at /home/user/base_node_modules/node_modules" + echo "WARNING: node-foxx binary not found in base node_modules" fi -# Install additional project dependencies if package.json exists +# Install project dependencies if package.json exists if [ -f "package.json" ]; then echo "Installing project dependencies..." - - # Always ensure base packages are installed first (they provide node-foxx binary) - # This ensures node-foxx is available even if npm install does a clean install - echo "Ensuring base node-foxx packages are installed..." - npm install --production --no-save \ - @arangodb/node-foxx@^0.0.1-alpha.0 \ - @arangodb/node-foxx-launcher@^0.0.1-alpha.0 \ - @arangodb/arangodb@^0.0.1-alpha.0 || { - echo "Warning: Failed to install base packages, continuing anyway..." - } - - # Verify node-foxx exists after installing base packages - if [ ! -f "node_modules/.bin/node-foxx" ]; then - echo "ERROR: node-foxx binary not found after installing base packages!" - echo "Listing node_modules/.bin contents:" - ls -la node_modules/.bin/ 2>/dev/null || echo "node_modules/.bin directory does not exist" - exit 1 - fi - echo "✓ Base node-foxx packages installed" - - # Install project dependencies (this should preserve base packages) + echo "npm will automatically:" + echo " - Check base node_modules at /home/user/node_modules" + echo " - Install only missing or incompatible packages" + echo " - Handle version conflicts (project version takes precedence)" + + # npm install will: + # 1. Check if packages exist in base (/home/user/node_modules) + # 2. Compare versions with package.json requirements + # 3. Install only missing or incompatible packages to ./node_modules + # 4. Handle version conflicts automatically npm install --production --no-save - # Final verification that node-foxx still exists - if [ -f "node_modules/.bin/node-foxx" ]; then - echo "✓ node-foxx binary exists after installing project dependencies" - else - echo "ERROR: node-foxx binary missing after npm install!" - echo "Reinstalling base packages..." - npm install --production --no-save \ - @arangodb/node-foxx@^0.0.1-alpha.0 \ - @arangodb/node-foxx-launcher@^0.0.1-alpha.0 \ - @arangodb/arangodb@^0.0.1-alpha.0 - - # Final check - if [ ! -f "node_modules/.bin/node-foxx" ]; then - echo "ERROR: Failed to install node-foxx binary!" - echo "Current directory: $(pwd)" - echo "Listing node_modules/.bin contents:" - ls -la node_modules/.bin/ 2>/dev/null || echo "node_modules/.bin directory does not exist" - echo "Listing node_modules contents:" - ls -la node_modules/ 2>/dev/null | head -20 - exit 1 - fi - echo "✓ node-foxx binary restored" - fi - - # Find all files in node_modules and create checksums - find node_modules -type f -print0 | xargs -0 sha256sum > /tmp/node_modules_sha256_new 2>/dev/null || true + echo "✓ Project dependencies installed" - # Find new files (files in current node_modules that weren't in base) - if [ -f "/home/user/sums_sha256" ] && [ -f "/tmp/node_modules_sha256_new" ]; then - cat /tmp/node_modules_sha256_new /home/user/sums_sha256 | sort | uniq -c | grep "^ 1 " | awk '{ print $3 }' > /tmp/newfiles || true + # Verify node-foxx is accessible (either from base or project node_modules) + if [ -f "node_modules/.bin/node-foxx" ]; then + echo "✓ node-foxx binary found in project node_modules" + elif [ -f "/home/user/node_modules/.bin/node-foxx" ]; then + echo "✓ node-foxx binary available from base node_modules (will be resolved via NODE_PATH)" else - # If no base checksums, all files are new - if [ -f "/tmp/node_modules_sha256_new" ]; then - awk '{ print $3 }' /tmp/node_modules_sha256_new > /tmp/newfiles || true - fi - fi - - # Move new files to /project/node_modules (similar to Python approach) - if [ -f "/tmp/newfiles" ]; then - mkdir -p /project/node_modules - while IFS= read -r filename; do - # Skip empty lines - [[ -z "$filename" ]] && continue - - # Check if source file exists and is relative to current dir - if [[ "$filename" == node_modules/* ]] && [ -f "$filename" ]; then - # Get the directory part of the filename - DIR=$(dirname "$filename") - - # Create the destination directory structure - mkdir -p "/project/$DIR" - - # Copy the file preserving the directory hierarchy - cp "$filename" "/project/$filename" - fi - done < "/tmp/newfiles" + echo "ERROR: node-foxx binary not found in base or project node_modules!" + exit 1 fi - - rm -f /tmp/node_modules_sha256_new /tmp/newfiles +else + echo "No package.json found, skipping dependency installation" fi echo "Node.js project prepared successfully" diff --git a/src/main.rs b/src/main.rs index f094bc5..8be69f2 100644 --- a/src/main.rs +++ b/src/main.rs @@ -6,6 +6,10 @@ use std::path::{Path, PathBuf}; use std::process::Command; use toml::Value; +// Default base images +const DEFAULT_PYTHON_BASE_IMAGE: &str = "arangodb/py13base:latest"; +const DEFAULT_NODEJS_BASE_IMAGE: &str = "arangodb/node22base:latest"; + // Embedded chart files struct ChartFile { path: &'static str, @@ -73,8 +77,8 @@ struct Args { project_home: Option, /// Base Docker image - #[arg(long, default_value = "arangodb/py13base:latest")] - base_image: String, + #[arg(long)] + base_image: Option, /// Exposed port number #[arg(long)] @@ -95,15 +99,14 @@ struct Args { /// Whether to create a tar.gz file with project files and virtual environment changes #[arg(long, default_value = "false")] make_tar_gz: bool, - - /// Mount path for the service (required for Foxx services, e.g., /itz) - #[arg(long)] - mount_path: Option, } fn main() -> Result<(), Box> { let mut args = Args::parse(); + // Track if base_image was explicitly set by user + let base_image_explicitly_set = args.base_image.is_some(); + // Get project home first (prompt if needed) if args.project_home.is_none() { let path_str = prompt("Project home path")?; @@ -141,6 +144,11 @@ fn main() -> Result<(), Box> { if args.entrypoint.is_none() { args.entrypoint = Some(prompt("Python entrypoint script (e.g., main.py)")?); } + + // Set default base image for Python if not explicitly set + if !base_image_explicitly_set { + args.base_image = Some(DEFAULT_PYTHON_BASE_IMAGE.to_string()); + } } "foxx" => { // Multi-service structure (has services.json) @@ -151,13 +159,13 @@ fn main() -> Result<(), Box> { args.name = Some(name); } - // Set default base image for Node.js if not specified - if args.base_image == "arangodb/py13base:latest" { - args.base_image = "arangodb/node22base:latest".to_string(); + // Set default base image for Node.js if not explicitly set + if !base_image_explicitly_set { + args.base_image = Some(DEFAULT_NODEJS_BASE_IMAGE.to_string()); } } - "foxx-service" | "nodejs" => { - // Single service directory - will create wrapper structure + "foxx-service" => { + // Single service directory - will generate services.json let service_name = project_home.file_name().unwrap().to_str().unwrap(); // Try to get name from package.json if not provided @@ -169,20 +177,9 @@ fn main() -> Result<(), Box> { } } - // Set default base image for Node.js if not specified - if args.base_image == "arangodb/py13base:latest" { - args.base_image = "arangodb/node22base:latest".to_string(); - } - - // Prompt for mount path if not provided (for foxx-service) - if project_type == "foxx-service" && args.mount_path.is_none() { - let default_mount = format!("/{}", service_name.to_lowercase()); - let mount_input = prompt(&format!("Mount path (default: {})", default_mount))?; - args.mount_path = Some(if mount_input.is_empty() { - default_mount - } else { - mount_input - }); + // Set default base image for Node.js if not explicitly set + if !base_image_explicitly_set { + args.base_image = Some(DEFAULT_NODEJS_BASE_IMAGE.to_string()); } } _ => { @@ -210,21 +207,19 @@ fn main() -> Result<(), Box> { let project_home = args.project_home.as_ref().unwrap(); let image_name = args.image_name.as_ref().unwrap(); let port = args.port.unwrap(); + let base_image = args.base_image.as_ref().unwrap(); println!("\n=== Configuration ==="); println!("Project name: {}", name); println!("Project type: {}", project_type); println!("Project home: {}", project_home.display()); println!("Project directory name: {}", initial_project_dir); - println!("Base image: {}", args.base_image); + println!("Base image: {}", base_image); println!("Port: {}", port); println!("Image name: {}", image_name); if let Some(ref entrypoint) = args.entrypoint { println!("Entrypoint: {}", entrypoint); } - if let Some(ref mount_path) = args.mount_path { - println!("Mount path: {}", mount_path); - } println!("Push: {}", args.push); println!("Make tar.gz: {}", args.make_tar_gz); println!("=====================\n"); @@ -242,42 +237,27 @@ fn main() -> Result<(), Box> { // Copy scripts to temp directory with executable permissions copy_scripts_to_temp(&temp_dir)?; - // Handle Node.js single service directory differently - create wrapper structure - let project_dir = if project_type == "foxx-service" { + // Handle Node.js single service directory - copy directly and generate services.json + let (service_name_opt, project_dir) = if project_type == "foxx-service" { let service_name = initial_project_dir; - // Create wrapper directory - let wrapper_dir = temp_dir.join("wrapper"); - fs::create_dir_all(&wrapper_dir)?; - - // Copy service directory into wrapper - let service_dest = wrapper_dir.join(service_name); + // Copy service directory directly (no wrapper folder) + let project_dest = temp_dir.join(service_name); println!( - "Creating wrapper structure: copying {} to wrapper/{}", + "Copying service from {} to {}", project_home.display(), - service_name + project_dest.display() ); - copy_dir_recursive(project_home, &service_dest)?; - - // Copy package.json from service to wrapper root for dependency installation - let service_package_json = service_dest.join("package.json"); - if service_package_json.exists() { - let wrapper_package_json = wrapper_dir.join("package.json"); - fs::copy(&service_package_json, &wrapper_package_json)?; - println!("Copied package.json to wrapper root for dependency installation"); - } + copy_dir_recursive(project_home, &project_dest)?; - // Generate services.json with mount path - let mount_path = args - .mount_path - .as_deref() - .ok_or("Mount path is required for foxx-service")?; - let services_json_content = generate_services_json(service_name, mount_path); - let services_json_path = wrapper_dir.join("services.json"); + // Generate services.json with mount path "/" directly in service directory + let services_json_content = generate_services_json(service_name); + let services_json_path = project_dest.join("services.json"); fs::write(&services_json_path, services_json_content)?; println!("Generated services.json: {}", services_json_path.display()); - "wrapper".to_string() + // Store service_name for later use in Dockerfile generation + (Some(service_name.to_string()), service_name.to_string()) } else { // Normal case - copy project directory as-is let project_dest = temp_dir.join(initial_project_dir); @@ -287,27 +267,33 @@ fn main() -> Result<(), Box> { project_dest.display() ); copy_dir_recursive(project_home, &project_dest)?; - initial_project_dir.to_string() + (None, initial_project_dir.to_string()) }; // Choose Dockerfile template and modify based on project type let modified_dockerfile = match project_type.as_str() { "python" => { - let python_version = extract_python_version(&args.base_image); + let python_version = extract_python_version(base_image); let entrypoint = args.entrypoint.as_ref().unwrap(); let dockerfile_template = include_str!("../Dockerfile.template"); modify_dockerfile_python( dockerfile_template, - &args.base_image, + base_image, &project_dir, entrypoint, port, &python_version, ) } - "foxx" | "foxx-service" | "nodejs" => { + "foxx" | "foxx-service" => { let dockerfile_template = include_str!("../Dockerfile.nodejs.template"); - modify_dockerfile_nodejs(dockerfile_template, &args.base_image, &project_dir, port) + modify_dockerfile_nodejs( + dockerfile_template, + base_image, + &project_dir, + port, + service_name_opt.as_deref(), + ) } _ => return Err("Unsupported project type".into()), }; @@ -443,7 +429,7 @@ fn main() -> Result<(), Box> { println!("Version from pyproject.toml: {}", ver); (name, ver) } - "foxx" | "foxx-service" | "nodejs" => { + "foxx" | "foxx-service" => { let (name, ver) = read_service_info_from_package_json(project_home)?; println!("Service name from package.json: {}", name); println!("Version from package.json: {}", ver); @@ -548,11 +534,20 @@ fn modify_dockerfile_nodejs( base_image: &str, project_dir: &str, port: u16, + _service_name: Option<&str>, ) -> String { + // Simplified structure: + // - COPY copies the service directory directly (services.json is inside) + // - WORKDIR is /project/{service-name} (where services.json is located) + // - node_modules is in /project/{service-name}/node_modules + let node_path = format!("/project/{}/node_modules:/home/user/node_modules", project_dir); + template .replace("{BASE_IMAGE}", base_image) .replace("{PROJECT_DIR}", project_dir) + .replace("{WORKDIR}", project_dir) .replace("{PORT}", &port.to_string()) + .replace("{NODE_PATH}", &node_path) } fn detect_project_type(project_home: &Path) -> Result> { @@ -576,15 +571,16 @@ fn detect_project_type(project_home: &Path) -> Result String { +fn generate_services_json(_service_name: &str) -> String { + // basePath is "." because services.json is in the same directory as the service + // and node-foxx runs from that directory (WORKDIR) format!( r#"[ {{ - "mount": "{}", - "basePath": "{}" + "mount": "/", + "basePath": "." }} -]"#, - mount_path, service_name +]"# ) } diff --git a/testprojects/itzpapalotl-node/LICENSE b/testprojects/itzpapalotl-node/LICENSE deleted file mode 100644 index 7a4a3ea..0000000 --- a/testprojects/itzpapalotl-node/LICENSE +++ /dev/null @@ -1,202 +0,0 @@ - - Apache License - Version 2.0, January 2004 - http://www.apache.org/licenses/ - - TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION - - 1. Definitions. - - "License" shall mean the terms and conditions for use, reproduction, - and distribution as defined by Sections 1 through 9 of this document. - - "Licensor" shall mean the copyright owner or entity authorized by - the copyright owner that is granting the License. - - "Legal Entity" shall mean the union of the acting entity and all - other entities that control, are controlled by, or are under common - control with that entity. For the purposes of this definition, - "control" means (i) the power, direct or indirect, to cause the - direction or management of such entity, whether by contract or - otherwise, or (ii) ownership of fifty percent (50%) or more of the - outstanding shares, or (iii) beneficial ownership of such entity. - - "You" (or "Your") shall mean an individual or Legal Entity - exercising permissions granted by this License. - - "Source" form shall mean the preferred form for making modifications, - including but not limited to software source code, documentation - source, and configuration files. - - "Object" form shall mean any form resulting from mechanical - transformation or translation of a Source form, including but - not limited to compiled object code, generated documentation, - and conversions to other media types. - - "Work" shall mean the work of authorship, whether in Source or - Object form, made available under the License, as indicated by a - copyright notice that is included in or attached to the work - (an example is provided in the Appendix below). - - "Derivative Works" shall mean any work, whether in Source or Object - form, that is based on (or derived from) the Work and for which the - editorial revisions, annotations, elaborations, or other modifications - represent, as a whole, an original work of authorship. For the purposes - of this License, Derivative Works shall not include works that remain - separable from, or merely link (or bind by name) to the interfaces of, - the Work and Derivative Works thereof. - - "Contribution" shall mean any work of authorship, including - the original version of the Work and any modifications or additions - to that Work or Derivative Works thereof, that is intentionally - submitted to Licensor for inclusion in the Work by the copyright owner - or by an individual or Legal Entity authorized to submit on behalf of - the copyright owner. For the purposes of this definition, "submitted" - means any form of electronic, verbal, or written communication sent - to the Licensor or its representatives, including but not limited to - communication on electronic mailing lists, source code control systems, - and issue tracking systems that are managed by, or on behalf of, the - Licensor for the purpose of discussing and improving the Work, but - excluding communication that is conspicuously marked or otherwise - designated in writing by the copyright owner as "Not a Contribution." - - "Contributor" shall mean Licensor and any individual or Legal Entity - on behalf of whom a Contribution has been received by Licensor and - subsequently incorporated within the Work. - - 2. Grant of Copyright License. Subject to the terms and conditions of - this License, each Contributor hereby grants to You a perpetual, - worldwide, non-exclusive, no-charge, royalty-free, irrevocable - copyright license to reproduce, prepare Derivative Works of, - publicly display, publicly perform, sublicense, and distribute the - Work and such Derivative Works in Source or Object form. - - 3. Grant of Patent License. Subject to the terms and conditions of - this License, each Contributor hereby grants to You a perpetual, - worldwide, non-exclusive, no-charge, royalty-free, irrevocable - (except as stated in this section) patent license to make, have made, - use, offer to sell, sell, import, and otherwise transfer the Work, - where such license applies only to those patent claims licensable - by such Contributor that are necessarily infringed by their - Contribution(s) alone or by combination of their Contribution(s) - with the Work to which such Contribution(s) was submitted. If You - institute patent litigation against any entity (including a - cross-claim or counterclaim in a lawsuit) alleging that the Work - or a Contribution incorporated within the Work constitutes direct - or contributory patent infringement, then any patent licenses - granted to You under this License for that Work shall terminate - as of the date such litigation is filed. - - 4. Redistribution. You may reproduce and distribute copies of the - Work or Derivative Works thereof in any medium, with or without - modifications, and in Source or Object form, provided that You - meet the following conditions: - - (a) You must give any other recipients of the Work or - Derivative Works a copy of this License; and - - (b) You must cause any modified files to carry prominent notices - stating that You changed the files; and - - (c) You must retain, in the Source form of any Derivative Works - that You distribute, all copyright, patent, trademark, and - attribution notices from the Source form of the Work, - excluding those notices that do not pertain to any part of - the Derivative Works; and - - (d) If the Work includes a "NOTICE" text file as part of its - distribution, then any Derivative Works that You distribute must - include a readable copy of the attribution notices contained - within such NOTICE file, excluding those notices that do not - pertain to any part of the Derivative Works, in at least one - of the following places: within a NOTICE text file distributed - as part of the Derivative Works; within the Source form or - documentation, if provided along with the Derivative Works; or, - within a display generated by the Derivative Works, if and - wherever such third-party notices normally appear. The contents - of the NOTICE file are for informational purposes only and - do not modify the License. You may add Your own attribution - notices within Derivative Works that You distribute, alongside - or as an addendum to the NOTICE text from the Work, provided - that such additional attribution notices cannot be construed - as modifying the License. - - You may add Your own copyright statement to Your modifications and - may provide additional or different license terms and conditions - for use, reproduction, or distribution of Your modifications, or - for any such Derivative Works as a whole, provided Your use, - reproduction, and distribution of the Work otherwise complies with - the conditions stated in this License. - - 5. Submission of Contributions. Unless You explicitly state otherwise, - any Contribution intentionally submitted for inclusion in the Work - by You to the Licensor shall be under the terms and conditions of - this License, without any additional terms or conditions. - Notwithstanding the above, nothing herein shall supersede or modify - the terms of any separate license agreement you may have executed - with Licensor regarding such Contributions. - - 6. Trademarks. This License does not grant permission to use the trade - names, trademarks, service marks, or product names of the Licensor, - except as required for reasonable and customary use in describing the - origin of the Work and reproducing the content of the NOTICE file. - - 7. Disclaimer of Warranty. Unless required by applicable law or - agreed to in writing, Licensor provides the Work (and each - Contributor provides its Contributions) on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or - implied, including, without limitation, any warranties or conditions - of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A - PARTICULAR PURPOSE. You are solely responsible for determining the - appropriateness of using or redistributing the Work and assume any - risks associated with Your exercise of permissions under this License. - - 8. Limitation of Liability. In no event and under no legal theory, - whether in tort (including negligence), contract, or otherwise, - unless required by applicable law (such as deliberate and grossly - negligent acts) or agreed to in writing, shall any Contributor be - liable to You for damages, including any direct, indirect, special, - incidental, or consequential damages of any character arising as a - result of this License or out of the use or inability to use the - Work (including but not limited to damages for loss of goodwill, - work stoppage, computer failure or malfunction, or any and all - other commercial damages or losses), even if such Contributor - has been advised of the possibility of such damages. - - 9. Accepting Warranty or Additional Liability. While redistributing - the Work or Derivative Works thereof, You may choose to offer, - and charge a fee for, acceptance of support, warranty, indemnity, - or other liability obligations and/or rights consistent with this - License. However, in accepting such obligations, You may act only - on Your own behalf and on Your sole responsibility, not on behalf - of any other Contributor, and only if You agree to indemnify, - defend, and hold each Contributor harmless for any liability - incurred by, or claims asserted against, such Contributor by reason - of your accepting any such warranty or additional liability. - - END OF TERMS AND CONDITIONS - - APPENDIX: How to apply the Apache License to your work. - - To apply the Apache License to your work, attach the following - boilerplate notice, with the fields enclosed by brackets "[]" - replaced with your own identifying information. (Don't include - the brackets!) The text should be enclosed in the appropriate - comment syntax for the file format. We also recommend that a - file or class name and description of purpose be included on the - same "printed page" as the copyright notice for easier - identification within third-party archives. - - Copyright [yyyy] [name of copyright owner] - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. \ No newline at end of file From 991c44e6428cf71eb3c8b230d199077345ae6913 Mon Sep 17 00:00:00 2001 From: bluepal-yaswanth-peravali Date: Tue, 2 Dec 2025 14:57:20 +0530 Subject: [PATCH 4/8] Fixed clipy warning --- baseimages/Dockerfile.node22base | 7 ------- src/main.rs | 10 ++++------ 2 files changed, 4 insertions(+), 13 deletions(-) diff --git a/baseimages/Dockerfile.node22base b/baseimages/Dockerfile.node22base index 39d3dda..3c44229 100644 --- a/baseimages/Dockerfile.node22base +++ b/baseimages/Dockerfile.node22base @@ -1,13 +1,6 @@ FROM debian:trixie-slim COPY ./scripts /scripts - -# FIX: Convert Windows CRLF → LF and make scripts executable -RUN apt-get update && apt-get install -y dos2unix \ - && dos2unix /scripts/*.sh \ - && chmod +x /scripts/*.sh - -# Now the script will run successfully RUN /scripts/debinstall.sh # Install Node.js 22 diff --git a/src/main.rs b/src/main.rs index 8be69f2..369263c 100644 --- a/src/main.rs +++ b/src/main.rs @@ -574,14 +574,12 @@ fn detect_project_type(project_home: &Path) -> Result String { // basePath is "." because services.json is in the same directory as the service // and node-foxx runs from that directory (WORKDIR) - format!( - r#"[ - {{ + r#"[ + { "mount": "/", "basePath": "." - }} -]"# - ) + } +]"#.to_string() } fn read_name_from_package_json(project_home: &Path) -> Result> { From 45d7bf063036cdcc5ba34bb7e3aa8fa1033bea89 Mon Sep 17 00:00:00 2001 From: bluepal-yaswanth-peravali Date: Tue, 2 Dec 2025 15:29:58 +0530 Subject: [PATCH 5/8] Added versions for npm packages in node base dockerfile --- baseimages/Dockerfile.node22base | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/baseimages/Dockerfile.node22base b/baseimages/Dockerfile.node22base index 3c44229..715c250 100644 --- a/baseimages/Dockerfile.node22base +++ b/baseimages/Dockerfile.node22base @@ -21,21 +21,21 @@ RUN npm init -y && \ @arangodb/node-foxx-launcher@^0.0.1-alpha.0 \ @arangodb/arangodb@^0.0.1-alpha.0 \ # Essential utilities - lodash \ - dayjs \ - uuid \ - dotenv \ + lodash@^4.17.21 \ + dayjs@^1.11.10 \ + uuid@^9.0.1 \ + dotenv@^16.4.5 \ # HTTP clients - axios \ + axios@^1.7.2 \ # Validation - joi \ + joi@^17.13.3 \ # Logging - winston \ + winston@^3.15.0 \ # Async utilities - async \ + async@^3.2.5 \ # Security - jsonwebtoken \ - bcrypt + jsonwebtoken@^9.0.2 \ + bcrypt@^5.1.1 # Create checksums for base node_modules (for tracking, base remains immutable) RUN find node_modules -type f -print0 | \ From de1bb90e0d8be31180c6097a59fe0fe5bde59d9b Mon Sep 17 00:00:00 2001 From: bluepal-yaswanth-peravali Date: Tue, 2 Dec 2025 16:25:12 +0530 Subject: [PATCH 6/8] Implement dependency checking to avoid duplicating base packages --- CHANGELOG.md | 74 +++++++---- baseimages/Dockerfile.node22base | 2 + docs/ARCHITECTURE.md | 203 +++++++++++++++++++++++++++-- scripts/check-base-dependencies.js | 94 +++++++++++++ scripts/prepareproject-nodejs.sh | 69 +++++++--- src/main.rs | 4 + 6 files changed, 388 insertions(+), 58 deletions(-) create mode 100644 scripts/check-base-dependencies.js diff --git a/CHANGELOG.md b/CHANGELOG.md index 417879c..61a9b62 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -10,65 +10,73 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Added #### Node.js/Foxx Service Support -- **Node.js Base Image**: Added `Dockerfile.node22base` for Node.js 22 base image with pre-installed ArangoDB packages +- **Node.js Base Image**: Added `Dockerfile.node22base` for Node.js 22 base image with pre-installed packages - Installs Node.js 22 from NodeSource - - Pre-installs `@arangodb/node-foxx`, `@arangodb/node-foxx-launcher`, and `@arangodb/arangodb` packages - - Creates base `node_modules` with checksums for dependency tracking + - Pre-installs ArangoDB packages: `@arangodb/node-foxx`, `@arangodb/node-foxx-launcher`, `@arangodb/arangodb` + - Pre-installs standard packages with version pinning: `lodash`, `dayjs`, `uuid`, `dotenv`, `axios`, `joi`, `winston`, `async`, `jsonwebtoken`, `bcrypt`, `semver` + - Creates base `node_modules` at `/home/user/node_modules` with checksums for dependency tracking + - Base image is immutable and pre-scanned for security vulnerabilities - Added to `baseimages/imagelist.txt` as `node22base` - **Node.js Dockerfile Template**: Created `Dockerfile.nodejs.template` for building Node.js/Foxx service images - - Supports wrapper structure for single service directories + - Copies service directory directly to `/project/{service-name}/` - Configures working directory and user permissions + - Sets NODE_PATH environment variable for module resolution - Executes `prepareproject-nodejs.sh` for dependency management -- **Dependency Management Script**: Added `scripts/prepareproject-nodejs.sh` +- **Dependency Management Script**: Added `scripts/prepareproject-nodejs.sh` and `scripts/check-base-dependencies.js` - Base `node_modules` at `/home/user/node_modules` is immutable and never copied - - Installs only missing or incompatible packages to project's `node_modules` + - Pre-install check: `check-base-dependencies.js` analyzes project dependencies against base packages + - Version compatibility: Uses `semver` to verify if base package versions satisfy project requirements + - Avoids duplication: Only installs packages that are missing or have incompatible versions - Uses NODE_PATH for module resolution (project first, then base) - - npm automatically handles version conflicts (project version takes precedence) - Verifies `node-foxx` binary accessibility from either location - Keeps base image immutable for security scanning + - Results in smaller project `node_modules` and `project.tar.gz` files - **Project Type Detection**: Extended `detect_project_type()` to support: - `python`: Projects with `pyproject.toml` - - `foxx`: Multi-service projects with `package.json` and `services.json` - - `foxx-service`: Single service directory with `package.json` (creates wrapper structure) - - `nodejs`: Generic Node.js projects + - `foxx`: Multi-service projects with `package.json` and `services.json` (both required) + - `foxx-service`: Single service directory with `package.json` only (auto-generates `services.json`) + - Execution stops with error if `services.json` is missing for Node.js projects -- **Wrapper Structure Generation**: Automatic wrapper creation for single service directories - - Creates `wrapper/` directory structure - - Copies service directory directly to `/project/{service-name}/` - - Generates `services.json` automatically with mount path "/" in the service directory +- **Service Structure Generation**: Simplified structure for single service directories + - Copies service directory directly to `/project/{service-name}/` (no wrapper folder) + - Generates `services.json` automatically with mount path "/" and basePath "." in the service directory - `package.json` and `services.json` are in the same directory where `node_modules` will be created -- **CLI Arguments**: - - **Services JSON Generation**: Added `generate_services_json()` function - - Automatically generates `services.json` for single service directories - - Configures mount path as "/" and base path for Foxx services + - Automatically generates `services.json` for single service directories (`foxx-service` type) + - Configures mount path as "/" (routing handled by Helm chart at deployment) + - Sets basePath to "." (relative to WORKDIR where `node-foxx` runs) - **Package.json Support**: Added functions to read Node.js project metadata - `read_name_from_package_json()`: Extracts project name from `package.json` - `read_service_info_from_package_json()`: Extracts name and version for Helm charts - **Entrypoint Enhancement**: Updated `baseimages/scripts/entrypoint.sh` to support Node.js/Foxx services - - Detects service type based on project files - - Automatically runs `node-foxx` for Foxx services - - Falls back to generic Node.js execution for non-Foxx services + - Detects Foxx services by checking for both `package.json` and `services.json` + - Automatically runs `node-foxx` for Foxx services (checks project `node_modules` first, then base) + - Only supports Foxx services (requires both `package.json` and `services.json`) - Maintains backward compatibility with Python services - **Test Service**: Added `itzpapalotl-node` test service in `testprojects/` - Example Node.js/Foxx service for testing ServiceMaker functionality - - Demonstrates wrapper structure generation and dependency management + - Demonstrates service structure generation and dependency management ### Changed -- **Main Application Logic**: Extended `src/main.rs` to support Node.js projects - - Added project type detection for Node.js/Foxx services - - Updated file copying logic to handle wrapper structure - - Modified Dockerfile generation to use appropriate template based on project type - - Updated Helm chart generation to support Node.js projects - - Added `prepareproject-nodejs.sh` to embedded scripts list +- **Main Application Logic**: Extended `src/main.rs` to support Node.js/Foxx projects + - Added project type detection requiring both `package.json` and `services.json` for `foxx` type + - Error handling: execution stops if `services.json` is missing for Node.js projects + - Simplified file copying: projects are copied as-is (no wrapper structure generation) + - Base image default handling: Introduced compile-time constants (`DEFAULT_PYTHON_BASE_IMAGE`, `DEFAULT_NODEJS_BASE_IMAGE`) + - Explicit user intent tracking: Changed `base_image` to `Option` to detect explicit user choices + - Smart defaults: Only sets project-type-specific defaults when user hasn't explicitly set base image + - Modified Dockerfile generation to use Node.js template for Foxx projects + - Updated Helm chart generation to support Node.js/Foxx projects + - Added `prepareproject-nodejs.sh` and `check-base-dependencies.js` to embedded scripts list + - No entrypoint required for Foxx services (uses `node-foxx` from base image) - **Entrypoint Script**: Enhanced `baseimages/scripts/entrypoint.sh` to support Node.js/Foxx services - Added service type detection based on project files @@ -78,9 +86,19 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - **File Copying Logic**: Updated `copy_dir_recursive()` to skip `node_modules` directories - Prevents copying local `node_modules` which should be installed in Docker build + - Ensures only project source code is copied, dependencies are installed fresh in Docker ### Fixed +- **services.json basePath**: Fixed incorrect basePath in generated `services.json` + - Changed from service name to "." (current directory) + - Fixes path resolution issue where `node-foxx` was looking in wrong location for `manifest.json` + +- **Base Image Default Handling**: Improved robustness of base image default selection + - Replaced magic string comparisons with compile-time constants + - Added explicit tracking of user intent (whether base image was explicitly set) + - Prevents breakage if default values change in the future + - **Windows Compatibility**: Fixed Windows build issues in `src/main.rs` - Added conditional compilation for Unix-specific file permissions (`#[cfg(unix)]`) - Windows builds now skip `set_mode()` calls that are Unix-only diff --git a/baseimages/Dockerfile.node22base b/baseimages/Dockerfile.node22base index 715c250..e023852 100644 --- a/baseimages/Dockerfile.node22base +++ b/baseimages/Dockerfile.node22base @@ -20,6 +20,8 @@ RUN npm init -y && \ @arangodb/node-foxx@^0.0.1-alpha.0 \ @arangodb/node-foxx-launcher@^0.0.1-alpha.0 \ @arangodb/arangodb@^0.0.1-alpha.0 \ + # Dependency checking utility + semver@^7.6.3 \ # Essential utilities lodash@^4.17.21 \ dayjs@^1.11.10 \ diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index fbb14d7..bbb38ae 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -6,26 +6,201 @@ This document describes the technical architecture and design decisions for Serv ### Base Image Structure -- Base `node_modules` located at `/home/user/node_modules` (immutable, pre-scanned) -- Checksums stored at `/home/user/sums_sha256` for dependency tracking -- Base packages: `@arangodb/node-foxx@^0.0.1-alpha.0`, `@arangodb/node-foxx-launcher@^0.0.1-alpha.0`, `@arangodb/arangodb@^0.0.1-alpha.0` -- Base image is never modified - projects install only missing/incompatible packages to their own `node_modules` +The base image (`arangodb/node22base:latest`) provides an immutable foundation for all Node.js/Foxx services: + +**File System Layout:** +``` +/home/user/ +├── node_modules/ # Immutable base packages (pre-scanned for security) +│ ├── @arangodb/ +│ │ ├── node-foxx@^0.0.1-alpha.0 +│ │ ├── node-foxx-launcher@^0.0.1-alpha.0 +│ │ └── arangodb@^0.0.1-alpha.0 +│ ├── lodash@^4.17.21 +│ ├── dayjs@^1.11.10 +│ ├── axios@^1.7.2 +│ ├── joi@^17.13.3 +│ ├── winston@^3.15.0 +│ ├── semver@^7.6.3 # Required for dependency checking +│ └── ... (other standard packages) +└── sums_sha256 # SHA256 checksums of all base node_modules files +``` + +**Key Properties:** +- **Immutability**: Base `node_modules` is never modified after image creation +- **Pre-scanning**: Base packages are security-scanned before deployment +- **Version Pinning**: All base packages use caret ranges (e.g., `^4.17.21`) for reproducible builds +- **Checksum Tracking**: `sums_sha256` file enables change detection and verification ### Service Structure +Each service is deployed with the following structure: + ``` /project/{service-name}/ -├── services.json # Auto-generated with mount path "/" -├── package.json # Service package.json -├── node_modules/ # Project-specific packages ONLY (missing/incompatible) -└── ... # Service code files +├── services.json # Auto-generated Foxx service configuration +│ # Format: [{"mount": "/", "basePath": "."}] +├── package.json # Project dependencies and metadata +├── node_modules/ # Project-specific packages ONLY +│ # Contains only packages that are: +│ # - Missing from base node_modules +│ # - Have incompatible versions with base +└── ... # Service code files (manifest.json, routes, etc.) +``` + +**Build-Time Generation:** +- `services.json` is auto-generated for `foxx-service` project types +- Mount path is hardcoded to `"/"` (routing handled by Helm chart at deployment) +- `basePath` is set to `"."` (relative to WORKDIR where `node-foxx` runs) + +### Dependency Resolution Algorithm + +The dependency resolution process ensures no duplication while maintaining compatibility: + +**Phase 1: Pre-Install Analysis (`check-base-dependencies.js`)** + +1. **Read Project Dependencies**: Parse `package.json` to extract all `dependencies` +2. **Check Base Availability**: For each dependency: + - Check if package exists at `/home/user/node_modules/{package-name}/` + - Read version from base package's `package.json` +3. **Version Compatibility Check**: Use `semver.satisfies()` to verify: + - Base version satisfies project's version range (e.g., `^4.17.21` satisfies `^4.17.0`) + - If satisfied → skip installation (use base version) + - If not satisfied → add to install list +4. **Output**: JSON array of packages to install (missing or incompatible) + +**Phase 2: Selective Installation (`prepareproject-nodejs.sh`)** + +1. **Parse Install List**: Extract package specifications from JSON output +2. **Install Missing/Incompatible**: Run `npm install --production --no-save` for each: + ```bash + npm install --production --no-save package-name@version-range + ``` +3. **Result**: Project `node_modules` contains only packages not available/compatible in base + +**Example Scenario:** +``` +Project requires: lodash@^4.17.0, axios@^1.7.0, custom-pkg@^1.0.0 +Base has: lodash@4.17.21, axios@1.7.2 + +Result: +- lodash: ✓ Base version 4.17.21 satisfies ^4.17.0 → Use base +- axios: ✓ Base version 1.7.2 satisfies ^1.7.0 → Use base +- custom-pkg: ✗ Not in base → Install to project/node_modules +``` + +### Module Resolution at Runtime + +Node.js module resolution uses `NODE_PATH` environment variable: + +**Configuration:** +```dockerfile +ENV NODE_PATH=/project/{service-name}/node_modules:/home/user/node_modules +``` + +**Resolution Order:** +1. **Project `node_modules`** (checked first) + - Contains project-specific packages + - Takes precedence for version conflicts +2. **Base `node_modules`** (checked second) + - Contains standard packages + - Used when package not found in project + +**Runtime Behavior:** +- `require('lodash')` → Resolves from base (if compatible version exists) +- `require('custom-pkg')` → Resolves from project (not in base) +- `require('axios')` → Resolves from base (if compatible) or project (if incompatible version installed) + +### Build Process Flow + +**Dockerfile Build Steps:** + +1. **Base Image**: `FROM arangodb/node22base:latest` +2. **Copy Scripts**: Embed `prepareproject-nodejs.sh` and `check-base-dependencies.js` +3. **Copy Project**: Copy service directory to `/project/{service-name}/` + - Local `node_modules` are excluded (not copied) +4. **Set Working Directory**: `WORKDIR /project/{service-name}` +5. **Configure NODE_PATH**: Set environment variable for module resolution +6. **Run Preparation Script**: Execute `prepareproject-nodejs.sh` + - Analyzes dependencies + - Installs only missing/incompatible packages +7. **Set Entrypoint**: `CMD ["/home/user/node_modules/.bin/node-foxx"]` + +**Script Execution Flow:** + +``` +prepareproject-nodejs.sh +├── Verify base node_modules exists +├── Run check-base-dependencies.js +│ ├── Parse package.json +│ ├── Check each dependency against base +│ ├── Verify version compatibility (semver) +│ └── Output JSON: packages to install +├── Parse JSON output +├── Install missing/incompatible packages +└── Verify node-foxx binary accessibility ``` -### Module Resolution +### Runtime Execution + +**Container Startup:** + +1. **Entrypoint**: `/home/user/node_modules/.bin/node-foxx` (from base image) +2. **Working Directory**: `/project/{service-name}/` (where `services.json` is located) +3. **Service Discovery**: `node-foxx` reads `services.json` to determine: + - Mount path: `"/"` + - Base path: `"."` (current directory) +4. **Module Resolution**: Uses `NODE_PATH` to resolve dependencies from both locations +5. **Service Launch**: Foxx service starts with access to both base and project packages + +### Security Considerations + +**Base Image Scanning:** +- Base `node_modules` is pre-scanned for vulnerabilities before deployment +- Checksums (`sums_sha256`) enable change detection +- Immutability ensures base packages cannot be modified + +**Project Package Scanning:** +- Only project-specific packages need scanning (smaller surface area) +- Project `node_modules` is mutable and can be scanned separately +- Version conflicts resolved by installing project version (explicit choice) + +**Benefits:** +- Reduced scan time (only scan project packages) +- Base image can be pre-approved and reused +- Clear separation between base (trusted) and project (variable) packages + +### Performance Implications + +**Build Time:** +- Faster builds: Only install missing/incompatible packages +- Reduced network: Fewer packages to download +- Smaller layers: Project `node_modules` is minimal + +**Runtime:** +- Faster startup: Fewer packages to load +- Smaller images: Reduced image size +- Efficient resolution: NODE_PATH lookup is fast (filesystem-based) + +**Storage:** +- Smaller `project.tar.gz`: Only project-specific packages archived +- Base image reuse: Single base image shared across all services +- Layer caching: Base image layers cached, only project layer changes + +### Technical Constraints + +**Version Compatibility:** +- Uses semantic versioning (semver) for compatibility checks +- Caret ranges (^) in base allow patch/minor updates +- Project can override with specific versions if needed + +**NODE_PATH Limitations:** +- Only affects `require()` resolution, not `npm install` behavior +- Requires explicit pre-install check to avoid duplication +- Binary resolution: `.bin` executables must be in accessible `node_modules` -- Base packages: `/home/user/node_modules` (immutable, pre-scanned) -- Project packages: `/project/{service-name}/node_modules` (mutable) -- NODE_PATH configured to resolve from project first, then base -- npm automatically installs only missing or incompatible packages -- Version conflicts handled automatically (project version takes precedence) +**File System:** +- Base `node_modules` must be read-only (immutability requirement) +- Project `node_modules` must be writable (for installation) +- Both locations must be accessible via NODE_PATH diff --git a/scripts/check-base-dependencies.js b/scripts/check-base-dependencies.js new file mode 100644 index 0000000..0e47250 --- /dev/null +++ b/scripts/check-base-dependencies.js @@ -0,0 +1,94 @@ +#!/usr/bin/env node +/** + * Checks which dependencies from package.json are already satisfied by base node_modules. + * Returns a list of packages that need to be installed (missing or incompatible versions). + * + * This script ensures we don't duplicate packages that already exist in the base image. + */ + +const fs = require('fs'); +const path = require('path'); +const semver = require('semver'); + +const BASE_NODE_MODULES = '/home/user/node_modules'; +const PROJECT_PACKAGE_JSON = './package.json'; + +// Read project package.json +let projectPackageJson; +try { + projectPackageJson = JSON.parse(fs.readFileSync(PROJECT_PACKAGE_JSON, 'utf8')); +} catch (error) { + console.error(`Error reading ${PROJECT_PACKAGE_JSON}:`, error.message); + process.exit(1); +} + +// Get all dependencies (dependencies + devDependencies, but we only care about production) +const allDependencies = { + ...(projectPackageJson.dependencies || {}), + // Note: We're in production mode, but checking all for completeness +}; + +if (Object.keys(allDependencies).length === 0) { + console.log('No dependencies found in package.json'); + process.exit(0); +} + +// Check which packages need to be installed +const packagesToInstall = []; +const packagesFromBase = []; + +for (const [packageName, versionRange] of Object.entries(allDependencies)) { + const basePackagePath = path.join(BASE_NODE_MODULES, packageName); + const basePackageJsonPath = path.join(basePackagePath, 'package.json'); + + // Check if package exists in base node_modules + if (fs.existsSync(basePackageJsonPath)) { + try { + const basePackageJson = JSON.parse(fs.readFileSync(basePackageJsonPath, 'utf8')); + const baseVersion = basePackageJson.version; + + // Check if base version satisfies project requirement + if (semver.satisfies(baseVersion, versionRange)) { + packagesFromBase.push(`${packageName}@${baseVersion} (satisfies ${versionRange})`); + // Package is available in base and version is compatible - skip installation + continue; + } else { + // Package exists but version is incompatible - need to install project version + packagesToInstall.push(`${packageName}@${versionRange}`); + } + } catch (error) { + // Error reading base package.json - install to be safe + console.warn(`Warning: Could not read base package.json for ${packageName}, will install`); + packagesToInstall.push(`${packageName}@${versionRange}`); + } + } else { + // Package doesn't exist in base - need to install + packagesToInstall.push(`${packageName}@${versionRange}`); + } +} + +// Output results to stderr (for user visibility, won't interfere with JSON parsing) +const output = { + packagesToInstall: packagesToInstall, + packagesFromBase: packagesFromBase.length, + totalDependencies: Object.keys(allDependencies).length +}; + +process.stderr.write(`\n=== Dependency Analysis ===\n`); +process.stderr.write(`Total dependencies: ${Object.keys(allDependencies).length}\n`); +process.stderr.write(`From base node_modules: ${packagesFromBase.length}\n`); +process.stderr.write(`To install: ${packagesToInstall.length}\n`); + +if (packagesFromBase.length > 0) { + process.stderr.write(`\nPackages using base version:\n`); + packagesFromBase.forEach(pkg => process.stderr.write(` ✓ ${pkg}\n`)); +} + +if (packagesToInstall.length > 0) { + process.stderr.write(`\nPackages to install:\n`); + packagesToInstall.forEach(pkg => process.stderr.write(` → ${pkg}\n`)); +} + +// Write JSON to stdout (shell script will capture this separately) +process.stdout.write(JSON.stringify(output)); + diff --git a/scripts/prepareproject-nodejs.sh b/scripts/prepareproject-nodejs.sh index 24d497b..8912690 100644 --- a/scripts/prepareproject-nodejs.sh +++ b/scripts/prepareproject-nodejs.sh @@ -1,7 +1,7 @@ #!/bin/bash -# This script installs only missing project dependencies to the project's node_modules. +# This script installs only missing or incompatible project dependencies to the project's node_modules. # Base node_modules at /home/user/node_modules is immutable and never copied. -# npm automatically resolves from both locations via NODE_PATH. +# Uses check-base-dependencies.js to avoid duplicating packages that exist in base. set -e @@ -28,20 +28,57 @@ fi # Install project dependencies if package.json exists if [ -f "package.json" ]; then - echo "Installing project dependencies..." - echo "npm will automatically:" - echo " - Check base node_modules at /home/user/node_modules" - echo " - Install only missing or incompatible packages" - echo " - Handle version conflicts (project version takes precedence)" - - # npm install will: - # 1. Check if packages exist in base (/home/user/node_modules) - # 2. Compare versions with package.json requirements - # 3. Install only missing or incompatible packages to ./node_modules - # 4. Handle version conflicts automatically - npm install --production --no-save - - echo "✓ Project dependencies installed" + echo "Analyzing dependencies against base node_modules..." + + # Check which packages need to be installed + CHECK_SCRIPT="/scripts/check-base-dependencies.js" + if [ ! -f "$CHECK_SCRIPT" ]; then + echo "ERROR: check-base-dependencies.js not found at $CHECK_SCRIPT" + exit 1 + fi + + # Run dependency check script + # Capture stdout (JSON) only, stderr (user messages) automatically goes to console + INSTALL_DATA=$(node "$CHECK_SCRIPT") + CHECK_RESULT=$? + + if [ $CHECK_RESULT -ne 0 ]; then + echo "ERROR: Failed to check base dependencies (exit code: $CHECK_RESULT)" + exit 1 + fi + + if [ -z "$INSTALL_DATA" ] || ! echo "$INSTALL_DATA" | grep -q '^{'; then + echo "ERROR: Could not parse dependency check output" + echo "Received: $INSTALL_DATA" + exit 1 + fi + PACKAGES_TO_INSTALL=$(echo "$INSTALL_DATA" | node -e "const data=JSON.parse(require('fs').readFileSync(0,'utf8')); console.log(data.packagesToInstall.map(p => p.split('@')[0]).join(' '))") + + # Count packages + TOTAL_DEPS=$(echo "$INSTALL_DATA" | node -e "const data=JSON.parse(require('fs').readFileSync(0,'utf8')); console.log(data.totalDependencies)") + FROM_BASE=$(echo "$INSTALL_DATA" | node -e "const data=JSON.parse(require('fs').readFileSync(0,'utf8')); console.log(data.packagesFromBase)") + TO_INSTALL_COUNT=$(echo "$INSTALL_DATA" | node -e "const data=JSON.parse(require('fs').readFileSync(0,'utf8')); console.log(data.packagesToInstall.length)") + + echo "" + echo "Dependency summary:" + echo " Total dependencies: $TOTAL_DEPS" + echo " Available in base: $FROM_BASE" + echo " To install: $TO_INSTALL_COUNT" + echo "" + + # Install only packages that are missing or incompatible + if [ -n "$PACKAGES_TO_INSTALL" ] && [ "$PACKAGES_TO_INSTALL" != "" ]; then + echo "Installing missing/incompatible packages: $PACKAGES_TO_INSTALL" + # Install packages individually to avoid installing everything + for package_spec in $(echo "$INSTALL_DATA" | node -e "const data=JSON.parse(require('fs').readFileSync(0,'utf8')); console.log(data.packagesToInstall.join(' '))"); do + npm install --production --no-save "$package_spec" + done + echo "✓ Project-specific dependencies installed" + else + echo "✓ All dependencies satisfied by base node_modules (no installation needed)" + # Create empty node_modules directory if it doesn't exist (for consistency) + mkdir -p node_modules + fi # Verify node-foxx is accessible (either from base or project node_modules) if [ -f "node_modules/.bin/node-foxx" ]; then diff --git a/src/main.rs b/src/main.rs index 369263c..f2ff9f1 100644 --- a/src/main.rs +++ b/src/main.rs @@ -58,6 +58,10 @@ const SCRIPT_FILES: &[ScriptFile] = &[ path: "prepareproject-nodejs.sh", content: include_str!("../scripts/prepareproject-nodejs.sh"), }, + ScriptFile { + path: "check-base-dependencies.js", + content: include_str!("../scripts/check-base-dependencies.js"), + }, ScriptFile { path: "zipper.sh", content: include_str!("../scripts/zipper.sh"), From 30a2ce6fe996c702a1cdbf396275a250ac53dcaf Mon Sep 17 00:00:00 2001 From: bluepal-yaswanth-peravali Date: Tue, 2 Dec 2025 16:31:35 +0530 Subject: [PATCH 7/8] Minor correction in Changelog --- CHANGELOG.md | 9 --------- 1 file changed, 9 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 61a9b62..23f49a0 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -90,15 +90,6 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Fixed -- **services.json basePath**: Fixed incorrect basePath in generated `services.json` - - Changed from service name to "." (current directory) - - Fixes path resolution issue where `node-foxx` was looking in wrong location for `manifest.json` - -- **Base Image Default Handling**: Improved robustness of base image default selection - - Replaced magic string comparisons with compile-time constants - - Added explicit tracking of user intent (whether base image was explicitly set) - - Prevents breakage if default values change in the future - - **Windows Compatibility**: Fixed Windows build issues in `src/main.rs` - Added conditional compilation for Unix-specific file permissions (`#[cfg(unix)]`) - Windows builds now skip `set_mode()` calls that are Unix-only From 32637c63657be929438d9f823fde8696ff18e01e Mon Sep 17 00:00:00 2001 From: bluepal-yaswanth-peravali Date: Fri, 5 Dec 2025 12:47:44 +0530 Subject: [PATCH 8/8] added auth label in deployment.yaml --- charts/templates/deployment.yaml | 1 + 1 file changed, 1 insertion(+) diff --git a/charts/templates/deployment.yaml b/charts/templates/deployment.yaml index bca055d..1c67c73 100644 --- a/charts/templates/deployment.yaml +++ b/charts/templates/deployment.yaml @@ -31,6 +31,7 @@ spec: release: {{ .Release.Name }} type: deployment profiles.arangodb.com/deployment: deployment + integration.profiles.arangodb.com/authn: v1 spec: containers: - name: {SERVICE_NAME}