diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 8c5cbeb..e5bf774 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -74,6 +74,11 @@ jobs: - name: Install dependencies run: npm ci + - name: Validate package before publish + run: npm run validate-publish + env: + DENO_PATH: deno + - name: Build run: npm run build diff --git a/docs/NPM_PUBLISH_VALIDATION.md b/docs/NPM_PUBLISH_VALIDATION.md new file mode 100644 index 0000000..e28015a --- /dev/null +++ b/docs/NPM_PUBLISH_VALIDATION.md @@ -0,0 +1,194 @@ +# NPM Publishing Validation + +## Overview + +This document explains the npm publishing validation system for PluresDB and how to use it. + +## The Problem + +PluresDB supports both Node.js/npm and Deno ecosystems, which have different import conventions: + +- **TypeScript/Node.js**: Imports should NOT include `.ts` extensions (e.g., `import { foo } from "./bar"`) +- **Deno**: Imports MUST include `.ts` extensions (e.g., `import { foo } from "./bar.ts"`) + +This creates a conflict when: +1. Files are compiled for npm using TypeScript +2. The same files are tested using Deno during the `npm test` command +3. Deno type-checking fails because imports lack `.ts` extensions + +## The Solution + +We use Deno's `--sloppy-imports` flag, which allows Deno to automatically resolve imports without explicit extensions, similar to Node.js behavior. + +### Changes Made + +1. **Updated test command** in `package.json`: + ```json + "test": "deno test -A --unstable-kv --sloppy-imports" + ``` + +2. **Created validation script** at `scripts/validate-npm-publish.js`: + - Validates the package before publishing to npm + - Runs comprehensive checks on TypeScript compilation, file structure, and Deno compatibility + +3. **Updated release workflow** (`.github/workflows/release.yml`): + - Added validation step before npm publish + - Ensures packages are verified before release + +## Using the Validation Script + +### Manual Validation + +Run the validation script manually before publishing: + +```bash +npm run validate-publish +``` + +### What It Checks + +The validation script performs the following checks: + +1. **package.json validation**: Ensures required fields are present +2. **TypeScript compilation**: Builds the library with `tsc` +3. **Required files**: Checks that all expected dist files exist +4. **Deno type checking**: Validates key files with Deno using `--sloppy-imports` +5. **Tests**: Runs the Deno test suite (if available) +6. **Package size**: Reports the size of the npm package + +### Example Output + +``` +๐Ÿš€ NPM Publish Validation + +๐Ÿ“ฆ Validating package.json... +โœ“ Package: @plures/pluresdb@1.6.9 + +๐Ÿ”จ Building TypeScript... +โœ“ TypeScript compilation + +๐Ÿ“ Checking required files... +โœ“ Required file: dist/node-index.js +โœ“ Required file: dist/node-index.d.ts +... + +๐Ÿฆ• Deno type checking... +โœ“ Deno type check: legacy/local-first/unified-api.ts +... + +๐Ÿงช Running tests... +โœ“ Deno tests + +๐Ÿ“Š Package size check... +โœ“ Package size: 137.3 kB + +๐Ÿ“‹ Validation Summary +โœ“ All critical checks passed! โœจ + +The package is ready to be published to npm. +``` + +## CI/CD Integration + +The validation runs automatically in the release workflow: + +1. When a tag is pushed (e.g., `v1.0.0`) +2. Before `npm publish` is executed +3. If validation fails, the publish is aborted + +### Workflow Steps + +```yaml +- name: Validate package before publish + run: npm run validate-publish + env: + DENO_PATH: deno +``` + +## Import Conventions + +To maintain compatibility with both ecosystems: + +### Files Compiled for npm (in `tsconfig.json`) + +Use imports **without** `.ts` extensions: + +```typescript +// โœ… Correct for npm-compiled files +import { debugLog } from "../util/debug"; +import { PluresNode } from "./node-wrapper"; +``` + +### Deno-only Files (not in `tsconfig.json`) + +Use imports **with** `.ts` extensions: + +```typescript +// โœ… Correct for Deno-only files +import { debugLog } from "../util/debug.ts"; +import { PluresDBLocalFirst } from "../../local-first/unified-api.ts"; +``` + +## Troubleshooting + +### "Cannot find module" errors in Deno + +If you see errors like: +``` +TS2307 [ERROR]: Cannot find module 'file:///.../debug' +``` + +**Solution**: Ensure `--sloppy-imports` flag is used: +```bash +deno check --sloppy-imports your-file.ts +deno test -A --unstable-kv --sloppy-imports +``` + +### "Cannot end with .ts extension" errors in TypeScript + +If you see errors like: +``` +TS5097: An import path can only end with a '.ts' extension when 'allowImportingTsExtensions' is enabled. +``` + +**Solution**: Remove `.ts` extensions from imports in files that are compiled for npm (listed in `tsconfig.json`). + +### Package Size Warnings + +If the validation warns about package size: + +1. Check the `files` array in `package.json` +2. Ensure build artifacts and dependencies are excluded +3. Use `.npmignore` to exclude unnecessary files + +## Testing Locally + +Before pushing changes that affect imports or the build process: + +1. **Clean build**: + ```bash + rm -rf dist node_modules + npm ci + ``` + +2. **Run validation**: + ```bash + npm run validate-publish + ``` + +3. **Test the build**: + ```bash + npm run build + npm test + ``` + +4. **Dry run publish**: + ```bash + npm pack --dry-run + ``` + +## References + +- [Deno Import Resolution](https://docs.deno.com/runtime/manual/basics/modules/) +- [TypeScript Module Resolution](https://www.typescriptlang.org/docs/handbook/module-resolution.html) +- [npm prepublishOnly Hook](https://docs.npmjs.com/cli/v9/using-npm/scripts#life-cycle-scripts) diff --git a/package.json b/package.json index c69fe71..cb57cb3 100644 --- a/package.json +++ b/package.json @@ -51,7 +51,7 @@ "build:web": "cd web/svelte && npm install && npm run build", "dev": "deno run -A --unstable-kv --watch src/main.ts serve --port 34567", "start": "node dist/cli.js serve", - "test": "deno test -A --unstable-kv", + "test": "deno test -A --unstable-kv --sloppy-imports", "test:azure:relay": "deno test --allow-net --allow-env azure/tests/relay-tests.ts", "test:azure:full": "npm run test:azure:relay", "lint": "eslint . --ext .js,.ts,.tsx", @@ -60,6 +60,7 @@ "verify": "npm run build:lib && npm test", "prepare": "npm run build:lib", "prepublishOnly": "npm run verify && npm run build:web", + "validate-publish": "node scripts/validate-npm-publish.js", "postinstall": "node scripts/postinstall.js", "release-check": "node scripts/release-check.js", "update-changelog": "node scripts/update-changelog.js" diff --git a/scripts/validate-npm-publish.js b/scripts/validate-npm-publish.js new file mode 100755 index 0000000..2d84d3c --- /dev/null +++ b/scripts/validate-npm-publish.js @@ -0,0 +1,228 @@ +#!/usr/bin/env node + +/** + * Pre-publish Validation Script for NPM + * + * This script validates that the package is ready to be published to npm. + * It checks: + * 1. TypeScript compilation succeeds + * 2. Deno type checking passes + * 3. All tests pass + * 4. Required files exist in dist/ + * 5. package.json is valid + */ + +const { execSync } = require("node:child_process"); +const fs = require("node:fs"); +const path = require("node:path"); + +const RED = "\x1b[31m"; +const GREEN = "\x1b[32m"; +const YELLOW = "\x1b[33m"; +const RESET = "\x1b[0m"; +const BOLD = "\x1b[1m"; + +// Configuration +const MAX_PACKAGE_SIZE_MB = 10; + +function log(message, color = RESET) { + console.log(`${color}${message}${RESET}`); +} + +function error(message) { + log(`โœ— ${message}`, RED); +} + +function success(message) { + log(`โœ“ ${message}`, GREEN); +} + +function info(message) { + log(`โ„น ${message}`, YELLOW); +} + +function title(message) { + log(`\n${BOLD}${message}${RESET}`); +} + +function runCommand(command, description) { + try { + info(`Running: ${description}...`); + execSync(command, { stdio: "inherit", cwd: process.cwd() }); + success(description); + return true; + } catch (err) { + error(`${description} failed`); + return false; + } +} + +function checkFileExists(filePath, description) { + const fullPath = path.join(process.cwd(), filePath); + if (fs.existsSync(fullPath)) { + success(`${description}: ${filePath}`); + return true; + } else { + error(`${description} missing: ${filePath}`); + return false; + } +} + +async function main() { + title("๐Ÿš€ NPM Publish Validation"); + + let allChecksPassed = true; + + // 1. Check package.json is valid + title("๐Ÿ“ฆ Validating package.json..."); + try { + const packageJson = JSON.parse( + fs.readFileSync(path.join(process.cwd(), "package.json"), "utf-8"), + ); + if (!packageJson.name || !packageJson.version) { + error("package.json missing required fields (name or version)"); + allChecksPassed = false; + } else { + success( + `Package: ${packageJson.name}@${packageJson.version}`, + ); + } + } catch (err) { + error(`Invalid package.json: ${err.message}`); + allChecksPassed = false; + } + + // 2. TypeScript compilation + title("๐Ÿ”จ Building TypeScript..."); + if (!runCommand("npm run build:lib", "TypeScript compilation")) { + allChecksPassed = false; + } + + // 3. Check required dist files exist + title("๐Ÿ“ Checking required files..."); + const requiredFiles = [ + "dist/node-index.js", + "dist/node-index.d.ts", + "dist/better-sqlite3.js", + "dist/better-sqlite3.d.ts", + "dist/cli.js", + "dist/cli.d.ts", + "dist/local-first/unified-api.js", + "dist/local-first/unified-api.d.ts", + "dist/vscode/extension.js", + "dist/vscode/extension.d.ts", + "dist/types/node-types.js", + "dist/types/node-types.d.ts", + ]; + + for (const file of requiredFiles) { + if (!checkFileExists(file, "Required file")) { + allChecksPassed = false; + } + } + + // 4. Deno type checking + title("๐Ÿฆ• Deno type checking..."); + const denoPath = process.env.DENO_PATH || "deno"; + const denoCheckFiles = [ + "legacy/local-first/unified-api.ts", + "legacy/node-index.ts", + "legacy/better-sqlite3.ts", + "legacy/cli.ts", + "legacy/vscode/extension.ts", + ]; + + // Check if Deno is available + let denoAvailable = false; + try { + execSync(`${denoPath} --version`, { stdio: "pipe" }); + denoAvailable = true; + } catch (err) { + info("Deno not available - skipping Deno type checks"); + } + + if (denoAvailable) { + let denoChecksFailed = false; + for (const file of denoCheckFiles) { + if ( + !runCommand( + `${denoPath} check --sloppy-imports ${file}`, + `Deno type check: ${file}`, + ) + ) { + error(`Deno type check failed for ${file}`); + denoChecksFailed = true; + allChecksPassed = false; + // Continue checking other files to show all failures + } + } + if (!denoChecksFailed) { + success("All Deno type checks passed"); + } + } + + // 5. Run tests (if Deno is available) + title("๐Ÿงช Running tests..."); + if (denoAvailable) { + // Set DENO_PATH environment variable so npm test can find deno + const testEnv = { ...process.env }; + const denoPathEnv = process.env.DENO_PATH; + if (denoPathEnv && denoPathEnv.includes(path.sep)) { + // If DENO_PATH was provided as a path, make sure its directory is in PATH for npm test + const denoBinDir = path.dirname(denoPathEnv); + // Use path.delimiter for cross-platform compatibility (: on Unix, ; on Windows) + testEnv.PATH = `${denoBinDir}${path.delimiter}${process.env.PATH}`; + } + + try { + execSync("npm test", { stdio: "inherit", cwd: process.cwd(), env: testEnv }); + success("Deno tests"); + } catch (err) { + error("Tests failed"); + allChecksPassed = false; + } + } else { + info("Deno tests skipped (Deno not available)"); + } + + // 6. Check package size + title("๐Ÿ“Š Package size check..."); + try { + const output = execSync("npm pack --dry-run 2>&1", { encoding: "utf-8" }); + const sizeMatch = output.match(/package size:\s+(\d+\.?\d*)\s*(\w+)/i); + if (sizeMatch) { + const size = parseFloat(sizeMatch[1]); + const unit = sizeMatch[2]; + success(`Package size: ${size} ${unit}`); + + // Warn if package is larger than configured threshold + if (unit.toLowerCase() === "mb" && size > MAX_PACKAGE_SIZE_MB) { + info( + `Warning: Package size is quite large (${size} ${unit}). Consider excluding unnecessary files.`, + ); + } + } + } catch (err) { + info("Could not determine package size"); + } + + // Summary + title("๐Ÿ“‹ Validation Summary"); + if (allChecksPassed) { + success("All critical checks passed! โœจ"); + log( + "\nThe package is ready to be published to npm.", + GREEN, + ); + process.exit(0); + } else { + error("Some checks failed. Please fix the issues before publishing."); + process.exit(1); + } +} + +main().catch((err) => { + error(`Validation failed with error: ${err.message}`); + console.error(err); + process.exit(1); +});