Improve lang tooling: Docker, tree-sitter, IntelliJ plugin, CI#384
Merged
Improve lang tooling: Docker, tree-sitter, IntelliJ plugin, CI#384
Conversation
Add path filtering to conditionally run tree-sitter grammar validation only when relevant files change. Uses dorny/paths-filter to detect changes in: - lang/** (grammar DSL, tree-sitter, LSP server) - lib_ecstasy/**/*.x (XTC source files used as parsing corpus) When no lang-related files change, the tree-sitter validation step is skipped to save CI time. Gradle's content-based caching ensures that even when the step runs, only tasks with actual input changes execute.
- Move JRE cache from IntelliJ system path to GRADLE_USER_HOME/caches/xtc-jre/ (persists across IDE sessions, not cleared by "Invalidate Caches") - Add metadata tracking (package ID, platform, timestamp) for cache validation - Check Foojay for newer JRE versions every 7 days - Default to downloading IntelliJ Community instead of using local IDE (avoids incompatible bundled plugins like Full Line Code Completion) - Use -PuseLocalIde=true to override and use local installation - Fix macOS JRE extraction: search recursively for bin/java instead of hardcoding paths (handles Contents/Home structure) - Update plan docs with current implementation status
- Add lang/Dockerfile demonstrating runIde works with clean environment (only JDK 21/24 and IntelliJ pre-installed, all other tools downloaded) - Add lang/.dockerignore to exclude build artifacts from Docker context - Add logging and assertions to tree-sitter tasks to verify we're using build-local CLI and grammar.js (not system-installed tree-sitter) The assertions will fail the build if anyone accidentally uses a system-installed tree-sitter or grammar.js from outside the build tree.
Remove manual JDK 24 download - let Gradle auto-provision it like JDK 25. Add assertions to verify: - Base image JDK is version 21 (for IntelliJ) - Gradle provisions JDK 24 (org.xtclang.kotlin.jdk from version.properties) - Gradle provisions JDK 25 (org.xtclang.java.jdk from version.properties) This ensures the Dockerfile stays in sync with version.properties and demonstrates that Gradle handles all JDK provisioning automatically.
- Explain headless operation (Xvfb virtual display, nothing on host) - Add command to capture output to log file for debugging - Add instructions for X11 forwarding on Mac (XQuartz) - Fix inline comment that broke multi-line RUN
- Update build instructions to run from project root with -f lang/Dockerfile - Keep .git in build context for palantir-git-version plugin - Sort apt packages alphabetically
- Use eclipse-temurin:25-jdk-noble (was 21-jdk-jammy) - Run ./gradlew build to provision JDK 24 via Foojay - JDK 25 comes from base image, only JDK 24 needs provisioning
- Download architecture-specific IntelliJ (arm64/x86_64) - Pre-install JDK 24 to avoid Foojay API dependency - Add BuildKit cache mounts for faster rebuilds - Set JVM memory limits to prevent OOM in constrained environments - Add build caching documentation
- Add :tree-sitter to coreProjects in lang/build.gradle.kts - Use IntelliJ bundled JBR-21 instead of separate JDK 21 install
…idiomatic Kotlin - Make local IntelliJ IDE the default instead of downloading (use -PdownloadIde=true to override) - Switch to Provider API for Gradle properties (CC-compatible) - Add lifecycle-level build logging for IDE selection, TextMate/LSP resource copying, and sandbox config - Add humanSize() and logCopiedFiles() utilities to build-logic for CC-safe task instrumentation - Use idiomatic Kotlin patterns: buildList, in operator, method references, two-arg System.getProperty
Make includeBuildManualTests and includeBuildAttachManualTests false by default for workflow_dispatch, enabled only when include-lang is true. Push and PR events continue to include manual tests automatically.
Add @EnabledIfEnvironmentVariable annotations to XDK integration tests so they only run when RUN_INTEGRATION_TESTS=true is set, and set that env var in the CI workflow. Fix FileUtil.delete() calls in JreProvisioner to pass File instead of Path.
Flip the default IDE resolution: download IntelliJ Community (cached in ~/.gradle by version) instead of auto-detecting a local installation. Use -PuseLocalIde=true or -PintellijLocalPath=/path to opt in to local. Log the sandbox directory and idea.log path when runIde starts so the paths are clickable in the IDE console. Extend the clean task to also delete build/idea-sandbox/ so that version or dependency changes are picked up on the next run. Re-enable lang composite build includes in gradle.properties.
… searchable options Fix SlowOperations EDT assertion in project wizard by making VFS refresh async. Add buildPlugin task logging with ZIP path and installation instructions. Make buildSearchableOptions controllable via -Plsp.buildSearchableOptions property. Expand semantic tokens plan into two phases (lexer-based and compiler-based).
The lang/ subtree (IntelliJ plugin, LSP server, tree-sitter) adds significant build time and has dependencies (Java 25, native libs) that not all contributors need. Default to includeBuildLang=false so regular XDK development is unaffected. Enable with -PincludeBuildLang=true or by editing gradle.properties.
The env vars were overriding gradle.properties and forcing includeBuildLang=true on all push/PR events. Now lang is only included when explicitly requested via workflow_dispatch with include-lang=true.
…arsing Add LspIntegrationTest that exercises the LSP server against real .x source files (Boolean.x, Exception.x, Closeable.x, TestSimple.x) using the TreeSitter adapter with native parsing. Tests cover hover, completion, definition, references, documentSymbol, formatting, and structural features. The build passes xtc.composite.root to the test JVM (same marker-file approach as XdkPropertiesService) and enables FFM native access to suppress JDK warnings for tree-sitter's SymbolLookup calls. Update PLAN_TREE_SITTER.md: mark Phase 3 query tests and Phase 4 LSP features as complete.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This branch improves the
lang/tooling subsystem across several areas:Docker build environment
lang/Dockerfilefor reproducible builds of the full lang stack (LSP server, tree-sitter, IntelliJ plugin)Tree-sitter improvements
cleantasklang/files are unchangedIntelliJ plugin
~/.gradleby version) instead of requiring a local installationSlowOperationsEDT assertion in New Project wizard (async VFS refresh)buildPlugintask logging with distribution ZIP path and installation instructionsbuildSearchableOptionscontrollable via-Plsp.buildSearchableOptions=truepropertyidea.loglocation duringrunIde./gradlew cleanJRE provisioning
{GRADLE_USER_HOME}/caches/xtc-jre/so it survives IDE cache invalidationCI & build
RUN_INTEGRATION_TESTSenv varinclude-langworkflow dispatch flaglang/composite build by default (includeBuildLang=false) to avoid impacting regular XDK developmentPlan updates
Test plan
./gradlew :lang:intellij-plugin:buildPluginsucceeds with configuration cacherunIdelaunches sandbox IDE withoutSlowOperationsassertion on project creationbuildSearchableOptionsproperty toggles correctlylang/disabled by default🤖 Generated with Claude Code