Skip to content

chore(yql): highlight using monaco-yql-languages [YTFRONT-4282]#1409

Open
vityaman wants to merge 1 commit intoytsaurus:mainfrom
vityaman:YTFRONT-4282
Open

chore(yql): highlight using monaco-yql-languages [YTFRONT-4282]#1409
vityaman wants to merge 1 commit intoytsaurus:mainfrom
vityaman:YTFRONT-4282

Conversation

@vityaman
Copy link

@vityaman vityaman commented Dec 27, 2025

https://nda.ya.ru/t/WwcKXZO07Qv4JL

Current YQL syntax highlighting at the YT Web UI seems so outdated. This patch updates it by using the package `monaco-yql-languages` maintained by YQL UI team members.

Testing Query:

https://github.com/ydb-platform/ydb/blob/main/yql/essentials/tools/yql_highlight/ut/query.yql

Old vs New syntax highlighting:

image image

Summary by Sourcery

Update YQL Monaco integration to use the shared monaco-yql-languages package and align editor theming and autocomplete with it.

New Features:

  • Integrate the external monaco-yql-languages package to provide YQL language configuration and tokenization in the Monaco editor.

Enhancements:

  • Align light and dark Monaco editor themes with monaco-yql-languages token colors for consistent YQL syntax highlighting.
  • Adjust autocomplete suggestion weighting to include variable suggestions at an appropriate priority.

Build:

  • Add monaco-yql-languages as a UI dependency and bump @gravity-ui/websql-autocomplete to a newer version.

@sourcery-ai
Copy link

sourcery-ai bot commented Dec 27, 2025

Reviewer's Guide

Replaces the locally maintained YQL Monaco language definition with the upstream monaco-yql-languages package, aligns editor themes with the new token set, and updates dependencies related to Monaco/YQL autocomplete and suggestions weighting.

Sequence diagram for YQL Monaco language and theme usage

sequenceDiagram
    actor User
    participant YTWebUI
    participant MonacoEditor
    participant LocalYqlModule as MonacoYqlYql
    participant ExternalYqlPkg as MonacoYqlLanguagesPkg

    User ->> YTWebUI: Open YQL editor
    YTWebUI ->> MonacoEditor: Initialize editor instance
    MonacoEditor ->> LocalYqlModule: Request conf and language
    LocalYqlModule ->> ExternalYqlPkg: getLanguage(ansi: false) and conf
    ExternalYqlPkg -->> LocalYqlModule: LanguageConfiguration and MonarchLanguage
    LocalYqlModule -->> MonacoEditor: Provide conf and language
    YTWebUI ->> MonacoEditor: Apply YT_LIGHT_HC and YT_DARK_HC themes

    loop Typing query
        User ->> MonacoEditor: Type YQL query
        MonacoEditor ->> MonacoEditor: Tokenize using monaco-yql-languages
        MonacoEditor ->> MonacoEditor: Colorize tokens via updated rules
    end
Loading

Class diagram for Monaco YQL language module, themes, and suggestions

classDiagram
    class MonacoYqlYql {
        +conf
        +language
        +language = getLanguage(ansi: boolean)
    }

    class MonacoEditorThemes {
        +lightRules: TokenRule[]
        +darkRules: TokenRule[]
        +defineLightTheme(): void
        +defineDarkTheme(): void
    }

    class TokenRule {
        +token: string
        +foreground: string
        +fontStyle: string
    }

    class GenerateSuggestions {
        +SuggestionsWeight: Record~SuggestionType, number~
    }

    class SuggestionType {
        <<enumeration>>
        suggestTables
        suggestPragmas
        binding
        connection
        suggestVariables
        suggestColumns
        suggestColumnAliases
        suggestKeywords
    }

    MonacoEditorThemes --> TokenRule : uses
    GenerateSuggestions --> SuggestionType : uses
    MonacoYqlYql ..> MonacoEditorThemes : token names align
Loading

File-Level Changes

Change Details Files
Switch YQL Monaco language configuration to use upstream monaco-yql-languages implementation instead of in-repo definition.
  • Remove custom LanguageConfiguration and Monarch language definition previously implemented in the repo.
  • Import getLanguage from monaco-yql-languages/build/yql/yql and export language via getLanguage({ansi: false}).
  • Re-export conf directly from monaco-yql-languages/build/yql/yql instead of defining it locally.
packages/ui/src/ui/libs/monaco-yql-languages/yql/yql.ts
Adjust Monaco editor themes to support tokens produced by monaco-yql-languages.
  • Add light theme token color rules copied from ydb-platform/monaco-yql-languages for YQL-specific tokens (string.tablepath, constant.yql, keyword.type, string.sql, support.function, constant.other.color, comment).
  • Add dark theme token color rules for the same YQL-specific tokens plus storage.type to match new highlighting behavior.
  • Keep existing custom tablepath/path/string.sql rules for backward compatibility and additional styling.
packages/ui/src/ui/components/MonacoEditor/MonacoEditorThemes.tsx
Update dependencies to bring in monaco-yql-languages and a newer websql autocomplete version.
  • Add monaco-yql-languages@^2.1.0 to dependencies to supply the YQL language definition and tokens.
  • Bump @gravity-ui/websql-autocomplete from ^12.3.0 to ^13.7.0, aligning autocomplete with the updated language tooling.
  • Regenerate package-lock.json to reflect the new/updated dependencies.
packages/ui/package.json
packages/ui/package-lock.json
Tune suggestion weights to support variable suggestions in Monaco YQL autocomplete.
  • Add suggestVariables to SuggestionsWeight map with weight 4, aligning it relative to bindings and connections.
  • Preserve existing weights for other suggestion types to keep current ordering semantics.
packages/ui/src/ui/libs/monaco-yql-languages/helpers/generateSuggestions.ts

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@vityaman vityaman force-pushed the YTFRONT-4282 branch 2 times, most recently from 727d687 to 83fd408 Compare December 27, 2025 21:03
@vityaman vityaman marked this pull request as ready for review December 27, 2025 21:12
Copy link

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've found 1 issue, and left some high level feedback:

  • The getLanguage and conf imports are taken from monaco-yql-languages/build/..., which looks like an internal path; if the package exposes stable top-level exports, prefer those to avoid breakage on internal layout changes.
  • The light and dark theme rules now contain duplicate token entries (e.g. string.sql, comment) and copy‑pasted color definitions; consider consolidating these in a shared constants module and removing duplicates to keep the theme configuration easier to maintain.
Prompt for AI Agents
Please address the comments from this code review:

## Overall Comments
- The `getLanguage` and `conf` imports are taken from `monaco-yql-languages/build/...`, which looks like an internal path; if the package exposes stable top-level exports, prefer those to avoid breakage on internal layout changes.
- The light and dark theme rules now contain duplicate token entries (e.g. `string.sql`, `comment`) and copy‑pasted color definitions; consider consolidating these in a shared constants module and removing duplicates to keep the theme configuration easier to maintain.

## Individual Comments

### Comment 1
<location> `packages/ui/src/ui/components/MonacoEditor/MonacoEditorThemes.tsx:13-15` </location>
<code_context>
+    {token: 'string.tablepath', foreground: '338186'},
+    {token: 'constant.yql', foreground: '608b4e'},
+    {token: 'keyword.type', foreground: '4d932d'},
+    {token: 'string.sql', foreground: 'a31515'},
+    {token: 'support.function', foreground: '7a3e9d'},
+    {token: 'constant.other.color', foreground: '7a3e9d'},
</code_context>

<issue_to_address>
**suggestion:** Avoid duplicating the same `string.sql` rule in `lightRules`.

`lightRules` now has two identical entries for `token: 'string.sql', foreground: 'a31515'` (one in the new block and one in the existing rules). Please consolidate to a single `string.sql` rule to avoid confusion and simplify future changes.

```suggestion
    {token: 'tablepath', foreground: '3e999f'},
    {token: 'path', foreground: '3e999f', fontStyle: 'underline'},
});
```
</issue_to_address>

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

Comment on lines 13 to 15
{token: 'tablepath', foreground: '3e999f'},
{token: 'path', foreground: '3e999f', fontStyle: 'underline'},
{token: 'string.sql', foreground: 'a31515'},
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion: Avoid duplicating the same string.sql rule in lightRules.

lightRules now has two identical entries for token: 'string.sql', foreground: 'a31515' (one in the new block and one in the existing rules). Please consolidate to a single string.sql rule to avoid confusion and simplify future changes.

Suggested change
{token: 'tablepath', foreground: '3e999f'},
{token: 'path', foreground: '3e999f', fontStyle: 'underline'},
{token: 'string.sql', foreground: 'a31515'},
{token: 'tablepath', foreground: '3e999f'},
{token: 'path', foreground: '3e999f', fontStyle: 'underline'},
});

@vityaman
Copy link
Author

@Raubzeug, is it possible somehow to share the YQL themes? Now I just copy-pasted a JSON-object with rules. Are there other ways to share, except a rules extraction to a separate variables? How the theme is shared between YDB Web UI and YQL Web UI now?

Copy link
Collaborator

@SimbiozizV SimbiozizV Jan 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I installed all the dependencies, but I get the error Cannot find module 'monaco-yql-languages/build/yql/yql' when starting the project.

Copy link
Author

@vityaman vityaman Feb 1, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I removed it. Package monaco-yql-languages is intended to be used only with monaco-editor-webpack-plugin.

I noticed that this plugin is already added to the project. Surprisingly, only an addition of the monaco-yql-languages as a dependency to package.json triggered theirs YQL definition registration.

Theirs registration was overwritten by the registration at libs/monaco-yql-languages/_.contribution.ts, so I disabled it for MonacoLanguage.YQL. It seems, that it also let us to overwrite the yql_ansi, which is actually YT QL (LOL), and s-expressions, which is actually SPYT (LOL 2).

I tried to make autocompletion work, but failed. I noticed that it does not work even on the main branch. I hope, that it should be registered as always and also overwrite autocompletion from the monaco-yql-languages (if it registers it somehow).

@SimbiozizV, can you, please, test the solution again?

@vityaman vityaman force-pushed the YTFRONT-4282 branch 4 times, most recently from 7ba47dd to 9312ddc Compare February 1, 2026 08:56
Signed-off-by: vityaman <vityaman.dev@yandex.ru>
lazyLanguageLoader.load().then((mod) => {
languages.setLanguageConfiguration(languageId, mod.conf);

if (languageId !== MonacoLanguage.YQL) {
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if (languageId !== MonacoLanguage.YQL) {
// YQL is registered via `ydb-platform/monaco-yql-languages`
if (languageId !== MonacoLanguage.YQL) {


{token: 'tablepath', foreground: '3e999f'},
{token: 'path', foreground: '3e999f', fontStyle: 'underline'},
{token: 'string.sql', foreground: 'a31515'},
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There were 2 rules for string.sql in one list.

lazyLanguageLoader.load().then((mod) => {
languages.setLanguageConfiguration(languageId, mod.conf);

if (languageId !== MonacoLanguage.YQL) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps because of this rule, yql has now lost its highlighting.


{token: 'tablepath', foreground: '3e999f'},
{token: 'path', foreground: '3e999f', fontStyle: 'underline'},
{token: 'string.sql', foreground: 'ce9178'},
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

Comments