Skip to content

reverse_transform not handling nested functions in attribute value #235

@t0mmili

Description

@t0mmili

Description

When working on a Terraform parameterizer for imported Datadog resources, the synthetics_test resource exhibits a complex structure that poses challenges for reverse transformation. Specifically, the browser_step.params.element field contains a value wrapped in jsonencode, which itself contains another nested jsonencode.

This nested jsonencode structure creates complications during the reverse transformation process. When the nested jsonencode is removed, the reverse transformation works as expected. However, with the nested structure in place, the transformation fails.

Software

  • OS: Windows
  • Python version: 3.12.7
  • python-hcl2 version: 7.2.1

Snippet of HCL2 code causing the unexpected behaviour

resource "datadog_synthetics_test" "tfer--synthetics_123-abc-456" {
  ...
  browser_step {
    allow_failure   = false
    always_execute  = false
    exit_if_succeed = false
    is_critical     = true
    local_key       = null
    name            = "Click on Layers"
    no_screenshot   = false
    timeout         = 0
    type            = "click"

    params {
      attribute  = null
      check      = null
      click_type = null
      code       = null
      delay      = 0
      element = jsonencode(
        {
          multiLocator = {
            ab  = "/*[local-name()=\"html\"][1]/*[local-name()=\"body\"][1]/*[local-name()=\"div\"][1]/*[local-name()=\"div\"][1]/*[local-name()=\"div\"][1]/*[local-name()=\"div\"][1]/*[local-name()=\"ef-button\"][1]"
            at  = "/descendant::*[@role=\"button\" and @title=\"Display Layers panel\"]"
            cl  = "/descendant::*[contains(concat(' ', normalize-space(@class), ' '), \" toolbar__container--left \")]/*[local-name()=\"ef-button\"][1]"
            clt = "/descendant::*[contains(concat(' ', normalize-space(@class), ' '), \" toolbar__container--left \")]/descendant::*[text()[normalize-space(translate(., 'ABCDEFGHIJKLMNOPQRSTUVWXYZÀÁÂÃÄÅÆÇÈÉÊËÌÍÎÏÐÑÒÓÔÕÖØÙÚÛÜÝÞŸŽŠŒ', 'abcdefghijklmnopqrstuvwxyzàáâãäåæçèéêëìíîïðñòóôõöøùúûüýþÿžšœ')) = \"layers\"]]"
            co = jsonencode(
              [
                {
                  text     = "layers"
                  textType = "directText"
                },
              ]
            )
            ro = "//*[text()[normalize-space(translate(., 'ABCDEFGHIJKLMNOPQRSTUVWXYZÀÁÂÃÄÅÆÇÈÉÊËÌÍÎÏÐÑÒÓÔÕÖØÙÚÛÜÝÞŸŽŠŒ', 'abcdefghijklmnopqrstuvwxyzàáâãäåæçèéêëìíîïðñòóôõöøùúûüýþÿžšœ')) = \"layers\"]]"
          }
          targetOuterHTML = "<ef-button tabindex=\"0\" role=\"button\" title=\"Display Layers panel\" toggles=\"\" active=\"\" icon=\"map-layers\" aria-pressed=\"true\" textpos=\"after\">Layers</ef-button>"
          url             = "https://myapp.corp.com/"
        }
      )
      email             = null
      file              = null
      files             = null
      modifiers         = []
      playing_tab_id    = null
      request           = null
      subtest_public_id = null
      value             = null
      with_click        = false
      x                 = 0
      y                 = 0
    }
  }
  ...
}

Exception

$ python ./scripts/hcl2-poc/hcl2-poc.py                                                                                                          
Traceback (most recent call last):
  File "/home/vscode/.local/lib/python3.12/site-packages/lark/parsers/lalr_parser_state.py", line 77, in feed_token
    action, arg = states[state][token.type]
                  ~~~~~~~~~~~~~^^^^^^^^^^^^
KeyError: 'STRING_LIT'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/workspaces/./scripts/hcl2-poc/hcl2-poc.py", line 20, in <module>
    example_ast = hcl2.reverse_transform(tf_dict)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/vscode/.local/lib/python3.12/site-packages/hcl2/api.py", line 60, in reverse_transform
    return HCLReverseTransformer().transform(hcl2_dict)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/vscode/.local/lib/python3.12/site-packages/hcl2/reconstructor.py", line 383, in transform
    body = self._transform_dict_to_body(hcl_dict, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/vscode/.local/lib/python3.12/site-packages/hcl2/reconstructor.py", line 492, in _transform_dict_to_body
    block_body = self._transform_dict_to_body(
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/vscode/.local/lib/python3.12/site-packages/hcl2/reconstructor.py", line 492, in _transform_dict_to_body
    block_body = self._transform_dict_to_body(
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/vscode/.local/lib/python3.12/site-packages/hcl2/reconstructor.py", line 492, in _transform_dict_to_body
    block_body = self._transform_dict_to_body(
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/vscode/.local/lib/python3.12/site-packages/hcl2/reconstructor.py", line 511, in _transform_dict_to_body
    expr_term = self._transform_value_to_expr_term(value, level)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/vscode/.local/lib/python3.12/site-packages/hcl2/reconstructor.py", line 659, in _transform_value_to_expr_term
    ast = reconstruction_parser().parse(f"value = {wrapped_value}")
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/vscode/.local/lib/python3.12/site-packages/lark/lark.py", line 655, in parse
    return self.parser.parse(text, start=start, on_error=on_error)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/vscode/.local/lib/python3.12/site-packages/lark/parser_frontends.py", line 104, in parse
    return self.parser.parse(stream, chosen_start, **kw)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/vscode/.local/lib/python3.12/site-packages/lark/parsers/lalr_parser.py", line 42, in parse
    return self.parser.parse(lexer, start)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/vscode/.local/lib/python3.12/site-packages/lark/parsers/lalr_parser.py", line 88, in parse
    return self.parse_from_state(parser_state)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/vscode/.local/lib/python3.12/site-packages/lark/parsers/lalr_parser.py", line 111, in parse_from_state
    raise e
  File "/home/vscode/.local/lib/python3.12/site-packages/lark/parsers/lalr_parser.py", line 102, in parse_from_state
    state.feed_token(token)
  File "/home/vscode/.local/lib/python3.12/site-packages/lark/parsers/lalr_parser_state.py", line 80, in feed_token
    raise UnexpectedToken(token, expected, state=self, interactive_parser=None)
lark.exceptions.UnexpectedToken: Unexpected token Token('STRING_LIT', '"]]"') at line 1, column 1705.
Expected one of: 
        * EQ
        * DOT
        * COLON

Metadata

Metadata

Labels

bugSomething isn't workingfixed-in-8.0this issue has been fixed in `transformer-overhaul` feature branch, to be released as v8.0reverse transformationtransformationThe issue is related to logic of transforming Lark AST into Python data structure

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions