Skip to content

Empty string dictionary keys can not be parsed twice #249

@mcasperson

Description

@mcasperson

Describe the bug

A dictionary with an empty key fails to parse, deserialize, and parse again.

Software:

  • OS: Linux
  • Python version 3.11
  • python-hcl2 version 7.3.1

Snippet of HCL2 code causing the unexpected behaviour:

resource "octopusdeploy_process_step" "process_step_azure_web_app_validate_setup" {
  git_dependencies      = { "" = { default_branch = "main", file_path_filters = null, git_credential_id = "", git_credential_type = "Anonymous", repository_uri = "https://github.com/OctopusSolutionsEngineering/Octopub.git" } }
}

Expected behavior

The following script should work:

import hcl2

hcl = """resource "octopusdeploy_process_step" "process_step_azure_web_app_validate_setup" {
  git_dependencies      = { "" = { default_branch = "main", file_path_filters = null, git_credential_id = "", git_credential_type = "Anonymous", repository_uri = "https://github.com/OctopusSolutionsEngineering/Octopub.git" } }
}"""

parsed_config = hcl2.loads(hcl, with_meta=True)
example_ast = hcl2.reverse_transform(parsed_config)
converted_hcl = hcl2.writes(example_ast)

hcl2.loads(converted_hcl, with_meta=True)

The generated HCL from the first hcl2.writes() function is this:

resource "octopusdeploy_process_step" "process_step_azure_web_app_validate_setup" {
  git_dependencies = {
     = {
      default_branch = "main"
      file_path_filters = null
      git_credential_id = ""
      git_credential_type = "Anonymous"
      repository_uri = "https://github.com/OctopusSolutionsEngineering/Octopub.git"
    }
  }
}

It should be this:

resource "octopusdeploy_process_step" "process_step_azure_web_app_validate_setup" {
  git_dependencies = {
   ""  = {
      default_branch = "main"
      file_path_filters = null
      git_credential_id = ""
      git_credential_type = "Anonymous"
      repository_uri = "https://github.com/OctopusSolutionsEngineering/Octopub.git"
    }
  }
}

Exception traceback (if applicable):

Traceback (most recent call last):
  File "/home/matthew/Desktop/hcl/venv/lib/python3.12/site-packages/lark/lexer.py", line 689, in lex
    yield lexer.next_token(lexer_state, parser_state)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/matthew/Desktop/hcl/venv/lib/python3.12/site-packages/lark/lexer.py", line 622, in next_token
    raise UnexpectedCharacters(lex_state.text.text, line_ctr.char_pos, line_ctr.line, line_ctr.column,
lark.exceptions.UnexpectedCharacters: No terminal matches '=' in the current parser context, at line 3 col 6

     = {
     ^
Expected one of: 
	* LSQB
	* LBRACE
	* FOR_OBJECT_ARROW
	* /<<-(?P<heredoc_trim>[a-zA-Z][a-zA-Z0-9._-]+)\n?(?:.|\n)*?\n\s*(?P=heredoc_trim)\n/
	* BINARY_OP
	* NEGATIVE_DECIMAL
	* LPAR
	* NL_OR_COMMENT
	* RSQB
	* RPAR
	* COLON
	* /<<(?P<heredoc>[a-zA-Z][a-zA-Z0-9._-]+)\n?(?:.|\n)*?\n\s*(?P=heredoc)\n/
	* BANG
	* QMARK
	* DBLQUOTE
	* COMMA
	* "..."
	* NAME
	* DECIMAL
	* RBRACE

Previous tokens: Token('NL_OR_COMMENT', '\n     ')


During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/matthew/Desktop/hcl/main.py", line 11, in <module>
    hcl2.loads(converted_hcl, with_meta=True)
  File "/home/matthew/Desktop/hcl/venv/lib/python3.12/site-packages/hcl2/api.py", line 29, in loads
    tree = parser().parse(text + "\n")
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/matthew/Desktop/hcl/venv/lib/python3.12/site-packages/lark/lark.py", line 677, in parse
    return self.parser.parse(text, start=start, on_error=on_error)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/matthew/Desktop/hcl/venv/lib/python3.12/site-packages/lark/parser_frontends.py", line 131, in parse
    return self.parser.parse(stream, chosen_start, **kw)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/matthew/Desktop/hcl/venv/lib/python3.12/site-packages/lark/parsers/lalr_parser.py", line 42, in parse
    return self.parser.parse(lexer, start)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/matthew/Desktop/hcl/venv/lib/python3.12/site-packages/lark/parsers/lalr_parser.py", line 88, in parse
    return self.parse_from_state(parser_state)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/matthew/Desktop/hcl/venv/lib/python3.12/site-packages/lark/parsers/lalr_parser.py", line 111, in parse_from_state
    raise e
  File "/home/matthew/Desktop/hcl/venv/lib/python3.12/site-packages/lark/parsers/lalr_parser.py", line 100, in parse_from_state
    for token in state.lexer.lex(state):
  File "/home/matthew/Desktop/hcl/venv/lib/python3.12/site-packages/lark/lexer.py", line 698, in lex
    raise UnexpectedToken(token, e.allowed, state=parser_state, token_history=[last_token], terminals_by_name=self.root_lexer.terminals_by_name)
lark.exceptions.UnexpectedToken: Unexpected token Token('STRING_CHARS', '= {\n      default_branch = ') at line 3, column 6.
Expected one of: 
	* DBLQUOTE
	* IN
	* NAME
	* NEGATIVE_DECIMAL
	* FOR_EACH
	* LPAR
	* IF
	* FOR
	* DECIMAL
	* NL_OR_COMMENT
	* RBRACE
Previous tokens: [Token('NL_OR_COMMENT', '\n     ')]

Metadata

Metadata

Labels

bugSomething isn't workingfixed-in-8.0this issue has been fixed in `transformer-overhaul` feature branch, to be released as v8.0

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions