Populate Outputs section for nmr

[cecizag]

No description provided.

[cecizag]

Hi @ladinesa,

I tried to replicate the structure used in the pwscf parser to create an ElectronicEigenvalues section inside Outputs, but for the gipaw files and the MagneticShielding section.
However, in my case the Outputs section remains empty. While debugging, I can’t even get the debug print at

https://github.com/exactlab/nomad-parser-plugins-simulation/blob/62f920e557a745052e48001957bd5c569f27ec1a/src/nomad_simulation_parsers/parsers/quantumespresso/gipaw/parser.py#L24

Am I missing or doing something wrong? Thanks for your help!

[ladinesa]

Hi, you should also add annotation to the Outputs section from the nmr schema package i.e

add_mapping_annotations(
        schema_package.Outputs.m_def,
        OUT_KEY,
        '.@'
    )

The annotation for the outputs sub-section in common.py refers to the original Outputs section in nomad-simulations which does not contain Magnetic Shielding.

[cecizag]

Thanks @ladinesa! I tried to fix it as you suggested, following the structure of AtomsState in common.py, but it still doesn’t work (not even the debug print).

[ladinesa]

Hi @cecizag , I found several issues

1. Do you have an xml file in the same folder? If yes, I put there a check that for gipaw calculations v>=7.4.0, it will parse the xml file so make sure you do not have it. Otherwise you add the parsing of magnetic shielding also for xml define the mapping for xml.
2. I forgot but you also have to add the annotation for the magnetic shieldings sub section i.e. add_mapping_annotations(schema_package.Outputs.magnetic_shieldings, OUT_KEY, '.@')
3. The annotation for Simulation.outputs clashes with the nmr Outputs.m_def annotation, Mapping parser will use the annotation for Simulation.outputs. You have two solutions for this, you define another annotation key for gipaw but you have to the parsing logic in the parser or simpler you remove the annotation with remove_mapping_annotations(Simulation.outputs) before add_mapping_annotations(schema_package.Outputs.m_def,OUT_KEY,('get_magnetic_shieldings', ['.@']),).

[cecizag]

Thanks. For now, I’m only working with text files, and those are the only ones I have in the test folder. I applied the changes you suggested, and indeed a MagneticShielding is now added to the Output section. However, I can’t figure out why only MagneticShielding.name is being added, and not the others like MagneticShielding.value. How can I fix this? Thanks.

[ladinesa]

Thanks. For now, I’m only working with text files, and those are the only ones I have in the test folder. I applied the changes you suggested, and indeed a MagneticShielding is now added to the Output section. However, I can’t figure out why only MagneticShielding.name is being added, and not the others like MagneticShielding.value. How can I fix this? Thanks.

get_magnetic_shields is returning a list of dict with key magnetic_shieldings so your annotation for magnetic_shieldings should be .magnetic_shielding snot '.@'. sorry i was the one who wrote @

[cecizag]

Hi @ladinesa,

I made the suggested change, but the situation is still the same. Would it be possible to schedule a short call to go over the situation together? I think I don’t fully understand how this new version of the parser works, and I believe a quick discussion would be helpful.

Thanks!

[cecizag]

Hi @ladinesa ,

I tried to parse the XML files, following the approach used here:

https://github.com/FAIRmat-NFDI/nomad-parser-plugins-simulation/blob/1e67a5ad4d3556f40d0c74ab706aadca213e1cfb/src/nomad_simulation_parsers/schema_packages/quantumespresso/common.py#L58C7-L58C17

I couldn’t get it to work. To understand the issue, I tried debugging and noticed that even for non-GIPAW XML files, nothing gets parsed. Here’s the debug output:

tests/parsers/test_quantumespresso_parser.py:24 test_pwscf_xml
    archive.data: Simulation(steps, program, model_method, model_system, outputs) (Simulation) len=17
tests/parsers/test_quantumespresso_parser.py:23 test_pwscf_xml
    archive.data.model_system: [] (MSubSectionList) len=0

You can find all the files I used for testing and debugging in the tests/data/quantumespresso folder.

[ladinesa]

Hi @ladinesa ,

I tried to parse the XML files, following the approach used here:

https://github.com/FAIRmat-NFDI/nomad-parser-plugins-simulation/blob/1e67a5ad4d3556f40d0c74ab706aadca213e1cfb/src/nomad_simulation_parsers/schema_packages/quantumespresso/common.py#L58C7-L58C17

I couldn’t get it to work. To understand the issue, I tried debugging and noticed that even for non-GIPAW XML files, nothing gets parsed. Here’s the debug output:

tests/parsers/test_quantumespresso_parser.py:24 test_pwscf_xml
    archive.data: Simulation(steps, program, model_method, model_system, outputs) (Simulation) len=17
tests/parsers/test_quantumespresso_parser.py:23 test_pwscf_xml
    archive.data.model_system: [] (MSubSectionList) len=0

You can find all the files I used for testing and debugging in the tests/data/quantumespresso folder.

I tried to parse the same pwscf xml file and it was able to yield

[ModelSystem(positions, cell, particle_states), ModelSystem(positions, cell, particle_states)]

can you try to print at what annotation_key is used?

[ladinesa]

can you rebase it again?

[ladinesa]

As much as possible, we reduce the amount of transformation of the source data, Could you maybe annotate directly to the file parser data. And if transformation is necessary split it into smaller functions. This way it is more readable.

[cecizag]

I merged the latest commits as you suggested yesterday. I tried again to process some uploads from the GUI, and these are the results:

• quartz-scf.out + quartz-nmr.out: in the quartz-nmr.out entry the data section is present, but inside it there is only program, not outputs as expected.
• quartz-scf.xml + quartz-nmr.xml: in the quartz-nmr.xml entry the data section is not present.
• quartz-scf.out + quartz-nmr.out + quartz-nmr.xml version 7.4.1: in the quartz-nmr.out entry the data section is not present.

[ladinesa]

I merged the latest commits as you suggested yesterday. I tried again to process some uploads from the GUI, and these are the results:

• quartz-scf.out + quartz-nmr.out: in the quartz-nmr.out entry the data section is present, but inside it there is only program, not outputs as expected.
• quartz-scf.xml + quartz-nmr.xml: in the quartz-nmr.xml entry the data section is not present.
• quartz-scf.out + quartz-nmr.out + quartz-nmr.xml version 7.4.1: in the quartz-nmr.out entry the data section is not present.

I checked out 5c2b0a7 but It has l458 self.mainfile_parser.filepath = self.mainfile pelase remove this.

[cecizag]

Hi @ladinesa ,

thank you. I tried the new version, but I’m getting the following result:

• When I run the nomad parse command from the command line, the files are parsed correctly, both .out and .xml.
• When processing the uploads from the GUI, the entry for quartz-nmr.out or quartz-nmr.xml is created correctly, and the program, model system, and outputs sections are populated. However, in outputs I only see the quantities inherited from the scf; all the NMR-specific quantities from the file are missing.

[ladinesa]

Would it be possible for you to rebase your branch so I can test it thanks

[cecizag]

I merged, but now I get this error:

ERROR    nomad_simulation_par 2025-12-05T13:25:25Z Could not load parser class nomad_simulation_parsers.parsers.abinit.parser.AbinitParser
  - exception: Traceback (most recent call last):
      File "/home/cecilia/lavoro/fairy-magic-nomad-distro-dev/packages/nomad-parser-plugins-simulation/src/nomad_simulation_parsers/parsers/__init__.py", line 21, in load
        module = importlib.import_module(module_path)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/home/cecilia/.local/share/uv/python/cpython-3.12.9-linux-x86_64-gnu/lib/python3.12/importlib/__init__.py", line 90, in import_module
        return _bootstrap._gcd_import(name[level:], package, level)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
      File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
      File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
      File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
      File "<frozen importlib._bootstrap_external>", line 999, in exec_module
      File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
      File "/home/cecilia/lavoro/fairy-magic-nomad-distro-dev/packages/nomad-parser-plugins-simulation/src/nomad_simulation_parsers/parsers/abinit/parser.py", line 26, in <module>
        from nomad_simulation_parsers.schema_packages import abinit
      File "/home/cecilia/lavoro/fairy-magic-nomad-distro-dev/packages/nomad-parser-plugins-simulation/src/nomad_simulation_parsers/schema_packages/abinit.py", line 54, in <module>
        class Representation(model_system.Representation):
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^
    AttributeError: module 'nomad_simulations.schema_packages.model_system' has no attribute 'Representation'
  - exception_hash: _1slL0W6c7zboUYmTj-eJth_4gd1
  - nomad.commit: 
  - nomad.deployment: oasis
  - nomad.service: cli
  - nomad.version: 1.4.1.dev95+g852d11600.d20251111
  - taskName: None
Traceback (most recent call last):
  File "/home/cecilia/lavoro/fairy-magic-nomad-distro-dev/.venv/bin/nomad", line 10, in <module>
    sys.exit(run_cli())
             ^^^^^^^^^
  File "/home/cecilia/lavoro/fairy-magic-nomad-distro-dev/packages/nomad-FAIR/nomad/cli/cli.py", line 77, in run_cli
    return cli(obj=POPO())  # pylint: disable=E1120,E1123
           ^^^^^^^^^^^^^^^
  File "/home/cecilia/lavoro/fairy-magic-nomad-distro-dev/.venv/lib/python3.12/site-packages/click/core.py", line 1462, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/cecilia/lavoro/fairy-magic-nomad-distro-dev/.venv/lib/python3.12/site-packages/click/core.py", line 1383, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/home/cecilia/lavoro/fairy-magic-nomad-distro-dev/.venv/lib/python3.12/site-packages/click/core.py", line 1850, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/cecilia/lavoro/fairy-magic-nomad-distro-dev/.venv/lib/python3.12/site-packages/click/core.py", line 1246, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/cecilia/lavoro/fairy-magic-nomad-distro-dev/.venv/lib/python3.12/site-packages/click/core.py", line 814, in invoke
    return callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/cecilia/lavoro/fairy-magic-nomad-distro-dev/packages/nomad-FAIR/nomad/cli/parse.py", line 110, in _parse
    entry_archives = parse(mainfile, **kwargs)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/cecilia/lavoro/fairy-magic-nomad-distro-dev/packages/nomad-FAIR/nomad/client/processing.py", line 53, in parse
    from nomad.parsing import parsers
  File "/home/cecilia/lavoro/fairy-magic-nomad-distro-dev/packages/nomad-FAIR/nomad/parsing/parsers.py", line 249, in <module>
    assert isinstance(instance, Parser), (
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: Error loading entry point "nomad_simulation_parsers.parsers:abinit_parser": The load method of a parser entry point must return a Parser instance

[ladinesa]

please pull the most recent nomad-simulations develop branch

[cecizag]

Thanks, now I can start it. I get the same situation described here:

Hi @ladinesa ,

thank you. I tried the new version, but I’m getting the following result:

* When I run the `nomad parse` command from the command line, the files are parsed correctly, both `.out` and `.xml`.

* When processing the uploads from the GUI, the entry for `quartz-nmr.out` or `quartz-nmr.xml` is created correctly, and the `program`, `model system`, and `outputs` sections are populated. However, in `outputs` I only see the quantities inherited from the `scf`; all the NMR-specific quantities from the file are missing.

[ladinesa]

I tried your branch now and it seems to work

[cecizag]

Hi @ladinesa ,

I figured out why we weren’t able to get the same outputs:

• If I run nomad-distro-dev without modifying nomad.yaml, I get the same situation you showed:

  • the GIPAW quantities are present in outputs, but program and model_system are not inherited from the self-consistent run.

• If I add this to nomad.yaml:

temporal:
  enabled: false

• then program and model_system are inherited from the self-consistent run, but the GIPAW quantities are missing from outputs.

[ladinesa]

@cecizag are you done with the implementation? If yes, I will merge this and fix the remaining technical issues.

[cecizag]

@ladinesa yes, you can proceed with the merge. Thank you!