How to get a list of Jobs fields, ReservationFlags and ReservationReoccurrence?

[mrvn]

I have a GUI that interfaces with pyslurm.

Firstly the GUI can display Jobs from a cluster and allows sorting, filtering and manipulating them in a few ways. I'm getting the jobs from pyslurm and can display all the data raw in a table. But then I have columns like derived_exit_code_signal or federation_siblings_viable. To make things nicer I have a list of the possible fields with a nicer looking name for the column and a tooltip text and also hints how to make the data more human readable (e.g. time durations are turned into days/hours/minutes).

The problem I face is that with each new slurm / pyslurm version the jobs fields can change. So I want to get a list of fields a Job can have at build time and check if that differs from what the GUI has listed. One Idea I have is to use

import pyslurm
dir(pyslurm.Job)

Secondly the GUI also manages Reservations. So it needs to know about pyslurm.ReservationFlags and pyslurm.ReservationReoccurrence. There I can use:

import pyslurm
list(pyslurm.ReservationReoccurrence)
list(pyslurm.ReservationFlags)

But I'm left with one BIG and one small problem:

1. There is no slurm configured on the build system:

   import pyslurm
   python3: error: resolve_ctls_from_dns_srv: res_nsearch error: Unknown host
   python3: error: fetch_config: DNS SRV lookup failed
   python3: error: _establish_config_source: failed to fetch config
   python3: fatal: Could not establish a configuration source

Pyslurm can't be imported when slurm is not configured. I really don't want to have to setup slurm on the build system and automated CI checks. Can this be fixed?

2. I need to come up with good tooltips for each job field and find out the data content. Similar for reservation flags.

Any tips how to best find that info from either pyslurm source or slurm source?
Currently I'm manually checking slurm/slurm.h and use the comment behind each struct job_info member or #define for reservation flags as tooltip.

[tazend]

Hi,

For your first concern, just a little reasoning why it is like this: every slurm API call requires that first the slurm_init function is called once, so Slurm can perform some internal setup. Since this has to be done anyway to do anything substantial with pyslurm, it is automatically executed on import of pyslurm (at the end of the __init__.py file).

However, in your case, if you just want to inspect the classes without having slurm configured as a whole, it makes sense to actually not run this automatically, as it will fail if there is no slurm.conf file.

I could make this slurm_init call depend on an environment variable, for example PYSLURM_INIT. If you set it to something like PYSLURM_INIT=0 before executing your script, or via os.environ before importing pyslurm, then slurm_init will not be run. Would this work for you?
Of course if you would want to use the API afterwards in the same script, you would need to call pyslurm.slurm_init() manually. Otherwise it would segfault.

---

For your second problem:

You could potentially rely on the Docstrings of each class. pyslurm uses Google-Docstring style. You could either use any google docstring parser you'd like, or use the parser that is also utilized by pyslurm when building the documentation, which is griffe (install with something like pip install griffe==1.15.0)

Here is an example:

import pyslurm
import griffe

docstring = griffe.Docstring(pyslurm.Job.__doc__)
text, params, attrs = griffe.parse_google(docstring)

for attr in attrs.value:
    print(attr.name, attr.description, attr.annotation)

This way you would get every attribute available on a specific Class, alongside its type and the description of it. Would that work?

[mrvn]

PYSLURM_INIT=0 sounds perfect. The doc string also looks good, for Jobs. Thanks.

But not so good for e.g. ReservationFlags:

  >>> pyslurm.ReservationFlags.__doc__
  'Flags for Reservations that can be set.\n\n    See {scontrol#OPT_Flags} for more info.\n    '
  >>> pyslurm.ReservationFlags.MAINTENANCE.__doc__
  ''

[tazend]

Alright, I will get the option to disable the auto init in.
Unfortunately, I think you would need to use a different way for these Enums. Last time I tried, I wasn't able to document the enum members properly (so that it doesn't look ugly in the generated documentation)

You might need to use your list(pyslurm.ReservationFlags) approach here, and create your own tooltip in this case :).

[mrvn]

On second thought maybe creating a slurm.conf for the build system wouldn't be so bad. I've got it running now with just:

ClusterName=ClusterName
SlurmctldHost=SlurmctldHost

Both are totally made up. Obviously doing anything with pyslurm that needs to contact slurm doesn't work since the values are nonesense but that doesn't matter if all I need is to inspect the data structures / doc strings.