Review of submission Ristin_RE_AE

[LloydThinks]

Author: @mristin
Reviewer 1: @AnonymousPenguin123
Reviewer 2: tba

[AnonymousPenguin123]

Hi, I have a Mac and I am stuck trying to run the code on my machine. Is there a specific order run the python scripts?
Thanks :)

[mristin]

Hi @AnonymousPenguin123 ,
Sorry, I'd don't have a Mac. Could you maybe post what commands you are trying to run and also post the output / error messages?

Maybe you have conflicting virtual environments?

Once you installed rasaeco in your virtual environment, there is a single console executable, pyrasaeco-render and you only need to run:

pyrasaeco-render once --scenarios_dir /path/to/some/scenarios

[AnonymousPenguin123]

Thank you for the answer :)
I now got the same error when using:

• pyrasaeco-render once --scenarios_dir /sample_scenarios/scaffolding/scenario.md
• pyrasaeco-render once --scenarios_dir /sample_scenarios/scaffolding/
• pyrasaeco-render once --scenarios_dir /Users/coolPinguin123/Desktop/Reviews/artifacts/rasaeco-0.0.14/sample_scenarios/scaffolding/scenario.md
• pyrasaeco-render once --scenarios_dir /Users/coolPinguin123/Desktop/Reviews/artifacts/rasaeco-0.0.14/sample_scenarios/z_dummy_scenario/

The stack trace:

(RASAECOvenv) Pinguins-Macbook:artifacts coolPinguin123$ pyrasaeco-render once --scenarios_dir /sample_scenarios/scaffolding/scenario.md
Traceback (most recent call last):
  File "/Users/coolPinguin123/Desktop/Reviews/artifacts/RASAECOvenv/bin/pyrasaeco-render", line 11, in <module>
    load_entry_point('rasaeco==0.0.14', 'console_scripts', 'pyrasaeco-render')()
  File "/Users/coolPinguin123/Desktop/Reviews/artifacts/RASAECOvenv/lib/python3.8/site-packages/rasaeco/pyrasaeco_render.py", line 386, in entry_point
    return run(argv=sys.argv[1:], stdout=sys.stdout, stderr=sys.stderr)
  File "/Users/coolPinguin123/Desktop/Reviews/artifacts/RASAECOvenv/lib/python3.8/site-packages/rasaeco/pyrasaeco_render.py", line 335, in run
    errors = rasaeco.render.once(scenarios_dir=command.scenarios_dir)
  File "/Users/coolPinguin123/Desktop/Reviews/artifacts/RASAECOvenv/lib/python3.8/site-packages/rasaeco/render.py", line 910, in once
    ontology, errors = rasaeco.intermediate.load_ontology(scenarios_dir=scenarios_dir)
  File "/Users/coolPinguin123/Desktop/Reviews/artifacts/RASAECOvenv/lib/python3.8/site-packages/icontract/_checkers.py", line 631, in wrapper
    raise violation_error
icontract.errors.ViolationError: File /Users/coolPinguin123/Desktop/Reviews/artifacts/RASAECOvenv/lib/python3.8/site-packages/rasaeco/intermediate.py, line 361 in <module>:
scenarios_dir.is_dir():
scenarios_dir was PosixPath('/sample_scenarios/scaffolding/scenario.md')
scenarios_dir.is_dir() was False

Can you help me? Is the "path/to/some/scenario" correct?

Thank you

[mristin]

Hi @AnonymousPenguin123 ,
The error message is telling that the --scenarios_dir is not a directory. As far as I can see, you supplied a file (/sample_scenarios/scaffolding/scenario.md).

Could you please try the following call:

pyrasaeco-render once --scenarios_dir /sample_scenarios

?

[AnonymousPenguin123]

I tried the call:
(RASAECOvenv) Pinguins-Macbook:artifacts coolPinguin123$ pyrasaeco-render once --scenarios_dir /sample_scenarios

and received:

Traceback (most recent call last):
  File "/Users/coolPinguin123/Desktop/Reviews/artifacts/RASAECOvenv/bin/pyrasaeco-render", line 11, in <module>
    load_entry_point('rasaeco==0.0.14', 'console_scripts', 'pyrasaeco-render')()
  File "/Users/coolPinguin123/Desktop/Reviews/artifacts/RASAECOvenv/lib/python3.8/site-packages/rasaeco/pyrasaeco_render.py", line 386, in entry_point
    return run(argv=sys.argv[1:], stdout=sys.stdout, stderr=sys.stderr)
  File "/Users/coolPinguin123/Desktop/Reviews/artifacts/RASAECOvenv/lib/python3.8/site-packages/rasaeco/pyrasaeco_render.py", line 335, in run
    errors = rasaeco.render.once(scenarios_dir=command.scenarios_dir)
  File "/Users/coolPinguin123/Desktop/Reviews/artifacts/RASAECOvenv/lib/python3.8/site-packages/rasaeco/render.py", line 910, in once
    ontology, errors = rasaeco.intermediate.load_ontology(scenarios_dir=scenarios_dir)
  File "/Users/coolPinguin123/Desktop/Reviews/artifacts/RASAECOvenv/lib/python3.8/site-packages/icontract/_checkers.py", line 631, in wrapper
    raise violation_error
icontract.errors.ViolationError: File /Users/coolPinguin123/Desktop/Reviews/artifacts/RASAECOvenv/lib/python3.8/site-packages/rasaeco/intermediate.py, line 361 in <module>:
scenarios_dir.is_dir():
scenarios_dir was PosixPath('/sample_scenarios')
scenarios_dir.is_dir() was False

[mristin]

@AnonymousPenguin123 could you please check that the path /sample_scenarios is actually a directory on your system?

[AnonymousPenguin123]

These are the directories:

I tried it from inside artifacts and inside rasaeco-0.0.14.

[AnonymousPenguin123]

I think I found a mistake, the first "/" should not be in the command. I played around with the path. If I do
(RASAECOvenv) Pinguins-Macbook:artifacts coolPinguin123 $ pyrasaeco-render once --scenarios_dir rasaeco-0.0.14/sample_scenarios
I get the following warning:

/Users/coolPinguin123/Desktop/Reviews/artifacts/RASAECOvenv/lib/python3.8/site-packages/rasaeco/render.py:176: MatplotlibDeprecationWarning: Calling gca() with keyword arguments was deprecated in Matplotlib 3.4. Starting two minor releases later, gca() will take no keyword arguments. The gca() function should only be used to get the current axes, or if no axes exist, create new axes with default keyword arguments. To create a new axes with non-default arguments, use plt.axes() or plt.subplot().
  ax = fig.gca(projection="3d")
/Users/coolPinguin123/Desktop/Reviews/artifacts/RASAECOvenv/lib/python3.8/site-packages/rasaeco/render.py:218: MatplotlibDeprecationWarning: Calling gca() with keyword arguments was deprecated in Matplotlib 3.4. Starting two minor releases later, gca() will take no keyword arguments. The gca() function should only be used to get the current axes, or if no axes exist, create new axes with default keyword arguments. To create a new axes with non-default arguments, use plt.axes() or plt.subplot().
  ax = fig.gca(projection="3d")

and if I switch directories:
(RASAECOvenv) Pinguins-Macbook:artifacts coolPinguin123 $ cd rasaeco-0.0.14
(RASAECOvenv) Pinguins-Macbook:rasaeco-0.0.14 coolPinguin123 $ pyrasaeco-render once --scenarios_dir sample_scenarios

I get the same warning again:

/Users/coolPinguin123/Desktop/Reviews/artifacts/RASAECOvenv/lib/python3.8/site-packages/rasaeco/render.py:176: MatplotlibDeprecationWarning: Calling gca() with keyword arguments was deprecated in Matplotlib 3.4. Starting two minor releases later, gca() will take no keyword arguments. The gca() function should only be used to get the current axes, or if no axes exist, create new axes with default keyword arguments. To create a new axes with non-default arguments, use plt.axes() or plt.subplot().
  ax = fig.gca(projection="3d")
/Users/coolPinguin123/Desktop/Reviews/artifacts/RASAECOvenv/lib/python3.8/site-packages/rasaeco/render.py:218: MatplotlibDeprecationWarning: Calling gca() with keyword arguments was deprecated in Matplotlib 3.4. Starting two minor releases later, gca() will take no keyword arguments. The gca() function should only be used to get the current axes, or if no axes exist, create new axes with default keyword arguments. To create a new axes with non-default arguments, use plt.axes() or plt.subplot().
  ax = fig.gca(projection="3d")

But other than that I do not get more errors.

[mristin]

Ah, sorry, I supposed you had an absolute directory /sample_scenarios on your system. Please let me know if there are any other issues.

You can ignore the warnings.

[AnonymousPenguin123]

Artifacts Available - Available Badge - yes

• Artifacts are hosted online. Yes
• The URL to access the artifacts is immutable (cannot be altered by the author). Yes
• The organisation hosting the URL plans to maintain it for the foreseeable future. Yes, it is on Zenodo
• Artifacts have a Digital Object Identifier (DOI) redirecting to the immutable URL. DOI available
• Anyone can access the artifacts, without the need for registration. Yes

Artifacts Reusable - Reusable Badge - almost

• Artifacts can be obtained via the “Artifact Location” README section. Yes
• Artifacts are documented in the “Descriptions of Artifacts” README section. Yes, with suggestions for improvement

• I assume this is the first section of the GitHub, it would be nice if you could label it accordingly, but not a must-have
• The description is a bit brief, it would be nice if you would put in a few more details.

• Artifacts can be installed and run via the “Installation Instructions” README section. Yes
• Artifacts generate the results presented in the article following the “Steps to Reproduce” README section. Not applicable
• Artifacts have a proper open-source license attached as a LICENSE.md file. Yes
• The artifacts are very carefully documented and well-structured to the extent that reuse and repurposing is facilitated. Norms and standards of the research community for artifacts of this type are strictly adhered to. Almost

• The current README of the artifact's GitHub is very well documented for Windows, but there are no specific instructions for Linux / Mac, I got it running after our discussion, please add specific commands for Linux and Mac users
• Suggestion: Python -m venv venv it would be nicer if the venv had a more descriptive name, f.e. python3.8 -m venv RASAECOvenv, it might be confusing for people that have no experience using virtual environments.

[mristin]

Hi @AnonymousPenguin123,

Thanks for your feedback!

Could you help me a bit with this point:

The current README of the artifact's GitHub is very well documented for Windows, but there are no specific instructions for Linux / Mac, I got it running after our discussion, please add specific commands for Linux and Mac users

If I understood you correctly, you actually ran the exactly same commands as they are written in the readme? Or did I miss any other steps?

The path to the scenarios directory is of course user specific, not system specific (whether on Windows, Mac, Linux etc.). I can try to add a more user-friendly error message if you think that would help?

I'll try to address the other points. Should I simply update the GitHub repository till we converge and then update the Zenodo only in the end?

[AnonymousPenguin123]

Hi @mristin,

I ran the exact same steps with these slightly altered commands:

• pyrasaeco-render once --scenarios_dir some\path\to\scenarios
• pyrasaeco-render continuously --scenarios_dir some\path\to\scenarios
• pyrasaeco-render continuously --scenarios_dir some\path\to\scenarios --port 8000

Yes a more user-friendly error message would also help.

Updating the GitHub repo and the Zenodo only in the end makes sense.