CGenerator::Library::CommitError

[ErikDeBruijn]

I'd love to see if I can use redshift for simulating a smart grid with EV chargers. I'm intrigued of the DSL you created compared to other tools that have a hard to understand syntax.
However, I'm running into this issue that's probably because the project is a bit older and ruby is now at version above 2.6.1 (my particular instance is at ruby 2.6.1p33 (2019-01-30 revision 66950) [x86_64-darwin18]).

git clone https://github.com/vjoel/redshift.git
cd redshift/examples
ruby thermostat.rb

I got this result:

Traceback (most recent call last):
	17: from thermostat.rb:29:in `<main>'
	16: from /Users/erik/.rbenv/versions/2.6.1/lib/ruby/gems/2.6.0/gems/redshift-1.3.29/lib/redshift/world.rb:70:in `new'
	15: from /Users/erik/.rbenv/versions/2.6.1/lib/ruby/gems/2.6.0/gems/redshift-1.3.29/lib/redshift/target/c/world-gen.rb:150:in `new'
	14: from /Users/erik/.rbenv/versions/2.6.1/lib/ruby/gems/2.6.0/gems/cgen-0.16.12/lib/cgen/cshadow.rb:363:in `commit'
	13: from /Users/erik/.rbenv/versions/2.6.1/lib/ruby/gems/2.6.0/gems/redshift-1.3.29/lib/redshift/target/c/library.rb:66:in `commit'
	12: from /Users/erik/.rbenv/versions/2.6.1/lib/ruby/gems/2.6.0/gems/cgen-0.16.12/lib/cgen/cgen.rb:646:in `use_work_dir'
	11: from /Users/erik/.rbenv/versions/2.6.1/lib/ruby/gems/2.6.0/gems/cgen-0.16.12/lib/cgen/cgen.rb:646:in `chdir'
	10: from /Users/erik/.rbenv/versions/2.6.1/lib/ruby/gems/2.6.0/gems/cgen-0.16.12/lib/cgen/cgen.rb:647:in `block in use_work_dir'
	 9: from /Users/erik/.rbenv/versions/2.6.1/lib/ruby/gems/2.6.0/gems/redshift-1.3.29/lib/redshift/target/c/library.rb:69:in `block in commit'
	 8: from /Users/erik/.rbenv/versions/2.6.1/lib/ruby/gems/2.6.0/gems/cgen-0.16.12/lib/cgen/cgen.rb:744:in `commit'
	 7: from /Users/erik/.rbenv/versions/2.6.1/lib/ruby/gems/2.6.0/gems/cgen-0.16.12/lib/cgen/cgen.rb:784:in `show_times'
	 6: from /Users/erik/.rbenv/versions/2.6.1/lib/ruby/gems/2.6.0/gems/cgen-0.16.12/lib/cgen/cgen.rb:744:in `block in commit'
	 5: from /Users/erik/.rbenv/versions/2.6.1/lib/ruby/gems/2.6.0/gems/redshift-1.3.29/lib/redshift/target/c/library.rb:58:in `make'
	 4: from /Users/erik/.rbenv/versions/2.6.1/lib/ruby/gems/2.6.0/gems/cgen-0.16.12/lib/cgen/cgen.rb:982:in `make'
	 3: from /Users/erik/.rbenv/versions/2.6.1/lib/ruby/gems/2.6.0/gems/cgen-0.16.12/lib/cgen/cgen.rb:1025:in `build_wrapper'
	 2: from /Users/erik/.rbenv/versions/2.6.1/lib/ruby/gems/2.6.0/gems/cgen-0.16.12/lib/cgen/cgen.rb:1025:in `chdir'
	 1: from /Users/erik/.rbenv/versions/2.6.1/lib/ruby/gems/2.6.0/gems/cgen-0.16.12/lib/cgen/cgen.rb:1025:in `block in build_wrapper'
/Users/erik/.rbenv/versions/2.6.1/lib/ruby/gems/2.6.0/gems/cgen-0.16.12/lib/cgen/cgen.rb:984:in `block in make':  (CGenerator::Library::CommitError)
  Make   failed for thermostat_clib_261.
  Transcript is saved in thermostat_clib_261/make.log and follows:
  ____________________________________________________________
 | creating Makefile
 | linking shared-object thermostat_clib_261.bundle
 | duplicate symbol '_dummy' in:
 |     Component.o
 |     Thermostat.o
 | duplicate symbol '_dummy' in:
 |     Component.o
 |     flow_Thermostat_temp_Heat.o
 | duplicate symbol '_dummy' in:
 |     Component.o
 |     flow_Thermostat_temp_Off.o
 | duplicate symbol '_dummy' in:
 |     Component.o
 |     guard_Thermostat_1.o
 | duplicate symbol '_dummy' in:
 |     Component.o
 |     guard_Thermostat_0.o
 | duplicate symbol '_dummy' in:
 |     Component.o
 |     World.o
 | duplicate symbol '_dummy' in:
 |     Component.o
 |     libmain.o
 | ld: 7 duplicate symbols for architecture x86_64
 | clang: error: linker command failed with exit code 1 (use -v to see invocation)
 | make: *** [thermostat_clib_261.bundle] Error 1
  ____________________________________________________________
➜  examples git:(master) clang

make.log has the following contents:

creating Makefile
linking shared-object thermostat_clib_261.bundle
duplicate symbol '_dummy' in:
    Component.o
    Thermostat.o
duplicate symbol '_dummy' in:
    Component.o
    flow_Thermostat_temp_Heat.o
duplicate symbol '_dummy' in:
    Component.o
    flow_Thermostat_temp_Off.o
duplicate symbol '_dummy' in:
    Component.o
    guard_Thermostat_1.o
duplicate symbol '_dummy' in:
    Component.o
    guard_Thermostat_0.o
duplicate symbol '_dummy' in:
    Component.o
    World.o
duplicate symbol '_dummy' in:
    Component.o
    libmain.o
ld: 7 duplicate symbols for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [thermostat_clib_261.bundle] Error 1

I looked at the thermostat_clib_261 directory that was generated and I didn't know where to look to see if I could fix something.

I'm curious to know if you have any pointers of what to try next to get this working. Let me know if you need additional info. Thanks!

[ErikDeBruijn]

FYI: a friend of mine tried it on his MacBook (OS X) and got the same error. I don't think it has to do with Ruby as much as it has with the build process initiated by the Makefile, but it's a bit over my head to debug a.t.m.

If anyone knows of another hybrid automaton way to simulate physical + digital processes, I'm all ears (please mention me here! :) )!

[vjoel]

I'll try it with both ruby 2.6 and 2.7. I've gotten behind on matz's xmas releases...

If your network of components (aka blocks) is static, then there are many other options. If you need dynamic reconfiguration (discrete transitions cause changes in which outputs feed into which inputs), then I do not know. That was why SHIFT was invented in the mid-90s, and RedShift followed.

[vjoel]

The example works for me with ruby 2.7 and gcc. Maybe clang has some differences. Is it possible for you to use gcc instead?

[ErikDeBruijn]

Hi @vjoel , thanks for taking the time to respond!

I now have /usr/local/bin in my PATH before /usr/bin and linked gcc-9 to gcc, so when I run gcc in a shell I get the Homebrew GCC 9.2.0. Changing CC and CXX in the makefile was needed to get it to use Homebew's version of GCC.

The result (except that all warnings are now gone) is very similar:

make
linking shared-object sandbox_clib_261.bundle
duplicate symbol '_dummy' in:
    Component.o
    Planet.o
duplicate symbol '_dummy' in:
    Component.o
    World.o
duplicate symbol '_dummy' in:
    Component.o
    flow_Planet_ax_Enter.o
duplicate symbol '_dummy' in:
    Component.o
    flow_Planet_ay_Enter.o
duplicate symbol '_dummy' in:
    Component.o
    flow_Planet_r_Enter.o
duplicate symbol '_dummy' in:
    Component.o
    flow_Planet_vx_Enter.o
duplicate symbol '_dummy' in:
    Component.o
    flow_Planet_vy_Enter.o
duplicate symbol '_dummy' in:
    Component.o
    flow_Planet_w_Enter.o
duplicate symbol '_dummy' in:
    Component.o
    flow_Planet_x_Enter.o
duplicate symbol '_dummy' in:
    Component.o
    flow_Planet_y_Enter.o
duplicate symbol '_dummy' in:
    Component.o
    libmain.o
ld: 11 duplicate symbols for architecture x86_64
collect2: error: ld returned 1 exit status
make: *** [sandbox_clib_261.bundle] Error 1

My Ruby version is ruby 2.6.1p33 (2019-01-30 revision 66950). Using rbenv install 2.7.0-dev I somehow ended up with ruby 2.8.0dev. There was no difference in the result w.r.t. compilation. Let me know if you want me to try to get 2.7 working to try to converge to the setup you currently have...

Interestingly, I didn't need to change the makefile with this version of Ruby.

Anything else I should change/try?

if your network of components (aka blocks) is static, then there are many other options. If you need dynamic reconfiguration (discrete transitions cause changes in which outputs feed into which inputs), then I do not know. That was why SHIFT was invented in the mid-90s, and RedShift followed.

Not so sure whether my setup is so special that it requires dynamic reconfiguration. Apart from energy flows from renewable sources, we will have multiple EVs (basically buffers/batteries) that are either connected or not, that can be charging if they are not already 100% full. So there are conditions and constraints, but I don't think things require dynamic reconfiguration... O.t.o.h. if you are up for the challenge to get redshift working together with me, I'm happy to try a few more things to make that happen!

[vjoel]

Can you construct a minimal example which exhibits this problem, and then post (or email) the foo.rb file as well as the foo_clib dir (after a make clean)?

[vjoel]

Hi, @ErikDeBruijn, just wondering if it's possible to send me your sandbox_clib files? This dir should be in a tmp dir adjacent to your source sandbox.rb file You can make clean in that dir first--I don't think the object files would be helpful. Thanks!

[vjoel]

Also, please try applying this patch:

diff --git a/lib/redshift/target/c/world-gen.rb b/lib/redshift/target/c/world-gen.rb
index 637cc15..95839bf 100644
--- a/lib/redshift/target/c/world-gen.rb
+++ b/lib/redshift/target/c/world-gen.rb
@@ -20,7 +20,6 @@ class World
             1);                                        \\
           list_i--)
 
-    int dummy;
   '.tabto(0)
   # Note: EACH_COMP_DO(lc) block may use move_comp and remove_comp
   # but it should (re)move none or all. Must have declarations for comp

This doesn't break any tests for me, and it may fix the errors you are getting.

I don't recall what the dummy var was for. Strange not to declare it as extern in that file.

Possibly it was to make some old compiler happy. RedShift worked with msvc, for example, though I don't test it any more.

[ErikDeBruijn]

Hi @vjoel : I tried to apply this patch to my fork and want to write about my results, however given that it's late I'll do this tomorrow. I wanted to let you know that I didn't abandon this issue yet and really appreciate you helping me out!

[ErikDeBruijn]

Hi @vjoel !

Initially it didn't actually use the updated gem, so I got a false negative testing it with the patch.Now your patch did appear to (removing int dummy;) solve the build problem for me!

1. I fetched the gem, patched it and installed it
2. I ran the tests through rake (see below)
3. It works!

I could now run ruby ball.rb succesfully (using GNUTERM=canvas):

I think it would make sense to publish an update to this gem, don't you agree?

Just FYI, one of the tests does give a failure. This is probably unrelated to the patch, though:
rake test:
Everything succeeded except:

Starting test_strict_continuity.rb...
Run options: --seed 9253

# Running:

.F

Failure:
TestStrictContinuity#test_strict_continuity [test_strict_continuity.rb:376]:
Expected: 1
  Actual: 4

And:

Starting test_numerics.rb...
Traceback (most recent call last):
	2: from test_numerics.rb:1:in `<main>'
	1: from /Users/erik/.rbenv/versions/2.6.1/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:in `require'
/Users/erik/.rbenv/versions/2.6.1/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:in `require': cannot load such file -- mathn (LoadError)
__________________________________________________
Starting test_queue.rb...
Run options: --seed 64687

# Running:

..

[vjoel]

That's great to hear, @ErikDeBruijn.

I'll take a look at thetest failures and update the gem.

[ErikDeBruijn]

Hi! Sorry if I'm bugging you. Is it possible for you to update the gem? I've been able to successfully use redshift for simulations of EV charging and was experimenting with heat pumps as well. However, on my old laptop I could use a gem I had patched, I'm having trouble repeating them on my new laptop. This makes me unable to use the patched version.

The patch that resolved the problem for me was simply the one you provided (removing the line with int dummy;). I doubt whether the failed tests have anything to do with the patch, so hopefully it isn't much work for you to bump the version to 1.3.30 for you?

It's really exciting to be able to state machines in conjunction with physics equations in ruby, so that's why I can't wait to use it again!

[vjoel]

Erik, I'd be happy to take a look and update the gem if possible. Glad to hear you find it useful. Joel

…

On 3/4/21 4:49 AM, Erik de Bruijn wrote: Hi! Sorry if I'm bugging you. Is it possible for you to update the gem? I've been able to successfully use redshift for simulations of EV charging and was experimenting with heat pumps as well. However, on my old laptop I could use a gem I had patched, I'm having trouble repeating them on my new laptop. This makes me unable to use the patched version. The patch that resolved the problem for me was simply the one you provided (removing the line with |int dummy;|). I doubt whether the failed tests have anything to do with the patch, so hopefully it isn't much work for you to bump the version to 1.3.30 for you? It's really exciting to be able to state machines in conjunction with physics equations in ruby, so that's why I can't wait to use it again! — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#2 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAN3FLQBPVC77FEWIQPUH3TB56VVANCNFSM4J6NSBCA>.

[vjoel]

Erik, I just pushed redshift-1.3.30. The only change is removing the "int dummy" that was causing the problem. It was a workaround that is apparently not needed any more. I am testing with the following: ruby 2.7.2p137 (2020-10-01 revision 5445e04352) [x86_64-linux] gcc version 7.5.0 (Ubuntu 7.5.0-3ubuntu1~18.04) To get the tests to pass, I had to install some gems that (I guess) ruby no longer includes by default: gem install cmath mathn There are some new test deprecation warnings, and the (probably not important) failure in test_strict_continuity.rb. Other than that, it seem to work for me. Hope it does for you too. Joel

…

On 3/4/21 10:59 AM, Joel VanderWerf wrote: Erik, I'd be happy to take a look and update the gem if possible. Glad to hear you find it useful. Joel On 3/4/21 4:49 AM, Erik de Bruijn wrote: > Hi! Sorry if I'm bugging you. Is it possible for you to update the > gem? I've been able to successfully use redshift for simulations of EV > charging and was experimenting with heat pumps as well. However, on my > old laptop I could use a gem I had patched, I'm having trouble > repeating them on my new laptop. This makes me unable to use the > patched version. > > The patch that resolved the problem for me was simply the one you > provided (removing the line with |int dummy;|). I doubt whether the > failed tests have anything to do with the patch, so hopefully it isn't > much work for you to bump the version to 1.3.30 for you? > > It's really exciting to be able to state machines in conjunction with > physics equations in ruby, so that's why I can't wait to use it again! > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > <#2 (comment)>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/AAAN3FLQBPVC77FEWIQPUH3TB56VVANCNFSM4J6NSBCA>. > >

[ErikDeBruijn]

Thanks, Joel! B.t.w. I'm amazed at how well it works to simulate things!