Repurpose the datastores to merge values, not replace

[joepavitt]

Description

• Rather than replacing the latest msg in the datastore, we merge the objects, overriding new values where relevant, but persistent values from previous msg's that have not had new values set.
• This then enables a significant clean of most widgets, such that they can rely on their respective value being available in the centralised vuex store.

Related Issue(s)

Closes #1294

[bartbutenaers]

Hi @joepavitt,
Thanks for having a look at this!!
Not the most fun job to do, but hopefully it will make things more stable and easy to understand.
I see that you managed to simplify the codebase a lot.
Much appreciated!!

[joepavitt]

Still a lot more I can clean out I think, was just taking a snapshot to share progress.

[colinl]

Does this change the contents of the 'last' message replay, that is received when a page connects or refreshes?

[bartbutenaers]

@colinl,
If I understand correctly it works like this (but I have only reviewed very quickly):

1. You inject an msg with payload "xxx".
2. That msg is stored for the ui-node id.
3. You inject an msg with some ui_update stuff to change some dynamic properties.
4. That msg is merged with the previous last msg, and the ui_update stuff is removed from the merged msg before replacing the last msg by the merged msg. So this new last msg still contains a payload "xxx".
5. You inject an msg with payload "yyy".
6. That msg is merged with the previous last msg, and the ui_update stuff is removed from the merged msg before replacing the last msg by the merged msg. So this new last msg will contain the updated payload "yyy".

So you have 1 last msg per ui node. That last msg is continiously updated by merging new input messages into it (while ignoring all ui_update stuff).

Which means it is in fact not correct anymore to talk about the last msg, because it is in fact build by merging N successive input messages.

• If these messages have different properties, then the merged message will contain all those properties.
• If these messages have the same properties, then the merged message will only contain the last values for that properties (because the previous values have been overwritten).

As a result you can replay that (last) merged message, because it always contains the (last) complete state.

Although I am in a very negative mood at the moment, I can't make up any scenario at the moment that would fail with this new setup ;-)

It looks like a decent solution to me.

[colinl]

OK, so potentially this could break existing contributed nodes. I think I will probably be ok because that is basically what I do myself anyway.

[bartbutenaers]

Not sure if it will break existing nodes. Like you say you (and I) already have added some code to workaround the data loss.

I assume there will be some obsolete code in the third-party ui nodes.

[colinl]

I assume there will be some obsolete code in the third-party ui nodes.

Yes, I should be able to simplify the code.

[colinl]

Will this affect ui-template node operation?

[joepavitt]

Will this affect ui-template node operation?

No, anything already built will work as-is, this will mostly be cleaning the underlying core nodes.

[Steve-Mcl]

LGTM. Will likely break #1237 again sigh