TypeError: self._context,self._fmu,self.callBackFunctions,self.callbacks cannot be converted to a Python object for pickling

[jschueller]

Describe the bug
PyFMI model objects cannot be serialized, which makes it impossible from using multiprocessing-like parallel evaluations (joblib etc)

To Reproduce

import pyfmi
import pickle
model = pyfmi.load_fmu("BouncingBall.fmu")
with open("study.pkl", "wb") as f:
    pickle.dump(model, f)
with open("study.pkl", "rb") as f:
    model2 = pickle.load(f)

Observed behavior

model = <pyfmi.fmi2.FMUModelCS2 object at 0x5fe351a1a6d0>
>   ???
E   TypeError: self._context,self._fmu,self.callBackFunctions,self.callbacks cannot be converted to a Python object for pickling

Expected behavior
script passes

Versions

• PyFMI 2.18.3 (latest)
• openmodelica fmu

additional context
what usually causes pain is the cython attributes
maybe a path forward is override getstate/setstate operators, pass pure-python arguments only and recreate a model instance from scratch so that cython attributes are not really copied but reinstanciated

[PeterMeisrimelModelon]

Can you elaborate on exactly what you are trying to do, maybe a complete example? In terms of serializing FMUs, there is the fmiGetFMUState/fmiSetFMUState functionality.

/Peter

[jschueller]

I'm interested in parallelizing fmu simulations with different input values across multiple processes, something like that

import concurrent.futures
import pyfmi

model = pyfmi.load_fmu("BouncingBall.fmu")
X = [1, 2, 3]
with concurrent.futures.ThreadPoolExecutor(max_workers=10) as executor:
    future_to_x = {executor.submit(model.simulate, {"input": x}): x for x in X}
    for future in concurrent.futures.as_completed(future_to_x):
        x = future_to_x[future]
        y = future.result()

internally Python uses serialization of the context so it fails to pickle the model variable which is an instance of a PyFMI class

Do you have an example of fmiSetFMUState ?
I dont see anything in the doc.
I dont know if it this will help as it should be a pure python object also in order to be serialized.

[PeterMeisrimelModelon]

I have doubts what you are trying to do is going to work this way. I think the safest best is to simply load the FMU N times.

Given e.g., the example of BouncingBall, how would this deal with events? One needs to change the FMU state to appropriate deal with events + the FMU might have internal states that can be affected by events and differ based the external inputs.

On FMUState, first I recommend checking the specification here.

Example on how to use in PyFMI: https://github.com/modelon-community/PyFMI/blob/master/tests/test_fmi3_states.py#L152 (works the same for FMI2)
Note that this is optional functionality, you'll need to check if your FMU generating tool supports generation of FMUs that support this.

Essentially, get_fmu_state creates a Python object housing an opaque pointer of the FMU, that be used to (re)set the FMUs internal state.
de(serialization) allows to convert this internal data structure to a bytes for e.g., external storage.

/Peter

[jschueller]

yes, reloading works
but it would be convenient to do it at the pyfmi level