Skip to content

Conversation

@MauroAbidalCarrer
Copy link
Contributor

Hello, this is a follow up to my previous pull request.

The new code allows the user to pick up from a conversation saved into a notebook, or from any notebook even if it wasn't created by Local-Code-Interpreter.

The code cells get rerun to recreate the jupyter kernel state.
This behavior assumes that the output of the code is deterministic/reproducible which is usually the case for notebooks.

As of making this PR, if there happened to be a discrepancy between the old code output and the new one, the behavior is undefined.
Maybe we should ask the user what tho do when that happens, but it's not clear what options we would give him to remedy such discrepancy.

The code of jupyter_backend and bot_backend modules have been modified so that the working directory of the kernel is the same as the notebook this is to avoid output discrepancy due to file inputs.

As of now, this branch is more of a place to discuss the implementation of jupyter kernel state deserialization than the actual final implementation.

The other possible solutions to this problem are:

  • docker: use a container and pause it whenever needed.
    This is effective but might add a layer of complexity though on every other feature.
    It would also be more comber sum for sharing conversation compared to just sharing a notebook (and potentially some input files).
  • dill: enhanced pickle that can save the kernel state according to this reddit post.
    The main issues with this option is that dill cannot serialized every thing and that.
    The serialized objects are kind of black boxes and might have compatibility issues between versions of Local-Code-Interperter .

Overall I believe that rerunning the code cell is the best option.
Let me know what you think, hopw this will be useful.

@MrGreyfun
Copy link
Owner

Hello, thanks for this PR. Deserializing a notebook and restoring its kernel state is indeed a challenging task. Actually, I've tried several methods of restoring kernel state before, including the dill package you mentioned. I was satisfied with this package, but I haven't fully tested it. Additionally, using this package introduces the need for users to store two files (notebook and kernel state file) for recovering a conversion.

Deserialization through code rerunning is a simple approach, but there are at least three problem:

  • As you pointed out, the output may not be reproducible.
  • User-uploaded files must be in the working directory, or else it will result in a FileNotFoundError during rerunning.
  • The code rerunning process may take a considerable amount of time.

As of now, all known methods have their limitations, and it might be necessary to find a suitable trade-off or discover a better approach. Given my current busy schedule, it may take me some time (maybe a week or longer) to thoroughly review and test the code. I will also carefully consider the implementation approach for kernel state deserialization. I'll reply as soon as I have completed this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants