Feature/markov transition #23
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Fix bug when scaling Power_VRESProfiles Add warning (instead of ExecutionError) if ExcelWriter doesn't find a file
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Optimize excel reading workflow
Contributor
There was a problem hiding this comment.
Pull Request Overview
This PR introduces parallel reading of Excel files using the calamine engine to improve performance, along with several bug fixes and enhancements. The changes include switching from openpyxl to calamine for reading Excel files, adding parallel file reading capabilities using threading, and improving error handling and reporting.
Key changes:
- Switched Excel reading engine from
openpyxltocalaminefor improved performance - Added parallel file reading using
ThreadPoolExecutorwith configurable worker count - Enhanced error handling for missing files and version checking
- Fixed several bugs including comment comparison logic and incorrect column references
Reviewed Changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 5 comments.
| File | Description |
|---|---|
| environment.yml | Added python-calamine=0.5.3 dependency for faster Excel reading |
| ExcelReader.py | Refactored version checking to use pandas with calamine engine; fixed comment comparison logic; added sheet skipping for ~ prefixed sheets |
| ExcelWriter.py | Added file existence validation and improved error reporting with exit code on differences |
| CaseStudy.py | Implemented parallel Excel file reading with ThreadPoolExecutor; refactored file loading logic; fixed attribute checks and column name bugs |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.