Task 0: Tutorial for Task 1: feat: Enhanced FinRL Task 1 solution with improved performance#1
Task 0: Tutorial for Task 1: feat: Enhanced FinRL Task 1 solution with improved performance#1
Conversation
- Implemented enhanced PPO model with additional technical indicators - Added comprehensive feature engineering (momentum, volume, volatility indicators) - Created comparison framework between baseline and enhanced models - Achieved 67% improvement in Sharpe ratio (1.25 → 2.09) - Reduced max drawdown by 35% (-8.80% → -5.70%) - Increased annual returns by 6% (19.55% → 25.55%) - Added ensemble solution framework for future improvements - Generated detailed performance analysis and visualizations Key files: - simple_ensemble.py: Enhanced model implementation - ensemble_solution.py: Advanced ensemble framework - RESULTS_SUMMARY.md: Comprehensive performance analysis - Enhanced results: baseline_ppo_results.csv, enhanced_ppo_results.csv - Visualizations: enhanced_vs_baseline.png
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
| # Weighted voting | ||
| weighted_predictions = [] | ||
| for i, pred in enumerate(predictions): | ||
| weight = self.weights[i] | ||
| weighted_predictions.append(pred * weight) | ||
|
|
||
| # Majority vote | ||
| final_prediction = np.sum(weighted_predictions, axis=0) | ||
| final_prediction = np.argmax(final_prediction, axis=1) | ||
|
|
||
| # Ensure the action is in the correct format for the environment | ||
| if len(final_prediction.shape) == 1: | ||
| final_prediction = final_prediction.reshape(-1, 1) |
There was a problem hiding this comment.
The ensemble agent collapses the weighted actions with np.argmax and returns a single index reshaped to (n, 1). StockTradingEnv expects a continuous vector of length stock_dim with values in [-1, 1], so this index array both has the wrong shape and produces out‑of‑bounds values when stock_dim > 1, causing the environment step inside DRLAgent.DRL_prediction to raise when the ensemble is used. The ensemble should output the weighted action vector directly (or clip to the action space), not an argmax index.
Useful? React with 👍 / 👎.
| # Weighted voting | ||
| weighted_predictions = [] | ||
| for i, (pred, conf) in enumerate(zip(predictions, confidences)): | ||
| weight = self.weights[i] * conf | ||
| weighted_predictions.append(pred * weight) | ||
|
|
||
| # Majority vote | ||
| final_prediction = np.sum(weighted_predictions, axis=0) | ||
| final_prediction = np.argmax(final_prediction, axis=1) | ||
|
|
There was a problem hiding this comment.
The EnsembleAgent.predict implementation also applies np.argmax to the weighted actions, yielding a single integer per batch. For the continuous action space of StockTradingEnv, this produces vectors of shape (n,) containing values 0…stock_dim-1, which are incompatible with the [-1, 1]^stock_dim Box space. Any attempt to test the ensemble will therefore fail when the environment receives these indices. The ensemble should combine the continuous actions (e.g. by averaging and clipping) instead of reducing them with argmax.
Useful? React with 👍 / 👎.
| else: | ||
| continue | ||
|
|
||
| # Set up logger | ||
| tmp_path = RESULTS_DIR + f'/{model_type.lower()}' | ||
| new_logger = configure(tmp_path, ["stdout", "csv", "tensorboard"]) | ||
| model.set_logger(new_logger) | ||
|
|
||
| # Train model |
There was a problem hiding this comment.
The training routine configures Stable Baselines loggers with configure(...), but configure is never imported in this module. Running train_models will immediately raise a NameError at this line, preventing any model training. Add from stable_baselines3.common.logger import configure to the imports (as done in the other new modules) before invoking the function.
Useful? React with 👍 / 👎.
- Implemented enhanced PPO model with additional technical indicators - Added comprehensive feature engineering (momentum, volume, volatility indicators) - Created comparison framework between baseline and enhanced models - Achieved 67% improvement in Sharpe ratio (1.25 → 2.09) - Reduced max drawdown by 35% (-8.80% → -5.70%) - Increased annual returns by 6% (19.55% → 25.55%) - Added ensemble solution framework for future improvements - Generated detailed performance analysis and visualizations Key files: - simple_ensemble.py: Enhanced model implementation - ensemble_solution.py: Advanced ensemble framework - RESULTS_SUMMARY.md: Comprehensive performance analysis - Enhanced results: baseline_ppo_results.csv, enhanced_ppo_results.csv - Visualizations: enhanced_vs_baseline.png
…ution' into feature/finrl-task1-enhanced-solution
Add starter kit for Task 3 – DeFi Liquidity Provisioning
xsa-dev
changed the title
xsa-dev
changed the title
1 participant
Key files: