Optimize network performance for remote server deployment

[Copilot]

• Investigate network lag issue when hosting server on VM
• Analyze current client-server communication patterns
• Identify and implement optimizations
• Document changes comprehensively
• Code review completed and all feedback addressed:
  • Fixed backwards tick_update_frequency comments
  • Integrated api_client.close() in quitApplication()
  • Documented entity movement visibility tradeoff
  • Added tests for updateTick optimization
• Fixed failing PersistenceIntegrationTest (player stats not persisted)
• Await user feedback on network performance
• Consider WebSocket implementation for future improvements

---

💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

[Copilot]

Pull request overview

This pull request implements significant network performance optimizations for remote server deployment scenarios. The game was experiencing severe lag when the server ran on a VM and clients connected over a network, primarily due to excessive HTTP requests (~60/sec during normal gameplay).

Changes:

• Reduced tick update frequency from every 3 frames to every 15 frames, decreasing requests from ~20/sec to ~4/sec
• Eliminated redundant player data requests by extracting player state from tick response instead of separate GET calls
• Implemented smart room reloading that only fetches room data on room transitions rather than every tick
• Added debounced room refresh mechanism with 500ms cooldown for gather/place actions to prevent request spam
• Introduced HTTP connection pooling with persistent connections and 5-second request timeouts

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 5 comments.

File	Description
src/screen/serverBackedWorldScreen.py	Implements reduced tick frequency (15 frames), player data extraction from tick response, conditional room loading on transitions, and debounced room refresh with tracking variables for optimization
src/client/api_client.py	Adds HTTP session with connection pooling (HTTPAdapter), configurable request timeout (5s default), and close() method for resource cleanup
docs/PERFORMANCE.md	Comprehensive documentation of optimization strategies, impact measurements, configuration options, and future improvement opportunities
docs/OPTIMIZATION_SUMMARY.md	User-friendly summary of optimizations with before/after comparisons, performance metrics, and testing recommendations

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

The comment states "Lower = fewer updates, higher = more responsive" but this is backwards. Lower values (e.g., 5) mean the tick updates MORE frequently (every 5 frames), making it more responsive but using more network. Higher values (e.g., 30) mean the tick updates LESS frequently (every 30 frames), using less network but being less responsive. The comment should be: "Lower = more frequent updates (more responsive, higher network usage), higher = less frequent updates (less responsive, lower network usage)".

Suggested change

	tick_update_frequency = 15 # Lower = fewer updates, higher = more responsive
	tick_update_frequency = 15 # Lower = more frequent updates (more responsive, higher network usage), higher = less frequent updates (less responsive, lower network usage)

[Copilot]

Fixed in ae554b7. Comment now correctly states: "Lower = more frequent updates (more responsive, higher network usage), higher = less frequent updates (less responsive, lower network usage)".

[Copilot]

The optimization to only reload rooms on room transitions means that entity movements within the same room will no longer be visible in real-time. The previous implementation reloaded the room every tick specifically to show entity movements. This tradeoff significantly impacts gameplay visibility and should be clearly documented in the PR description and performance documentation. Consider adding a note about this limitation and potentially implementing a separate mechanism for entity position updates if real-time entity tracking is important for gameplay.

[Copilot]

Documented in ae554b7. Added clear warnings about this tradeoff in both PERFORMANCE.md and OPTIMIZATION_SUMMARY.md. The docs now explain that entity movements within the same room are not visible in real-time and recommend WebSocket integration for real-time entity tracking if needed.

[Copilot]

The comment suggests to "Increase for less network usage, decrease for more responsive feel" but the relationship is inverted. Increasing tick_update_frequency (e.g., from 15 to 30) means updating LESS frequently (every 30 frames instead of 15), which reduces network usage. Decreasing it (e.g., from 15 to 5) means updating MORE frequently, which increases responsiveness but also network usage. The comment should be: "Increase for less frequent updates (lower network usage), decrease for more frequent updates (more responsive)".

Suggested change

	tick_update_frequency = 15 # Increase for less network usage, decrease for more responsive feel
	tick_update_frequency = 15 # Increase for less frequent updates (lower network usage), decrease for more frequent updates (more responsive)

[Copilot]

Fixed in ae554b7. Comment now correctly states: "Increase for less frequent updates (lower network usage), decrease for more frequent updates (more responsive)".

[dmccoystephenson]

@copilot address comments

[dmccoystephenson]

@copilot Create sub PR for this PR for the following:

### 1. WebSocket Integration (High Priority) 
  
 The server already has WebSocket support (`WebSocketMessageService`) but the client doesn't use it. 
  
 **Recommended approach**: 
 - Use WebSocket for server-to-client state updates (push instead of poll) 
 - Keep REST API for client-to-server actions 
 - Would eliminate periodic tick polling entirely 
  
 **Expected impact**: Additional 30-50% reduction in network overhead

[Copilot]

@copilot Create sub PR for this PR for the following:

I cannot directly create pull requests, but I can help implement the WebSocket integration work on a new branch that you can then use to create a PR.

Would you like me to:

1. Create a new branch off the current branch (copilot/sub-pr-266)
2. Implement WebSocket client integration for real-time state updates
3. Keep REST API for client actions (move, gather, place, etc.)
4. Add configuration to switch between polling and WebSocket modes

Please confirm and I'll proceed with the implementation.

[dmccoystephenson]

@copilot no, just provide a detailed ticket title/description for the future in a comment

[Copilot]

@copilot no, just provide a detailed ticket title/description for the future in a comment

WebSocket Integration for Real-Time Game State Updates

Priority

High - This would provide an additional 30-50% reduction in network overhead on top of current optimizations

Summary

Implement WebSocket-based real-time communication for server-to-client state updates, eliminating the need for periodic tick polling while maintaining REST API for client-initiated actions.

Background

Current implementation uses REST API polling every 15 frames (~4 requests/sec) to fetch game state updates. The server already has full WebSocket support via WebSocketMessageService that broadcasts:

• Player position updates (/topic/session/{id}/player)
• Entity state updates (/topic/session/{id}/entity)
• World events (/topic/session/{id}/world)
• Tick updates (/topic/session/{id}/tick)

Technical Approach

Client-Side Implementation:

1. Add WebSocket client library (e.g., websocket-client or python-socketio)

2. Create WebSocketClient class in src/client/websocket_client.py:

   • Connect to ws://server:8080/ws on session init
   • Subscribe to session-specific topics
   • Handle reconnection with exponential backoff
   • Parse and dispatch incoming messages to handlers

3. Integrate with ServerBackedWorldScreen:

   • Remove periodic updateTick() calls from game loop
   • Add WebSocket message handlers for:
     • TickUpdate → Update server_tick
     • PlayerPositionUpdate → Update player position/state
     • EntityStateUpdate → Update entity positions in current room
     • WorldEventMessage → Handle world events
   • Keep REST API for player actions (move, gather, place, etc.)

Configuration:

• Add use_websocket flag to Config (default: True)
• Fallback to polling mode if WebSocket connection fails
• Configurable WebSocket reconnection parameters

Error Handling:

• Graceful degradation to REST polling on connection loss
• Automatic reconnection with exponential backoff
• Health check mechanism to detect stale connections

Implementation Tasks

1. Add WebSocket client dependency to requirements.txt
2. Implement WebSocketClient class with connection management
3. Add message handlers for all WebSocket message types
4. Integrate WebSocket client with ServerBackedWorldScreen
5. Remove/disable tick polling when WebSocket is active
6. Add configuration options for WebSocket vs REST modes
7. Implement reconnection logic with exponential backoff
8. Add comprehensive error handling and logging
9. Write unit tests for WebSocket client
10. Write integration tests for WebSocket message handling
11. Update documentation (PERFORMANCE.md, README.md)
12. Add metrics/logging for WebSocket connection health

Expected Impact

• Network Requests: Reduce from ~8-12/sec to ~1-2/sec (actions only)
• Latency: Immediate state updates (no polling delay)
• Bandwidth: ~30-50% reduction in total network traffic
• Responsiveness: Real-time entity movements visible without room reloads

Testing Strategy

• Unit tests for WebSocket client connection/disconnection
• Integration tests for message handling
• Manual testing with server on remote VM
• Network latency simulation testing
• Reconnection scenario testing (server restart, network interruption)

Documentation Updates

• Add WebSocket architecture diagram
• Document configuration options
• Update PERFORMANCE.md with new metrics
• Add troubleshooting guide for WebSocket issues

Dependencies

• WebSocket client library (Python)
• Server WebSocket endpoints (already implemented)
• Spring Boot STOMP WebSocket support (already configured)

Risks & Mitigation

• Risk: WebSocket connection instability
  • Mitigation: Automatic fallback to REST polling
• Risk: Message ordering issues
  • Mitigation: Use sequence numbers in messages
• Risk: Increased server load
  • Mitigation: Connection pooling, message batching

Success Criteria

• WebSocket connection establishes successfully on session init
• All game state updates received via WebSocket (no polling)
• Graceful fallback to REST mode on connection failure
• No increase in perceived latency
• 30-50% reduction in network requests measured
• All existing tests pass with WebSocket mode enabled

[dmccoystephenson]

@copilot fix failing unit test:

 Tests run: 4, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.109 s <<< FAILURE! -- in com.preponderous.roam.persistence.PersistenceIntegrationTest

[Copilot]

@copilot fix failing unit test:

Fixed in 18535bf. The test was failing because player stats (score, roomsExplored, foodEaten, numberOfDeaths, visitedRooms) were not being persisted to the database.

Root cause: The PersistenceService had three methods that needed updating:

1. savePlayer() - wasn't saving the stats to PlayerEntityData
2. convertToPlayer() - wasn't loading the stats from PlayerEntityData
3. copyPlayerData() - wasn't copying the stats between Player objects

All player stats are now correctly persisted and restored. The test now passes with all 4 tests successful.