More tests, better functionality, the server serves the dashboard now on configurable ports. websocket stuff fixed, though I dont think data is really being sent / recieved...
This commit is contained in:
Nate Anderson
@@ -0,0 +1,213 @@
|
||||
# Dashboard WebSocket Integration Tests
|
||||
|
||||
This directory contains integration tests for the dashboard's WebSocket functionality, focusing on real-time synchronization between the actual server and dashboard client.
|
||||
|
||||
## Overview
|
||||
|
||||
The integration tests verify that the dashboard correctly:
|
||||
- Connects to WebSocket endpoints on a real running server
|
||||
- Receives and processes WebSocket messages from the actual server
|
||||
- Maintains connection stability and handles reconnection
|
||||
- Loads all data types correctly through the real API
|
||||
|
||||
## Test Architecture
|
||||
|
||||
### Real Integration Testing
|
||||
- **Spins up actual xp_server process** with `--db` and `--port` flags
|
||||
- **Uses real ApiService and DashboardProvider** - no mocked implementations
|
||||
- **Tests actual WebSocket communication** between server and client
|
||||
- **Supports custom databases** for testing different scenarios
|
||||
|
||||
### Core Files
|
||||
|
||||
- **`dashboard_websocket_integration_test.dart`** - Main integration test suite
|
||||
- **`run_dashboard_test.sh`** - Test runner script with `--db` flag support
|
||||
- **`README.md`** - This documentation
|
||||
|
||||
### Golden Test Databases
|
||||
|
||||
Golden test databases are created using `xp_server/test/create_golden_db.dart` and contain pre-populated data for specific testing scenarios.
|
||||
|
||||
## Running Tests
|
||||
|
||||
### Basic Usage
|
||||
|
||||
```bash
|
||||
# Run with server's default database
|
||||
cd xp_dashboard
|
||||
./test/integration/run_dashboard_test.sh
|
||||
|
||||
# Or run directly with Flutter
|
||||
flutter test test/integration/dashboard_websocket_integration_test.dart
|
||||
```
|
||||
|
||||
### Using Custom Databases
|
||||
|
||||
```bash
|
||||
# Run with a specific database file
|
||||
./test/integration/run_dashboard_test.sh --db path/to/custom.db
|
||||
|
||||
# Run with golden test database
|
||||
./test/integration/run_dashboard_test.sh --db ../xp_server/test/high_achiever.db
|
||||
```
|
||||
|
||||
### Creating Golden Test Databases
|
||||
|
||||
Golden test databases are created in the server directory:
|
||||
|
||||
```bash
|
||||
cd xp_server/test
|
||||
|
||||
# Create different scenario databases
|
||||
dart create_golden_db.dart high_achiever high_achiever.db
|
||||
dart create_golden_db.dart new_user new_user.db
|
||||
dart create_golden_db.dart level_up_ready level_up.db
|
||||
dart create_golden_db.dart achievement_rich achievements.db
|
||||
dart create_golden_db.dart focus_master focus.db
|
||||
|
||||
# Then run tests with these databases
|
||||
cd ../../xp_dashboard
|
||||
./test/integration/run_dashboard_test.sh --db ../xp_server/test/high_achiever.db
|
||||
```
|
||||
|
||||
## Golden Test Scenarios
|
||||
|
||||
### `high_achiever`
|
||||
- **Level**: 8-9 (2500+ XP)
|
||||
- **Features**: Multiple achievements, extensive activity history, many focus sessions
|
||||
- **Use Case**: Testing high-level user experience, complex data handling
|
||||
|
||||
### `new_user`
|
||||
- **Level**: 1 (50 XP)
|
||||
- **Features**: Minimal data, first achievement, basic activities
|
||||
- **Use Case**: Testing new user onboarding, minimal data scenarios
|
||||
|
||||
### `level_up_ready`
|
||||
- **Level**: 4 (850 XP, close to level 5)
|
||||
- **Features**: Just below level threshold, good activity mix
|
||||
- **Use Case**: Testing level-up notifications and threshold behavior
|
||||
|
||||
### `achievement_rich`
|
||||
- **Level**: 5 (1200 XP)
|
||||
- **Features**: Many recent achievements (last 5 minutes to 2 hours)
|
||||
- **Use Case**: Testing achievement notification handling, recent achievement display
|
||||
|
||||
### `focus_master`
|
||||
- **Level**: 7 (1800 XP)
|
||||
- **Features**: 15 focus sessions, 9 hours focus time, focus-related achievements
|
||||
- **Use Case**: Testing focus session workflows, high focus activity scenarios
|
||||
|
||||
## Test Scenarios
|
||||
|
||||
The integration test covers these key scenarios:
|
||||
|
||||
1. **WebSocket Connection & Data Loading**
|
||||
- Server process startup and connection
|
||||
- Initial data synchronization
|
||||
- Connection state verification
|
||||
|
||||
2. **Connection Management**
|
||||
- Connection maintenance over time
|
||||
- Manual disconnection and reconnection
|
||||
- Heartbeat/ping-pong handling
|
||||
|
||||
3. **Real-time Updates**
|
||||
- Listening for server-generated updates
|
||||
- State change detection
|
||||
- Update processing verification
|
||||
|
||||
4. **Data Loading**
|
||||
- Stats loading through real API
|
||||
- Achievements, activities, focus sessions
|
||||
- XP breakdown and classifications
|
||||
- Error handling for API calls
|
||||
|
||||
## Server Configuration
|
||||
|
||||
The test automatically:
|
||||
- Starts xp_server with random port (8000-9000 range) to avoid conflicts
|
||||
- Passes `--db` flag to server if custom database specified
|
||||
- Waits for server startup confirmation
|
||||
- Configures ApiService to point to test server
|
||||
- Shuts down server process after tests complete
|
||||
|
||||
## Debugging Tests
|
||||
|
||||
### Verbose Output
|
||||
```bash
|
||||
flutter test test/integration/dashboard_websocket_integration_test.dart --verbose
|
||||
```
|
||||
|
||||
### Server Logs
|
||||
- Server output is printed during test execution
|
||||
- Look for "SERVER:" prefixed lines in test output
|
||||
- Server errors are printed with "SERVER ERROR:" prefix
|
||||
|
||||
### Common Issues
|
||||
|
||||
1. **Port Conflicts**: Tests use random ports (8000-9000 range)
|
||||
2. **Database Locks**: Ensure no other processes are using test databases
|
||||
3. **Server Startup**: Tests wait up to 30 seconds for server startup
|
||||
4. **Connection Timeouts**: WebSocket operations have 5-10 second timeouts
|
||||
|
||||
## Integration with CI/CD
|
||||
|
||||
The tests are designed to be CI/CD friendly:
|
||||
- No external dependencies beyond Flutter/Dart and the xp_server
|
||||
- Self-contained server process management
|
||||
- Deterministic test data with golden databases
|
||||
- Configurable timeouts
|
||||
- Clean setup/teardown
|
||||
|
||||
### Example CI Usage
|
||||
```yaml
|
||||
- name: Run Dashboard WebSocket Integration Tests
|
||||
run: |
|
||||
cd xp_dashboard
|
||||
./test/integration/run_dashboard_test.sh
|
||||
```
|
||||
|
||||
## Extending Tests
|
||||
|
||||
### Adding New Test Cases
|
||||
1. Add test case to `dashboard_websocket_integration_test.dart`
|
||||
2. Follow existing pattern: setup → action → verify
|
||||
3. Use `_waitForCondition()` helper for async state changes
|
||||
4. Ensure proper cleanup in test teardown
|
||||
|
||||
### Adding New Golden Databases
|
||||
1. Add scenario to `xp_server/test/create_golden_db.dart`
|
||||
2. Implement data population function
|
||||
3. Document scenario in this README
|
||||
4. Test with `./run_dashboard_test.sh --db new_scenario.db`
|
||||
|
||||
### Testing New WebSocket Messages
|
||||
1. Ensure server sends the message type during normal operation
|
||||
2. Add listener in test to detect message reception
|
||||
3. Verify dashboard state changes appropriately
|
||||
4. Test error conditions and edge cases
|
||||
|
||||
## Benefits of This Approach
|
||||
|
||||
1. **True Integration**: Tests actual server-client communication
|
||||
2. **No Fragile Mocks**: Uses real implementations, reducing test brittleness
|
||||
3. **Realistic Scenarios**: Golden databases provide real-world data patterns
|
||||
4. **Easy to Run**: Simple command-line interface with helpful options
|
||||
5. **Comprehensive Coverage**: Tests WebSocket, HTTP API, and state management
|
||||
6. **Debugging Friendly**: Server logs and verbose output for troubleshooting
|
||||
|
||||
## Performance Considerations
|
||||
|
||||
- Tests use random ports to avoid conflicts
|
||||
- Server startup timeout prevents hanging tests
|
||||
- WebSocket connections are properly cleaned up
|
||||
- Database operations are optimized for test speed
|
||||
- Memory usage is monitored for large datasets
|
||||
|
||||
## Security Notes
|
||||
|
||||
- Tests run on localhost only with random ports
|
||||
- No external network access required
|
||||
- Test databases contain no sensitive data
|
||||
- All server processes are cleaned up after tests
|
||||
- Temporary files are properly managed
|
||||
Reference in New Issue
Block a user