Skip to content

Conversation

@mrchocha
Copy link

@mrchocha mrchocha commented Aug 15, 2025

📋 Summary

This PR introduces a QA Benchmark Seed containing a set of intentional, toggleable bugs for use in QA benchmarking and performance evaluation. These bugs are safe, isolated, and can be enabled/disabled locally via localStorage variables without impacting production.


�� Included Demo Bugs

Bug Name Trigger Flag Scope Description
New Event Types Not Appearing localStorage.QA_BUG_SKIP_NEW_EVENT_TYPES = '1' Client Newly created event types don't appear in bookings list even after bookings are made
Booking Form Validation Persists localStorage.QA_BUG_VALIDATION_ERROR_PERSISTS = '1' Client Validation errors persist even after field correction and prevent form submission
Timezone Selector Not Updating localStorage.QA_BUG_TIMEZONE_NOT_UPDATING = '1' Client Timezone changes don't update displayed time slots, requiring page refresh
Deleted Recurring Events Showing NA Already present Deleted recurring events continue to appear in upcoming bookings list
Availability Update Not Cancelling NA Already present Updating availability after booking doesn't cancel conflicting booked events

�� Usage

Each bug is documented with:

  • Enable/Disable instructions - localStorage commands to control behavior
  • Reproduction steps - Detailed steps to reproduce the bug
  • Modified files - List of files that implement the bug
  • Expected incorrect behavior - Description of how the bug manifests

These can be found in qa-benchmarks/ along with tickets.csv, which lists the bugs as realistic QA tickets with difficulty levels and detailed reproduction steps.


🛡️ Safety

  • Bugs are opt-in only (disabled by default)
  • No side effects without explicitly enabling the respective localStorage flags
  • No changes to production behavior unless deliberately toggled
  • All bugs are client-side only and don't affect server-side functionality
  • Easy to disable by removing localStorage variables or refreshing the page

📁 Files Modified

  • apps/web/modules/bookings/views/bookings-listing-view.tsx - New event types filtering bug
  • packages/features/form-builder/FormBuilderField.tsx - Validation error persistence bug
  • packages/features/bookings/Booker/components/BookEventForm/BookEventForm.tsx - Form submission prevention bug
  • packages/features/bookings/Booker/components/EventMeta.tsx - Timezone selector bug
  • qa-benchmarks/README.md - Complete documentation for all bugs
  • qa-benchmarks/tickets.csv - QA tickets with reproduction steps and difficulty levels

🎯 Purpose

These bugs serve as:

  • QA Testing Scenarios - Realistic bug scenarios for testing QA processes
  • Performance Benchmarking - Controlled test cases for measuring QA team performance
  • Training Materials - Examples of common UI/UX issues for QA training
  • Regression Testing - Baseline bugs to ensure QA processes remain effective

Each bug simulates real-world issues that QA teams commonly encounter, providing a standardized set of test cases for benchmarking and evaluation purposes.

@mrchocha mrchocha changed the title feat(benchmarking): added bugs to QA benchmarking feat(benchmarking): added bugs to QA benchmarking [FOU-83] Aug 15, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants