test(e2e): census gets its own project, ordered before the mutating probes
The census baselines were captured after api.probe had already created its adversarial bills (Zero/Big/<script>) in the scratch DB, so snapshot content depended on spec ordering — the first census diffs were probe artifacts, not product changes. Now: a dedicated 'census' project (read-only, pristine seed data) runs via project dependencies before 'probe' (api.probe/webauthn mutate state), and the default UI projects ignore the spec. Re-baselined from a prepare-db-fresh DB; probe chain 33/33 across three consecutive runs; default e2e 27 passed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
parent
e7974cf4c0
commit
64ab37ae39
|
|
@ -6,7 +6,7 @@ button | Print / Save PDF
|
|||
button | Refresh
|
||||
button | Reset filters
|
||||
button | Search billsCtrl K
|
||||
button | Tracker#
|
||||
button | Tracker
|
||||
input:checkbox | (unnamed)
|
||||
input:number | (unnamed)
|
||||
link | About
|
||||
|
|
@ -17,7 +17,7 @@ link | Release Notes
|
|||
link | Skip to main content
|
||||
select | # months# months# months
|
||||
select | # months# months# months# months
|
||||
select | All bills<script>alert(#)</script>Adobe Creative CloudAmazon
|
||||
select | All billsAdobe Creative CloudAmazon PrimeAMC A-ListApple iCl
|
||||
select | All categoriesBeautyCoffeeCredit CardsDining OutEntertainmen
|
||||
select | JanuaryFebruaryMarchAprilMayJuneJulyAugustSeptemberOctoberNo
|
||||
select | LineArea
|
||||
|
|
@ -2,7 +2,7 @@ button | Current theme: Dark. Click to change.
|
|||
button | Open Tanstack query devtools
|
||||
button | Open user menu
|
||||
button | Search billsCtrl K
|
||||
button | Tracker#
|
||||
button | Tracker
|
||||
link | About
|
||||
link | Analytics
|
||||
link | Bill Tracker
|
||||
|
|
@ -1,13 +1,11 @@
|
|||
button | #st bucket
|
||||
button | #th bucket
|
||||
button | <script>alert(#)</script>
|
||||
button | AMC A-List
|
||||
button | Add Bill
|
||||
button | Adobe Creative Cloud
|
||||
button | Amazon Prime
|
||||
button | Apple iCloud+
|
||||
button | Autopay
|
||||
button | Big
|
||||
button | Cadence
|
||||
button | Capital One Quicksilver
|
||||
button | Car Insurance
|
||||
|
|
@ -33,8 +31,6 @@ button | Homeowners Insurance
|
|||
button | Inactive
|
||||
button | Internet Provider
|
||||
button | Mortgage
|
||||
button | Move <script>alert(#)</script> down
|
||||
button | Move <script>alert(#)</script> up
|
||||
button | Move AMC A-List down
|
||||
button | Move AMC A-List up
|
||||
button | Move Adobe Creative Cloud down
|
||||
|
|
@ -43,8 +39,6 @@ button | Move Amazon Prime down
|
|||
button | Move Amazon Prime up
|
||||
button | Move Apple iCloud+ down
|
||||
button | Move Apple iCloud+ up
|
||||
button | Move Big down
|
||||
button | Move Big up
|
||||
button | Move Capital One Quicksilver down
|
||||
button | Move Capital One Quicksilver up
|
||||
button | Move Car Insurance down
|
||||
|
|
@ -83,8 +77,6 @@ button | Move Student Loan down
|
|||
button | Move Student Loan up
|
||||
button | Move Trash Service down
|
||||
button | Move Trash Service up
|
||||
button | Move Zero down
|
||||
button | Move Zero up
|
||||
button | Netflix
|
||||
button | Open Tanstack query devtools
|
||||
button | Open user menu
|
||||
|
|
@ -92,9 +84,8 @@ button | Search & filters# of # shown · no filters
|
|||
button | Search billsCtrl K
|
||||
button | Spotify
|
||||
button | Student Loan
|
||||
button | Tracker#
|
||||
button | Tracker
|
||||
button | Trash Service
|
||||
button | Zero
|
||||
combobox | Filter by billing schedule
|
||||
combobox | Filter by category
|
||||
combobox | Use a bill template
|
||||
|
|
@ -7,7 +7,7 @@ button | Refresh calendar
|
|||
button | Search billsCtrl K
|
||||
button | Subscribe Calendar
|
||||
button | Today
|
||||
button | Tracker#
|
||||
button | Tracker
|
||||
button | View #/#/#
|
||||
link | About
|
||||
link | Analytics
|
||||
|
|
@ -52,7 +52,7 @@ button | Rename Subscriptions
|
|||
button | Rename Utilities
|
||||
button | Search billsCtrl K
|
||||
button | Show Empty
|
||||
button | Tracker#
|
||||
button | Tracker
|
||||
input:text | New category name...
|
||||
link | About
|
||||
link | Analytics
|
||||
|
|
@ -6,7 +6,7 @@ button | ImportBring in existing data
|
|||
button | Open Tanstack query devtools
|
||||
button | Open user menu
|
||||
button | Search billsCtrl K
|
||||
button | Tracker#
|
||||
button | Tracker
|
||||
button | TransactionsReview & match
|
||||
link | About
|
||||
link | Analytics
|
||||
|
|
@ -5,7 +5,7 @@ button | Open Tanstack query devtools
|
|||
button | Open user menu
|
||||
button | Refresh
|
||||
button | Search billsCtrl K
|
||||
button | Tracker#
|
||||
button | Tracker
|
||||
link | About
|
||||
link | Analytics
|
||||
link | Bill Tracker
|
||||
|
|
@ -7,7 +7,7 @@ button | Print
|
|||
button | Required Minimum: help
|
||||
button | Search billsCtrl K
|
||||
button | Time Savings: help
|
||||
button | Tracker#
|
||||
button | Tracker
|
||||
button | ▲
|
||||
button | ▼
|
||||
combobox | Capital One Quicksilver$#
|
||||
|
|
@ -1,14 +1,14 @@
|
|||
button | Add key
|
||||
button | Change Password
|
||||
button | Current theme: Dark. Click to change.
|
||||
button | Last LoginJul # # #:# PMDesktop · Chrome on Linux#
|
||||
button | Last LoginJul # # #:# AMDesktop · Chrome on Linux#
|
||||
button | Open Tanstack query devtools
|
||||
button | Open user menu
|
||||
button | Save Privacy Settings
|
||||
button | Save Profile
|
||||
button | Search billsCtrl K
|
||||
button | Set up
|
||||
button | Tracker#
|
||||
button | Tracker
|
||||
input:password | (unnamed)
|
||||
input:text | Display name
|
||||
link | About
|
||||
|
|
@ -15,7 +15,7 @@ button | Search billsCtrl K
|
|||
button | Send test
|
||||
button | System
|
||||
button | Theme: help
|
||||
button | Tracker#
|
||||
button | Tracker
|
||||
combobox | MM/DD/YYYY
|
||||
combobox | Sunday
|
||||
combobox | Tracker
|
||||
|
|
@ -26,7 +26,7 @@ button | Restore Ramsey Order
|
|||
button | Save Order
|
||||
button | Search billsCtrl K
|
||||
button | Toggle plan details
|
||||
button | Tracker#
|
||||
button | Tracker
|
||||
input:number | #
|
||||
link | About
|
||||
link | Analytics
|
||||
|
|
@ -17,7 +17,7 @@ button | Previous month
|
|||
button | Search billsCtrl K
|
||||
button | Shopping
|
||||
button | Shopping$# transactionavg $#/mo$# left
|
||||
button | Tracker#
|
||||
button | Tracker
|
||||
button | Uncategorized
|
||||
button | Uncategorized$# transaction
|
||||
button | View
|
||||
|
|
@ -40,7 +40,7 @@ button | Search subscriptions# of # shown · no filters
|
|||
button | Search transactionsall bank transactions · no filters
|
||||
button | Spotify
|
||||
button | Track Subscription
|
||||
button | Tracker#
|
||||
button | Tracker
|
||||
button | View recommendation details
|
||||
input:search | Search by merchant, description, or payee…
|
||||
input:search | Search recommendations…
|
||||
|
|
@ -1,15 +1,11 @@
|
|||
button | Current theme: Dark. Click to change.
|
||||
button | Edit
|
||||
button | Move <script>alert(#)</script> down
|
||||
button | Move <script>alert(#)</script> up
|
||||
button | Move AMC A-List down
|
||||
button | Move AMC A-List up
|
||||
button | Move Adobe Creative Cloud down
|
||||
button | Move Adobe Creative Cloud up
|
||||
button | Move Apple iCloud+ down
|
||||
button | Move Apple iCloud+ up
|
||||
button | Move Big down
|
||||
button | Move Big up
|
||||
button | Move Capital One Quicksilver down
|
||||
button | Move Capital One Quicksilver up
|
||||
button | Move Car Insurance down
|
||||
|
|
@ -46,8 +42,6 @@ button | Move Student Loan down
|
|||
button | Move Student Loan up
|
||||
button | Move Trash Service down
|
||||
button | Move Trash Service up
|
||||
button | Move Zero down
|
||||
button | Move Zero up
|
||||
button | Next month
|
||||
button | Open Tanstack query devtools
|
||||
button | Open user menu
|
||||
|
|
@ -55,7 +49,7 @@ button | Previous month
|
|||
button | Print / PDF
|
||||
button | Search billsCtrl K
|
||||
button | Today
|
||||
button | Tracker#
|
||||
button | Tracker
|
||||
link | About
|
||||
link | Analytics
|
||||
link | Bill Tracker
|
||||
|
|
@ -12,11 +12,9 @@ button | Current theme: Dark. Click to change.
|
|||
button | Debt
|
||||
button | Due
|
||||
button | Due Soon, click to mark paid
|
||||
button | Edit <script>alert(#)</script>
|
||||
button | Edit AMC A-List
|
||||
button | Edit Adobe Creative Cloud
|
||||
button | Edit Apple iCloud+
|
||||
button | Edit Big
|
||||
button | Edit Capital One Quicksilver
|
||||
button | Edit Car Insurance
|
||||
button | Edit Car Payment
|
||||
|
|
@ -35,25 +33,18 @@ button | Edit Netflix
|
|||
button | Edit Spotify
|
||||
button | Edit Student Loan
|
||||
button | Edit Trash Service
|
||||
button | Edit Zero
|
||||
button | Edit monthly starting amounts
|
||||
button | Expected
|
||||
button | Export or print this month
|
||||
button | Hide
|
||||
button | Hide overdue panel
|
||||
button | Keyboard shortcuts
|
||||
button | Last Month
|
||||
button | Late, click to mark paid
|
||||
button | Move <script>alert(#)</script> down
|
||||
button | Move <script>alert(#)</script> up
|
||||
button | Move AMC A-List down
|
||||
button | Move AMC A-List up
|
||||
button | Move Adobe Creative Cloud down
|
||||
button | Move Adobe Creative Cloud up
|
||||
button | Move Apple iCloud+ down
|
||||
button | Move Apple iCloud+ up
|
||||
button | Move Big down
|
||||
button | Move Big up
|
||||
button | Move Capital One Quicksilver down
|
||||
button | Move Capital One Quicksilver up
|
||||
button | Move Car Insurance down
|
||||
|
|
@ -90,8 +81,6 @@ button | Move Student Loan down
|
|||
button | Move Student Loan up
|
||||
button | Move Trash Service down
|
||||
button | Move Trash Service up
|
||||
button | Move Zero down
|
||||
button | Move Zero up
|
||||
button | Next month
|
||||
button | Open Tanstack query devtools
|
||||
button | Open calendar for this month
|
||||
|
|
@ -101,25 +90,19 @@ button | Paid
|
|||
button | Paid Date
|
||||
button | Paid, click to mark unpaid
|
||||
button | Pay all due (#)
|
||||
button | Pay now
|
||||
button | Pin Due
|
||||
button | Previous month
|
||||
button | Review overdue bills# items ready for pay, skip, or snooze$#
|
||||
button | Search & sort# of # shown · no filters
|
||||
button | Search billsCtrl K
|
||||
button | Skip
|
||||
button | Snooze
|
||||
button | Status
|
||||
button | Today
|
||||
button | Tracker#
|
||||
button | Tracker
|
||||
button | Unpaid
|
||||
button | Upcoming, click to mark paid
|
||||
button | —
|
||||
checkbox | Select <script>alert(#)</script>
|
||||
checkbox | Select AMC A-List
|
||||
checkbox | Select Adobe Creative Cloud
|
||||
checkbox | Select Apple iCloud+
|
||||
checkbox | Select Big
|
||||
checkbox | Select Capital One Quicksilver
|
||||
checkbox | Select Car Insurance
|
||||
checkbox | Select Car Payment
|
||||
|
|
@ -138,13 +121,10 @@ checkbox | Select Netflix
|
|||
checkbox | Select Spotify
|
||||
checkbox | Select Student Loan
|
||||
checkbox | Select Trash Service
|
||||
checkbox | Select Zero
|
||||
combobox | Filter by billing schedule
|
||||
combobox | Filter by category
|
||||
combobox | Sort by
|
||||
input:number | <script>alert(#)</script> payment amount
|
||||
input:number | Apple iCloud+ payment amount
|
||||
input:number | Big payment amount
|
||||
input:number | Capital One Quicksilver payment amount
|
||||
input:number | Car Payment payment amount
|
||||
input:number | Cell Phone payment amount
|
||||
|
|
@ -51,6 +51,8 @@ module.exports = defineConfig({
|
|||
// probe-only: enables WebAuthn for the seeded user mid-test, which would
|
||||
// race the password logins in these parallel UI projects
|
||||
/webauthn\.probe\.spec\.js/,
|
||||
// census-project-only: snapshots depend on pristine seed data
|
||||
/control-census\.spec\.js/,
|
||||
],
|
||||
},
|
||||
{
|
||||
|
|
@ -64,6 +66,8 @@ module.exports = defineConfig({
|
|||
// probe-only: enables WebAuthn for the seeded user mid-test, which would
|
||||
// race the password logins in these parallel UI projects
|
||||
/webauthn\.probe\.spec\.js/,
|
||||
// census-project-only: snapshots depend on pristine seed data
|
||||
/control-census\.spec\.js/,
|
||||
],
|
||||
},
|
||||
// Find-mode diagnostics (adversarial API probe + authenticated a11y scan).
|
||||
|
|
@ -71,12 +75,23 @@ module.exports = defineConfig({
|
|||
// so open findings here don't turn the default suite red. Run on demand:
|
||||
// `npm run test:e2e:probe`. Fold a spec into the default projects once its
|
||||
// findings are fixed (it becomes a passing regression guard).
|
||||
// Control census (Appendix E automation): read-only snapshots of every
|
||||
// page's interactive controls. Own project ordered BEFORE the probe so the
|
||||
// snapshots always see pristine seed data — api.probe/webauthn mutate the
|
||||
// scratch DB (adversarial bills, passkey enrollment) and would otherwise
|
||||
// make the baselines depend on spec ordering.
|
||||
{
|
||||
name: 'probe',
|
||||
testMatch: /(api\.probe|a11y\.authed|webauthn\.probe|control-census)\.spec\.js/,
|
||||
name: 'census',
|
||||
testMatch: /control-census\.spec\.js/,
|
||||
use: { ...devices['Desktop Chrome'] },
|
||||
dependencies: ['setup'],
|
||||
},
|
||||
{
|
||||
name: 'probe',
|
||||
testMatch: /(api\.probe|a11y\.authed|webauthn\.probe)\.spec\.js/,
|
||||
use: { ...devices['Desktop Chrome'] },
|
||||
dependencies: ['setup', 'census'],
|
||||
},
|
||||
// Cross-browser (WebAuthn/Select behavior differs) — enable when ready:
|
||||
// { name: 'firefox', use: { ...devices['Desktop Firefox'] }, dependencies: ['setup'], testIgnore: /auth\.setup\.js/ },
|
||||
// { name: 'webkit', use: { ...devices['Desktop Safari'] }, dependencies: ['setup'], testIgnore: /auth\.setup\.js/ },
|
||||
|
|
|
|||
Loading…
Reference in New Issue