test(e2e): census gets its own project, ordered before the mutating probes

The census baselines were captured after api.probe had already created
its adversarial bills (Zero/Big/<script>) in the scratch DB, so snapshot
content depended on spec ordering — the first census diffs were probe
artifacts, not product changes. Now: a dedicated 'census' project
(read-only, pristine seed data) runs via project dependencies before
'probe' (api.probe/webauthn mutate state), and the default UI projects
ignore the spec. Re-baselined from a prepare-db-fresh DB; probe chain
33/33 across three consecutive runs; default e2e 27 passed.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
null 2026-07-10 22:23:35 -05:00
parent e7974cf4c0
commit 64ab37ae39
16 changed files with 34 additions and 54 deletions

View File

@ -6,7 +6,7 @@ button | Print / Save PDF
button | Refresh
button | Reset filters
button | Search billsCtrl K
button | Tracker#
button | Tracker
input:checkbox | (unnamed)
input:number | (unnamed)
link | About
@ -17,7 +17,7 @@ link | Release Notes
link | Skip to main content
select | # months# months# months
select | # months# months# months# months
select | All bills<script>alert(#)</script>Adobe Creative CloudAmazon
select | All billsAdobe Creative CloudAmazon PrimeAMC A-ListApple iCl
select | All categoriesBeautyCoffeeCredit CardsDining OutEntertainmen
select | JanuaryFebruaryMarchAprilMayJuneJulyAugustSeptemberOctoberNo
select | LineArea

View File

@ -2,7 +2,7 @@ button | Current theme: Dark. Click to change.
button | Open Tanstack query devtools
button | Open user menu
button | Search billsCtrl K
button | Tracker#
button | Tracker
link | About
link | Analytics
link | Bill Tracker

View File

@ -1,13 +1,11 @@
button | #st bucket
button | #th bucket
button | <script>alert(#)</script>
button | AMC A-List
button | Add Bill
button | Adobe Creative Cloud
button | Amazon Prime
button | Apple iCloud+
button | Autopay
button | Big
button | Cadence
button | Capital One Quicksilver
button | Car Insurance
@ -33,8 +31,6 @@ button | Homeowners Insurance
button | Inactive
button | Internet Provider
button | Mortgage
button | Move <script>alert(#)</script> down
button | Move <script>alert(#)</script> up
button | Move AMC A-List down
button | Move AMC A-List up
button | Move Adobe Creative Cloud down
@ -43,8 +39,6 @@ button | Move Amazon Prime down
button | Move Amazon Prime up
button | Move Apple iCloud+ down
button | Move Apple iCloud+ up
button | Move Big down
button | Move Big up
button | Move Capital One Quicksilver down
button | Move Capital One Quicksilver up
button | Move Car Insurance down
@ -83,8 +77,6 @@ button | Move Student Loan down
button | Move Student Loan up
button | Move Trash Service down
button | Move Trash Service up
button | Move Zero down
button | Move Zero up
button | Netflix
button | Open Tanstack query devtools
button | Open user menu
@ -92,9 +84,8 @@ button | Search & filters# of # shown · no filters
button | Search billsCtrl K
button | Spotify
button | Student Loan
button | Tracker#
button | Tracker
button | Trash Service
button | Zero
combobox | Filter by billing schedule
combobox | Filter by category
combobox | Use a bill template

View File

@ -7,7 +7,7 @@ button | Refresh calendar
button | Search billsCtrl K
button | Subscribe Calendar
button | Today
button | Tracker#
button | Tracker
button | View #/#/#
link | About
link | Analytics

View File

@ -52,7 +52,7 @@ button | Rename Subscriptions
button | Rename Utilities
button | Search billsCtrl K
button | Show Empty
button | Tracker#
button | Tracker
input:text | New category name...
link | About
link | Analytics

View File

@ -6,7 +6,7 @@ button | ImportBring in existing data
button | Open Tanstack query devtools
button | Open user menu
button | Search billsCtrl K
button | Tracker#
button | Tracker
button | TransactionsReview & match
link | About
link | Analytics

View File

@ -5,7 +5,7 @@ button | Open Tanstack query devtools
button | Open user menu
button | Refresh
button | Search billsCtrl K
button | Tracker#
button | Tracker
link | About
link | Analytics
link | Bill Tracker

View File

@ -7,7 +7,7 @@ button | Print
button | Required Minimum: help
button | Search billsCtrl K
button | Time Savings: help
button | Tracker#
button | Tracker
button | ▲
button | ▼
combobox | Capital One Quicksilver$#

View File

@ -1,14 +1,14 @@
button | Add key
button | Change Password
button | Current theme: Dark. Click to change.
button | Last LoginJul # # #:# PMDesktop · Chrome on Linux#
button | Last LoginJul # # #:# AMDesktop · Chrome on Linux#
button | Open Tanstack query devtools
button | Open user menu
button | Save Privacy Settings
button | Save Profile
button | Search billsCtrl K
button | Set up
button | Tracker#
button | Tracker
input:password | (unnamed)
input:text | Display name
link | About

View File

@ -15,7 +15,7 @@ button | Search billsCtrl K
button | Send test
button | System
button | Theme: help
button | Tracker#
button | Tracker
combobox | MM/DD/YYYY
combobox | Sunday
combobox | Tracker

View File

@ -26,7 +26,7 @@ button | Restore Ramsey Order
button | Save Order
button | Search billsCtrl K
button | Toggle plan details
button | Tracker#
button | Tracker
input:number | #
link | About
link | Analytics

View File

@ -17,7 +17,7 @@ button | Previous month
button | Search billsCtrl K
button | Shopping
button | Shopping$# transactionavg $#/mo$# left
button | Tracker#
button | Tracker
button | Uncategorized
button | Uncategorized$# transaction
button | View

View File

@ -40,7 +40,7 @@ button | Search subscriptions# of # shown · no filters
button | Search transactionsall bank transactions · no filters
button | Spotify
button | Track Subscription
button | Tracker#
button | Tracker
button | View recommendation details
input:search | Search by merchant, description, or payee…
input:search | Search recommendations…

View File

@ -1,15 +1,11 @@
button | Current theme: Dark. Click to change.
button | Edit
button | Move <script>alert(#)</script> down
button | Move <script>alert(#)</script> up
button | Move AMC A-List down
button | Move AMC A-List up
button | Move Adobe Creative Cloud down
button | Move Adobe Creative Cloud up
button | Move Apple iCloud+ down
button | Move Apple iCloud+ up
button | Move Big down
button | Move Big up
button | Move Capital One Quicksilver down
button | Move Capital One Quicksilver up
button | Move Car Insurance down
@ -46,8 +42,6 @@ button | Move Student Loan down
button | Move Student Loan up
button | Move Trash Service down
button | Move Trash Service up
button | Move Zero down
button | Move Zero up
button | Next month
button | Open Tanstack query devtools
button | Open user menu
@ -55,7 +49,7 @@ button | Previous month
button | Print / PDF
button | Search billsCtrl K
button | Today
button | Tracker#
button | Tracker
link | About
link | Analytics
link | Bill Tracker

View File

@ -12,11 +12,9 @@ button | Current theme: Dark. Click to change.
button | Debt
button | Due
button | Due Soon, click to mark paid
button | Edit <script>alert(#)</script>
button | Edit AMC A-List
button | Edit Adobe Creative Cloud
button | Edit Apple iCloud+
button | Edit Big
button | Edit Capital One Quicksilver
button | Edit Car Insurance
button | Edit Car Payment
@ -35,25 +33,18 @@ button | Edit Netflix
button | Edit Spotify
button | Edit Student Loan
button | Edit Trash Service
button | Edit Zero
button | Edit monthly starting amounts
button | Expected
button | Export or print this month
button | Hide
button | Hide overdue panel
button | Keyboard shortcuts
button | Last Month
button | Late, click to mark paid
button | Move <script>alert(#)</script> down
button | Move <script>alert(#)</script> up
button | Move AMC A-List down
button | Move AMC A-List up
button | Move Adobe Creative Cloud down
button | Move Adobe Creative Cloud up
button | Move Apple iCloud+ down
button | Move Apple iCloud+ up
button | Move Big down
button | Move Big up
button | Move Capital One Quicksilver down
button | Move Capital One Quicksilver up
button | Move Car Insurance down
@ -90,8 +81,6 @@ button | Move Student Loan down
button | Move Student Loan up
button | Move Trash Service down
button | Move Trash Service up
button | Move Zero down
button | Move Zero up
button | Next month
button | Open Tanstack query devtools
button | Open calendar for this month
@ -101,25 +90,19 @@ button | Paid
button | Paid Date
button | Paid, click to mark unpaid
button | Pay all due (#)
button | Pay now
button | Pin Due
button | Previous month
button | Review overdue bills# items ready for pay, skip, or snooze$#
button | Search & sort# of # shown · no filters
button | Search billsCtrl K
button | Skip
button | Snooze
button | Status
button | Today
button | Tracker#
button | Tracker
button | Unpaid
button | Upcoming, click to mark paid
button | —
checkbox | Select <script>alert(#)</script>
checkbox | Select AMC A-List
checkbox | Select Adobe Creative Cloud
checkbox | Select Apple iCloud+
checkbox | Select Big
checkbox | Select Capital One Quicksilver
checkbox | Select Car Insurance
checkbox | Select Car Payment
@ -138,13 +121,10 @@ checkbox | Select Netflix
checkbox | Select Spotify
checkbox | Select Student Loan
checkbox | Select Trash Service
checkbox | Select Zero
combobox | Filter by billing schedule
combobox | Filter by category
combobox | Sort by
input:number | <script>alert(#)</script> payment amount
input:number | Apple iCloud+ payment amount
input:number | Big payment amount
input:number | Capital One Quicksilver payment amount
input:number | Car Payment payment amount
input:number | Cell Phone payment amount

View File

@ -51,6 +51,8 @@ module.exports = defineConfig({
// probe-only: enables WebAuthn for the seeded user mid-test, which would
// race the password logins in these parallel UI projects
/webauthn\.probe\.spec\.js/,
// census-project-only: snapshots depend on pristine seed data
/control-census\.spec\.js/,
],
},
{
@ -64,6 +66,8 @@ module.exports = defineConfig({
// probe-only: enables WebAuthn for the seeded user mid-test, which would
// race the password logins in these parallel UI projects
/webauthn\.probe\.spec\.js/,
// census-project-only: snapshots depend on pristine seed data
/control-census\.spec\.js/,
],
},
// Find-mode diagnostics (adversarial API probe + authenticated a11y scan).
@ -71,12 +75,23 @@ module.exports = defineConfig({
// so open findings here don't turn the default suite red. Run on demand:
// `npm run test:e2e:probe`. Fold a spec into the default projects once its
// findings are fixed (it becomes a passing regression guard).
// Control census (Appendix E automation): read-only snapshots of every
// page's interactive controls. Own project ordered BEFORE the probe so the
// snapshots always see pristine seed data — api.probe/webauthn mutate the
// scratch DB (adversarial bills, passkey enrollment) and would otherwise
// make the baselines depend on spec ordering.
{
name: 'probe',
testMatch: /(api\.probe|a11y\.authed|webauthn\.probe|control-census)\.spec\.js/,
name: 'census',
testMatch: /control-census\.spec\.js/,
use: { ...devices['Desktop Chrome'] },
dependencies: ['setup'],
},
{
name: 'probe',
testMatch: /(api\.probe|a11y\.authed|webauthn\.probe)\.spec\.js/,
use: { ...devices['Desktop Chrome'] },
dependencies: ['setup', 'census'],
},
// Cross-browser (WebAuthn/Select behavior differs) — enable when ready:
// { name: 'firefox', use: { ...devices['Desktop Firefox'] }, dependencies: ['setup'], testIgnore: /auth\.setup\.js/ },
// { name: 'webkit', use: { ...devices['Desktop Safari'] }, dependencies: ['setup'], testIgnore: /auth\.setup\.js/ },