1320 lines
52 KiB
Markdown
1320 lines
52 KiB
Markdown
# Engineering Reference Manual — Bill Tracker
|
|
|
|
**Status:** Current code reference
|
|
**Last Updated:** 2026-05-10
|
|
**Version:** 0.23.2
|
|
**Primary stack:** Node.js + Express, React + Vite, SQLite via `better-sqlite3`
|
|
|
|
This manual reflects the current application code in `server.js`, `routes/`, `services/`, `middleware/`, `db/`, `client/`, `package.json`, `Dockerfile`, and `docker-compose.yml`. It is written as a current-state reference, not a changelog.
|
|
|
|
---
|
|
|
|
## 1. System Overview
|
|
|
|
Bill Tracker is a self-hosted bill management application. It supports:
|
|
|
|
- Local username/password authentication, optional single-user mode, and optional Authentik/OIDC login.
|
|
- User-scoped bills, categories, payments, monthly bill overrides, monthly income, and starting cash buckets.
|
|
- Admin user management, backup/restore, cleanup, auth-mode/OIDC configuration, status checks, and migration rollback.
|
|
- Spreadsheet and user-SQLite import workflows.
|
|
- CSV, Excel, and user-SQLite export workflows.
|
|
- SMTP-based bill due notifications.
|
|
- React SPA frontend with protected user/admin routes and CSRF-protected JSON APIs.
|
|
|
|
Runtime flow:
|
|
|
|
1. `server.js` initializes SQLite through `db/database.js`, runs schema/migrations, seeds defaults/admin user, cleans expired sessions, then starts Express.
|
|
2. Express applies security headers, JSON body parsing, cookies, CSRF token provisioning, route-level CSRF/auth/rate-limit middleware, static `dist/` serving, and JSON error formatting.
|
|
3. Route files under `routes/` validate input, enforce ownership through `req.user.id`, and use service modules for business logic.
|
|
4. React under `client/` calls `/api/*` through `client/api.js` with `credentials: include` and CSRF headers on mutating methods.
|
|
|
|
---
|
|
|
|
## 2. Project Layout
|
|
|
|
- `server.js` — Express entry point and route mounting.
|
|
- `routes/` — HTTP API handlers.
|
|
- `services/` — auth, OIDC, backup, cleanup, notification, import, status, audit business logic.
|
|
- `middleware/` — auth guards, CSRF, rate limits, security headers, error formatting.
|
|
- `db/schema.sql` — base SQLite schema.
|
|
- `db/database.js` — DB connection, migrations, defaults, settings, rollback support.
|
|
- `workers/dailyWorker.js` — scheduled notification/cleanup worker entry.
|
|
- `client/` — React SPA.
|
|
- `dist/` — generated Vite build output served by Express.
|
|
- `Dockerfile`, `docker-compose.yml`, `docker-entrypoint.sh` — container deployment.
|
|
|
|
---
|
|
|
|
## 3. Backend Entry Point and Middleware
|
|
|
|
### `server.js`
|
|
|
|
Server defaults:
|
|
|
|
- Port: `PORT || 3000`.
|
|
- Static frontend: `dist/`.
|
|
- Optional CORS: enabled when `CORS_ORIGIN` is set; credentials allowed.
|
|
- Session cleanup: runs on startup and every `SESSION_CLEANUP_INTERVAL_MS || 86400000` ms.
|
|
- Admin seed: `INIT_ADMIN_USER || admin`; `INIT_ADMIN_PASS` or generated/default behavior in `db/database.js`.
|
|
- Environment variables INIT_ADMIN_USER and INIT_ADMIN_PASS (or INIT_REGULAR_USER + INIT_REGULAR_PASS) skip the first-login flow entirely by pre-seeding users with first_login=0 flags via setup/firstRun.js.
|
|
- Optional regular-user seed: `INIT_REGULAR_USER` + `INIT_REGULAR_PASS`; password must be at least 8 chars.
|
|
|
|
Global middleware order:
|
|
|
|
1. `securityHeaders`
|
|
2. optional `cors`
|
|
3. `express.json()`
|
|
4. `cookieParser()`
|
|
5. `csrfTokenProvider`
|
|
6. mounted API routers with route-level rate-limit/auth/CSRF middleware
|
|
7. static `legacy/`, redirect `/login.html` to `/login`, static `dist/`
|
|
8. SPA fallback `GET *` serving `dist/index.html` after ensuring a CSRF token cookie
|
|
9. `errorFormatter`
|
|
10. final JSON error handler for malformed JSON/body size/runtime errors
|
|
|
|
### Authentication middleware
|
|
|
|
`middleware/requireAuth.js` exports:
|
|
|
|
- `requireAuth`: attaches `req.user` from `bt_session`; in single-user mode attaches the configured active regular user without a session.
|
|
- `requireUser`: permits `user` and `admin` roles but blocks the default admin account from tracker access.
|
|
- `requireAdmin`: requires `req.user.role === 'admin'`.
|
|
|
|
### CSRF middleware
|
|
|
|
`middleware/csrf.js`:
|
|
|
|
- Cookie name: `CSRF_COOKIE_NAME || bt_csrf_token`.
|
|
- Header name: `x-csrf-token`.
|
|
- Defaults: `CSRF_HTTP_ONLY !== false`, `CSRF_SAME_SITE || strict`, `CSRF_SECURE !== false`.
|
|
- `csrfTokenProvider` sets a token cookie on responses.
|
|
- `csrfMiddleware` validates mutating requests unless `req.csrfSkip` is set. Token may come from header, query, or body and must match cookie.
|
|
- Failures return 403 and audit `csrf.failure`.
|
|
|
|
### Rate limits
|
|
|
|
Defined in `middleware/rateLimiter.js`:
|
|
|
|
- Login: 10 / 15 min.
|
|
- Password changes: 5 / 15 min.
|
|
- Import: 20 / 15 min.
|
|
- Export: 30 / 15 min.
|
|
- Admin actions: 30 / 15 min.
|
|
- OIDC: 20 / 15 min.
|
|
- Backup operations: 5 / 60 min.
|
|
- Demo-data clear: 3 / 15 min.
|
|
|
|
Rate-limit responses are JSON: `{ "error": "..." }`.
|
|
|
|
### Security headers
|
|
|
|
`middleware/securityHeaders.js` sets CSP with a per-request nonce plus common hardening headers. Static SPA scripts/styles must comply with the generated CSP.
|
|
|
|
### Error formatting
|
|
|
|
`middleware/errorFormatter.js` standardizes route responses into JSON with `error`, `code`, and optional `field`. Common statuses: 400 validation, 401 auth, 403 forbidden, 404 not found, 409 conflict, 429 rate limit, 500 server error.
|
|
|
|
---
|
|
|
|
## 4. Services
|
|
|
|
### `services/authService.js`
|
|
|
|
- Cookie: `bt_session`.
|
|
- Session lifetime: 7 days.
|
|
- Password hashing: bcrypt, cost 12.
|
|
- Functions:
|
|
- `login(username, password)` — verifies active local user, rejects OIDC-only accounts, cleans expired sessions for that user, creates a new session, updates `last_login_at`, and returns `{sessionId, user}` or `null`.
|
|
- `createSession(userId)` — creates a session for an OIDC-provisioned or existing local user after validating that the user is active.
|
|
- `logout(sessionId)` — deletes one session.
|
|
- `getSessionUser(sessionId)` — returns public user fields when the session exists, is unexpired, and belongs to an active user.
|
|
- `rotateSessionId(oldSessionId, userId)` — validates the current session, deletes it, and inserts a replacement session in a transaction. Password changes use this to prevent session fixation.
|
|
- `invalidateOtherSessions(userId, keepSessionId)` — deletes all sessions for a user except the supplied current session; when `keepSessionId` is `null`, deletes every session for that user.
|
|
- `pruneExpiredSessions()` — deletes expired sessions.
|
|
- `publicUser(user)` — strips password/session secrets and normalizes booleans.
|
|
|
|
Password changes through `/api/auth/change-password` update the password hash, clear `must_change_password`, stamp `last_password_change_at`, invalidate all other sessions, rotate the current session ID when a valid session cookie is present, set a replacement `bt_session` cookie, and audit `password.change`.
|
|
|
|
`/api/auth/logout-all` calls `invalidateOtherSessions(req.user.id, null)`, deletes the current cookie session if present, audits `logout.all`, clears `bt_session`, and returns `{success:true}`.
|
|
|
|
### `services/oidcService.js`
|
|
|
|
Handles Authentik/OIDC login with `openid-client`:
|
|
|
|
- Reads settings first, then env fallbacks: issuer URL, client ID/secret, redirect URI, token auth method, scopes, provider name, admin group, auto-provision flag.
|
|
- Builds PKCE login state in `oidc_states`.
|
|
- Discovers provider and caches OIDC client for 1 hour.
|
|
- Exchanges callback code, verifies nonce/state, maps claims to local user.
|
|
- Role mapping uses configured admin group; default role is `user`.
|
|
- Client secret is write-only through admin settings.
|
|
|
|
### `services/backupService.js`
|
|
|
|
- Backup directory: `BACKUP_PATH || backups/`.
|
|
- Valid backup IDs match managed prefixes only: `bill-tracker-backup`, `pre-restore`, `imported-backup`, `scheduled-backup`.
|
|
- Creates SQLite backups with checksum and metadata.
|
|
- Validates uploads by SQLite `integrity_check` and optional SHA-256 checksum.
|
|
- Restore creates a pre-restore backup before swapping DB.
|
|
- Path traversal is prevented by ID regex and `path.relative` checks.
|
|
|
|
### `services/backupScheduler.js`
|
|
|
|
- Validates daily/weekly schedule, `HH:MM` time, retention count 1-365.
|
|
- Stores schedule settings in `settings`.
|
|
- Uses `node-cron`, supports run-now, reload, next-run status, and retention cleanup.
|
|
|
|
### `services/cleanupService.js`
|
|
|
|
Cleanup tasks:
|
|
|
|
- Expired import sessions.
|
|
- Stale temp SQLite export files in OS temp dir.
|
|
- Orphan `.partial`/`.upload` backup files.
|
|
- Optional old import-history pruning.
|
|
|
|
Settings are stored in `settings`; run results are stored as JSON.
|
|
|
|
### `services/notificationService.js`
|
|
|
|
- Builds SMTP transport from global notification settings.
|
|
- Sends test email to an admin-provided address.
|
|
- Runs due-bill notifications for 3 days, 1 day, due today, and overdue.
|
|
- De-duplicates sends via `notifications` unique key.
|
|
- Recipients come from user notification settings when enabled/allowed or global recipient settings.
|
|
|
|
### `services/spreadsheetImportService.js`
|
|
|
|
- Accepts XLSX buffers only, max 10 MB, max 5,000 rows.
|
|
- Detects monthly sheets, headers, categories, bills, amounts, and ambiguous rows.
|
|
- Creates `import_sessions` preview records with 24-hour TTL.
|
|
- Apply step creates/updates user-owned categories, bills, payments, monthly state, and import history.
|
|
|
|
### `services/userDbImportService.js`
|
|
|
|
- Accepts user SQLite export files up to 50 MB.
|
|
- Requires export metadata and known tables.
|
|
- Sanitizes categories, bills, payments, monthly state, and starting amounts.
|
|
- Preview stores an import session; apply maps export IDs to current user-owned IDs.
|
|
|
|
### `services/statusService.js`
|
|
|
|
Shared tracker/calendar logic:
|
|
|
|
- `resolveDueDate(bill, year, month)` clamps due day to month length.
|
|
- `resolveBucket(bill)` uses bucket or due-day threshold.
|
|
- `getCycleRange(year, month)` returns first/last day of month.
|
|
- `calculateStatus(...)` returns paid/autodraft/upcoming/due/overdue-style status.
|
|
- `buildTrackerRow(...)` returns row data for the monthly tracker.
|
|
|
|
### `services/auditService.js`
|
|
|
|
Writes `audit_log` rows for security-sensitive events such as login, logout, password changes, role changes, CSRF failures, user seed flag resets, migration runs, and migration rollback attempts.
|
|
|
|
`db/database.js` does not import `logAudit` at module load time. It uses a lazy `getLogAudit()` helper so migration code can write audit rows without creating a circular dependency: `auditService` imports `database.js`, and `database.js` needs audit logging during migrations. If the lazy require fails, the helper degrades to a no-op audit function while console logging still records migration progress/errors.
|
|
|
|
---
|
|
|
|
## 5. API Reference
|
|
|
|
All routes are prefixed with `/api` unless stated otherwise. Most mutating endpoints require CSRF token header `x-csrf-token`. Auth is cookie-based. User routes are scoped to the authenticated user unless noted.
|
|
|
|
Response conventions:
|
|
|
|
- Success: JSON object/array unless endpoint downloads a file.
|
|
- Validation: `400 {error, code?, field?}`.
|
|
- Unauthenticated: `401`.
|
|
- Forbidden: `403`.
|
|
- Missing resource: `404`.
|
|
- Conflict: `409`.
|
|
- Rate-limited: `429`.
|
|
|
|
### 5.1 Auth
|
|
|
|
Mounted under `/api/auth`.
|
|
|
|
- `POST /auth/login`
|
|
- Body: `{username, password}`.
|
|
- Validation: both required; local login must be enabled.
|
|
- Response: sets `bt_session`; `{user}`.
|
|
|
|
- `POST /auth/logout`
|
|
- Auth: required.
|
|
- Body: none.
|
|
- Response: clears cookie; `{success:true}`.
|
|
|
|
- `POST /auth/logout-all`
|
|
- Auth: required; CSRF skip is set before the router mount, matching other auth/session mutation routes.
|
|
- Body: none.
|
|
- Behavior: deletes every session for the current user by calling `invalidateOtherSessions(userId, null)`, also deletes the current cookie session, audits `logout.all`, clears `bt_session`, and returns `{success:true}`.
|
|
|
|
- `GET /auth/me`
|
|
- Auth: required unless single-user mode supplies user.
|
|
- Response: public user object.
|
|
|
|
- `GET /auth/mode`
|
|
- Public.
|
|
- Response: auth mode, local-login flag, OIDC public info, single-user status.
|
|
|
|
- `POST /auth/restore-multi-user-mode`
|
|
- Auth: required.
|
|
- Body: none.
|
|
- Response: restores multi-user mode where allowed.
|
|
|
|
- `POST /auth/acknowledge-privacy`
|
|
- Auth: required.
|
|
- Body: none.
|
|
- Response: updates first-login/privacy acknowledgement flags.
|
|
|
|
- `POST /auth/change-password`
|
|
- Auth: required; password limiter; CSRF skip is set before the router mount.
|
|
- Body: `{current_password, new_password}`.
|
|
- Validation: current password required unless `must_change_password` is set; new password min 8.
|
|
- Behavior: updates password hash, clears `must_change_password`, updates `last_password_change_at`, invalidates all other sessions, rotates the current session ID when a valid `bt_session` exists, sets the new cookie, audits `password.change`, and returns `{success:true}`.
|
|
|
|
- `GET /auth/has-users`
|
|
- Response: whether non-default users exist.
|
|
|
|
- `GET /auth/users`
|
|
- Auth: admin.
|
|
- Response: safe user list.
|
|
|
|
- `POST /auth/users`
|
|
- Auth: admin.
|
|
- Body: `{username, password}`.
|
|
- Validation: username min 3, password min 8, unique username.
|
|
- Response: created safe user.
|
|
|
|
### 5.2 OIDC Auth
|
|
|
|
Mounted under `/api/auth/oidc`; OIDC rate limiter applies.
|
|
|
|
- `GET /auth/oidc/login?redirect_to=/path`
|
|
- Public when OIDC active.
|
|
- Creates PKCE state and redirects to provider authorization URL.
|
|
|
|
- `GET /auth/oidc/callback?code=&state=`
|
|
- Public callback.
|
|
- Validates state/nonce, exchanges code, provisions/fetches user, creates session, redirects to saved path or app root.
|
|
- Error redirects use query errors such as `access_denied` or `authentication_failed`.
|
|
|
|
### 5.3 Admin
|
|
|
|
Mounted under `/api/admin`; all require `requireAuth + requireAdmin + adminActionLimiter` at the mount level. Backup subroutes also use `backupOperationLimiter`.
|
|
|
|
- `GET /admin/has-users`
|
|
- Response: `{has_users:boolean}` for users other than current admin.
|
|
|
|
- `GET /admin/users`
|
|
- Response: safe users ordered by default-admin, role, username.
|
|
|
|
- `POST /admin/users`
|
|
- Body: `{username, password}`.
|
|
- Validation: username min 3, password min 8, unique.
|
|
- Response 201: created user.
|
|
|
|
- `PUT /admin/users/:id/password`
|
|
- Body: `{password}`.
|
|
- Validation: password min 8; user exists.
|
|
- Response: `{success:true}`; invalidates target sessions and requires password change.
|
|
|
|
- `PUT /admin/users/:id/role`
|
|
- Body: `{role:"admin"|"user"}`.
|
|
- Validation: cannot change own role; cannot remove last admin.
|
|
- Response: updated safe user; invalidates target sessions; audits role change.
|
|
|
|
- `PUT /admin/users/:id/active`
|
|
- Body: `{active:boolean}`.
|
|
- Validation: user exists; cannot deactivate self.
|
|
- Response: updated safe user; deactivation invalidates sessions.
|
|
|
|
- `DELETE /admin/users/:id`
|
|
- Validation: user exists; cannot delete self.
|
|
- Response: `{success:true, deleted_user_id}`; transaction deletes import sessions/history, sessions, and user.
|
|
|
|
- `POST /admin/backups`
|
|
- Body: none.
|
|
- Response 201: backup metadata `{id, filename, size_bytes, checksum, ...}`.
|
|
|
|
- `GET /admin/backups`
|
|
- Response: `{backups:[metadata...]}`.
|
|
|
|
- `GET /admin/backups/settings`
|
|
- Response: backup schedule status/settings.
|
|
|
|
- `PUT /admin/backups/settings`
|
|
- Body: `{enabled, frequency:"daily"|"weekly", time:"HH:MM", retention_count}`.
|
|
- Validation: frequency, valid time, retention 1-365.
|
|
- Response: saved schedule status.
|
|
|
|
- `POST /admin/backups/run-scheduled-now`
|
|
- Body: none.
|
|
- Response 201: scheduled backup result.
|
|
|
|
- `POST /admin/backups/import`
|
|
- Content-Type: `application/octet-stream`, `application/x-sqlite3`, or `application/vnd.sqlite3`.
|
|
- Body: raw SQLite backup, max 100 MB.
|
|
- Optional checksum: `X-Checksum-Sha256` header or `?checksum=`.
|
|
- Response 201: imported backup metadata.
|
|
|
|
- `GET /admin/backups/:id/download`
|
|
- Response: file download. ID must be a managed backup filename.
|
|
|
|
- `POST /admin/backups/:id/restore`
|
|
- Response: `{restored_from, pre_restore_backup}`; validates and restores managed backup.
|
|
|
|
- `DELETE /admin/backups/:id`
|
|
- Response: `{deleted:true, id, deleted_at}`.
|
|
|
|
- `GET /admin/cleanup`
|
|
- Response: cleanup settings and last result.
|
|
|
|
- `PUT /admin/cleanup`
|
|
- Body: any of `{import_sessions_enabled, temp_exports_enabled, temp_export_max_age_hours, backup_partials_enabled, import_history_enabled, import_history_max_age_days}`.
|
|
- Validation: booleans; temp export age 1-72 hours; import history age 30-3650 days.
|
|
- Response: updated cleanup status.
|
|
|
|
- `POST /admin/cleanup/run`
|
|
- Response: cleanup run result by task.
|
|
|
|
- `GET /admin/auth-mode`
|
|
- Response: local/single-user/OIDC settings and lockout warnings. Client secret is not returned.
|
|
|
|
- `POST /admin/auth-mode/oidc-test`
|
|
- Body: submitted or saved OIDC config fields.
|
|
- Response: `{ok:true,...}` or 400 with test error. Never returns secret/token material.
|
|
|
|
- `PUT /admin/auth-mode`
|
|
- Body: legacy `{auth_mode, default_user_id}` plus OIDC/local settings such as `local_login_enabled`, `oidc_login_enabled`, `oidc_issuer_url`, `oidc_client_id`, `oidc_client_secret`, `oidc_redirect_uri`, `oidc_scopes`, `oidc_admin_group`, `oidc_auto_provision`.
|
|
- Validation: cannot disable all login methods; cannot disable local login until OIDC is configured, enabled, and has an admin group; cannot enable incomplete OIDC.
|
|
- Response: `{success:true, ...authModeStatus}`.
|
|
|
|
- `POST /admin/migrations/rollback`
|
|
- Body: `{version:"v0.44"|"v0.45"|"v0.46"}`.
|
|
- Validation: version required; migration must be present in `schema_migrations`; `ROLLBACK_SQL_MAP` must define rollback SQL for that version.
|
|
- Behavior: calls `rollbackMigration(version)` from `db/database.js`, audits success as `migration.rollback`, and returns `{success:true, version, description, elapsed_ms}`.
|
|
- Error mapping: `NOT_APPLIED` becomes HTTP 404 with `{error}`; `ROLLBACK_NOT_SUPPORTED` becomes HTTP 422 with `{error}`; other rollback failures become HTTP 500 with `{error:"Rollback failed", details}` and are audited as `migration.rollback.failure`.
|
|
|
|
### 5.4 Bills
|
|
|
|
Mounted under `/api/bills`; auth: user/admin tracker access.
|
|
|
|
Bill object fields include `id`, `user_id`, `name`, `category_id`, `category_name`, `due_day`, `override_due_date`, `bucket`, `expected_amount`, `interest_rate`, `billing_cycle`, `autopay_enabled`, `autodraft_status`, `website`, `username`, `account_info`, `has_2fa`, `active`, `notes`, `history_visibility`, `is_seeded`, `cycle_type`, `cycle_day`, timestamps, and `has_history_ranges` on list/detail queries.
|
|
|
|
Validation shared by create/update:
|
|
|
|
- `name` required for create.
|
|
- `due_day`: integer 1-31.
|
|
- `expected_amount`: numeric, defaults to 0.
|
|
- `interest_rate`: null/empty or number 0-100.
|
|
- `category_id`: must belong to current user.
|
|
- `history_visibility`: `default`, `all`, `ranges`, or `none`.
|
|
- `cycle_type`: `monthly`, `weekly`, `biweekly`, `quarterly`, `annual`.
|
|
- `cycle_day`: monthly 1-31; weekly/biweekly day name; quarterly/annual text up to 50 chars.
|
|
|
|
Endpoints:
|
|
|
|
- `GET /bills?inactive=true`
|
|
- Response: current user's bills; inactive excluded unless `inactive=true`.
|
|
|
|
- `GET /bills/:id`
|
|
- Response: one owned bill or 404.
|
|
|
|
- `POST /bills`
|
|
- Body: bill fields.
|
|
- Response 201: created bill.
|
|
|
|
- `PUT /bills/:id`
|
|
- Body: partial bill fields.
|
|
- Response: updated bill.
|
|
|
|
- `DELETE /bills/:id`
|
|
- Hard delete; cascades payments, monthly state, history ranges.
|
|
- Response: `{success:true, deleted_bill_id, deleted_bill_name, warning}`.
|
|
|
|
- `GET /bills/:id/monthly-state?year=&month=`
|
|
- Validation: year 2000-2100; month 1-12.
|
|
- Response: `{bill_id, year, month, actual_amount, notes, is_skipped}`.
|
|
|
|
- `PUT /bills/:id/monthly-state`
|
|
- Body: `{year, month, actual_amount, notes, is_skipped}`.
|
|
- Validation: year/month; actual_amount null or non-negative.
|
|
- Response: saved monthly state with timestamps.
|
|
|
|
- `GET /bills/:id/payments?page=1&limit=20`
|
|
- Validation: limit capped at 100.
|
|
- Response: `{bill_id, bill_name, total, page, limit, pages, payments}`.
|
|
|
|
- `POST /bills/:id/toggle-paid`
|
|
- Body optional: `{amount, paid_date, method, notes}`.
|
|
- If latest live payment exists, soft-deletes it. Otherwise creates a payment for amount or bill expected amount.
|
|
- Response: `{success, isPaid, action, payment?}`.
|
|
|
|
- `GET /bills/:id/history-ranges`
|
|
- Response: `{bill_id, history_visibility, ranges}`.
|
|
|
|
- `POST /bills/:id/history-ranges`
|
|
- Body: `{start_year, start_month, end_year?, end_month?, label?}`.
|
|
- Validation: years 2000-2100, months 1-12, end both present/absent and not before start.
|
|
- Response 201: created range.
|
|
|
|
- `PUT /bills/:id/history-ranges/:rangeId`
|
|
- Body: partial range fields.
|
|
- Response: updated range.
|
|
|
|
- `DELETE /bills/:id/history-ranges/:rangeId`
|
|
- Response: `{success:true}`.
|
|
|
|
### 5.5 Payments
|
|
|
|
Mounted under `/api/payments`; auth: user/admin tracker access. All queries are user-owned through joined bill ownership. Delete is soft delete via `deleted_at`.
|
|
|
|
- `GET /payments?bill_id=&year=&month=`
|
|
- Validation: year and month must be supplied together; year 2000-2100; month 1-12.
|
|
- Response: live payments ordered descending by `paid_date`.
|
|
|
|
- `GET /payments/:id`
|
|
- Response: live payment or 404.
|
|
|
|
- `POST /payments`
|
|
- Body: `{bill_id, amount, paid_date, method?, notes?}`.
|
|
- Validation: bill exists and owned; amount > 0; required fields present.
|
|
- Response 201: created payment.
|
|
|
|
- `POST /payments/quick`
|
|
- Body: `{bill_id, amount?, paid_date?, method?, notes?}`.
|
|
- Defaults amount to bill expected amount and date to today; confirms autodraft status for autopay bills.
|
|
- Response 201: created payment.
|
|
|
|
- `POST /payments/bulk`
|
|
- Body: `{payments:[{bill_id, amount, paid_date, method?, notes?}]}`.
|
|
- Validation: array required; max 50; bill_id integer; `paid_date` `YYYY-MM-DD`; amount finite >= 0.
|
|
- Duplicate live payments by user/bill/date/amount are skipped.
|
|
- Response 201: `{created:[...], skipped:[...], errors:[...]}`.
|
|
|
|
- `PUT /payments/:id`
|
|
- Body: partial `{amount, paid_date, method, notes}`.
|
|
- Response: updated payment. Current code preserves existing fields when omitted.
|
|
|
|
- `DELETE /payments/:id`
|
|
- Response: `{success:true}` after setting `deleted_at`.
|
|
|
|
- `POST /payments/:id/restore`
|
|
- Response: restored payment with `deleted_at:null`.
|
|
|
|
### 5.6 Categories
|
|
|
|
Mounted under `/api/categories`; auth: user/admin tracker access.
|
|
|
|
- `GET /categories`
|
|
- Seeds default categories for user if needed.
|
|
- Response: categories with bill counts/payment counts and bill summaries.
|
|
|
|
- `POST /categories`
|
|
- Body: `{name}`.
|
|
- Validation: non-empty name; unique per user case-insensitive.
|
|
- Response 201: created category.
|
|
|
|
- `PUT /categories/:id`
|
|
- Body: `{name}`.
|
|
- Validation: category belongs to user; non-empty unique name.
|
|
- Response: updated category.
|
|
|
|
- `DELETE /categories/:id`
|
|
- Validation: category belongs to user.
|
|
- Behavior: transaction nulls category on owned bills, then deletes category.
|
|
- Response: deletion summary.
|
|
|
|
### 5.7 Tracker and Calendar
|
|
|
|
- `GET /tracker?year=&month=`
|
|
- Auth: user/admin tracker access.
|
|
- Defaults to current year/month.
|
|
- Response includes `year`, `month`, tracker `rows`, totals, starting amount info, previous month paid total, three-month averages/trends, and generated timestamp.
|
|
- Row fields come from `buildTrackerRow` plus monthly override state and previous-month payment data.
|
|
|
|
- `GET /tracker/upcoming?days=30`
|
|
- Auth: user/admin tracker access.
|
|
- Response: upcoming active bills in the requested horizon.
|
|
|
|
- `GET /calendar?year=&month=`
|
|
- Auth: user/admin tracker access.
|
|
- Defaults current year/month.
|
|
- Response: month days with payment entries, bills/due-status entries, and totals `{expectedTotal, paidTotal, remainingTotal, paidPercent}`.
|
|
|
|
### 5.8 Summary and Starting Amounts
|
|
|
|
- `GET /summary?year=&month=`
|
|
- Auth: user/admin tracker access.
|
|
- Validation: valid year/month.
|
|
- Response: `{year, month, income, expenses, starting_amounts, previous_month, summary, chart, generated_at}`.
|
|
|
|
- `PUT /summary/income`
|
|
- Body: `{year, month, amount, label?}`.
|
|
- Validation: valid year/month; amount 0-1,000,000,000; label trimmed to 80 chars.
|
|
- Response: `{year, month, income}` after upsert into `monthly_income`.
|
|
|
|
- `GET /monthly-starting-amounts?year=&month=`
|
|
- Response: `{year, month, first_amount, fifteenth_amount, other_amount, combined_amount, paid deductions, remaining values, notes}`.
|
|
|
|
- `PUT /monthly-starting-amounts`
|
|
- Body: `{year, month, first_amount, fifteenth_amount, other_amount, notes?}`.
|
|
- Validation: valid year/month; numeric amounts.
|
|
- Response: recomputed starting-amount response after upsert.
|
|
|
|
### 5.9 Analytics
|
|
|
|
- `GET /analytics/summary?year=&month=&months=&category_id=&bill_id=&include_inactive=true&include_skipped=false`
|
|
- Auth: user/admin tracker access.
|
|
- Validation: year/month valid; months clamped by route validation; IDs parsed as integers.
|
|
- Response includes monthly spending, expected vs actual, category totals, bill totals, filters, and generated timestamp.
|
|
|
|
### 5.10 Settings
|
|
|
|
- `GET /settings`
|
|
- Auth: user/admin tracker access.
|
|
- Response: user-visible settings from the allowed settings key list.
|
|
|
|
- `PUT /settings`
|
|
- Auth: user/admin tracker access.
|
|
- Body: key/value object for allowed user setting keys.
|
|
- Response: updated settings object.
|
|
|
|
- `POST /settings/seed-demo-data`
|
|
- Auth: user/admin tracker access.
|
|
- Response: demo seed result.
|
|
|
|
### 5.11 Notifications
|
|
|
|
Mounted under `/api/notifications`. Server mount requires `requireAuth`; route-level guards further restrict.
|
|
|
|
- `GET /notifications/admin`
|
|
- Auth: admin.
|
|
- Response: global SMTP/notification settings.
|
|
|
|
- `PUT /notifications/admin`
|
|
- Auth: admin.
|
|
- Body: allowed global SMTP and notification settings.
|
|
- Response: saved settings.
|
|
|
|
- `POST /notifications/test`
|
|
- Auth: admin.
|
|
- Body: `{to}`.
|
|
- Validation: recipient required.
|
|
- Response: send result or SMTP error.
|
|
|
|
- `GET /notifications/me`
|
|
- Auth: user/admin tracker access.
|
|
- Response: current user's notification email and toggle settings.
|
|
|
|
- `PUT /notifications/me`
|
|
- Auth: user/admin tracker access.
|
|
- Body: `{notification_email, notifications_enabled, notify_3d, notify_1d, notify_due, notify_overdue}`.
|
|
- Response: saved user notification settings.
|
|
|
|
### 5.12 Profile
|
|
|
|
Mounted under `/api/profile`; auth: user/admin tracker access; password limiter applies at mount.
|
|
|
|
- `GET /profile`
|
|
- Response: safe profile, notification settings, export URLs, import-history URL.
|
|
|
|
- `PATCH /profile`
|
|
- Body: `{display_name}`.
|
|
- Validation: string, max 100 chars.
|
|
- Response: `{success:true, profile}`.
|
|
|
|
- `GET /profile/settings`
|
|
- Response: user notification preferences only.
|
|
|
|
- `PATCH /profile/settings`
|
|
- Body: `{notification_email|email, notifications_enabled, notify_3d, notify_1d, notify_due, notify_overdue}`.
|
|
- Validation: email value string/null, max 255 chars.
|
|
- Response: `{success:true}`.
|
|
|
|
- `POST /profile/change-password`
|
|
- Body: `{current_password, new_password, confirm_new_password}`.
|
|
- Validation: all required; confirmation matches; new password min 8; current password must verify.
|
|
- Response: rotates current session, invalidates others, `{success:true}`.
|
|
|
|
- `GET /profile/exports`
|
|
- Response: metadata for `user_db` and `user_excel` export URLs.
|
|
|
|
- `GET /profile/import-history`
|
|
- Response: `{history:[...]}`.
|
|
|
|
### 5.13 User Demo Data
|
|
|
|
Mounted under `/api/user`; auth: user/admin tracker access.
|
|
|
|
- `POST /user/seed-demo-data`
|
|
- Response: seed result for current user.
|
|
|
|
- `POST /user/clear-demo-data`
|
|
- Rate-limited by demo-data limiter.
|
|
- Behavior: deletes `is_seeded=1` bills/categories for current user and records import history.
|
|
- Response: deletion counts.
|
|
|
|
### 5.14 Import
|
|
|
|
Mounted under `/api/import`; auth: user/admin tracker access; import limiter applies.
|
|
|
|
- `POST /import/spreadsheet/preview?parse_all_sheets=true&year=&month=`
|
|
- Content-Type: `application/octet-stream`.
|
|
- Headers: optional `X-Filename`.
|
|
- Body: XLSX file buffer, max 10 MB.
|
|
- Response: preview session, parsed rows, categories/bills/payment candidates, ambiguous decisions, errors.
|
|
|
|
- `POST /import/spreadsheet/apply`
|
|
- Body: `{import_session_id, decisions, options}`.
|
|
- Validation: session exists, not expired, belongs to user.
|
|
- Response: created/updated/skipped/error counts and import history summary.
|
|
|
|
- `POST /import/user-db/preview`
|
|
- Content-Type: `application/octet-stream`.
|
|
- Headers: optional `X-Filename`.
|
|
- Body: user SQLite export file, max 50 MB.
|
|
- Response: preview session and sanitized import plan.
|
|
|
|
- `POST /import/user-db/apply`
|
|
- Body: `{import_session_id, options}`.
|
|
- Validation: session exists, not expired, belongs to user.
|
|
- Response: apply result with ID mappings/counts.
|
|
|
|
- `GET /import/history`
|
|
- Response: current user's import history.
|
|
|
|
### 5.15 Export
|
|
|
|
Mounted under `/api/export`; auth: user/admin tracker access; export limiter applies.
|
|
|
|
- `GET /export?year=YYYY&format=csv|xlsx`
|
|
- Response: file download of payment/bill history for the requested year. CSV includes date, bill, category, expected, paid, method, notes, actual amount, monthly notes. XLSX includes enriched rows.
|
|
|
|
- `GET /export/user-excel`
|
|
- Response: Excel workbook with user categories, bills, payments, monthly state, monthly starting amounts, and notes/metadata.
|
|
|
|
- `GET /export/user-db`
|
|
- Response: portable SQLite file with export metadata and user-owned categories, bills, payments, monthly state, monthly starting amounts, and notes.
|
|
|
|
### 5.16 Status
|
|
|
|
- `GET /status`
|
|
- Auth: admin.
|
|
- Response: app version, uptime, runtime worker state, DB health/counts/path/size, SMTP configuration status, backup status/schedule, current-month tracker health, recent errors.
|
|
|
|
### 5.17 About and Version
|
|
|
|
- `GET /about`
|
|
- Public.
|
|
- Response: package version and public project/about metadata.
|
|
|
|
- `GET /about-admin`
|
|
- Auth: admin; admin action limiter; CSRF middleware.
|
|
- Response: package version plus sanitized/redacted `FUTURE.md` and `DEVELOPMENT_LOG.md` content.
|
|
- File allowlist only: `FUTURE.md`, `DEVELOPMENT_LOG.md`.
|
|
|
|
- `GET /version`
|
|
- Public.
|
|
- Response: current package version and latest structured notes from `HISTORY.md`.
|
|
|
|
- `GET /version/history`
|
|
- Public.
|
|
- Response: package version and raw history text, or error if unavailable.
|
|
|
|
---
|
|
|
|
## 6. Database Reference
|
|
|
|
SQLite uses WAL mode and foreign keys. Base schema is in `db/schema.sql`; `db/database.js` applies migrations to reach the current schema.
|
|
|
|
### Tables and columns
|
|
|
|
#### `users`
|
|
|
|
- `id INTEGER PRIMARY KEY`
|
|
- `username TEXT NOT NULL UNIQUE COLLATE NOCASE`
|
|
- `password_hash TEXT NOT NULL`
|
|
- `role TEXT NOT NULL DEFAULT 'user'` (`admin` or `user`)
|
|
- `must_change_password INTEGER NOT NULL DEFAULT 0`
|
|
- `first_login INTEGER NOT NULL DEFAULT 1`
|
|
- `created_at TEXT DEFAULT datetime('now')`
|
|
- `updated_at TEXT DEFAULT datetime('now')`
|
|
- `notification_email TEXT`
|
|
- `notifications_enabled INTEGER NOT NULL DEFAULT 0`
|
|
- `notify_3d INTEGER NOT NULL DEFAULT 1`
|
|
- `notify_1d INTEGER NOT NULL DEFAULT 1`
|
|
- `notify_due INTEGER NOT NULL DEFAULT 1`
|
|
- `notify_overdue INTEGER NOT NULL DEFAULT 1`
|
|
- `display_name TEXT`
|
|
- `last_password_change_at TEXT`
|
|
- `auth_provider TEXT NOT NULL DEFAULT 'local'`
|
|
- `external_subject TEXT`
|
|
- `email TEXT`
|
|
- `last_login_at TEXT`
|
|
- `active INTEGER NOT NULL DEFAULT 1`
|
|
- `is_default_admin INTEGER NOT NULL DEFAULT 0`
|
|
|
|
#### `sessions`
|
|
|
|
- `id TEXT PRIMARY KEY`
|
|
- `user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE`
|
|
- `expires_at TEXT NOT NULL`
|
|
- `created_at TEXT DEFAULT datetime('now')`
|
|
|
|
#### `categories`
|
|
|
|
- `id INTEGER PRIMARY KEY`
|
|
- `user_id INTEGER REFERENCES users(id) ON DELETE CASCADE`
|
|
- `name TEXT NOT NULL`
|
|
- `created_at TEXT DEFAULT datetime('now')`
|
|
- `updated_at TEXT DEFAULT datetime('now')`
|
|
- `is_seeded INTEGER NOT NULL DEFAULT 0`
|
|
|
|
#### `bills`
|
|
|
|
- `id INTEGER PRIMARY KEY`
|
|
- `user_id INTEGER REFERENCES users(id) ON DELETE CASCADE`
|
|
- `name TEXT NOT NULL`
|
|
- `category_id INTEGER REFERENCES categories(id) ON DELETE SET NULL`
|
|
- `due_day INTEGER NOT NULL CHECK 1-31`
|
|
- `override_due_date TEXT`
|
|
- `bucket TEXT CHECK ('1st','15th')`
|
|
- `expected_amount REAL NOT NULL DEFAULT 0`
|
|
- `interest_rate REAL CHECK null or 0-100`
|
|
- `billing_cycle TEXT DEFAULT 'monthly' CHECK ('monthly','quarterly','annually','irregular')`
|
|
- `autopay_enabled INTEGER NOT NULL DEFAULT 0`
|
|
- `autodraft_status TEXT NOT NULL DEFAULT 'none' CHECK ('none','pending','assumed_paid','confirmed')`
|
|
- `website TEXT`
|
|
- `username TEXT`
|
|
- `account_info TEXT`
|
|
- `has_2fa INTEGER NOT NULL DEFAULT 0`
|
|
- `active INTEGER NOT NULL DEFAULT 1`
|
|
- `notes TEXT`
|
|
- `created_at TEXT DEFAULT datetime('now')`
|
|
- `updated_at TEXT DEFAULT datetime('now')`
|
|
- `history_visibility TEXT NOT NULL DEFAULT 'default'`
|
|
- `is_seeded INTEGER NOT NULL DEFAULT 0`
|
|
- `cycle_type TEXT NOT NULL DEFAULT 'monthly'`
|
|
- `cycle_day TEXT`
|
|
|
|
#### `payments`
|
|
|
|
- `id INTEGER PRIMARY KEY`
|
|
- `bill_id INTEGER NOT NULL REFERENCES bills(id) ON DELETE CASCADE`
|
|
- `amount REAL NOT NULL`
|
|
- `paid_date TEXT NOT NULL`
|
|
- `method TEXT`
|
|
- `notes TEXT`
|
|
- `created_at TEXT DEFAULT datetime('now')`
|
|
- `updated_at TEXT DEFAULT datetime('now')`
|
|
- `deleted_at TEXT`
|
|
|
|
#### `monthly_bill_state`
|
|
|
|
- `id INTEGER PRIMARY KEY`
|
|
- `bill_id INTEGER NOT NULL REFERENCES bills(id) ON DELETE CASCADE`
|
|
- `year INTEGER NOT NULL CHECK 2000-2100`
|
|
- `month INTEGER NOT NULL CHECK 1-12`
|
|
- `actual_amount REAL`
|
|
- `notes TEXT`
|
|
- `is_skipped INTEGER NOT NULL DEFAULT 0`
|
|
- `created_at TEXT DEFAULT datetime('now')`
|
|
- `updated_at TEXT DEFAULT datetime('now')`
|
|
- Unique: `(bill_id, year, month)`
|
|
|
|
#### `monthly_income`
|
|
|
|
- `id INTEGER PRIMARY KEY`
|
|
- `user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE`
|
|
- `year INTEGER NOT NULL`
|
|
- `month INTEGER NOT NULL`
|
|
- `label TEXT NOT NULL DEFAULT 'Salary'`
|
|
- `amount REAL NOT NULL DEFAULT 0`
|
|
- `created_at TEXT DEFAULT datetime('now')`
|
|
- `updated_at TEXT DEFAULT datetime('now')`
|
|
- Unique intended/current logic: `(user_id, year, month)` via migration/index.
|
|
|
|
#### `monthly_starting_amounts`
|
|
|
|
- `id INTEGER PRIMARY KEY`
|
|
- `user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE`
|
|
- `year INTEGER NOT NULL`
|
|
- `month INTEGER NOT NULL`
|
|
- `first_amount REAL NOT NULL DEFAULT 0`
|
|
- `fifteenth_amount REAL NOT NULL DEFAULT 0`
|
|
- `other_amount REAL NOT NULL DEFAULT 0`
|
|
- `notes TEXT`
|
|
- `created_at TEXT DEFAULT datetime('now')`
|
|
- `updated_at TEXT DEFAULT datetime('now')`
|
|
- Unique intended/current logic: `(user_id, year, month)` via migration/index.
|
|
|
|
#### `bill_history_ranges`
|
|
|
|
- `id INTEGER PRIMARY KEY`
|
|
- `bill_id INTEGER NOT NULL REFERENCES bills(id) ON DELETE CASCADE`
|
|
- `start_year INTEGER NOT NULL`
|
|
- `start_month INTEGER NOT NULL`
|
|
- `end_year INTEGER`
|
|
- `end_month INTEGER`
|
|
- `label TEXT`
|
|
- `created_at TEXT DEFAULT datetime('now')`
|
|
- `updated_at TEXT DEFAULT datetime('now')`
|
|
|
|
#### `settings`
|
|
|
|
- `key TEXT PRIMARY KEY`
|
|
- `value TEXT NOT NULL`
|
|
- `updated_at TEXT DEFAULT datetime('now')`
|
|
|
|
Used for app settings, auth mode, OIDC settings, SMTP settings, backup schedule, cleanup settings, and worker state.
|
|
|
|
#### `notifications`
|
|
|
|
- `id INTEGER PRIMARY KEY`
|
|
- `bill_id INTEGER NOT NULL REFERENCES bills(id) ON DELETE CASCADE`
|
|
- `user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE`
|
|
- `year INTEGER NOT NULL`
|
|
- `month INTEGER NOT NULL`
|
|
- `type TEXT NOT NULL` (`due_3d`, `due_1d`, `due_today`, `overdue`)
|
|
- `sent_date TEXT NOT NULL DEFAULT date('now')`
|
|
- Unique: `(bill_id, user_id, year, month, type, sent_date)`
|
|
|
|
#### `import_sessions`
|
|
|
|
- `id TEXT PRIMARY KEY`
|
|
- `user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE`
|
|
- `created_at TEXT NOT NULL`
|
|
- `expires_at TEXT NOT NULL`
|
|
- `preview_json TEXT NOT NULL`
|
|
|
|
#### `import_history`
|
|
|
|
- `id INTEGER PRIMARY KEY`
|
|
- `user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE`
|
|
- `imported_at TEXT NOT NULL`
|
|
- `source_filename TEXT`
|
|
- `file_type TEXT DEFAULT 'xlsx'`
|
|
- `sheet_name TEXT`
|
|
- `rows_parsed INTEGER DEFAULT 0`
|
|
- `rows_created INTEGER DEFAULT 0`
|
|
- `rows_updated INTEGER DEFAULT 0`
|
|
- `rows_skipped INTEGER DEFAULT 0`
|
|
- `rows_ambiguous INTEGER DEFAULT 0`
|
|
- `rows_errored INTEGER DEFAULT 0`
|
|
- `options_json TEXT`
|
|
- `summary_json TEXT`
|
|
|
|
#### `oidc_states`
|
|
|
|
- `id TEXT PRIMARY KEY`
|
|
- `nonce TEXT NOT NULL`
|
|
- `code_verifier TEXT NOT NULL`
|
|
- `redirect_to TEXT`
|
|
- `created_at TEXT NOT NULL`
|
|
- `expires_at TEXT NOT NULL`
|
|
|
|
#### `audit_log`
|
|
|
|
- `id INTEGER PRIMARY KEY`
|
|
- `user_id INTEGER`
|
|
- `action TEXT NOT NULL`
|
|
- `entity_type TEXT`
|
|
- `entity_id INTEGER`
|
|
- `details_json TEXT`
|
|
- `ip_address TEXT`
|
|
- `user_agent TEXT`
|
|
- `created_at TEXT DEFAULT datetime('now')`
|
|
|
|
#### `schema_migrations`
|
|
|
|
- `id INTEGER PRIMARY KEY`
|
|
- `version TEXT NOT NULL UNIQUE`
|
|
- `description TEXT NOT NULL`
|
|
- `applied_at TEXT NOT NULL DEFAULT datetime('now')`
|
|
|
|
### Indexes
|
|
|
|
Important indexes include:
|
|
|
|
- `idx_bills_active(active)`
|
|
- `idx_bills_user_active(user_id, active)`
|
|
- `idx_bills_user_name(user_id, name)`
|
|
- `idx_categories_user_name(user_id, name)`
|
|
- `idx_categories_user_name_unique(user_id, name COLLATE NOCASE)`
|
|
- `idx_payments_bill_id(bill_id)`
|
|
- `idx_payments_paid_date(paid_date)`
|
|
- `idx_payments_bill_date_del(bill_id, paid_date, deleted_at)`
|
|
- `idx_payments_deleted(deleted_at)`
|
|
- `idx_payments_method(method)`
|
|
- `idx_sessions_user_id(user_id)`
|
|
- `idx_sessions_expires(expires_at)`
|
|
- `idx_monthly_bill_state_lookup(bill_id, year, month)`
|
|
- `idx_monthly_income_user_month(user_id, year, month)`
|
|
- `idx_monthly_starting_amounts_user(user_id)`
|
|
- `idx_monthly_starting_amounts_user_month(user_id, year, month)`
|
|
- `idx_notifications_lookup(bill_id, user_id, year, month)`
|
|
- `idx_import_sessions_user(user_id)`, `idx_import_sessions_expires(expires_at)`
|
|
- `idx_import_history_user(user_id)`, `idx_import_history_imported_at(imported_at)`
|
|
- `idx_oidc_states_expires(expires_at)`
|
|
- `idx_bill_history_ranges_bill(bill_id)`
|
|
- `idx_audit_log_user(user_id, created_at)`, `idx_audit_log_action(action, created_at)`
|
|
|
|
### Migration system
|
|
|
|
`db/database.js`:
|
|
|
|
- Reads `db/schema.sql` in `initSchema()`.
|
|
- Creates `schema_migrations`.
|
|
- Detects and reconciles legacy databases.
|
|
- Applies ordered migrations only when not already recorded.
|
|
- Validates dependency chains before applying dependent migrations.
|
|
- Uses a column whitelist for dynamic `ALTER TABLE` statements.
|
|
- Wraps versioned migrations in transactions, with special handling for `v0.40` because it uses PRAGMA-driven table rebuild work.
|
|
- Supports rollback SQL for selected migrations through `ROLLBACK_SQL_MAP` and `rollbackMigration(version)`.
|
|
|
|
Current migration set:
|
|
|
|
- `v0.2` payments soft delete.
|
|
- `v0.3` tracker payment compound index.
|
|
- `v0.4` monthly bill state.
|
|
- `v0.13` user profile columns.
|
|
- `v0.14` bill history visibility.
|
|
- `v0.14.4` bill interest rate.
|
|
- `v0.15` import sessions/history.
|
|
- `v0.17` OIDC/external identity columns and state table.
|
|
- `v0.18.1` monthly income.
|
|
- `v0.18.2` monthly starting amounts.
|
|
- `v0.18.3` other starting amount bucket.
|
|
- `v0.38` per-user import audit history.
|
|
- `v0.40` ownership for bills/categories.
|
|
- `v0.41` seeded demo-data flags.
|
|
- `v0.42` bill history ranges.
|
|
- `v0.43` session `created_at`.
|
|
- `v0.44` performance indexes.
|
|
- `v0.45` audit log.
|
|
- `v0.46` bill `cycle_type` and `cycle_day`.
|
|
- Unversioned user notification columns are also reconciled.
|
|
|
|
Migration logging is both console-based and audit-backed:
|
|
|
|
- `runMigrations()` logs start, dependency status, transaction begin/commit, per-migration elapsed time, skips for already-applied migrations, failures with rollback messages, and total elapsed time.
|
|
- Audit events use the lazy `getLogAudit()` helper to avoid the `auditService -> database.js -> auditService` circular dependency.
|
|
- Audit actions include `migration.start`, `migration.complete`, and `migration.failure`.
|
|
- Rollback paths audit `migration.rollback` and `migration.rollback.failure`.
|
|
|
|
Rollback support is defined by `ROLLBACK_SQL_MAP`:
|
|
|
|
- `v0.44` — drops selected performance indexes: `idx_bills_user_name`, `idx_payments_method`, `idx_monthly_starting_amounts_user`, and `idx_import_history_imported_at`.
|
|
- `v0.45` — drops `idx_audit_log_user`, `idx_audit_log_action`, and the `audit_log` table.
|
|
- `v0.46` — drops `bills.cycle_day` and `bills.cycle_type`.
|
|
|
|
`rollbackMigration(version)` requires an initialized database, verifies the version exists in `schema_migrations`, looks up rollback SQL in `ROLLBACK_SQL_MAP`, executes all rollback statements inside a transaction, deletes the migration record, logs elapsed time, audits success, and returns `{success:true, version, description, elapsed_ms}`. If the migration is not recorded, it throws `NOT_APPLIED`. If no rollback definition exists, it throws `ROLLBACK_NOT_SUPPORTED`. Execution failures roll back the transaction and are audited as `migration.rollback.failure`.
|
|
|
|
The admin API exposes rollback through `POST /api/admin/migrations/rollback`. The route requires admin auth through the `/api/admin` mount. It maps `NOT_APPLIED` to HTTP 404, `ROLLBACK_NOT_SUPPORTED` to HTTP 422, and unexpected rollback failures to HTTP 500.
|
|
|
|
The lazy audit helper in `db/database.js` is:
|
|
|
|
```javascript
|
|
let _logAudit = null;
|
|
function getLogAudit() {
|
|
if (!_logAudit) {
|
|
try { _logAudit = require('../services/auditService').logAudit; } catch { _logAudit = () => {}; }
|
|
}
|
|
return _logAudit;
|
|
}
|
|
```
|
|
|
|
Use this pattern for database-layer audit calls instead of a top-level `require('../services/auditService')`.
|
|
|
|
---
|
|
|
|
## 7. Frontend Reference
|
|
|
|
### Frontend stack
|
|
|
|
- React `^18.3.1`
|
|
- Vite `^5.4.10`
|
|
- React Router `^6.26.2`
|
|
- TanStack Query `^5.100.9`
|
|
- Tailwind CSS `^3.4.14`
|
|
- Radix/shadcn-style UI primitives
|
|
- `sonner` for toasts
|
|
- `react-markdown`, `remark-gfm`, `rehype-sanitize` for markdown rendering
|
|
|
|
### `client/main.jsx`
|
|
|
|
Creates the React root and wraps `App` with global providers including auth/theme where defined.
|
|
|
|
### `client/App.jsx`
|
|
|
|
- Creates a TanStack `QueryClient` with `staleTime` 2 minutes, retry 1, no refetch on focus.
|
|
- Uses lazy loading and `Suspense` with `PageLoader` for most pages.
|
|
- Wraps route elements in `ErrorBoundary`.
|
|
- Exposes `ReactQueryDevtools`.
|
|
- Provides skip link for keyboard users.
|
|
|
|
Routes:
|
|
|
|
- `/login` → `LoginPage` public.
|
|
- `/about` → `AboutPage` public.
|
|
- `/release-notes` → `ReleaseNotesPage` public.
|
|
- `/admin` → `AdminPage`, admin only.
|
|
- `/admin/about` → admin shell + `AboutPage admin`, admin only.
|
|
- `/admin/roadmap` → admin shell + `AboutPage admin`, admin only.
|
|
- `/admin/status` → admin shell + `StatusPage`, admin only.
|
|
- `/status` → redirects admin to `/admin/status`.
|
|
- `/` → authenticated user layout index, `TrackerPage`.
|
|
- `/calendar` → `CalendarPage`.
|
|
- `/summary` → `SummaryPage`.
|
|
- `/bills` → `BillsPage`.
|
|
- `/categories` → `CategoriesPage`.
|
|
- `/analytics` → `AnalyticsPage`.
|
|
- `/settings` → `SettingsPage`.
|
|
- `/data` → `DataPage`.
|
|
- `/profile` → `ProfilePage`.
|
|
- `*` under user layout → redirect `/`.
|
|
|
|
`RequireAuth` behavior:
|
|
|
|
- Shows loading while auth state is `undefined`.
|
|
- Redirects unauthenticated users to `/login`.
|
|
- Allows admins to access user routes except default admin is redirected to `/admin`.
|
|
- Allows single-user mode only for user routes.
|
|
- Redirects role mismatches to `/admin` or `/`.
|
|
|
|
### API client
|
|
|
|
`client/api.js`:
|
|
|
|
- `_fetch(method, path, body)` calls `/api${path}` with JSON headers and `credentials: include`.
|
|
- Mutating methods read `bt_csrf_token` cookie and send `x-csrf-token`.
|
|
- Non-OK responses throw `Error` with `status`, `data`, `details`, and `code`.
|
|
- Exposes grouped functions for auth, admin, notifications, profile, tracker, calendar, summary, bills, payments, categories, settings, analytics, status, version/about, import, and export.
|
|
- File download/upload endpoints use raw `fetch` because responses/bodies are blobs or octet streams.
|
|
|
|
### Auth state
|
|
|
|
`client/hooks/useAuth.jsx`:
|
|
|
|
- Maintains `user`, `singleUserMode`, and loading state.
|
|
- Calls `api.authMode()` and `api.me()` on startup.
|
|
- Exposes `logout()` and `refresh()`.
|
|
|
|
### Query hooks
|
|
|
|
`client/hooks/useQueries.js`:
|
|
|
|
- `useTracker(year, month)` → `api.tracker(year, month)`.
|
|
- `useBills()` → `api.allBills()`.
|
|
- `useCategories()` → `api.categories()`.
|
|
|
|
These use TanStack Query keys and cache server data for common pages.
|
|
|
|
### Pages
|
|
|
|
- `LoginPage.jsx` — local login plus OIDC login availability based on `/auth/mode`.
|
|
- `TrackerPage.jsx` — monthly tracker, payment interactions, upcoming bills, starting amount awareness, bulk/session logout action.
|
|
- `CalendarPage.jsx` — calendar grid backed by `/calendar`.
|
|
- `SummaryPage.jsx` — monthly plan, income, starting amounts, expenses, chart data.
|
|
- `BillsPage.jsx` — bill CRUD, categories, monthly state/history range controls.
|
|
- `CategoriesPage.jsx` — category list/create/update/delete and related bill info.
|
|
- `AnalyticsPage.jsx` — analytics summary filters and charts.
|
|
- `SettingsPage.jsx` — user/app settings and demo data seed.
|
|
- `DataPage.jsx` — export, spreadsheet import, user DB import, import history.
|
|
- `ProfilePage.jsx` — display name, notification preferences, password change, export/import-history links.
|
|
- `AdminPage.jsx` — users, auth mode/OIDC, backups, cleanup, notifications, migrations, admin settings.
|
|
- `StatusPage.jsx` — admin system status.
|
|
- `AboutPage.jsx` — public or admin markdown/about view; admin mode uses `/about-admin`; markdown is sanitized.
|
|
- `ReleaseNotesPage.jsx` — release history display.
|
|
|
|
### Components
|
|
|
|
- Layout: `Layout`, `Sidebar`, `BrandBlock`, `NavPill`.
|
|
- Domain UI: `AdminDashboard`, `BillModal`, `BillsTableInner`, `MobileBillRow`, `MobileTrackerRow`, `StatusBadge`, `SummaryCard`, `MarkdownText`, `ReleaseNotesDialog`.
|
|
- Reliability: `ErrorBoundary`, `PageLoader`.
|
|
- UI primitives: alert dialog, badge, button, card, checkbox, confirm dialog, dialog, dropdown menu, input dialog, input, label, select, separator, skeleton, switch, table, tabs, theme toggle, tooltip.
|
|
|
|
### Frontend security notes
|
|
|
|
- CSRF header is sent on POST/PUT/PATCH/DELETE.
|
|
- Auth is cookie/session based; no tokens are stored in localStorage.
|
|
- Admin routes are client-guarded and server-guarded.
|
|
- Markdown rendering uses `rehype-sanitize`.
|
|
- Error boundaries prevent route crashes from taking down the whole SPA.
|
|
|
|
---
|
|
|
|
## 8. Infrastructure and Deployment
|
|
|
|
### `package.json`
|
|
|
|
Version: `0.23.2`.
|
|
|
|
Scripts:
|
|
|
|
- `npm run dev:api` — `node --watch server.js`.
|
|
- `npm run dev:ui` — Vite dev server.
|
|
- `npm run dev` — concurrently runs API and UI.
|
|
- `npm run build` — Vite production build.
|
|
- `npm start` — `node server.js`.
|
|
|
|
Key runtime dependencies:
|
|
|
|
- Express, cookie-parser, cors, express-rate-limit.
|
|
- better-sqlite3.
|
|
- bcryptjs.
|
|
- openid-client.
|
|
- nodemailer.
|
|
- node-cron.
|
|
- React, React DOM, React Router, TanStack Query.
|
|
- Radix UI primitives, lucide-react, Tailwind utilities.
|
|
- xlsx for spreadsheet import/export.
|
|
|
|
### Dockerfile
|
|
|
|
Multi-stage build:
|
|
|
|
1. Builder: `node:18-alpine`, installs build deps `python3 make g++`, runs `npm install`, copies source, runs `npm run build`.
|
|
2. Runtime: `node:18-alpine`, installs `bash nano su-exec`, creates non-root `bill` user, copies built app, creates `/data/db`, `/data/backups`, `/app/backups`, sets ownership and restrictive permissions.
|
|
|
|
Runtime environment:
|
|
|
|
- `NODE_ENV=production`
|
|
- `PORT=3000`
|
|
- `DB_PATH=/data/db/bills.db`
|
|
- `BACKUP_PATH=/data/backups`
|
|
|
|
Exposes port 3000, declares volume `/data`, entrypoint `docker-entrypoint.sh`, command `node server.js`.
|
|
|
|
### `docker-compose.yml`
|
|
|
|
Service: `bill-tracker`.
|
|
|
|
- Image: `dream.scheller.ltd/null/billtracker:latest`.
|
|
- Container name: `bill-tracker`.
|
|
- Ports: host `3030` to container `3000`.
|
|
- Volume: `/portainer/hosting/bill-tracker/data:/data`.
|
|
- Restart: `unless-stopped`.
|
|
- Environment includes `INIT_ADMIN_USER`, `INIT_ADMIN_PASS`, and CSRF cookie settings.
|
|
|
|
Important deployment note: the compose file currently sets `CSRF_SECURE: "true"`; for plain HTTP development this prevents CSRF cookies from being sent by browsers. Use HTTPS or override to `false` only in local/dev.
|
|
|
|
---
|
|
|
|
## 9. Auth and Security Flows
|
|
|
|
### Local login
|
|
|
|
1. Client calls `GET /auth/mode` to determine local/OIDC visibility.
|
|
2. Client submits `POST /auth/login`.
|
|
3. Server checks `local_login_enabled`, validates credentials through bcrypt, rejects inactive users, cleans expired sessions, creates a 7-day session.
|
|
4. Server sets `bt_session` cookie using `cookieOpts(req)`.
|
|
5. Client calls `GET /auth/me` to populate auth state.
|
|
|
|
### OIDC login
|
|
|
|
1. Client navigates to `/api/auth/oidc/login`.
|
|
2. Server verifies active OIDC config, creates PKCE state, redirects to provider.
|
|
3. Provider returns to `/api/auth/oidc/callback`.
|
|
4. Server validates state/nonce, exchanges code, maps/provisions user, creates local session cookie, redirects back into SPA.
|
|
|
|
### Password change
|
|
|
|
1. Current password and matching new password are required.
|
|
2. New password must be at least 8 chars.
|
|
3. Server updates hash, clears `must_change_password`, sets `last_password_change_at`.
|
|
4. Other sessions are invalidated and current session is rotated when possible.
|
|
|
|
### Authorization
|
|
|
|
- All user data routes enforce owner scope by `req.user.id` in SQL.
|
|
- Admin-only routes require `requireAdmin` on server.
|
|
- Default admin cannot use tracker routes.
|
|
- Role changes invalidate target sessions.
|
|
- Deactivation invalidates target sessions.
|
|
|
|
### Backup safety
|
|
|
|
- Managed filename regex and path checks prevent traversal.
|
|
- Uploads are written to temp paths first, validated, then moved.
|
|
- Restore creates a pre-restore backup.
|
|
|
|
### Import safety
|
|
|
|
- Spreadsheet import accepts only XLSX and validates size, sheets, rows, cells, headers, and decisions.
|
|
- User DB import validates SQLite magic, size, required metadata/tables, and maps all data to current user ownership.
|
|
|
|
---
|
|
|
|
## 10. Operational Notes
|
|
|
|
### Startup behavior
|
|
|
|
- DB path is `DB_PATH` or `db/bills.db`.
|
|
- DB open logging prints only `path.basename(DB_PATH)` instead of the full database path, so startup logs identify the file without exposing the full filesystem location.
|
|
- SQLite WAL and foreign keys are enabled.
|
|
- Schema and migrations run automatically.
|
|
- Default categories/settings are seeded.
|
|
- Expired sessions are purged at startup.
|
|
- A periodic expired-session cleanup interval is scheduled.
|
|
- Backup scheduler and daily worker are started where server code imports/starts them.
|
|
|
|
Environment-seeded regular users use `INIT_REGULAR_USER` and `INIT_REGULAR_PASS`. New seeded users are inserted with `first_login = 0` and `must_change_password = 0`. Existing seeded regular users have their password hash updated and both flags reset to `0`, then the server audits `seed.flag_reset` with the username, reset flags, and `source: "server-seed"`. This lets ENV-managed users skip first-login/privacy/password-change gates after seed refreshes.
|
|
|
|
### Environment variables
|
|
|
|
Common variables used by current code:
|
|
|
|
- `PORT`
|
|
- `DB_PATH`
|
|
- `BACKUP_PATH`
|
|
- `INIT_ADMIN_USER`
|
|
- `INIT_ADMIN_PASS`
|
|
- `INIT_REGULAR_USER`
|
|
- `INIT_REGULAR_PASS`
|
|
- `SESSION_CLEANUP_INTERVAL_MS`
|
|
- `CORS_ORIGIN`
|
|
- `COOKIE_SECURE`
|
|
- `HTTPS`
|
|
- `CSRF_HTTP_ONLY`
|
|
- `CSRF_SAME_SITE`
|
|
- `CSRF_SECURE`
|
|
- `CSRF_COOKIE_NAME`
|
|
- `OIDC_ISSUER_URL`
|
|
- `OIDC_CLIENT_ID`
|
|
- `OIDC_CLIENT_SECRET`
|
|
- `OIDC_REDIRECT_URI`
|
|
- `OIDC_TOKEN_AUTH_METHOD`
|
|
|
|
Most notification, OIDC, backup, cleanup, and auth-mode settings are also stored in the `settings` table and managed from Admin UI.
|
|
|
|
### Known code characteristics to preserve
|
|
|
|
- Use transactions for multi-step destructive or bulk DB changes.
|
|
- Keep user-owned SQL scoped by `req.user.id`.
|
|
- Keep admin lockout protection before changing login methods.
|
|
- Do not expose `password_hash`, session IDs, OIDC client secret, or internal backup paths in API responses.
|
|
- Keep import preview/apply separated so users can resolve ambiguous spreadsheet data before DB writes.
|
|
- Prefer soft delete for payments; bill deletion is intentionally hard delete and returns an explicit warning.
|
|
- DB path support: db/database.js uses path.basename(DB_PATH) in logging to anonymize the DB path while still providing useful diagnostic information. Absolute and relative paths are both supported.
|
|
|
|
---
|
|
|
|
## 11. Verification Checklist Used for This Reference
|
|
|
|
Reviewed current code sources:
|
|
|
|
- `server.js`
|
|
- all route files under `routes/`
|
|
- service files under `services/`
|
|
- middleware files under `middleware/`
|
|
- `db/schema.sql` and `db/database.js`
|
|
- actual initialized SQLite schema via `better-sqlite3` introspection
|
|
- `client/App.jsx`, `client/api.js`, `client/hooks/useAuth.jsx`, `client/hooks/useQueries.js`
|
|
- page/component inventory under `client/pages/` and `client/components/`
|
|
- `package.json`
|
|
- `Dockerfile`
|
|
- `docker-compose.yml`
|
|
|
|
The previous manual contained large historical update sections and stale route/page descriptions. This version replaces those with a current-state engineering reference.
|