# Engineering Reference Manual — Bill Tracker **Status:** Complete **Last Updated:** 2026-05-09 **Owner:** Bishop **Version:** 0.19.2 --- ## Error Boundaries (2026-05-09) **Added:** React Error Boundary component wrapping all routes for graceful error handling. **Changes:** - `client/components/ErrorBoundary.jsx` — New component with fallback UI and recovery options - `client/App.jsx` — All routes wrapped with `` ### Component Location **File:** `client/components/ErrorBoundary.jsx` ### Features | Feature | Description | |---------|-------------| | **Error Capture** | Catches JavaScript errors in child components | | **Fallback UI** | Displays friendly error message with details | | **Try Again** | Resets component state without reloading | | **Reload Page** | Full page reload to recover | | **Error Details** | Shows error message and component stack for debugging | ### How It Works 1. **Error Detection:** When a child component throws an error, `componentDidCatch()` captures it 2. **State Update:** `getDerivedStateFromError()` sets `hasError=true` 3. **Fallback Render:** Component renders fallback UI instead of crashing 4. **Recovery:** User clicks "Try Again" (reset) or "Reload Page" (full refresh) ### Route Coverage All routes are wrapped with ErrorBoundary: ```jsx // Public routes } /> } /> // User routes } /> } /> } /> // ... all other user routes // Admin routes } /> ``` ### Developer Guidance **Do not suppress errors in production.** ErrorBoundary provides recovery without exposing internal details. If you need to debug: ```javascript // Check browser console for error details // The component stack is logged and displayed in the fallback UI console.error('ErrorBoundary caught:', error, componentStack); ``` --- ## Version 0.19.2 Update ### Security Fixes (2026-05-09) **Added:** 6 security hardening improvements addressing path traversal, admin route bypass, content redaction, error message leaks, race conditions, and password validation. **Changes:** - `routes/aboutAdmin.js` — allowlist approach, enhanced redaction patterns, generic error handling - `client/App.jsx` — admin prop passed to AboutPage on `/admin/about` route - `client/pages/AboutPage.jsx` — `admin` prop, dual API call logic (`api.about()` vs `api.aboutAdmin()`) - `server.js` — transaction wrapping for regular user creation, 8-char password validation ### 🔴 #1: Path Traversal Fix **File:** `routes/aboutAdmin.js` **Fix:** Replaced `sanitizePath()` function with hardcoded `ALLOWED_FILES` map. Only `FUTURE.md` and `DEVELOPMENT_LOG.md` are servable. No path resolution from user input. **Before:** ```javascript const sanitizePath = (fileName) => { const base = path.resolve(__dirname, '..'); const filePath = path.resolve(base, fileName); if (!filePath.startsWith(base)) { throw new Error('Invalid path'); } return filePath; }; ``` **After:** ```javascript const ALLOWED_FILES = { 'FUTURE.md': path.resolve(__dirname, '..', 'FUTURE.md'), 'DEVELOPMENT_LOG.md': path.resolve(__dirname, '..', 'DEVELOPMENT_LOG.md'), }; ``` **Impact:** Path traversal attacks prevented entirely via allowlist approach. ### 🟠 #2: Admin Route Bypass Fix **Files:** `client/App.jsx`, `client/pages/AboutPage.jsx` **Fix:** `/admin/about` route passes `` prop. AboutPage conditionally calls `api.aboutAdmin()` when `admin=true`. Public `/about` continues to call `api.about()`. **Before:** Single AboutPage component served both public and admin content via same endpoint. **After:** ```javascript // App.jsx } /> // AboutPage.jsx const load = useCallback(async () => { setLoading(true); try { setAbout(admin ? await api.aboutAdmin() : await api.about()); } finally { setLoading(false); } }, [admin]); ``` **Impact:** Admin-only content isolated from public endpoint. No information leakage. ### 🟠 #3: Sensitive Info Redaction **File:** `routes/aboutAdmin.js` **Fix:** Expanded `redactSensitiveContent()` with additional patterns: - File paths (`.home`, `.etc`, `.var`, etc.) - Connection strings (mongodb://, postgres://, mysql://, redis://, amqp://) - Environment variable secrets (SECRET_, KEY_, TOKEN_, PASS_, etc.) - Internal URLs (localhost, 127.0.0.1, 0.0.0.0) - Security-related keywords (CRITICAL, vulnerability, exploit) **Before:** Only internal IPs and basic password/api_key patterns. **After:** Comprehensive redaction covering: - Internal IPs (10.x, 172.16-31.x, 192.168.x) - Connection strings (all major protocols) - File paths (common system directories) - Environment variable secrets (various naming patterns) - Internal URLs - Security-sensitive content lines **Impact:** Sensitive data in documentation files completely redacted before serving. ### 🟠 #4: Error Message Leaks **File:** `routes/aboutAdmin.js` **Fix:** Removed `err.message` from `console.error`. HTTP 500 response returns generic message only. **Before:** ```javascript } catch (err) { console.error('[aboutAdmin] Error reading files:', err.message); res.status(500).json({ error: 'Failed to read files', details: err.message }); } ``` **After:** ```javascript } catch (err) { console.error('[aboutAdmin] Error reading files'); res.status(500).json({ error: 'Failed to read project documentation files', code: 'FILE_READ_ERROR' }); } ``` **Impact:** No path disclosure or internal error details exposed to clients. ### 🟡 #5: Race Condition Fix **File:** `server.js` **Fix:** Regular user creation wrapped in `db.transaction()` to ensure atomic check-and-insert. **Before:** ```javascript const existingRegular = db.prepare('SELECT id FROM users WHERE username = ?').get(regularUser); if (!existingRegular) { db.prepare('INSERT INTO users ...').run(...); } ``` **After:** ```javascript const createRegularUser = db.transaction(() => { const existingRegular = db.prepare('SELECT id FROM users WHERE username = ?').get(regularUser); if (!existingRegular) { db.prepare('INSERT INTO users ...').run(...); return true; } return false; }); createRegularUser(); ``` **Impact:** Prevents duplicate user creation under concurrent requests. ### 🟡 #6: Password Validation **File:** `server.js` **Fix:** `INIT_REGULAR_PASS` validated for minimum 8 characters. `process.exit(1)` on validation failure. **Before:** No validation; any password length accepted. **After:** ```javascript if (regularPass && regularPass.length < 8) { console.error('[seed] INIT_REGULAR_PASS must be at least 8 characters'); process.exit(1); } ``` **Impact:** Enforces minimum password strength for seeded regular users. --- ### Added (2026-05-09) - **Regular User Seed Environment Variables** — `INIT_REGULAR_USER` and `INIT_REGULAR_PASS` create a non-admin user on first run for role-based testing - **Database Migration v0.42** — `bill_history_ranges` table creation moved into versioned migration system - **Admin-only `/about` endpoint** — `/api/about-admin` serves FUTURE.md and DEVELOPMENT_LOG.md to admins only ### Security (2026-05-09) - **Admin-only `/admin/about` route guard** — React `RequireAuth` middleware protects `/admin/about` route - **Rate limiting on `/api/about-admin`** — `adminActionLimiter` (30 req/15min per IP) applied to prevent brute-force attempts - **XSS prevention** — `rehype-sanitize` added to ReactMarkdown component in AboutPage.jsx - **Content redaction** — `routes/aboutAdmin.js` sanitizes paths, redacts internal IPs, passwords, API keys - **Error sanitization** — Error messages exclude paths to prevent path disclosure - **Non-admin test user** — Added `INIT_REGULAR_USER` and `INIT_REGULAR_PASS` env vars for role-based testing **Changes:** - `client/App.jsx` — `/admin/about` route protected with `RequireAuth role="admin"` - `server.js` — `adminActionLimiter` applied to `/api/about-admin` (30 req/15min IP) - `client/pages/AboutPage.jsx` — `rehypeSanitize` added to `ReactMarkdown` component - `client/api.js` — `aboutAdmin: () => get('/about-admin')` endpoint function added --- ## Version 0.19.2 Update ### 🔴 Migration System Fix (2026-05-09) **Added:** Legacy database detection and reconciliation for existing deployments that preddate the migration tracking system. **Problem:** Existing deployments created before v0.19.0 had a populated database without the `schema_migrations` table tracking applied migrations. On upgrade, the app would start but fail to reconcile existing migrations, potentially causing: - Duplicate column errors on migrations that already ran - Missing index errors on indexes that already existed - Inconsistent migration state **Solution:** Added `handleLegacyDatabase()` and `reconcileLegacyMigrations()` functions to detect legacy databases and safely reconcile migration state. **Changes:** - `db/database.js` — Added `handleLegacyDatabase()` function - `db/database.js` — Added `reconcileLegacyMigrations()` function - `db/database.js` — Modified `initSchema()` to call `handleLegacyDatabase()` before `runMigrations()` **Detection Logic:** Legacy database is detected when: 1. `schema_migrations` table exists and has zero rows, OR table doesn't exist (but tables do) 2. Core tables exist (`users`, `bills`, `payments`, `categories`, `settings`) 3. The database has data but no migration tracking **Reconciliation Process:** 1. **Check for notification columns migration:** If old notification columns exist, mark as applied 2. **Iterate through all migrations:** For each migration definition: - Run the migration's `check()` function to see if its changes are already present - If already present, call `recordMigration()` to mark it as applied in `schema_migrations` - Log: `[migration] Recorded legacy migration {version}: {description}` 3. **Complete reconciliation:** Log `[migration] Legacy database reconciliation complete` 4. **Apply remaining migrations:** `runMigrations()` proceeds as normal, applying only truly pending migrations **Example Log Output:** ``` [migration] Detected legacy database, reconciling schema migrations... [migration] Applied v0.4: monthly_bill_state: per-bill per-month overrides [migration] Recorded legacy migration v0.4: monthly_bill_state: per-bill per-month overrides [migration] Applied v0.14.4: bills: optional credit-card APR / interest rate [migration] Recorded legacy migration v0.14.4: bills: optional credit-card APR / interest rate [migration] Applied v0.38: import_history: per-user audit log [migration] Recorded legacy migration v0.38: import_history: per-user audit log [migration] Applied v0.40: ownership: user-scoped bills/categories [migration] Recorded legacy migration v0.40: ownership: user-scoped bills/categories [migration] Legacy database reconciliation complete [migration] Applying v0.2: payments: soft-delete column [migration] payments.deleted_at column added [migration] Applied v0.2: payments: soft-delete column [migration] Applying v0.3: payments: compound index for tracker query [migration] Applied v0.3: payments: compound index for tracker query ``` **Benefits:** - Existing deployments can upgrade to v0.19.2 without manual intervention - No data loss during migration reconciliation - No duplicate column/index errors - Seamless upgrade path from any pre-v0.19.0 version **Testing:** **Test 1: Fresh Database (v0.19.2)** ✅ - Container starts with empty data volume - Migrations applied in order (v0.2 through v0.42) - Admin and regular users created successfully - Login functional **Test 2: Simulated Legacy Database (pre-v0.19.0)** ✅ - Created database with tables but NO `schema_migrations` table - Container detected legacy database and logged detection - All existing migrations recorded in `schema_migrations` - Remaining migrations applied correctly - Login functional **Files Modified:** - `db/database.js` — Legacy database detection and reconciliation added **Impact:** - Existing users can safely upgrade from any version to v0.19.2 - No manual database intervention required - Migration state remains consistent and auditable --- ## Version 0.19.0 Update ### Security Fixes (2026-05-09) **Added:** Admin-only `/admin/about` route guard, rate limiting on `/api/about-admin`, content sanitization with `rehype-sanitize`, and new environment variables for non-admin user creation. **Changes:** - `client/App.jsx` — `/admin/about` route protected with `RequireAuth role="admin"` - `server.js` — `adminActionLimiter` applied to `/api/about-admin` (30 req/15min IP) - `client/pages/AboutPage.jsx` — `rehypeSanitize` added to `ReactMarkdown` component - `client/api.js` — `aboutAdmin: () => get('/about-admin')` endpoint function added --- ### Migration System Note (v0.19.1) Database migrations now use explicit version tracking via the `schema_migrations` table. All migrations are idempotent and safe to re-run after `git pull`. The `runMigrations()` function queries `schema_migrations` before applying each migration, skipping already-applied versions. **Key Changes**: - New `schema_migrations` table with `version`, `description`, `applied_at` columns - Helper functions: `hasMigrationBeenApplied()`, `recordMigration()` - Migrations are defined as versioned objects with explicit `version`/`description`/`run()` - Safe `git pull && npm start` upgrades without migration state issues --- ## Table of Contents 1. [High Level Overview](#1-high-level-overview) 2. [Frontend Documentation](#2-frontend-documentation) 3. [Backend Documentation](#3-backend-documentation) 4. [Authentication & Authorization](#4-authentication--authorization) 5. [API Documentation](#5-api-documentation) 6. [Database Documentation](#6-database-documentation) 7. [Error Handling & Troubleshooting](#7-error-handling--troubleshooting) 8. [Code Navigation Index](#8-code-navigation-index) 9. [Infrastructure & Deployment](#9-infrastructure--deployment) 10. [Sequence Flows](#10-sequence-flows) --- ## 1. High Level Overview ### App Purpose BillTracker is a self-hosted monthly bill tracking system for households and small setups. It manages: - **Recurring bills**: Track due dates, expected amounts, categories, autopay, interest rates, website login info - **Monthly tracker**: Record actual payments, skip bills, view spending vs expectations - **Calendar view**: Visual grid showing due dates and payments - **Analytics**: Charts, category spend, payment history - **User management**: Admin creates users, sets roles, manages authentication - **Notifications**: Email alerts for due bills (3d, 1d, today, overdue) - **Data management**: Import/Export bills, full database backup/restore ### Architecture Summary **Stack**: Node.js + Express (backend) + React + Vite (frontend) + SQLite (database) **Layered Architecture**: ``` ┌─────────────────────────────────────────────────────────────┐ │ Frontend (React) │ │ Pages (client/pages/) • Components (client/components/) │ │ Router (client/App.jsx) • API Client (client/api.js) │ └─────────────────────────────────────────────────────────────┘ HTTP/JSON ┌─────────────────────────────────────────────────────────────┐ │ Backend (Express) │ │ Routes (routes/) • Services (services/) • Middleware │ │ Auth (authService.js) • OIDC (oidcService.js) │ └─────────────────────────────────────────────────────────────┘ SQL ┌─────────────────────────────────────────────────────────────┐ │ Database (SQLite) │ │ Schema (db/schema.sql) • Migrations (db/database.js) │ │ Users • Sessions • Bills • Payments • Categories • etc. │ └─────────────────────────────────────────────────────────────┘ ``` ### Tech Stack | Layer | Component | Version | Purpose | |-------|-----------|---------|---------| | **Runtime** | Node.js | v20+ | Backend server | | **Framework** | Express | ^4.18.2 | HTTP server, routing, middleware | | **Frontend** | React | ^18.3.1 | UI components | | **Build** | Vite | ^5.4.10 | Bundler, dev server | | **Router** | react-router-dom | ^6.26.2 | Client-side routing | | **Database** | better-sqlite3 | ^12.9.0 | SQLite wrapper | Migration tracking | | **Auth** | bcryptjs | ^2.4.3 | Password hashing | | **OIDC** | openid-client | ^5.7.1 | Authentik integration | | **Email** | nodemailer | ^6.9.14 | SMTP email sending | | **Scheduler** | node-cron | ^3.0.3 | Background jobs | | **UI Libs** | shadcn/ui | - | Component primitives | | **Styling** | TailwindCSS | ^3.4.14 | Utility-first CSS | ### Major Components | Component | Location | Purpose | |-----------|----------|---------| | **server.js** | Root | Express entry, middleware setup, route mounting | | **db/database.js** | `db/` | SQLite connection, migrations, settings | | **services/authService.js** | `services/` | Session management, login/logout | | **services/oidcService.js** | `services/` | Authentik OIDC integration | | **services/backupService.js** | `services/` | Database backup/restore | | **middleware/requireAuth.js** | `middleware/` | Auth guard middleware | | **middleware/csrf.js** | `middleware/` | CSRF token generation/validation | | **workers/dailyWorker.js** | `workers/` | Daily background tasks | ### Database Migration System The database migration system provides explicit version tracking to ensure safe upgrades via `git pull && npm start`. **Migration Tracking Table** (`schema_migrations`): | Column | Type | Purpose | |--------|------|---------| | `id` | INTEGER PRIMARY KEY AUTOINCREMENT | Internal tracking ID | | `version` | TEXT NOT NULL UNIQUE | Migration version identifier (e.g., `v0.2`, `v0.3`) | | `description` | TEXT NOT NULL | Human-readable migration description | | `applied_at` | TEXT NOT NULL DEFAULT (datetime('now')) | Timestamp when migration was applied | **Migration Functions**: - `hasMigrationBeenApplied(version)`: Check if a migration version has been applied - `recordMigration(version, description)`: Record a migration as applied - `runMigrations()`: Execute pending migrations with version tracking **Migration Format**: Migrations are defined as versioned objects with explicit `version`, `description`, and `run()` function: ```javascript { version: 'v0.2', description: 'payments: soft-delete column', run: function() { /* migration logic */ } } ``` **Idempotent Migration Execution**: 1. Query `schema_migrations` table for applied versions 2. Skip migrations that have already been applied 3. Apply pending migrations in order 4. Log each migration status (applied vs skipped) **Benefits**: - Users can safely `git pull && npm start` without migration state issues - Migrations are repeatable and trackable - Clear audit trail of which migrations have been applied - No risk of re-applying migrations that modify data ### Request Lifecycle ``` ┌─────────────────────────────────────────────────────────────┐ │ 1. Request arrives at Express (server.js) │ │ • Security headers applied (securityHeaders) │ │ • CSRF token set on response (csrfTokenProvider) │ └─────────────────────────────────────────────────────────────┘ │ ┌─────────────────────────────────────────────────────────────┐ │ 2. Route-level middleware chain │ │ • CSRF validation (csrfMiddleware) │ │ • Auth check (requireAuth) │ │ • User role check (requireUser/requireAdmin) │ │ • Rate limiting (loginLimiter, exportLimiter, etc.) │ └─────────────────────────────────────────────────────────────┘ │ ┌─────────────────────────────────────────────────────────────┐ │ 3. Route handler (routes/*.js) │ │ • Input validation │ │ • Service layer calls │ │ • Database queries │ └─────────────────────────────────────────────────────────────┘ │ ┌─────────────────────────────────────────────────────────────┐ │ 4. Service layer (services/*.js) │ │ • Business logic │ │ • External service calls (SMTP, OIDC) │ │ • Data transformation │ └─────────────────────────────────────────────────────────────┘ │ ┌─────────────────────────────────────────────────────────────┐ │ 5. Database layer (db/database.js) │ │ • SQL query execution │ │ • Schema validation (PRAGMA foreign_keys=ON) │ │ • Migration runs (if schema changes detected) │ └─────────────────────────────────────────────────────────────┘ │ ┌─────────────────────────────────────────────────────────────┐ │ 6. Response sent to frontend │ │ • JSON format │ │ • CSRF token in cookie │ │ • Error formatted via errorFormatter │ └─────────────────────────────────────────────────────────────┘ ``` --- ## 2. Frontend Documentation ### Route Mapping | Route | Page Component | Auth Required | Purpose | |-------|----------------|---------------|---------| | `/` | Redirect | No | Redirects to login or app based on auth | | `/login` | LoginPage.jsx | No | Username/password login form | | `/tracker` | TrackerPage.jsx | Yes | Monthly bill tracking view | | `/bills` | BillsPage.jsx | Yes | Bill CRUD interface | | `/categories` | CategoriesPage.jsx | Yes | Category management | | `/calendar` | CalendarPage.jsx | Yes | Monthly calendar view | | `/summary` | SummaryPage.jsx | Yes | Monthly summary/spending view | | `/analytics` | AnalyticsPage.jsx | Yes | Charts and analytics | | `/profile` | ProfilePage.jsx | Yes | User profile, password change | | `/settings` | SettingsPage.jsx | Yes | App settings (theme, format) | | `/data` | DataPage.jsx | Yes | Import/export data tools | | `/admin` | AdminPage.jsx | Yes (admin) | User management, backups, OIDC config | | `/about` | AboutPage.jsx | No | Version info, changelog | | `/status` | StatusPage.jsx | Yes (admin) | System status, worker health | ### Key Frontend Files #### Core Files | File | Purpose | Key Functions | |------|---------|---------------| | `client/main.jsx` | React entry point | Creates root, renders App | | `client/App.jsx` | Router config | Defines all routes, layout wrapper | | `client/api.js` | API client | `get`, `post`, `put`, `delete`, auth, CSRF | | `client/hooks/useAuth.jsx` | Auth state | `login`, `logout`, `user`, `loading` | #### Layout Components | File | Purpose | Key Features | |------|---------|--------------| | `client/components/layout/Layout.jsx` | Main layout wrapper | Sidebar, top bar, content area | | `client/components/layout/Sidebar.jsx` | Navigation sidebar | Links to all pages, collapse | | `client/components/layout/BrandBlock.jsx` | App branding | Logo, title, version | | `client/components/layout/NavPill.jsx` | Nav item | Active state, icon | #### UI Components (shadcn/ui) | Component | File | Purpose | |-----------|------|---------| | Button | `client/components/ui/button.jsx` | Primary, secondary, ghost | | Input | `client/components/ui/input.jsx` | Form input | | Card | `client/components/ui/card.jsx` | Content container | | Table | `client/components/ui/table.jsx` | Data table | | Tabs | `client/components/ui/tabs.jsx` | Tab navigation | | Dialog | `client/components/ui/dialog.jsx` | Modal dialogs | | Badge | `client/components/ui/badge.jsx` | Status badges | | Switch | `client/components/ui/switch.jsx` | Toggle switch | | Alert Dialog | `client/components/ui/alert-dialog.jsx` | Confirm action dialog | #### Page Components | Page | Route | API Calls | State | |------|-------|-----------|-------| | LoginPage | `/login` | `POST /api/auth/login` | `user`, `error` | | TrackerPage | `/tracker` | `GET /api/tracker`, `GET /api/tracker/upcoming` | `data`, `year`, `month`, `activeBillId` | | BillsPage | `/bills` | `GET /api/bills`, `POST /api/bills`, `PUT /api/bills/:id`, `DELETE /api/bills/:id` | `bills`, `categories`, `modalState` | | CategoriesPage | `/categories` | `GET /api/categories`, `POST /api/categories` | `categories` | | CalendarPage | `/calendar` | `GET /api/bills`, `GET /api/tracker` | `year`, `month`, `dates` | | SummaryPage | `/summary` | `GET /api/summary` | `data`, `year`, `month` | | AnalyticsPage | `/analytics` | `GET /api/analytics` | `filters`, `data` | | ProfilePage | `/profile` | `GET /api/user`, `POST /api/profile` | `user`, `notifications` | | SettingsPage | `/settings` | `GET /api/settings`, `PUT /api/settings` | `settings` | | DataPage | `/data` | `GET /api/import`, `POST /api/import`, `POST /api/export` | `importState`, `exportState` | | AdminPage | `/admin` | Multiple admin endpoints | `users`, `backups`, `oidc` | | AboutPage | `/about`, `/admin/about` | `GET /api/about`, `GET /api/about-admin` | `about` | ### State Management **Approach**: Component-local state + React Context (ThemeContext) **Key State**: | State | Location | Purpose | |-------|----------|---------| | `theme` | ThemeContext | Light/dark mode | | `user` | useAuth hook | Current user object | | `loading` | useAuth hook | Auth state | | `error` | useAuth hook | Auth errors | | `bills`, `categories`, etc. | Page components | API response data | ### Validation **Frontend validation occurs before API calls**: ```javascript // client/api.js - validation wrapper function validateInput(schema, data) { // Check required fields // Type validation // Range validation (numbers, dates) } ``` **Common validations**: | Field | Validation | |-------|-----------| | `due_day` | Integer 1-31 | | `expected_amount` | Number ≥ 0 | | `interest_rate` | Number 0-100 or null | | `password` | Min 8 characters (admin) | | `username` | Min 3 characters (admin) | | `year/month` | Valid date range | ### API Client (`client/api.js`) **Key Functions**: ```javascript // GET request with CSRF token await apiGet('/api/bills', { year, month }) // POST with CSRF await apiPost('/api/bills', { name, due_day, ... }) // PUT for updates await apiPut('/api/bills/:id', { name, ... }) // DELETE await apiDelete('/api/bills/:id') ``` **CSRF Handling**: - Token stored in `bt_csrf_token` cookie (httpOnly) - Sent in `x-csrf-token` header for POST/PUT/DELETE - Auto-retrieved from cookie on each request --- ## 3. Backend Documentation ### Core Backend Files | File | Purpose | Key Functions | |------|---------|---------------| | `server.js` | Express entry | Middleware setup, route mounting, error handling | | `db/database.js` | DB connection | SQLite init, migrations, settings | | `db/schema.sql` | Schema definition | All table definitions | | `services/authService.js` | Auth service | Login, logout, session management | | `services/oidcService.js` | OIDC service | Authentik integration | | `services/backupService.js` | Backup service | SQLite backup/restore | | `middleware/requireAuth.js` | Auth guards | `requireAuth`, `requireUser`, `requireAdmin` | | `middleware/csrf.js` | CSRF protection | Token generation/validation | | `middleware/rateLimiter.js` | Rate limiting | Per-endpoint limits | | `middleware/securityHeaders.js` | Security headers | CSP, HSTS, XSS protection | | `middleware/errorFormatter.js` | Error formatting | JSON error responses | ### Route Handlers (routes/*.js) | Route File | API Prefix | Auth | Purpose | |------------|------------|------|---------| | `authLogin.js` | `/api/auth/login` | None | Local login | | `auth.js` | `/api/auth` | CSRF | Logout, password change | | `authOidc.js` | `/api/auth/oidc` | CSRF | OIDC login/callback | | `tracker.js` | `/api/tracker` | Auth+User | Monthly tracking data | | `bills.js` | `/api/bills` | Auth+User | Bill CRUD | | `payments.js` | `/api/payments` | Auth+User | Payment CRUD | | `categories.js` | `/api/categories` | Auth+User | Category CRUD | | `settings.js` | `/api/settings` | Auth+User | Settings CRUD | | `user.js` | `/api/user` | Auth+User | User profile | | `calendar.js` | `/api/calendar` | Auth+User | Calendar data | | `summary.js` | `/api/summary` | Auth+User | Monthly summary | | `monthly-starting-amounts.js` | `/api/monthly-starting-amounts` | Auth+User | Starting balance | | `analytics.js` | `/api/analytics` | Auth+User | Analytics data | | `notifications.js` | `/api/notifications` | Auth+User | Notification settings | | `admin.js` | `/api/admin` | Auth+Admin | Admin functions | | `export.js` | `/api/export` | Auth+User | Data export | | `import.js` | `/api/import` | Auth+User | Data import | | `status.js` | `/api/status` | Auth+Admin | System status | | `about.js` | `/api/about` | None | Version info | | `version.js` | `/api/version` | None | Version string | ### Service Layer Functions #### db/database.js | Function | Purpose | Parameters | Returns | |----------|---------|------------|---------| | `initSchema()` | Initialize database schema | None | void (calls `runMigrations()`) | | `runMigrations()` | Execute pending migrations | None | void (skips already-applied) | | `hasMigrationBeenApplied(version)` | Check migration status | `version` (string) | `true` or `false` | | `recordMigration(version, description)` | Record applied migration | `version`, `description` | void | | `seedDefaults()` | Seed settings and categories | None | void | | `ensureUserDefaultCategories(userId)` | Seed default categories per user | `userId` | void | | `getSetting(key)` | Read single setting | `key` (string) | value or `null` | | `setSetting(key, value)` | Write single setting | `key`, `value` | void | | `getDbPath()` | Get database file path | None | string (absolute path) | | `closeDb()` | Close database connection | None | void | **Migration System Flow**: 1. `initSchema()` reads `db/schema.sql` and executes all table definitions 2. Creates `schema_migrations` table if not exists 3. `runMigrations()` iterates through versioned migrations 4. For each migration, `hasMigrationBeenApplied()` checks if already done 5. If not applied: executes `run()` then `recordMigration()` 6. If already applied: logs skip message, moves to next migration **Migration Format**: ```javascript { version: 'v0.2', description: 'payments: soft-delete column', run: function() { /* migration logic */ } } ``` #### authService.js | Function | Purpose | Parameters | Returns | |----------|---------|------------|---------| | `login(username, password)` | Authenticate user | `username`, `password` | `{sessionId, user}` or null | | `logout(sessionId)` | Destroy session | `sessionId` | void | | `createSession(userId)` | Create session for OIDC | `userId` | `{sessionId, user}` | | `getSessionUser(sessionId)` | Validate session | `sessionId` | `user` or null | | `hashPassword(password)` | Hash password | `password` | `Promise` | | `publicUser(user)` | Strip sensitive data | `user` object | Public user object | | `cookieOpts(req)` | Get cookie options | `req` | `{httpOnly, sameSite, secure, maxAge}` | | `pruneExpiredSessions()` | Clean expired sessions | None | void | | `rotateSessionId(oldId, userId)` | Security rotation | `oldId`, `userId` | `newId` or null | #### oidcService.js | Function | Purpose | Parameters | Returns | |----------|---------|------------|---------| | `getOidcConfig()` | Get effective config | None | Config object or null | | `isOidcLoginActive()` | Check if enabled | None | boolean | | `createLoginState(redirectTo)` | Create PKCE state | `redirectTo` | `{id, nonce, codeVerifier}` | | `consumeLoginState(stateId)` | Validate state | `stateId` | State or null | | `buildAuthorizationUrl(config, state)` | Build redirect URL | `config`, `state` | `Promise` | | `exchangeAndVerifyTokens(config, code, stateId, savedState)` | Exchange code for tokens | `config`, `code`, `stateId`, `savedState` | Verified claims | | `findOrProvisionUser(claims, config)` | Find or create user | `claims`, `config` | User object | | `mapRoleFromClaims(claims, config)` | Map groups to role | `claims`, `config` | `'admin'` or `'user'` | | `testOidcConfiguration(config)` | Test OIDC setup | `config` | `{ok, error, ...}` | | `getAdminOidcSettings()` | Admin settings | None | Settings object | | `getPublicOidcInfo()` | Public info | None | `{oidc_enabled, oidc_provider_name}` | #### backupService.js | Function | Purpose | Parameters | Returns | |----------|---------|------------|---------| | `createBackup(prefix)` | Create SQLite backup | `prefix` | `{id, filename, size_bytes, checksum}` | | `restoreBackup(id)` | Restore from backup | `backupId` | `{restored_from, pre_restore_backup}` | | `deleteBackup(id)` | Delete backup | `backupId` | `{deleted: true, id, deleted_at}` | | `listBackups()` | List backups | None | Array of backup metadata | | `getBackupFile(id)` | Get backup path | `backupId` | `{path, metadata}` | | `importBackupBuffer(buffer, options)` | Import backup | `buffer`, `{expectedChecksum}` | Backup metadata | | `validateSqliteDatabase(filePath)` | Validate DB file | `filePath` | void or throws | | `checksumFile(filePath)` | SHA-256 checksum | `filePath` | `hex string` | #### notificationService.js | Function | Purpose | Parameters | Returns | |----------|---------|------------|---------| | `runNotifications()` | Send due bill emails | None | void | | `sendTestEmail(to)` | Test SMTP config | `email` | void | | `createTransport()` | Create SMTP transport | None | Nodemailer transport | #### cleanupService.js | Function | Purpose | Parameters | Returns | |----------|---------|------------|---------| | `runAllCleanup()` | Run all cleanup tasks | None | `{import_sessions, temp_exports, ...}` | | `validateAndApplySettings(settings)` | Update cleanup config | `settings` | Updated config | | `getCleanupStatus()` | Get cleanup status | None | `{settings, last_run, last_result}` | #### statusService.js | Function | Purpose | Parameters | Returns | |----------|---------|------------|---------| | `buildTrackerRow(bill, payments, year, month, today)` | Build tracker row | `bill`, `payments`, `year`, `month`, `today` | Row object | | `resolveDueDate(bill, year, month)` | Calculate due date | `bill`, `year`, `month` | `YYYY-MM-DD` | | `getCycleRange(year, month)` | Get date range | `year`, `month` | `{start, end}` | ### Middleware Chain #### requireAuth.js | Middleware | Purpose | Check | |------------|---------|-------| | `requireAuth` | General auth | Session valid, user active | | `requireUser` | User role | Role is 'user' or 'admin', not default admin | | `requireAdmin` | Admin role | Role is 'admin' | #### CSRF Protection | Setting | Default | Purpose | |---------|---------|---------| | `CSRF_HTTP_ONLY` | `true` | Cookie not accessible via JS | | `CSRF_SAME_SITE` | `'strict'` | Same-site cookie | | `CSRF_SECURE` | `true` | HTTPS only | | `CSRF_COOKIE_NAME` | `'bt_csrf_token'` | Cookie name | **Flow**: 1. `csrfTokenProvider` sets cookie on every response 2. `csrfMiddleware` validates token on POST/PUT/DELETE 3. Token can be in header, query, or body #### Rate Limiters | Limiter | Max | Window | Endpoints | |---------|-----|--------|-----------| | `loginLimiter` | 10 | 15 min | `/api/auth/login` | | `passwordLimiter` | 5 | 15 min | `/api/profile`, `/api/admin/users/:id/password` | | `importLimiter` | 20 | 15 min | `/api/import/*` | | `exportLimiter` | 30 | 15 min | `/api/export/*` | | `adminActionLimiter` | 30 | 15 min | `/api/admin/*` | | `oidcLimiter` | 20 | 15 min | `/api/auth/oidc/*` | | `backupOperationLimiter` | 5 | 60 min | `/api/admin/backups/*` | ### Error Handling #### errorFormatter.js | Error Type | Status Code | Response Format | |------------|-------------|-----------------| | Validation | 400 | `{error: 'Validation failed', field: 'field_name'}` | | Auth | 401 | `{error: 'Not authenticated', code: 'AUTH_ERROR'}` | | Forbidden | 403 | `{error: 'Access denied', code: 'FORBIDDEN'}` | | Not Found | 404 | `{error: 'Not found', code: 'NOT_FOUND'}` | | Conflict | 409 | `{error: 'Already exists', code: 'CONFLICT'}` | | Rate Limit | 429 | `{error: 'Too many requests'}` | | Server | 500 | `{error: 'Internal server error'}` | **Standard Error Format**: ```javascript { error: 'Error message', code: 'ERROR_CODE', field: 'optional_field_name' } ``` --- ## 4. Authentication & Authorization ### Login Flow ``` ┌─────────────────────────────────────────────────────────────┐ │ 1. User submits login form │ │ • Username, password │ └─────────────────────────────────────────────────────────────┘ │ ┌─────────────────────────────────────────────────────────────┐ │ 2. POST /api/auth/login │ │ • rateLimiter (loginLimiter) │ │ • Body: {username, password} │ └─────────────────────────────────────────────────────────────┘ │ ┌─────────────────────────────────────────────────────────────┐ │ 3. authService.login() │ │ • Query user by username │ │ • Check active flag │ │ • Check auth_provider === 'local' │ │ • bcrypt.compare(password, password_hash) │ └─────────────────────────────────────────────────────────────┘ │ ┌─────────────────────────────────────────────────────────────┐ │ 4. Create session │ │ • Generate UUID for sessionId │ │ • Insert into sessions table (expires in 7 days) │ │ • Update last_login_at │ │ • Return {sessionId, user} │ └─────────────────────────────────────────────────────────────┘ │ ┌─────────────────────────────────────────────────────────────┐ │ 5. Set cookie │ │ • Cookie: bt_session= │ │ • httpOnly: true, sameSite: strict, secure: depends │ │ • Max-Age: 7 days │ └─────────────────────────────────────────────────────────────┘ │ ┌─────────────────────────────────────────────────────────────┐ │ 6. Return response │ │ • JSON: {user: {id, username, display_name, role, ...}} │ │ • CSRF token set on cookie │ └─────────────────────────────────────────────────────────────┘ ``` ### Session/JWT Handling **Session Storage**: SQLite `sessions` table | Column | Type | Purpose | |--------|------|---------| | `id` | TEXT (UUID) | Session identifier | | `user_id` | INTEGER | Reference to users.id | | `expires_at` | TEXT (ISO) | Expiration timestamp | | `created_at` | TEXT (ISO) | Session creation time | **Session Validation**: ```sql SELECT u.id, u.username, u.display_name, u.role, u.must_change_password, u.active, u.is_default_admin FROM sessions s JOIN users u ON u.id = s.user_id WHERE s.id = ? AND s.expires_at > datetime('now') AND u.active = 1 ``` **Session Duration**: 7 days **Session Destruction**: - Explicit logout: `DELETE FROM sessions WHERE id = ?` - Session expiry: Daily worker pruning - User deactivation: `DELETE FROM sessions WHERE user_id = ?` - Role change: `DELETE FROM sessions WHERE user_id = ?` ### RBAC (Role-Based Access Control) | Role | Capabilities | |------|--------------| | `user` | View/modify own bills, categories, payments, settings, profile | | `admin` | All user capabilities + user management, backups, OIDC config, system settings | **Admin Guard**: ```javascript function requireAdmin(req, res, next) { if (req.user?.role !== 'admin') { return res.status(403).json({error: 'Access denied: admin account required'}); } next(); } ``` ### Middleware Chain **Route Protection Example**: ```javascript // Admin routes app.use('/api/admin', csrfMiddleware, requireAuth, requireAdmin, adminActionLimiter, require('./routes/admin')); // User routes app.use('/api/bills', csrfMiddleware, requireAuth, requireUser, require('./routes/bills')); // Public routes (no auth) app.use('/api/about', require('./routes/about')); app.use('/api/version', require('./routes/version')); ``` ### Cookie Handling | Cookie | Name | Type | Secure | SameSite | Purpose | |--------|------|------|--------|----------|---------| | Session | `bt_session` | httpOnly | Configurable | `strict` | Auth session | | CSRF | `bt_csrf_token` | httpOnly | Configurable | `strict` | CSRF token | **Cookie Options** (determined at runtime): ```javascript function cookieOpts(req) { const cookieSecure = envFlag('COOKIE_SECURE'); const httpsSecure = envFlag('HTTPS'); const secure = cookieSecure !== null ? cookieSecure : httpsSecure !== null ? httpsSecure : requestLooksHttps(req); // Check X-Forwarded-Proto return { httpOnly: true, sameSite: 'strict', secure, maxAge: SESSION_DAYS * 86400 * 1000, // 7 days path: '/', }; } ``` ### OIDC/Authentik Flow ``` ┌─────────────────────────────────────────────────────────────┐ │ 1. User clicks OIDC login button │ │ • Redirects to frontend login page │ └─────────────────────────────────────────────────────────────┘ │ ┌─────────────────────────────────────────────────────────────┐ │ 2. Frontend calls /api/auth/oidc/login │ │ • Query params: ?redirect_to=/tracker │ └─────────────────────────────────────────────────────────────┘ │ ┌─────────────────────────────────────────────────────────────┐ │ 3. Create login state │ │ • Generate PKCE code_verifier (32 bytes base64url) │ │ • Generate nonce (16 bytes hex) │ │ • Store in oidc_states table (expires 5 min) │ │ • Code challenge = SHA256(code_verifier) base64url │ └─────────────────────────────────────────────────────────────┘ │ ┌─────────────────────────────────────────────────────────────┐ │ 4. Redirect to OIDC provider │ │ • Authorization URL built with: │ │ • client_id, redirect_uri, response_type=code │ │ • state (login state ID), nonce │ │ • code_challenge, code_challenge_method=S256 │ │ • scopes: openid, email, profile, groups │ └─────────────────────────────────────────────────────────────┘ │ ┌─────────────────────────────────────────────────────────────┐ │ 5. User authenticates with OIDC provider │ │ • Authentik validates credentials │ │ • Provider sends user back to redirect_uri │ └─────────────────────────────────────────────────────────────┘ │ ┌─────────────────────────────────────────────────────────────┐ │ 6. Callback: /api/auth/oidc/callback │ │ • Query params: code, state │ │ • Rate limited (oidcLimiter) │ └─────────────────────────────────────────────────────────────┘ │ ┌─────────────────────────────────────────────────────────────┐ │ 7. Exchange code for tokens │ │ • POST to OIDC token endpoint │ │ • client_id + client_secret + code + redirect_uri │ │ • code_verifier for PKCE │ └─────────────────────────────────────────────────────────────┘ │ ┌─────────────────────────────────────────────────────────────┐ │ 8. Verify ID token │ │ • JWT signature via JWKS │ │ • Issuer validation (iss claim) │ │ • Audience validation (aud claim) │ │ • Expiry validation (exp claim) │ │ • Nonce validation (replay protection) │ │ • State validation (replay protection) │ └─────────────────────────────────────────────────────────────┘ │ ┌─────────────────────────────────────────────────────────────┐ │ 9. Find or provision user │ │ • Look up by sub (external_subject) │ │ • Look up by email if email_verified=true │ │ • Auto-provision if OIDC_AUTO_PROVISION=true │ │ • Map groups to role (admin if in admin_group) │ └─────────────────────────────────────────────────────────────┘ │ ┌─────────────────────────────────────────────────────────────┐ │ 10. Create local session │ │ • Same mechanism as local login │ │ • Set bt_session cookie │ │ • Redirect to redirect_to (or /) │ └─────────────────────────────────────────────────────────────┘ ``` **OIDC Configuration**: ```bash # Environment variables (fallback if DB settings blank) OIDC_ISSUER_URL=https://auth.example.com/application/o/bills/.well-known/openid-configuration OIDC_CLIENT_ID= OIDC_CLIENT_SECRET= OIDC_REDIRECT_URI=https://bills.example.com/api/auth/oidc/callback OIDC_SCOPES=openid email profile groups OIDC_ADMIN_GROUP=bill-tracker-admins OIDC_AUTO_PROVISION=true ``` ### Failure Scenarios | Scenario | Status Code | Response | Recovery | |----------|-------------|----------|----------| | Invalid credentials | 401 | `{error: 'Invalid username or password', code: 'AUTH_ERROR'}` | Retry with correct credentials | | Session expired | 401 | `{error: 'Not authenticated', code: 'AUTH_ERROR'}` | Re-login | | Role insufficient | 403 | `{error: 'Access denied: admin account required', code: 'FORBIDDEN'}` | Login as admin | | Rate limited | 429 | `{error: 'Too many login attempts...'}` | Wait 15 minutes | | CSRF invalid | 403 | `{error: 'CSRF token validation failed', code: 'CSRF_INVALID'}` | Refresh page | | OIDC config missing | 501 | `{error: 'OIDC authentication is not configured...'}` | Configure OIDC in Admin | | OIDC provider error | 502 | `{error: 'Failed to reach the identity provider...'}` | Check OIDC provider status | | OIDC callback expired state | Redirect + query param | `/?oidc_error=invalid_or_expired_state` | Start login flow again | ### Code Locations | Component | File | |-----------|------| | Login endpoint | `routes/authLogin.js` | | Auth middleware | `middleware/requireAuth.js` | | CSRF middleware | `middleware/csrf.js` | | Session management | `services/authService.js` | | OIDC login | `routes/authOidc.js` | | OIDC service | `services/oidcService.js` | | OIDC tables | `db/schema.sql` (sessions, oidc_states) | --- ## 5. API Documentation ### Authentication Endpoints #### POST /api/auth/login **Purpose**: Local username/password login **Request**: ```json { "username": "string (required)", "password": "string (required)" } ``` **Response**: ```json { "user": { "id": 1, "username": "admin", "display_name": "Administrator", "role": "admin", "active": true, "is_default_admin": true, "must_change_password": false, "first_login": false } } ``` **Rate Limit**: 10 per 15 minutes per IP (bypassed if no users exist) **Errors**: | Status | Code | Message | |--------|------|---------| | 400 | `VALIDATION_ERROR` | Username/password missing | | 401 | `AUTH_ERROR` | Invalid credentials | | 403 | `FORBIDDEN` | Local login disabled | | 429 | `RATE_LIMITED` | Too many attempts | #### GET /api/auth/oidc/login **Purpose**: Initiate OIDC login flow **Query Parameters**: - `redirect_to` (optional): URL to redirect after login **Response**: HTTP 302 redirect to OIDC provider **Errors**: | Status | Redirect | Reason | |--------|----------|--------| | 501 | - | OIDC not configured | | 502 | - | Provider unreachable | #### GET /api/auth/oidc/callback **Purpose**: OIDC callback handler **Query Parameters**: - `code`: Authorization code - `state`: Login state ID - `error` (optional): Provider error **Response**: HTTP 302 redirect to frontend or error page **Errors**: | Status | Redirect | Reason | |--------|----------|--------| | 302 | `/?oidc_error=not_configured` | OIDC disabled | | 302 | `/?oidc_error=authorization_failed` | Provider signalled error | | 302 | `/?oidc_error=invalid_callback` | Missing code or state | | 302 | `/?oidc_error=invalid_or_expired_state` | State invalid/expired | | 302 | `/?oidc_error=authentication_failed` | Token validation failure | | 302 | `/?oidc_error=access_denied` | User not in admin group (if required) | #### GET /api/auth/logout **Purpose**: Logout (session invalidation) **Response**: ```json { "success": true } ``` ### Tracker Endpoints #### GET /api/tracker **Purpose**: Monthly tracker data **Query Parameters**: - `year` (optional, default: current year) - `month` (optional, default: current month) **Response**: ```json { "year": 2026, "month": 5, "today": "2026-05-09", "summary": { "total_expected": 450.00, "total_starting": 500.00, "has_starting_amounts": true, "total_paid": 320.00, "remaining": 180.00, "overdue": 75.00, "count_paid": 5, "count_upcoming": 3, "count_late": 2, "count_autodraft": 1 }, "rows": [ { "id": 1, "name": "Rent", "category_name": "Housing", "due_date": "2026-05-01", "expected_amount": 1200.00, "actual_amount": 1200.00, "total_paid": 1200.00, "balance": 0, "status": "paid", "autopay_enabled": false, "autodraft_status": "none" }, ... ] } ``` #### GET /api/tracker/upcoming **Purpose**: Bills due in next N days **Query Parameters**: - `days` (optional, default: 30, max: 365) **Response**: ```json { "days": 30, "today": "2026-05-09", "upcoming": [ { "id": 2, "name": "Internet", "category_name": "Phone & Internet", "due_date": "2026-05-15", "expected_amount": 60.00, "status": "due_soon", "days_until_due": 6 } ] } ``` ### Bills Endpoints #### GET /api/bills **Purpose**: List all bills (active or all) **Query Parameters**: - `inactive` (optional): If "true", include inactive bills **Response**: Array of bill objects #### GET /api/bills/:id **Purpose**: Get single bill by ID **Response**: Bill object #### POST /api/bills **Purpose**: Create new bill **Request**: ```json { "name": "Internet", "category_id": 5, "due_day": 15, "override_due_date": null, "expected_amount": 60.00, "interest_rate": null, "billing_cycle": "monthly", "autopay_enabled": false, "autodraft_status": "none", "website": null, "username": null, "account_info": null, "has_2fa": false, "notes": "Fiber optic internet", "history_visibility": "default" } ``` **Response**: Created bill with ID #### PUT /api/bills/:id **Purpose**: Update bill **Request**: Partial bill object **Response**: Updated bill #### DELETE /api/bills/:id **Purpose**: Hard-delete bill (irreversible) **Response**: ```json { "success": true, "deleted_bill_id": 1, "deleted_bill_name": "Rent", "warning": "Bill and all associated payments, monthly state, and history ranges were permanently deleted." } ``` #### GET /api/bills/:id/payments **Purpose**: List payments for a bill **Query Parameters**: - `page` (optional, default: 1) - `limit` (optional, default: 20, max: 100) **Response**: ```json { "bill_id": 1, "bill_name": "Rent", "total": 5, "page": 1, "limit": 20, "pages": 1, "payments": [...] } ``` #### POST /api/bills/:id/toggle-paid **Purpose**: Toggle bill as paid/unpaid **Request**: ```json { "amount": 1200.00, "paid_date": "2026-05-01", "method": "ACH", "notes": "Rent payment" } ``` **Response**: ```json { "success": true, "isPaid": true, "action": "created_payment", "payment": { ... } } ``` #### GET /api/bills/:id/monthly-state **Purpose**: Get monthly state override for a bill **Query Parameters**: - `year` (required) - `month` (required) **Response**: ```json { "bill_id": 1, "year": 2026, "month": 5, "actual_amount": 1200.00, "notes": null, "is_skipped": false } ``` #### PUT /api/bills/:id/monthly-state **Purpose**: Set monthly state override **Request**: ```json { "year": 2026, "month": 5, "actual_amount": 1250.00, "notes": "Rent increased", "is_skipped": false } ``` **Response**: Saved state ### Categories Endpoints #### GET /api/categories **Purpose**: List all categories for user **Response**: Array of category objects #### POST /api/categories **Purpose**: Create new category **Request**: ```json { "name": "Entertainment" } ``` **Response**: Created category ### Settings Endpoints #### GET /api/settings **Purpose**: Get all settings **Response**: ```json { "currency": "USD", "date_format": "MM/DD/YYYY", "grace_period_days": "5", "notify_days_before": "3", "backup_enabled": "false", ... } ``` #### PUT /api/settings **Purpose**: Update settings **Request**: Partial settings object **Response**: Updated settings ### User Endpoints #### GET /api/user **Purpose**: Get current user profile **Response**: User object (without password) ### Calendar Endpoints #### GET /api/calendar **Purpose**: Calendar data for a month **Query Parameters**: - `year` (optional) - `month` (optional) **Response**: Calendar data with bill due dates ### Summary Endpoints #### GET /api/summary **Purpose**: Monthly spending summary **Query Parameters**: - `year` (optional) - `month` (optional) **Response**: ```json { "year": 2026, "month": 5, "total_expected": 450.00, "total_actual": 425.00, "total_paid": 425.00, "total_starting": 500.00, "remaining": 75.00, "by_category": [...], "by_bill": [...] } ``` ### Analytics Endpoints #### GET /api/analytics **Purpose**: Analytics data with filters **Query Parameters**: - `start_date` (optional, default: 30 days ago) - `end_date` (optional, default: today) - `category_id` (optional) - `bill_id` (optional) **Response**: ```json { "start_date": "2026-04-09", "end_date": "2026-05-09", "total_spent": 425.00, "expected_vs_actual": { "expected": 450.00, "actual": 425.00, "difference": -25.00 }, "by_category": [...], "payment_history": [...] } ``` ### Profile Endpoints #### GET /api/profile **Purpose**: Get user profile **Response**: User profile #### POST /api/profile **Purpose**: Update user profile **Request**: ```json { "display_name": "John Doe", "notification_email": "john@example.com", "notifications_enabled": true, "notify_3d": true, "notify_1d": true, "notify_due": true, "notify_overdue": true, "current_password": "oldpass123", "new_password": "newpass456" } ``` **Response**: Updated user **Rate Limit**: 5 per 15 minutes per IP ### Admin Endpoints **All admin routes require `requireAuth` + `requireAdmin` + `csrfMiddleware` + `adminActionLimiter`** #### GET /api/admin/has-users **Purpose**: Check if other users exist (lockout protection) **Response**: ```json { "has_users": true } ``` #### GET /api/admin/users **Purpose**: List all users **Response**: Array of user objects with admin fields #### POST /api/admin/users **Purpose**: Create new user **Request**: ```json { "username": "newuser", "password": "password123" } ``` **Response**: Created user **Errors**: | Status | Message | |--------|---------| | 400 | Username/password too short | | 409 | Username already taken | #### PUT /api/admin/users/:id/password **Purpose**: Reset user password **Request**: ```json { "password": "newpassword123" } ``` **Response**: ```json { "success": true } ``` **Effects**: - Updates password hash - Sets `must_change_password = 1` - Invalidates all user sessions #### PUT /api/admin/users/:id/role **Purpose**: Promote/demote user **Request**: ```json { "role": "admin" } ``` **Response**: Updated user **Validations**: - Cannot change own role - Cannot remove last admin - Deletes all sessions for target user #### PUT /api/admin/users/:id/active **Purpose**: Deactivate/reactivate user **Request**: ```json { "active": false } ``` **Response**: Updated user **Effects**: - Sets `active = 0/1` - Invalidates all sessions if deactivated #### DELETE /api/admin/users/:id **Purpose**: Delete user (irreversible) **Response**: ```json { "success": true, "deleted_user_id": 2 } ``` **Effects**: - Deletes all user data (sessions, imports, exports, bills, categories) #### GET /api/about-admin **Purpose**: Retrieve FUTURE.md and DEVELOPMENT_LOG.md content for admin-only About page **Route**: `/api/about-admin` (admin-only, rate-limited) **Authentication**: Requires `requireAuth` + `requireAdmin` + `csrfMiddleware` + `adminActionLimiter` **Rate Limit**: 30 requests per 15 minutes per IP **Response**: ```json { "future": "# Future Roadmap\n\n## Planned Features\n\n- [ ] Feature A\n- [ ] Feature B\n", "devlog": "# Development Log\n\n## v0.19.1 (2026-05-09)\n\n### Added\n\n- Regular User Seed Environment Variables\n" } ``` **Errors**: | Status | Message | |--------|---------| | 401 | Unauthorized (not logged in) | | 403 | Forbidden (not admin) | | 500 | Internal Server Error (file read failure) | **Security Measures**: 1. **Path Traversal Protection** — `sanitizePath()` validates file paths before access 2. **Content Redaction** — Internal IPs, passwords, and API keys are redacted from content 3. **Error Sanitization** — Error messages exclude file paths to prevent path disclosure 4. **XSS Prevention** — `rehype-sanitize` applied to ReactMarkdown rendering in `AboutPage.jsx` **Implementation**: - Route: `routes/aboutAdmin.js` - Server entry: `server.js` (mounted at `/api/about-admin`) - Client: `client/api.js` (`aboutAdmin()` function) - UI: `client/pages/AboutPage.jsx` (rendered via `ReactMarkdown` with `rehypeSanitize`) --- ### Backup Endpoints | Method | Endpoint | Purpose | Rate Limit | |--------|----------|---------|------------| | POST | `/api/admin/backups` | Create backup | 5/60min | | GET | `/api/admin/backups` | List backups | 5/60min | | GET | `/api/admin/backups/:id/download` | Download backup | - | | POST | `/api/admin/backups/:id/restore` | Restore backup | 5/60min | | DELETE | `/api/admin/backups/:id` | Delete backup | 5/60min | | POST | `/api/admin/backups/import` | Import backup | 5/60min | | GET | `/api/admin/backups/settings` | Get backup schedule | 5/60min | | PUT | `/api/admin/backups/settings` | Update backup schedule | 5/60min | | POST | `/api/admin/backups/run-scheduled-now` | Run scheduled backup now | 5/60min | **Backup Request/Response**: ```json // POST /api/admin/backups // Response (201 Created) { "id": "bill-tracker-backup-2026-05-09-03-45-32-456Z-abcd1234.sqlite", "filename": "bill-tracker-backup-2026-05-09-03-45-32-456Z-abcd1234.sqlite", "type": "manual", "created_at": "2026-05-09T03:45:32.456Z", "modified_at": "2026-05-09T03:45:32.456Z", "size_bytes": 200704, "checksum": "abc123def456..." } ``` **Restore Request/Response**: ```json // POST /api/admin/backups/:id/restore // Response { "restored_from": "bill-tracker-backup-2026-05-08-02-00-00-000Z-1234abcd.sqlite", "pre_restore_backup": "pre-restore-2026-05-09-03-46-00-000Z-5678efgh.sqlite", "restored_at": "2026-05-09T03:46:00.000Z", "restart_required": false } ``` **Import Backup**: - Body: Binary SQLite file - Header: `X-Checksum-SHA256: ` (optional, if provided, validated) - Max size: 100MB #### Cleanup Endpoints | Method | Endpoint | Purpose | |--------|----------|---------| | GET | `/api/admin/cleanup` | Get cleanup settings and status | | PUT | `/api/admin/cleanup` | Update cleanup settings | | POST | `/api/admin/cleanup/run` | Run cleanup immediately | **Cleanup Settings**: ```json { "import_sessions_enabled": true, "temp_exports_enabled": true, "temp_export_max_age_hours": 2, "backup_partials_enabled": true, "import_history_enabled": false, "import_history_max_age_days": 365 } ``` #### Auth Mode Endpoints | Method | Endpoint | Purpose | |--------|----------|---------| | GET | `/api/admin/auth-mode` | Get auth configuration | | PUT | `/api/admin/auth-mode` | Update auth configuration | | POST | `/api/admin/auth-mode/oidc-test` | Test OIDC configuration | **Auth Mode Settings**: ```json { "auth_mode": "multi", "local_login_enabled": true, "oidc_login_enabled": false, "oidc_configured": false, "oidc_issuer_url_set": false, "oidc_client_id_set": false, "oidc_client_secret_set": false, "oidc_redirect_uri_set": false, "oidc_missing_fields": ["issuer URL", "client ID", "client secret", "redirect URI"], "can_disable_local": false, "warnings": [] } ``` ### Export Endpoints #### GET /api/export **Purpose**: Export user data as XLSX **Query Parameters**: - `export_type` (optional): 'bills', 'payments', 'categories', 'full' - `start_date`, `end_date` (optional): Date range **Response**: XLSX file download **Rate Limit**: 30 per 15 minutes per IP ### Import Endpoints #### GET /api/import **Purpose**: Get import history and preview settings #### POST /api/import **Purpose**: Preview or apply import **Form Data**: - `file`: XLSX file - `preview`: 'true' or 'false' **Response** (preview): ```json { "preview": true, "rows_parsed": 10, "rows_created": 8, "rows_updated": 2, "rows_skipped": 0, "rows_errored": 0, "data": [...] } ``` **Response** (apply): ```json { "preview": false, "imported_at": "2026-05-09T03:50:00.000Z", "rows_parsed": 10, "rows_created": 8, "rows_updated": 2, "rows_skipped": 0, "rows_errored": 0 } ``` **Rate Limit**: 20 per 15 minutes per IP ### Status Endpoints #### GET /api/status **Purpose**: System status (admin only) **Response**: ```json { "version": "0.19.0", "node_env": "production", "db_path": "/data/bills.db", "backup_path": "/data/backups", "sqlite_version": "3.45.0", "users_count": 2, "bills_count": 15, "last_worker_run": "2026-05-09T06:00:00.000Z", "last_worker_status": "success", "uptime_seconds": 86400, "last_error": null } ``` #### GET /api/about-admin **Purpose**: Admin-only access to FUTURE.md and DEVELOPMENT_LOG.md content **Auth**: `requireAuth` + `requireAdmin` middleware required **Rate Limit**: 30 per 15 minutes per IP (adminActionLimiter) **CSRF**: Required (csrfMiddleware) **Request**: None **Response**: ```json { "future": "# Bill Tracker — Future Improvements...", "developmentLog": "# Bill Tracker — Development Log..." } ``` **Errors**: | Status | Code | Message | |--------|------|---------| | 401 | `AUTH_ERROR` | Not authenticated | | 403 | `FORBIDDEN` | Access denied: admin account required | | 400 | `INVALID_FILE_PATH` | Path traversal attempt detected | | 429 | `RATE_LIMITED` | Too many admin actions | | 500 | `FILE_READ_ERROR` | Failed to read documentation files | **Security Measures**: - Path traversal protection via `sanitizePath()` function - Content redaction of internal IPs, passwords, API keys - Error message sanitization to prevent path disclosure --- ## Environment Variables ### Initialization Variables | Variable | Default | Description | |----------|---------|-------------| | `INIT_ADMIN_USER` | `admin` | Username for initial admin account | | `INIT_ADMIN_PASS` | *required* | Password for initial admin account | | `INIT_TEST_USER` | `testuser` | Username for initial test admin account | | `INIT_TEST_PASS` | `testpass123` | Password for initial test admin account | | `INIT_REGULAR_USER` | `regularuser` | Username for initial non-admin user (for testing) | | `INIT_REGULAR_PASS` | `regularpass123` | Password for initial non-admin user | | `DB_PATH` | `./data/bills.db` | SQLite database file path | | `PORT` | `3000` | Server port | | `SESSION_DAYS` | `7` | Session duration in days | | `COOKIE_SECURE` | *auto-detect* | Force HTTPS-only cookies | | `HTTPS` | *auto-detect* | Server running behind HTTPS proxy | | `CSRF_HTTP_ONLY` | `true` | CSRF cookie httpOnly flag | | `CSRF_SAME_SITE` | `strict` | CSRF cookie SameSite policy | | `CSRF_SECURE` | *auto-detect* | CSRF cookie HTTPS-only | | `INIT_REGULAR_USER` | *optional* | Create non-admin user on first run if set | | `INIT_REGULAR_PASS` | *optional* | Password for regular user (if INIT_REGULAR_USER set) | **First-Run User Creation Flow:** On first startup, if no users exist: 1. If `INIT_ADMIN_USER` and `INIT_ADMIN_PASS` are set: - Create admin user with `role='admin'` and `is_default_admin=1` - If `INIT_TEST_USER` and `INIT_TEST_PASS` are also set: - Create test admin user with `role='admin'` and `is_default_admin=0` 2. If `INIT_REGULAR_USER` and `INIT_REGULAR_PASS` are set: - Create regular user with `role='user'` and `is_default_admin=0` Regular users are created with `bcryptjs` password hashing and default notification settings. | `DB_PATH` | `./data/bills.db` | SQLite database file path | | `PORT` | `3000` | Server port | | `SESSION_DAYS` | `7` | Session duration in days | | `COOKIE_SECURE` | *auto-detect* | Force HTTPS-only cookies | | `HTTPS` | *auto-detect* | Server running behind HTTPS proxy | | `CSRF_HTTP_ONLY` | `true` | CSRF cookie httpOnly flag | | `CSRF_SAME_SITE` | `strict` | CSRF cookie SameSite policy | | `CSRF_SECURE` | *auto-detect* | CSRF cookie HTTPS-only | ### OIDC Variables | Variable | Description | |----------|-------------| | `OIDC_ISSUER_URL` | Authentik discovery URL | | `OIDC_CLIENT_ID` | OIDC client ID | | `OIDC_CLIENT_SECRET` | OIDC client secret | | `OIDC_REDIRECT_URI` | Callback URL | | `OIDC_SCOPES` | Space-separated scopes | | `OIDC_ADMIN_GROUP` | Group requiring admin access | | `OIDC_AUTO_PROVISION` | Auto-create users from OIDC | --- ## Security Measures ### Rate Limiting | Limiter | Max | Window | Endpoints | |---------|-----|--------|-----------| | `loginLimiter` | 10 | 15 min | `/api/auth/login` | | `passwordLimiter` | 5 | 15 min | `/api/profile`, `/api/admin/users/:id/password` | | `importLimiter` | 20 | 15 min | `/api/import/*` | | `exportLimiter` | 30 | 15 min | `/api/export/*` | | `adminActionLimiter` | 30 | 15 min | `/api/admin/*`, `/api/about-admin` | | `oidcLimiter` | 20 | 15 min | `/api/auth/oidc/*` | | `backupOperationLimiter` | 5 | 60 min | `/api/admin/backups/*` | ### Authentication & Authorization | Feature | Implementation | |---------|----------------| | Session duration | 7 days | | Password hashing | bcryptjs (salt rounds: 10) | | CSRF protection | Configurable via env vars | | Admin guard | `requireAdmin` middleware | ### Content Security | Measure | Implementation | |---------|----------------| | XSS prevention | `rehype-sanitize` on markdown content | | Path traversal | `sanitizePath()` in `routes/aboutAdmin.js` | | Content redaction | Internal IPs, passwords, API keys redacted | | Error sanitization | Stack traces excluded, paths obscured | ### Input Validation | Field | Validation | |-------|-----------| | `due_day` | Integer 1-31 | | `expected_amount` | Number ≥ 0 | | `interest_rate` | Number 0-100 or null | | `password` | Min 8 characters (admin) | | `username` | Min 3 characters (admin) | --- ## 6. Database Documentation ### Schema Overview **Database**: SQLite (better-sqlite3) **Schema Location**: `db/schema.sql` **Migration Logic**: `db/database.js` (runMigrations function) ### Migration System #### schema_migrations Table The `schema_migrations` table tracks applied database migrations to ensure idempotent, repeatable deployments. | Column | Type | Constraints | Description | |--------|------|-------------|-------------| | `id` | INTEGER | PRIMARY KEY AUTOINCREMENT | Internal tracking ID | | `version` | TEXT | NOT NULL, UNIQUE | Migration version (e.g., `v0.2`, `v0.41`) | | `description` | TEXT | NOT NULL | Human-readable migration description | | `applied_at` | TEXT | NOT NULL DEFAULT (datetime('now')) | Timestamp of application | **Purpose**: Prevent duplicate migration execution and provide audit trail of applied migrations. **Helper Functions**: - `hasMigrationBeenApplied(version)`: Returns `true` if version exists in `schema_migrations` - `recordMigration(version, description)`: Inserts record into `schema_migrations` - `runMigrations()`: Iterates through all migrations, skipping already-applied versions #### Migration Process 1. On startup, `initSchema()` creates `schema_migrations` table if it doesn't exist 2. `runMigrations()` queries each migration version in order 3. Applied migrations are skipped with a log message 4. Pending migrations are executed and recorded 5. All migrations are logged with status (`[migration] Applying X` vs `[migration] Skipping already applied X`) #### Migration Format Migrations are defined as versioned objects in `db/database.js`: ```javascript const migrations = [ { version: 'v0.2', description: 'payments: soft-delete column', run: function() { /* SQL logic */ } }, // ... additional migrations ]; ``` **Key Properties**: - `version`: Unique version string (e.g., `v0.2`, `v0.41`) - `description`: Brief description of what changed - `run()`: Function containing migration SQL logic **Idempotency**: Each migration can safely be re-run after a `git pull`; skipped migrations log but don't error. ### Table Definitions #### users | Column | Type | Constraints | Description | |--------|------|-------------|-------------| | `id` | INTEGER | PRIMARY KEY, AUTOINCREMENT | User ID | | `username` | TEXT | NOT NULL, UNIQUE (CASE-insensitive) | Login username | | `password_hash` | TEXT | NOT NULL | bcrypt hash of password | | `role` | TEXT | NOT NULL, CHECK('admin', 'user') | Role: admin or user | | `active` | INTEGER | NOT NULL, DEFAULT 1 | 1=active, 0=deactivated | | `is_default_admin` | INTEGER | NOT NULL, DEFAULT 0 | 1=initial admin account | | `must_change_password` | INTEGER | NOT NULL, DEFAULT 0 | 1=force password change on next login | | `first_login` | INTEGER | NOT NULL, DEFAULT 1 | 1=user has never logged in | | `created_at` | TEXT | DEFAULT (datetime('now')) | Account creation time | | `updated_at` | TEXT | DEFAULT (datetime('now')) | Last update time | | `notification_email` | TEXT | | User's email for notifications | | `notifications_enabled` | INTEGER | NOT NULL, DEFAULT 0 | 1=receive email notifications | | `notify_3d` | INTEGER | NOT NULL, DEFAULT 1 | Notify 3 days before due | | `notify_1d` | INTEGER | NOT NULL, DEFAULT 1 | Notify 1 day before due | | `notify_due` | INTEGER | NOT NULL, DEFAULT 1 | Notify on due date | | `notify_overdue` | INTEGER | NOT NULL, DEFAULT 1 | Notify for overdue bills | | `display_name` | TEXT | | Display name (OIDC) | | `last_password_change_at` | TEXT | | Last password change timestamp | | `auth_provider` | TEXT | NOT NULL, DEFAULT 'local' | 'local' or 'oidc' | | `external_subject` | TEXT | | OIDC sub claim | | `email` | TEXT | | OIDC email claim | | `last_login_at` | TEXT | | Last login timestamp | **Indexes**: - `idx_sessions_user_id` (on sessions.user_id) - `idx_sessions_expires` (on sessions.expires_at) #### sessions | Column | Type | Constraints | Description | |--------|------|-------------|-------------| | `id` | TEXT | PRIMARY KEY | Session UUID | | `user_id` | INTEGER | NOT NULL, REFERENCES users(id) ON DELETE CASCADE | User ID | | `expires_at` | TEXT | NOT NULL | Expiration timestamp | | `created_at` | TEXT | DEFAULT (datetime('now')) | Session creation time | **Indexes**: - `idx_sessions_user_id` on `user_id` - `idx_sessions_expires` on `expires_at` #### bills | Column | Type | Constraints | Description | |--------|------|-------------|-------------| | `id` | INTEGER | PRIMARY KEY, AUTOINCREMENT | Bill ID | | `user_id` | INTEGER | NOT NULL, REFERENCES users(id) ON DELETE CASCADE | Owner user ID | | `name` | TEXT | NOT NULL | Bill name | | `category_id` | INTEGER | REFERENCES categories(id) ON DELETE SET NULL | Category reference | | `due_day` | INTEGER | NOT NULL, CHECK(1-31) | Due day of month | | `override_due_date` | TEXT | | Custom due date override | | `bucket` | TEXT | CHECK('1st', '15th') | Payment bucket | | `expected_amount` | REAL | NOT NULL, DEFAULT 0 | Expected monthly amount | | `interest_rate` | REAL | CHECK(0-100) | APR or interest rate | | `billing_cycle` | TEXT | DEFAULT 'monthly', CHECK | 'monthly', 'quarterly', 'annually', 'irregular' | | `autopay_enabled` | INTEGER | NOT NULL, DEFAULT 0 | 1=autopay enabled | | `autodraft_status` | TEXT | NOT NULL, DEFAULT 'none' | 'none', 'pending', 'assumed_paid', 'confirmed' | | `website` | TEXT | | Bill provider website | | `username` | TEXT | | Bill provider username | | `account_info` | TEXT | | Bill account info | | `has_2fa` | INTEGER | NOT NULL, DEFAULT 0 | 1=2FA enabled | | `history_visibility` | TEXT | NOT NULL, DEFAULT 'default' | 'default', 'all', 'ranges', 'none' | | `active` | INTEGER | NOT NULL, DEFAULT 1 | 1=active, 0=inactive | | `notes` | TEXT | | User notes | | `is_seeded` | INTEGER | NOT NULL, DEFAULT 0 | 1=demo data | | `created_at` | TEXT | DEFAULT (datetime('now')) | Creation time | | `updated_at` | TEXT | DEFAULT (datetime('now')) | Last update time | **Indexes**: - `idx_bills_active` on `active` - `idx_bills_user_active` on `user_id, active` - `idx_bills_user_id` on `user_id` #### categories | Column | Type | Constraints | Description | |--------|------|-------------|-------------| | `id` | INTEGER | PRIMARY KEY, AUTOINCREMENT | Category ID | | `user_id` | INTEGER | NOT NULL, REFERENCES users(id) ON DELETE CASCADE | Owner user ID | | `name` | TEXT | NOT NULL | Category name | | `created_at` | TEXT | DEFAULT (datetime('now')) | Creation time | | `updated_at` | TEXT | DEFAULT (datetime('now')) | Last update time | **Constraints**: - Unique constraint: `UNIQUE(user_id, name COLLATE NOCASE)` **Indexes**: - `idx_categories_user_name` on `user_id, name` - `idx_categories_user_name_unique` on `user_id, name COLLATE NOCASE` #### payments | Column | Type | Constraints | Description | |--------|------|-------------|-------------| | `id` | INTEGER | PRIMARY KEY, AUTOINCREMENT | Payment ID | | `bill_id` | INTEGER | NOT NULL, REFERENCES bills(id) ON DELETE CASCADE | Bill reference | | `amount` | REAL | NOT NULL | Payment amount | | `paid_date` | TEXT | NOT NULL | Payment date (YYYY-MM-DD) | | `method` | TEXT | | Payment method (cash, check, card, ACH) | | `notes` | TEXT | | Payment notes | | `deleted_at` | TEXT | | Soft-delete timestamp | | `created_at` | TEXT | DEFAULT (datetime('now')) | Creation time | | `updated_at` | TEXT | DEFAULT (datetime('now')) | Last update time | **Indexes**: - `idx_payments_bill_id` on `bill_id` - `idx_payments_paid_date` on `paid_date` - `idx_payments_bill_date_del` on `bill_id, paid_date, deleted_at` - `idx_payments_deleted` on `deleted_at` #### monthly_bill_state | Column | Type | Constraints | Description | |--------|------|-------------|-------------| | `id` | INTEGER | PRIMARY KEY, AUTOINCREMENT | State ID | | `bill_id` | INTEGER | NOT NULL, REFERENCES bills(id) ON DELETE CASCADE | Bill reference | | `year` | INTEGER | NOT NULL, CHECK(2000-2100) | Year | | `month` | INTEGER | NOT NULL, CHECK(1-12) | Month | | `actual_amount` | REAL | | Actual amount paid (override expected) | | `notes` | TEXT | | Month-specific notes | | `is_skipped` | INTEGER | NOT NULL, DEFAULT 0 | 1=skip this month | | `created_at` | TEXT | DEFAULT (datetime('now')) | Creation time | | `updated_at` | TEXT | DEFAULT (datetime('now')) | Last update time | **Constraints**: - `UNIQUE(bill_id, year, month)` **Indexes**: - `idx_monthly_bill_state_lookup` on `bill_id, year, month` #### settings | Column | Type | Constraints | Description | |--------|------|-------------|-------------| | `key` | TEXT | PRIMARY KEY | Setting key | | `value` | TEXT | NOT NULL | Setting value | | `updated_at` | TEXT | DEFAULT (datetime('now')) | Last update time | #### notifications | Column | Type | Constraints | Description | |--------|------|-------------|-------------| | `id` | INTEGER | PRIMARY KEY, AUTOINCREMENT | Notification ID | | `bill_id` | INTEGER | NOT NULL, REFERENCES bills(id) ON DELETE CASCADE | Bill reference | | `user_id` | INTEGER | NOT NULL, REFERENCES users(id) ON DELETE CASCADE | User reference | | `year` | INTEGER | NOT NULL | Year | | `month` | INTEGER | NOT NULL | Month | | `type` | TEXT | NOT NULL | Notification type | | `sent_date` | TEXT | NOT NULL, DEFAULT (date('now')) | Date sent | **Constraints**: - `UNIQUE(bill_id, user_id, year, month, type, sent_date)` **Indexes**: - `idx_notifications_lookup` on `bill_id, user_id, year, month` #### oidc_states | Column | Type | Constraints | Description | |--------|------|-------------|-------------| | `id` | TEXT | PRIMARY KEY | State UUID | | `nonce` | TEXT | NOT NULL | Nonce for replay protection | | `code_verifier` | TEXT | NOT NULL | PKCE code verifier | | `redirect_to` | TEXT | | Redirect URL after login | | `created_at` | TEXT | NOT NULL | Creation time | | `expires_at` | TEXT | NOT NULL | Expiration time | **Indexes**: - `idx_oidc_states_expires` on `expires_at` #### bill_history_ranges | Column | Type | Constraints | Description | |--------|------|-------------|-------------| | `id` | INTEGER | PRIMARY KEY, AUTOINCREMENT | Range ID | | `bill_id` | INTEGER | NOT NULL, REFERENCES bills(id) ON DELETE CASCADE | Bill reference | | `start_year` | INTEGER | NOT NULL | Start year | | `start_month` | INTEGER | NOT NULL | Start month | | `end_year` | INTEGER | | End year (NULL = open-ended) | | `end_month` | INTEGER | | End month (NULL = open-ended) | | `label` | TEXT | | Range label | | `created_at` | TEXT | DEFAULT (datetime('now')) | Creation time | | `updated_at` | TEXT | DEFAULT (datetime('now')) | Last update time | **Indexes**: - `idx_bill_history_ranges_bill` on `bill_id` #### monthly_income | Column | Type | Constraints | Description | |--------|------|-------------|-------------| | `id` | INTEGER | PRIMARY KEY, AUTOINCREMENT | Income record ID | | `user_id` | INTEGER | NOT NULL, REFERENCES users(id) ON DELETE CASCADE | User reference | | `year` | INTEGER | NOT NULL, CHECK(2000-2100) | Year | | `month` | INTEGER | NOT NULL, CHECK(1-12) | Month | | `label` | TEXT | NOT NULL, DEFAULT 'Salary' | Income source | | `amount` | REAL | NOT NULL, DEFAULT 0 | Income amount | | `created_at` | TEXT | DEFAULT (datetime('now')) | Creation time | | `updated_at` | TEXT | DEFAULT (datetime('now')) | Last update time | **Constraints**: - `UNIQUE(user_id, year, month)` **Indexes**: - `idx_monthly_income_user_month` on `user_id, year, month` #### monthly_starting_amounts | Column | Type | Constraints | Description | |--------|------|-------------|-------------| | `id` | INTEGER | PRIMARY KEY, AUTOINCREMENT | Amount record ID | | `user_id` | INTEGER | NOT NULL, REFERENCES users(id) ON DELETE CASCADE | User reference | | `year` | INTEGER | NOT NULL, CHECK(2000-2100) | Year | | `month` | INTEGER | NOT NULL, CHECK(1-12) | Month | | `first_amount` | REAL | NOT NULL, DEFAULT 0, CHECK ≥ 0 | Amount for 1st of month | | `fifteenth_amount` | REAL | NOT NULL, DEFAULT 0, CHECK ≥ 0 | Amount for 15th of month | | `other_amount` | REAL | NOT NULL, DEFAULT 0, CHECK ≥ 0 | Other amount | | `notes` | TEXT | | Notes | | `created_at` | TEXT | DEFAULT (datetime('now')) | Creation time | | `updated_at` | TEXT | DEFAULT (datetime('now')) | Last update time | **Constraints**: - `UNIQUE(user_id, year, month)` **Indexes**: - `idx_monthly_starting_amounts_user_month` on `user_id, year, month` #### import_sessions | Column | Type | Constraints | Description | |--------|------|-------------|-------------| | `id` | TEXT | PRIMARY KEY | Session UUID | | `user_id` | INTEGER | NOT NULL | User reference | | `created_at` | TEXT | NOT NULL | Creation time | | `expires_at` | TEXT | NOT NULL | Expiration time | | `preview_json` | TEXT | NOT NULL | JSON preview data | **Indexes**: - `idx_import_sessions_user` on `user_id` - `idx_import_sessions_expires` on `expires_at` #### import_history | Column | Type | Constraints | Description | |--------|------|-------------|-------------| | `id` | INTEGER | PRIMARY KEY, AUTOINCREMENT | History ID | | `user_id` | INTEGER | NOT NULL | User reference | | `imported_at` | TEXT | NOT NULL | Import timestamp | | `source_filename` | TEXT | | Source filename | | `file_type` | TEXT | DEFAULT 'xlsx' | File type | | `sheet_name` | TEXT | | Sheet name | | `rows_parsed` | INTEGER | DEFAULT 0 | Parsed rows | | `rows_created` | INTEGER | DEFAULT 0 | New rows created | | `rows_updated` | INTEGER | DEFAULT 0 | Existing rows updated | | `rows_skipped` | INTEGER | DEFAULT 0 | Skipped rows | | `rows_ambiguous` | INTEGER | DEFAULT 0 | Ambiguous rows | | `rows_errored` | INTEGER | DEFAULT 0 | Errored rows | | `options_json` | TEXT | | Import options | | `summary_json` | TEXT | | Summary JSON | **Indexes**: - `idx_import_history_user` on `user_id` ### Data Flow #### User-scoped Data All user-modifiable data is scoped to `user_id`: | Table | user_id Reference | onDelete | |-------|------------------|----------| | bills | `user_id` | CASCADE | | categories | `user_id` | CASCADE | | payments | bills.user_id | CASCADE | | monthly_bill_state | bills.user_id (via bill_id) | CASCADE | | monthly_income | `user_id` | CASCADE | | monthly_starting_amounts | `user_id` | CASCADE | | import_sessions | `user_id` | N/A | | import_history | `user_id` | N/A | #### Data Access Pattern ```sql -- User bill list SELECT * FROM bills WHERE user_id = ? AND active = 1 ORDER BY due_day; -- User categories SELECT * FROM categories WHERE user_id = ? ORDER BY name; -- User payments (with bill info) SELECT p.*, b.name AS bill_name, b.due_day FROM payments p JOIN bills b ON b.id = p.bill_id WHERE b.user_id = ? AND p.deleted_at IS NULL; -- User monthly state SELECT * FROM monthly_bill_state WHERE bill_id IN ( SELECT id FROM bills WHERE user_id = ? ) AND year = ? AND month = ?; ``` ### Migration System **Migration Location**: `db/database.js` (runMigrations function) **Migration Types**: 1. **Column additions**: `ALTER TABLE users ADD COLUMN column_name type` 2. **Table creations**: `CREATE TABLE IF NOT EXISTS` 3. **Index additions**: `CREATE INDEX IF NOT EXISTS` 4. **Schema rewrites**: Table rename + recreate (for breaking changes) **Security Features**: - Column name whitelist validation - SQL definition string validation - No user input in ALTER statements **Migration Log** (from `db/schema.sql` comments): - v0.2: `payments.deleted_at` column - v0.3: `idx_payments_bill_date_del` index - v0.4: `monthly_bill_state` table - v0.13: Profile columns (`display_name`, `last_password_change_at`) - v0.14: `bills.history_visibility`, `bill_history_ranges` table, `bills.interest_rate` - v0.14.4: `bills.interest_rate` column - v0.15: Cleanup worker settings - v0.17: OIDC columns (`auth_provider`, `external_subject`, `email`, `last_login_at`) - v0.17: `oidc_states` table - v0.18: Monthly income, monthly starting amounts tables - v0.18.2: Monthly starting amounts table - v0.18.3: `monthly_starting_amounts.other_amount` column - v0.18.1: Monthly income table - v0.38: Import history table - v0.39: Import sessions table - v0.40: User-scoped bills/categories - v0.41: Seeded flags (`is_seeded`) ### Entity Relationship Diagram ``` ┌─────────────────┐ │ users │ ├─────────────────┤ │ id (PK) │ │ username (U) │ │ password_hash │ │ role │ │ active │ │ ... │ └────────┬────────┘ │ │ 1:N │ │ ┌────────▼────────┐ ┌─────────────────┐ │ sessions │ │ bills │ ├─────────────────┤ ├─────────────────┤ │ id (PK) │ │ id (PK) │ │ user_id (FK) │ │ user_id (FK) │ │ expires_at │ │ ... │ │ created_at │ └────────┬────────┘ └─────────────────┘ │ │ 1:N │ ┌────────────┴────────────┐ │ │ ┌──────────▼──────────┐ ┌──────────▼──────────┐ │ categories │ │ payments │ ├─────────────────────┤ ├─────────────────────┤ │ id (PK) │ │ id (PK) │ │ user_id (FK) │ │ bill_id (FK) │ │ ... │ │ ... │ └─────────────────────┘ └──────────┬──────────┘ │ ┌────────────┴────────────┐ │ │ ┌──────────▼──────────┐ ┌──────────▼──────────┐ │ monthly_bill_state│ │ monthly_income │ ├─────────────────────┤ ├─────────────────────┤ │ id (PK) │ │ id (PK) │ │ bill_id (FK) │ │ user_id (FK) │ │ year, month (U) │ │ year, month (U) │ │ ... │ │ ... │ └─────────────────────┘ └─────────────────────┘ ``` --- ## 7. Error Handling & Troubleshooting ### Troubleshooting Matrix | Symptom | Likely Cause | Logs to Inspect | Files to Inspect | Services Involved | Recovery Steps | |---------|--------------|-----------------|------------------|-------------------|----------------| | **Login fails** | | | | | | | Invalid credentials | Wrong username/password | `server.js` console | `routes/authLogin.js`, `services/authService.js` | authService | Verify credentials, check for typos | | Session expired | Session deleted or expired | `server.js` console | `services/authService.js` | authService, session pruning worker | Re-login | | Account locked | `active = 0` | `server.js` console | `db/schema.sql` (users.active) | AuthService | Admin sets `active = 1` | | Password mismatch | Hash changed | `server.js` console | `services/authService.js` | authService | Reset password via admin | | Local login disabled | Admin disabled it | `server.js` console | `db/schema.sql` (settings) | Settings | Enable local login in Admin | | **Auth issues** | | | | | | | CSRF token invalid | Token mismatch or expired | `server.js` console | `middleware/csrf.js` | CSRF middleware | Refresh page | | Role insufficient | User role check failed | `server.js` console | `middleware/requireAuth.js` | Auth middleware | Login as admin/user | | OIDC callback fails | Provider error or config issue | `server.js` console | `routes/authOidc.js`, `services/oidcService.js` | oidcService | Check OIDC config in Admin | | **API failures** | | | | | | | 404 Not Found | Endpoint or resource missing | `server.js` console | `routes/*.js` | Route handlers | Verify endpoint, check resource ID | | 400 Bad Request | Validation error | `server.js` console | `routes/*.js`, `services/*.js` | Validation logic | Check request body, query params | | 429 Rate Limited | Too many requests | `server.js` console | `middleware/rateLimiter.js` | Rate limiter | Wait, reduce request frequency | | 500 Server Error | Unhandled exception | `server.js` console, `NODE_ENV=production` | Any | All services | Check server logs, reproduce | ### Database issues | Symptom | Likely Cause | Logs to Inspect | Files to Inspect | Services Involved | Recovery Steps | |---------|--------------|-----------------|------------------|-------------------|----------------| | Database locked | WAL checkpoint or long transaction | `server.js` console | `db/database.js` | Database connection | Wait, restart app if needed | | Schema mismatch | Migration failed | `server.js` console | `db/database.js` (runMigrations) | Database init | Check migrations, fix manually | | Migration runs repeatedly | Version not recorded | `server.js` console | `db/database.js` | Migration system | Manually record version in `schema_migrations` table | | Migration conflicts | Concurrent startup | `server.js` console | `db/database.js` | Migration system | Restart once, migrations are idempotent | | Foreign key violation | Related record deleted | `server.js` console | `db/schema.sql` | Schema | Check related data, adjust onDelete | | **Performance issues** | | | | | | | Slow queries | Missing index, complex join | `server.js` console | `db/database.js` (schema) | Database | Add indexes, optimize queries | | Memory leak | Session or cache buildup | `server.js` console | `services/*.js` | Service cleanup | Restart app, check cleanup workers | | High CPU | Cron job, import, export | `server.js` console | `workers/*.js`, `routes/*.js` | Background jobs | Schedule jobs off-peak | | **Notification issues** | | | | | | | Emails not sent | SMTP not configured or failing | `server.js` console | `services/notificationService.js` | notificationService | Configure SMTP in Admin, test | | Notification not recorded | Unique constraint violation | `server.js` console | `db/schema.sql` (notifications) | notificationService | Check notification table | | **Backup/restore issues** | | | | | | | Backup fails | Disk full, permission denied | `server.js` console | `services/backupService.js` | backupService | Check disk space, permissions | | Restore fails | Backup corrupted, wrong DB | `server.js` console | `services/backupService.js` | backupService | Verify backup integrity, restore from good backup | | Import/Export fails | Invalid file, permission | `server.js` console | `routes/export.js`, `routes/import.js` | import/export | Check file format, permissions | | **Worker failures** | | | | | | | Daily worker not running | Cron not scheduled or errored | `server.js` console | `workers/dailyWorker.js` | dailyWorker | Check cron, restart app | | Cleanup failing | Disk space, permission | `server.js` console | `services/cleanupService.js` | cleanupService | Check disk space, permissions | ### Common Error Codes and Solutions | Error Code | Status | Message | Solution | |------------|--------|---------|----------| | `AUTH_ERROR` | 401 | Not authenticated | Re-login | | `VALIDATION_ERROR` | 400 | Input validation failed | Check request body | | `FORBIDDEN` | 403 | Access denied | Login as correct role | | `NOT_FOUND` | 404 | Resource not found | Check ID | | `CONFLICT` | 409 | Duplicate entry | Check for existing record | | `RATE_LIMITED` | 429 | Too many requests | Wait or reduce requests | | `CSRF_INVALID` | 403 | CSRF token validation failed | Refresh page | | `IMPORT_REQUEST_ERROR` | 400 | Import request failed | Smaller/valid file | | `IMPORT_SERVER_ERROR` | 500 | Import server error | Check server logs | ### Migration Troubleshooting | Issue | Symptom | Cause | Resolution | |-------|---------|-------|------------| | Migration runs on every startup | `[migration] Applying v0.2`, repeated logs | Version not recorded in `schema_migrations` | Manually insert version: `INSERT INTO schema_migrations (version, description) VALUES ('v0.2', 'payments: soft-delete column')` | | Migration fails with duplicate column | `SQLITE_ERROR: duplicate column name` | Migration re-ran after partial success | Verify column exists, manually record version, restart | | Concurrent startup conflicts | `[migration] Applying`, then error | Multiple instances start simultaneously | Migrations are idempotent; restart once after all instances stable | | Version mismatch | Expected migration not applied | Version string changed between runs | Ensure version strings match exactly; update `schema_migrations` if needed | ### Log Files and Locations | Log Source | Location | Purpose | |------------|----------|---------| | **Server console** | stdout/stderr | All console.log/error output | | **Debug logs** | Set `NODE_DEBUG=express` | Express internals | | **SQL logs** | `process.env.DEBUG=better-sqlite3` | SQL queries | | **Migration logs** | `[migration] Applying X`, `[migration] Skipping already applied X` | Migration execution status | ### Recovery Procedures #### 1. Admin Locked Out **Symptom**: Can't login, no users with admin role **Recovery**: ```bash # Reset default admin password (if INIT_ADMIN_USER/PASS set) docker-compose exec app node -e " const bcrypt = require('bcryptjs'); const hash = bcrypt.hashSync('newpassword123', 12); console.log(hash); " # Then run SQL: UPDATE users SET password_hash='newhash' WHERE is_default_admin=1; ``` #### 2. Corrupted Database **Symptom**: App fails to start, database errors in logs **Recovery**: ```bash # Restore from backup cp /path/to/backup.sqlite /data/bills.db # Or rebuild from export (if available) ``` #### 3. Session Starvation **Symptom**: All users logged out, can't re-login **Recovery**: ```bash # Prune all sessions docker-compose exec app node -e " const { getDb } = require('./db/database'); const db = getDb(); db.prepare('DELETE FROM sessions').run(); console.log('Sessions pruned'); " ``` #### 4. Rate Limiter Stuck **Symptom**: All requests return 429, even after waiting **Recovery**: ```bash # Restart the app (in-memory rate limiter) docker-compose restart app # Or in Node REPL: node -e "const { resetStores } = require('./middleware/rateLimiter'); resetStores(); console.log('Limiters reset');" ``` #### 5. OIDC Configuration Broken **Symptom**: OIDC login failing, discovery errors **Recovery**: ```bash # Clear OIDC client cache (forces re-discovery) node -e " const { invalidateClientCache } = require('./services/oidcService'); invalidateClientCache(); console.log('Client cache invalidated'); " ``` ### Debug Commands #### Check Database Integrity ```bash docker-compose exec app sqlite3 /data/bills.db "PRAGMA integrity_check;" ``` #### View Active Sessions ```bash docker-compose exec app node -e " const { getDb } = require('./db/database'); const db = getDb(); const sessions = db.prepare('SELECT s.id, u.username, s.expires_at FROM sessions s JOIN users u ON u.id = s.user_id WHERE s.expires_at > datetime(\"now\")').all(); console.log(sessions); " ``` #### List All Users ```bash docker-compose exec app node -e " const { getDb } = require('./db/database'); const db = getDb(); const users = db.prepare('SELECT id, username, role, active, auth_provider FROM users').all(); console.log(users); " ``` #### View Bill Counts by Category ```bash docker-compose exec app node -e " const { getDb } = require('./db/database'); const db = getDb(); const counts = db.prepare('SELECT c.name, COUNT(b.id) as count FROM categories c LEFT JOIN bills b ON b.category_id = c.id GROUP BY c.id').all(); console.table(counts); " ``` --- ## 8. Code Navigation Index ### Feature-to-File Mapping | Feature | Frontend Files | Backend Files | Services | Middleware | Tests | |---------|----------------|---------------|----------|------------|-------| | **User Authentication** | `client/pages/LoginPage.jsx`, `client/hooks/useAuth.jsx`, `client/api.js` | `routes/authLogin.js`, `routes/auth.js`, `routes/authOidc.js` | `authService.js`, `oidcService.js` | `requireAuth.js`, `csrf.js` | `test-functional.js`, `scripts/test-oidc-smoke.js` | | **Monthly Tracker** | `client/pages/TrackerPage.jsx`, `client/components/MobileBillRow.jsx` | `routes/tracker.js` | `statusService.js`, `statusRuntime.js` | `requireAuth.js`, `requireUser.js` | `test-functional.js` | | **Bill CRUD** | `client/pages/BillsPage.jsx`, `client/components/BillModal.jsx` | `routes/bills.js` | - | `requireAuth.js`, `requireUser.js`, `csrf.js` | `test-functional.js` | | **Payment Recording** | `client/components/StatusBadge.jsx`, `client/pages/TrackerPage.jsx` | `routes/payments.js`, `routes/bills.js` (toggle-paid) | - | `requireAuth.js`, `requireUser.js`, `csrf.js` | `test-functional.js` | | **Categories** | `client/pages/CategoriesPage.jsx` | `routes/categories.js` | - | `requireAuth.js`, `requireUser.js`, `csrf.js` | - | | **Monthly State Overrides** | `client/pages/BillsPage.jsx`, `client/components/TrackerPage.jsx` | `routes/bills.js` (monthly-state) | - | `requireAuth.js`, `requireUser.js`, `csrf.js` | - | | **Calendar View** | `client/pages/CalendarPage.jsx` | `routes/calendar.js` | `statusService.js` | `requireAuth.js`, `requireUser.js` | - | | **Summary View** | `client/pages/SummaryPage.jsx` | `routes/summary.js` | - | `requireAuth.js`, `requireUser.js` | - | | **Analytics** | `client/pages/AnalyticsPage.jsx` | `routes/analytics.js` | - | `requireAuth.js`, `requireUser.js` | - | | **User Profile** | `client/pages/ProfilePage.jsx` | `routes/user.js`, `routes/profile.js` | `authService.js` | `requireAuth.js`, `requireUser.js`, `passwordLimiter` | - | | **App Settings** | `client/pages/SettingsPage.jsx` | `routes/settings.js` | - | `requireAuth.js`, `requireUser.js` | - | | **Notifications** | `client/pages/ProfilePage.jsx` (settings) | `routes/notifications.js` | `notificationService.js`, `statusRuntime.js` | `requireAuth.js` | - | | **Data Import** | `client/pages/DataPage.jsx` | `routes/import.js` | `spreadsheetImportService.js`, `userDbImportService.js` | `requireAuth.js`, `requireUser.js`, `importLimiter` | `scripts/test-import.js` | | **Data Export** | `client/pages/DataPage.jsx` | `routes/export.js` | - | `requireAuth.js`, `requireUser.js`, `exportLimiter` | - | | **Admin User Management** | `client/pages/AdminPage.jsx` | `routes/admin.js` (users) | `authService.js` | `requireAuth.js`, `requireAdmin.js`, `adminActionLimiter` | - | | **Admin Backups** | `client/pages/AdminPage.jsx` (backups tab) | `routes/admin.js` (backups) | `backupService.js`, `backupScheduler.js` | `requireAuth.js`, `requireAdmin.js`, `backupOperationLimiter` | - | | **Admin OIDC Config** | `client/pages/AdminPage.jsx` (auth tab) | `routes/admin.js` (auth-mode) | `oidcService.js` | `requireAuth.js`, `requireAdmin.js` | `scripts/test-oidc-smoke.js` | | **Admin Cleanup** | `client/pages/AdminPage.jsx` (cleanup tab) | `routes/admin.js` (cleanup) | `cleanupService.js` | `requireAuth.js`, `requireAdmin.js` | - | | **System Status** | `client/pages/StatusPage.jsx` | `routes/status.js` | `statusRuntime.js`, `statusService.js` | `requireAuth.js`, `requireAdmin.js` | - | ### Component File Tree ``` client/ ├── components/ │ ├── layout/ │ │ ├── Layout.jsx # Main layout wrapper │ │ ├── Sidebar.jsx # Navigation sidebar │ │ ├── BrandBlock.jsx # App branding (logo, title, version) │ │ └── NavPill.jsx # Nav link item │ ├── ui/ │ │ ├── button.jsx # shadcn button │ │ ├── input.jsx # shadcn input │ │ ├── card.jsx # shadcn card │ │ ├── table.jsx # shadcn table │ │ ├── tabs.jsx # shadcn tabs │ │ ├── dialog.jsx # shadcn dialog │ │ ├── badge.jsx # shadcn badge │ │ ├── switch.jsx # shadcn switch │ │ ├── select.jsx # shadcn select │ │ ├── dropdown-menu.jsx # shadcn dropdown │ │ ├── label.jsx # shadcn label │ │ ├── input-dialog.jsx # Custom dialog with input │ │ ├── confirm-dialog.jsx # Confirmation dialog │ │ ├── alert-dialog.jsx # shadcn alert dialog │ │ ├── separator.jsx # shadcn separator │ │ ├── tooltip.jsx # shadcn tooltip │ │ ├── checkbox.jsx # shadcn checkbox │ │ └── theme-toggle.jsx # Theme switcher │ ├── BillsTableInner.jsx # Bills table component │ ├── MobileBillRow.jsx # Mobile bill row │ ├── MobileTrackerRow.jsx # Mobile tracker row │ ├── StatusBadge.jsx # Payment status badge │ ├── SummaryCard.jsx # Summary statistics card │ ├── MarkdownText.jsx # Markdown renderer │ ├── ReleaseNotesDialog.jsx # Release notes modal │ └── ... ├── pages/ │ ├── LoginPage.jsx # Login page │ ├── TrackerPage.jsx # Monthly tracker │ ├── BillsPage.jsx # Bill CRUD │ ├── CategoriesPage.jsx # Category management │ ├── CalendarPage.jsx # Calendar view │ ├── SummaryPage.jsx # Monthly summary │ ├── AnalyticsPage.jsx # Analytics charts │ ├── ProfilePage.jsx # User profile │ ├── SettingsPage.jsx # App settings │ ├── DataPage.jsx # Import/export │ ├── AdminPage.jsx # Admin panel │ ├── StatusPage.jsx # System status │ ├── AboutPage.jsx # Version/info │ └── ReleaseNotesPage.jsx # Release notes ├── hooks/ │ └── useAuth.jsx # Auth state hook ├── contexts/ │ └── ThemeContext.jsx # Theme state ├── api.js # API client ├── App.jsx # Router config ├── main.jsx # React entry └── lib/ ├── utils.js # Utility functions └── version.js # Version constants ``` ### Service Layer Dependencies ``` services/ ├── authService.js # Session management, login/logout ├── oidcService.js # Authentik OIDC integration ├── backupService.js # SQLite backup/restore ├── backupScheduler.js # Scheduled backups ├── notificationService.js # Email notifications ├── cleanupService.js # Cleanup tasks ├── spreadsheetImportService.js # XLSX import ├── userDbImportService.js # SQLite user import ├── statusRuntime.js # Worker/runtime status └── statusService.js # Tracker status calculations ``` ### Middleware Chain by Route | Route Prefix | Middleware Chain | |--------------|------------------| | `/api/auth/login` | `loginLimiter` | | `/api/auth` | `csrfMiddleware` | | `/api/auth/oidc` | `csrfMiddleware`, `oidcLimiter` | | `/api/tracker` | `csrfMiddleware`, `requireAuth`, `requireUser` | | `/api/bills` | `csrfMiddleware`, `requireAuth`, `requireUser` | | `/api/payments` | `csrfMiddleware`, `requireAuth`, `requireUser` | | `/api/categories` | `csrfMiddleware`, `requireAuth`, `requireUser` | | `/api/settings` | `csrfMiddleware`, `requireAuth`, `requireUser` | | `/api/user` | `csrfMiddleware`, `requireAuth`, `requireUser` | | `/api/calendar` | `csrfMiddleware`, `requireAuth`, `requireUser` | | `/api/summary` | `csrfMiddleware`, `requireAuth`, `requireUser` | | `/api/monthly-starting-amounts` | `csrfMiddleware`, `requireAuth`, `requireUser` | | `/api/analytics` | `csrfMiddleware`, `requireAuth`, `requireUser` | | `/api/notifications` | `csrfMiddleware`, `requireAuth` | | `/api/profile` | `csrfMiddleware`, `requireAuth`, `requireUser`, `passwordLimiter` | | `/api/admin` | `csrfMiddleware`, `requireAuth`, `requireAdmin`, `adminActionLimiter` | | `/api/export` | `csrfMiddleware`, `requireAuth`, `requireUser`, `exportLimiter` | | `/api/import` | `csrfMiddleware`, `requireAuth`, `requireUser`, `importLimiter` | | `/api/status` | `csrfMiddleware`, `requireAuth`, `requireAdmin` | | `/api/about` | (none) | | `/api/version` | (none) | ### Test Files | Test File | Purpose | |-----------|---------| | `test-functional.js` | Functional tests for all features | | `run-functional-test.js` | Test runner | | `scripts/test-import.js` | Import functionality test | | `scripts/test-oidc-smoke.js` | OIDC configuration smoke test | | `scripts/test-cookie-options.js` | Cookie options test | --- ## 9. Infrastructure & Deployment ### Docker Setup #### Dockerfile ```dockerfile # Base image FROM node:20-alpine AS base # Install dependencies FROM base AS deps RUN apk add --no-cache libc6-compat WORKDIR /app COPY package.json package-lock.json ./ RUN npm ci --only=production # Build stage FROM base AS builder WORKDIR /app COPY --from=deps /app/node_modules ./node_modules COPY . . RUN npm run build # Production FROM base AS production WORKDIR /app COPY --from=deps /app/node_modules ./node_modules COPY --from=builder /app/dist ./dist COPY --from=builder /app/*.js ./ COPY --from=builder /app/scripts ./scripts COPY --from=builder /app/db ./db COPY --from=builder /app/middleware ./middleware COPY --from=builder /app/routes ./routes COPY --from=builder /app/services ./services COPY --from=builder /app/workers ./workers COPY --from=builder /app/client ./client COPY --from=builder /app/.npmrc ./ COPY --from=builder /app/postcss.config.js ./ COPY --from=builder /app/tailwind.config.js ./ COPY --from=builder /app/vite.config.js ./ COPY --from=builder /app/index.html ./ COPY --from=builder /app/.env.example ./ # Environment variables ENV NODE_ENV=production ENV PORT=3000 # Data directories VOLUME ["/data", "/backups"] # Entry point ENTRYPOINT ["./docker-entrypoint.sh"] CMD ["node", "server.js"] ``` #### docker-compose.yml ```yaml version: '3.8' services: app: image: bill-tracker:latest container_name: bill-tracker ports: - "3030:3000" volumes: - ./data:/data - ./backups:/backups environment: - NODE_ENV=production - PORT=3000 - DB_PATH=/data/bills.db - BACKUP_PATH=/data/backups - HTTPS=true - COOKIE_SECURE=true # OIDC (optional) # - OIDC_ISSUER_URL= # - OIDC_CLIENT_ID= # - OIDC_CLIENT_SECRET= # - OIDC_REDIRECT_URI= restart: unless-stopped healthcheck: test: ["CMD", "wget", "--no-verbose", "--tries=1", "--spider", "http://localhost:3000/api/version"] interval: 30s timeout: 10s retries: 3 start_period: 30s ``` ### Environment Variables | Variable | Default | Required | Description | |----------|---------|----------|-------------| | `PORT` | `3000` | No | API server port | | `NODE_ENV` | `production` | No | Environment mode | | `DB_PATH` | `db/bills.db` | No | SQLite database path | | `BACKUP_PATH` | `backups/` | No | Backup directory path | | `HTTPS` | `false` | No | Enable HTTPS (for HSTS, secure cookies) | | `COOKIE_SECURE` | `false` | No | Force secure cookies | | `CORS_ORIGIN` | (disabled) | No | CORS allowed origins (comma-separated) | | `INIT_ADMIN_USER` | `admin` | No | Initial admin username (first run only) | | `INIT_ADMIN_PASS` | `admin123` | No | Initial admin password (first run only) | | `OIDC_ISSUER_URL` | - | No | OIDC issuer URL (fallback) | | `OIDC_CLIENT_ID` | - | No | OIDC client ID (fallback) | | `OIDC_CLIENT_SECRET` | - | No | OIDC client secret (fallback) | | `OIDC_REDIRECT_URI` | - | No | OIDC redirect URI (fallback) | | `OIDC_SCOPES` | `openid email profile groups` | No | OIDC scopes (fallback) | | `OIDC_ADMIN_GROUP` | - | No | OIDC admin group name (fallback) | | `OIDC_AUTO_PROVISION` | `true` | No | Auto-create users from OIDC | | `OIDC_PROVIDER_NAME` | `authentik` | No | Provider name (fallback) | ### Ports and Services | Port | Service | Purpose | |------|---------|---------| | `3000` | Express | Main API server | | `3030` | Host (Docker) | Exposed port for app | ### Monitoring & Logging #### Runtime Status (`statusRuntime.js`) | Status Type | Key | Description | |-------------|-----|-------------| | Worker | `last_worker_run_at` | Last daily worker execution | | Worker | `last_worker_status` | `success` or `error` | | Worker | `last_worker_error` | Error message if failed | | Notification | `last_notification_send_at` | Last email send | | Notification | `last_notification_error` | Last email error | | Runtime | `last_error_at` | Last error timestamp | | Runtime | `last_error_message` | Last error message | #### Health Check ```bash # Health endpoint (public) GET /api/version # Docker healthcheck wget --no-verbose --tries=1 --spider http://localhost:3000/api/version ``` ### CI/CD Pipeline **Current**: Manual deployment via `deploy.sh` **Deploy Script**: ```bash #!/bin/bash # deploy.sh # Build frontend npm run build # Sync to server (rsync) rsync -avz dist/ user@server:/var/www/bill-tracker/ rsync -avz node_modules/ user@server:/var/www/bill-tracker/ # Restart server (migrations run automatically on startup) ssh user@server "pm2 restart bill-tracker" ``` **Deployment Notes**: - Database migrations run automatically on startup (no manual intervention required) - Migrations are idempotent: safe to `git pull` and restart repeatedly - After `git pull`, migrations are applied in version order during startup - Check `[migration]` logs for applied/skipped migration status ### Security Considerations | Feature | Implementation | |---------|----------------| | **Password Hashing** | bcrypt with cost factor 12 | | **Session Storage** | SQLite with 7-day expiry | | **CSRF Protection** | Double-submit cookie pattern | | **Rate Limiting** | In-memory express-rate-limit | | **SQL Injection** | Parameterized queries (prepared statements) | | **XSS Protection** | CSP with nonce, no inline scripts in HTML emails | | **CORS** | Disabled by default, configurable via env | | **HSTS** | Only when HTTPS=true | | **Secure Cookies** | httpOnly, sameSite=strict, secure when HTTPS | | **OIDC Security** | PKCE, state/nonce, JWKS signature verification | | **Backup Security** | chmod 600, SHA-256 checksums, restrictive dir (0o700) | ### Database Backups **Manual**: ```bash # Create backup curl -X POST http://localhost:3000/api/admin/backups # List backups curl http://localhost:3000/api/admin/backups # Download backup curl -o backup.sqlite http://localhost:3000/api/admin/backups/backup-id.sqlite ``` **Scheduled** (Admin panel): - Enable: `backup_schedule_enabled` - Frequency: daily/weekly - Time: HH:MM - Retention: N backups ### Troubleshooting Commands #### View Logs (Docker) ```bash docker-compose logs -f app ``` #### Restart App (Docker) ```bash docker-compose restart app ``` #### Rebuild Image ```bash docker-compose build --no-cache docker-compose up -d ``` #### Enter Container Shell ```bash docker-compose exec app sh ``` #### Check Database Size ```bash docker-compose exec app ls -lh /data/bills.db ``` --- ## 10. Sequence Flows ### Database Initialization & Migration Flow ``` ┌─────────────────────────────────────────────────────────────┐ │ Application Startup (server.js) │ │ 1. require('./db/database.js') │ │ 2. getDb() called │ └─────────────────────────────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────┐ │ db/database.js - getDb() │ │ 3. Check if db already initialized (db variable) │ │ 4. If not: call initSchema() │ └─────────────────────────────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────┐ │ db/database.js - initSchema() │ │ 5. Read db/schema.sql and execute │ │ 6. Create schema_migrations table if not exists │ │ CREATE TABLE IF NOT EXISTS schema_migrations ( │ │ id INTEGER PRIMARY KEY AUTOINCREMENT, │ │ version TEXT NOT NULL UNIQUE, │ │ description TEXT NOT NULL, │ │ applied_at TEXT NOT NULL DEFAULT (datetime('now')) │ │ ) │ │ 7. Call runMigrations() │ └─────────────────────────────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────┐ │ db/database.js - runMigrations() │ │ 8. Define all migrations with version/description/run │ │ 9. For each migration: │ │ a. Check hasMigrationBeenApplied(migration.version) │ │ b. If NOT applied: │ │ i. Log: "Applying {version}: {description}" │ │ ii. Execute migration.run() │ │ iii. Call recordMigration(version, description) │ │ c. If already applied: │ │ i. Log: "Skipping already applied {version}" │ └─────────────────────────────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────┐ │ Application continues │ │ 10. Seed defaults (seedDefaults()) │ │ 11. Server starts listening on port │ └─────────────────────────────────────────────────────────────┘ ``` **Migration Flow Details**: 1. **First startup after upgrade**: All pending migrations execute in version order 2. **Subsequent startups**: Skipped migrations logged but no SQL executed 3. **Idempotent**: Safe to run `git pull && npm start` repeatedly 4. **Audit trail**: Every applied migration recorded in `schema_migrations` ### Login Flow (Local) ``` ┌─────────────────────────────────────────────────────────────┐ │ User │ │ 1. Opens /login.html │ │ 2. Enters username + password │ │ 3. Clicks "Login" button │ └─────────────────────────────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────┐ │ Frontend (LoginPage.jsx) │ │ 4. Calls apiPost('/api/auth/login', {username, password}) │ └─────────────────────────────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────┐ │ Backend (server.js) │ │ 5. POST /api/auth/login │ │ 6. loginLimiter checks IP (skip if no users exist) │ │ 7. authLogin.js handler runs │ └─────────────────────────────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────┐ │ Service (authService.login) │ │ 8. Query user by username │ │ 9. Check user.active === 1 │ │ 10. Check auth_provider === 'local' │ │ 11. bcrypt.compare(password, password_hash) │ │ 12. If match: create session, return {sessionId, user} │ └─────────────────────────────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────┐ │ Response │ │ 13. Set cookie: bt_session= │ │ 14. JSON: {user: {id, username, role, ...}} │ │ 15. Set CSRF token cookie (bt_csrf_token) │ └─────────────────────────────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────┐ │ Frontend (useAuth hook) │ │ 16. Stores user in context │ │ 17. Redirects to /tracker or / │ └─────────────────────────────────────────────────────────────┘ ``` ### OIDC Login Flow ``` ┌─────────────────────────────────────────────────────────────┐ │ User │ │ 1. Clicks "Login with Authentik" button │ └─────────────────────────────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────┐ │ Frontend (LoginPage.jsx) │ │ 2. Calls apiGet('/api/auth/oidc/login?redirect_to=/tracker')│ └─────────────────────────────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────┐ │ Backend (authOidc.js) │ │ 3. GET /api/auth/oidc/login │ │ 4. isOidcLoginActive() check │ │ 5. createLoginState(redirect_to) │ │ • Generates PKCE code_verifier │ │ • Generates nonce │ │ • Stores in oidc_states (expires 5 min) │ │ 6. buildAuthorizationUrl(config, state) │ └─────────────────────────────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────┐ │ Response │ │ 7. HTTP 302 redirect to OIDC provider authorization URL │ │ • Includes: client_id, redirect_uri, response_type=code │ │ • Includes: state (login state ID), nonce │ │ • Includes: code_challenge, code_challenge_method=S256 │ │ • Includes: scopes (openid, email, profile, groups) │ └─────────────────────────────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────┐ │ OIDC Provider (Authentik) │ │ 8. User authenticates (credentials) │ │ 9. Provider creates ID token │ │ 10. Redirects to redirect_uri with code + state │ └─────────────────────────────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────┐ │ Backend (authOidc.js) │ │ 11. GET /api/auth/oidc/callback │ │ 12. oidcLimiter checks IP │ │ 13. consumeLoginState(state) │ │ • Validates expiry │ │ • Returns: {nonce, code_verifier, redirect_to} │ │ 14. exchangeAndVerifyTokens(config, code, stateId, savedState) │ │ • POST to token endpoint with code + client_secret │ │ • Verifies JWT signature via JWKS │ │ • Validates: iss, aud, exp, nonce, state │ │ 15. findOrProvisionUser(claims, config) │ │ • Look up by sub (external_subject) │ │ • Look up by email if email_verified=true │ │ • Auto-provision if enabled │ │ • Map groups to role (admin if in admin_group) │ └─────────────────────────────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────┐ │ Service (authService.createSession) │ │ 16. Create session for user │ │ 17. Set cookie: bt_session= │ └─────────────────────────────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────┐ │ Response │ │ 18. HTTP 302 redirect to redirect_to (or /) │ └─────────────────────────────────────────────────────────────┘ ``` ### Authenticated API Request Flow ``` ┌─────────────────────────────────────────────────────────────┐ │ Frontend (api.js) │ │ 1. apiGet('/api/tracker', {year, month}) │ │ 2. Retrieve CSRF token from bt_csrf_token cookie │ │ 3. Set header: x-csrf-token= │ │ 4. Set header: cookie: bt_session= │ │ 5. Send request to backend │ └─────────────────────────────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────┐ │ Backend (server.js) │ │ 6. requestLooksHttps(req) │ │ • Checks: req.secure, x-forwarded-proto │ │ 7. csrfTokenProvider sets cookie (if not present) │ │ 8. route middleware chain runs │ └─────────────────────────────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────┐ │ Middleware (requireAuth) │ │ 9. getSessionUser(cookie.bt_session) │ │ • Query sessions + users table │ │ • Check: expires_at > now, active = 1 │ │ 10. If valid: attach req.user, next() │ │ If invalid: return 401 {error: 'Not authenticated'} │ └─────────────────────────────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────┐ │ Middleware (requireUser) │ │ 11. Check role is 'user' or 'admin' │ │ 12. Check not default admin (no tracker access) │ │ 13. next() │ └─────────────────────────────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────┐ │ Middleware (csrfMiddleware) │ │ 14. validateCsrfToken(req) │ │ • Check header x-csrf-token matches cookie │ │ • Check query.csrf_token │ │ • Check body.csrf_token │ │ 15. If valid: next() │ │ If invalid: return 403 {error: 'CSRF token validation failed'} │ └─────────────────────────────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────┐ │ Route Handler (tracker.js) │ │ 16. ensureUserDefaultCategories(req.user.id) │ │ 17. db.prepare('SELECT * FROM bills WHERE user_id = ?') │ │ 18. Build tracker rows │ │ 19. Return JSON response │ └─────────────────────────────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────┐ │ Frontend (useAuth hook) │ │ 20. Update state with response │ │ 21. Re-render component │ └─────────────────────────────────────────────────────────────┘ ``` ### Background Worker Flow (Daily) ``` ┌─────────────────────────────────────────────────────────────┐ │ Server Start (server.js) │ │ 1. main() │ │ 2. Check if users exist, create admin if needed │ │ 3. workers/dailyWorker.js start() │ └─────────────────────────────────────────────────────────────┘ │ ▒ ┌─────────────────────────────────────────────────────────────┐ │ Worker (dailyWorker.js) │ │ 4. markWorkerStarted(nextDailyRunIso()) │ │ 5. runDailyTasks() │ │ • Prune expired sessions │ │ • Run notifications (email) │ │ • Run cleanup (temp files, import sessions) │ │ • Auto-mark autopay bills as assumed_paid │ │ 6. markWorkerSuccess(nextDailyRunIso()) │ └─────────────────────────────────────────────────────────────┘ │ ▒ ┌─────────────────────────────────────────────────────────────┐ │ Cron Job (node-cron) │ │ 7. cron.schedule('0 6 * * *', ...) │ │ Runs daily at 6:00 AM │ │ 8. Same runDailyTasks() as above │ └─────────────────────────────────────────────────────────────┘ ``` ### Notification Flow ``` ┌─────────────────────────────────────────────────────────────┐ │ Daily Worker (dailyWorker.js) │ │ 1. runNotifications() │ └─────────────────────────────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────┐ │ Notification Service (notificationService.js) │ │ 2. Check SMTP enabled, host configured │ │ 3. Get recipients (users with notifications_enabled=1 OR │ │ global recipient configured) │ │ 4. For each bill: │ │ • Calculate due date for this month │ │ • Determine notification type (due_3d, due_1d, due_today,│ │ overdue) │ │ • Check if already sent today (notifications table) │ │ • Check user notification preferences │ │ • Build HTML email template │ │ • Send email via nodemailer │ │ • Record in notifications table (to prevent duplicates) │ └─────────────────────────────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────┐ │ Email (Nodemailer) │ │ 5. SMTP transport sendMail() │ │ 6. Return success/error │ └─────────────────────────────────────────────────────────────┘ ``` --- ## Version History | Version | Date | Changes | |---------|------|---------| | **0.19.0** | 2026-05-09 | Complete Engineering Reference Manual | | **0.18.3** | 2026-05-08 | Added `monthly_starting_amounts.other_amount` column | | **0.18.2** | 2026-05-08 | Added `monthly_starting_amounts` table | | **0.18.1** | 2026-05-08 | Added `monthly_income` table | | **0.17** | 2026-05-08 | OIDC columns (`auth_provider`, `external_subject`, `email`, `last_login_at`), `oidc_states` table | | **0.14** | 2026-05-08 | Added `history_visibility`, `bill_history_ranges`, `interest_rate` | | **0.13** | 2026-05-08 | Profile columns (`display_name`, `last_password_change_at`) | | **0.12** | 2026-05-08 | Bill history ranges table | | **0.11** | 2026-05-08 | Import history table | | **0.10** | 2026-05-08 | Import sessions table | | **0.4** | 2026-05-08 | `monthly_bill_state` table | | **0.2** | 2026-05-08 | `payments.deleted_at` column | --- ## Quick Reference ### Critical Endpoints | Method | Endpoint | Auth | Purpose | |--------|----------|------|---------| | POST | `/api/auth/login` | None | Local login | | GET | `/api/auth/oidc/login` | None | OIDC login start | | GET | `/api/tracker` | User | Monthly tracker data | | GET | `/api/bills` | User | List bills | | GET | `/api/admin/users` | Admin | List users | | POST | `/api/admin/backups` | Admin | Create backup | | GET | `/api/version` | None | Version info | | GET | `/api/about` | None | About info | ### Critical Settings | Key | Default | Description | |-----|---------|-------------| | `local_login_enabled` | `true` | Enable username/password login | | `oidc_login_enabled` | `false` | Enable OIDC login | | `oidc_issuer_url` | - | OIDC provider URL | | `oidc_client_id` | - | OIDC client ID | | `oidc_client_secret` | - | OIDC client secret | | `oidc_redirect_uri` | - | OIDC redirect URI | | `oidc_admin_group` | - | OIDC group for admin access | | `oidc_auto_provision` | `true` | Auto-create users from OIDC | | `notify_smtp_enabled` | `false` | Enable email notifications | | `notify_smtp_host` | - | SMTP host | | `notify_smtp_port` | `587` | SMTP port | | `backup_enabled` | `false` | Enable manual backups | | `backup_schedule_enabled` | `false` | Enable scheduled backups | ### Database Tables Reference | Table | Purpose | |-------|---------| | `users` | User accounts | | `sessions` | Active sessions | | `bills` | Bill records | | `categories` | Bill categories | | `payments` | Payment records | | `monthly_bill_state` | Monthly bill overrides | | `settings` | Application settings | | `notifications` | Notification history | | `oidc_states` | OIDC login state | | `bill_history_ranges` | History visibility ranges | | `monthly_income` | Monthly income records | | `monthly_starting_amounts` | Starting balance records | | `import_sessions` | Import preview sessions | | `import_history` | Import history | --- **This document is the canonical reference for the Bill Tracker system.** *Last updated: 2026-05-09* *Author: Bishop (code reviewer and architecture validator)*