BillTracker/docs/Engineering_Reference_Manua...

4016 lines
163 KiB
Markdown
Raw Normal View History

2026-05-09 13:03:36 -05:00
# Engineering Reference Manual — Bill Tracker
**Status:** Complete
**Last Updated:** 2026-05-10
2026-05-09 13:03:36 -05:00
**Owner:** Bishop
**Version:** 0.23.1
---
## Version 0.23.1 Update
### Migration Rollback Feature (2026-05-10)
**Added:** Database migration rollback capability with transaction support, rollback SQL definitions, and admin API endpoint.
**Files Modified:**
- `db/database.js` — Added `rollbackMigration()` function, `ROLLBACK_SQLS` map, `hasRollbackSQL()`, `getRollbackSQL()`
- `routes/admin.js` — Added `POST /api/admin/migrations/rollback` endpoint
- `client/lib/version.js` — Version bumped to 0.23.1 with rollback highlights
- `package.json` — Version bumped to 0.23.1
### New Functions in database.js
**File:** `db/database.js`
#### RollbackSQLS Map
```javascript
const ROLLBACK_SQLS = {
'v0.44': [
'DROP INDEX IF EXISTS idx_bills_due_date',
'DROP INDEX IF EXISTS idx_payments_bill_id',
'DROP INDEX IF EXISTS idx_users_email',
],
'v0.45': [
'DROP TABLE IF EXISTS audit_log',
],
'v0.46': [
'ALTER TABLE bills DROP COLUMN cycle_start_day',
'ALTER TABLE bills DROP COLUMN cycle_end_day',
],
};
```
#### rollbackMigration(version) Function
**Signature:** `rollbackMigration(version: string) => { success: boolean, error?: string, message?: string }`
**Description:** Rolls back a previously applied migration by executing its rollback SQL statements within a transaction.
**Error Codes:**
- `NOT_APPLIED` (404): Migration hasn't been applied to the database
- `ROLLBACK_NOT_SUPPORTED` (422): No rollback SQL defined for this migration version
- `NO_ROLLBACK_STATEMENTS`: Migration has empty rollback SQL array
**Audit Events:**
- `migration.rollback`: Successful rollback with statement count
- `migration.rollback_failure`: Failed rollback with error details
**Security:** Admin-only endpoint via `requireAuth` + `requireAdmin` middleware
### API Endpoint
**Endpoint:** `POST /api/admin/migrations/rollback`
**Authentication:** Admin-only (`requireAuth` + `requireAdmin`)
**Request Body:**
```json
{
"version": "v0.46"
}
```
**Success Response (200):**
```json
{
"success": true,
"version": "v0.46",
"statements": 2
}
```
**Error Responses:**
**NOT_APPLIED (404):**
```json
{
"success": false,
"error": "NOT_APPLIED",
"message": "Migration v0.46 has not been applied"
}
```
**ROLLBACK_NOT_SUPPORTED (422):**
```json
{
"success": false,
"error": "ROLLBACK_NOT_SUPPORTED",
"message": "Rollback not supported for migration v0.40"
}
```
### Supported Rollback Versions
| Version | Description | Rollback SQL | Risk Level |
|---------|-------------|--------------|------------|
| v0.44 | Performance indexes | Drop 3 indexes | LOW - Indexes can be recreated |
| v0.45 | Audit log table | Drop table | MEDIUM - Data loss, but low-impact table |
| v0.46 | Cycle tracking columns | Drop 2 columns | LOW - Column data lost, but recoverable |
### Testing Rollback
**Docker Test:**
```bash
docker exec bill-tracker node -e "
const {getDb,rollbackMigration}=require('./db/database');
getDb();
console.log(JSON.stringify(rollbackMigration('v0.46')));
"
```
**Expected Output (v0.46 success):**
```json
{
"success": true,
"version": "v0.46",
"statements": 2
}
```
**Expected Output (v0.40 - no rollback):**
```json
{
"success": false,
"error": "ROLLBACK_NOT_SUPPORTED",
"message": "Rollback not supported for migration v0.40"
}
```
---
## Error Boundaries (2026-05-09)
**Added:** React Error Boundary component wrapping all routes for graceful error handling.
**Changes:**
- `client/components/ErrorBoundary.jsx` — New component with fallback UI and recovery options
- `client/App.jsx` — All routes wrapped with `<ErrorBoundary>`
### Component Location
**File:** `client/components/ErrorBoundary.jsx`
### Features
| Feature | Description |
|---------|-------------|
| **Error Capture** | Catches JavaScript errors in child components |
| **Fallback UI** | Displays friendly error message with details |
| **Try Again** | Resets component state without reloading |
| **Reload Page** | Full page reload to recover |
| **Error Details** | Shows error message and component stack for debugging |
### How It Works
1. **Error Detection:** When a child component throws an error, `componentDidCatch()` captures it
2. **State Update:** `getDerivedStateFromError()` sets `hasError=true`
3. **Fallback Render:** Component renders fallback UI instead of crashing
4. **Recovery:** User clicks "Try Again" (reset) or "Reload Page" (full refresh)
### Route Coverage
All routes are wrapped with ErrorBoundary:
```jsx
// Public routes
<Route path="/login" element={<ErrorBoundary><LoginPage /></ErrorBoundary>} />
<Route path="/about" element={<ErrorBoundary}><AboutPage /></ErrorBoundary>} />
// User routes
<Route index element={<ErrorBoundary><TrackerPage /></ErrorBoundary>} />
<Route path="bills" element={<ErrorBoundary><BillsPage /></ErrorBoundary>} />
<Route path="categories" element={<ErrorBoundary><CategoriesPage /></ErrorBoundary>} />
// ... all other user routes
// Admin routes
<Route
path="/admin"
element={<RequireAuth role="admin"><ErrorBoundary><AdminPage /></ErrorBoundary></RequireAuth>}
/>
```
### Developer Guidance
**Do not suppress errors in production.** ErrorBoundary provides recovery without exposing internal details. If you need to debug:
```javascript
// Check browser console for error details
// The component stack is logged and displayed in the fallback UI
console.error('ErrorBoundary caught:', error, componentStack);
```
---
## Version 0.19.2 Update
### Security Fixes (2026-05-09)
**Added:** 6 security hardening improvements addressing path traversal, admin route bypass, content redaction, error message leaks, race conditions, and password validation.
**Changes:**
- `routes/aboutAdmin.js` — allowlist approach, enhanced redaction patterns, generic error handling
- `client/App.jsx` — admin prop passed to AboutPage on `/admin/about` route
- `client/pages/AboutPage.jsx``admin` prop, dual API call logic (`api.about()` vs `api.aboutAdmin()`)
- `server.js` — transaction wrapping for regular user creation, 8-char password validation
### 🔴 #1: Path Traversal Fix
**File:** `routes/aboutAdmin.js`
**Fix:** Replaced `sanitizePath()` function with hardcoded `ALLOWED_FILES` map. Only `FUTURE.md` and `DEVELOPMENT_LOG.md` are servable. No path resolution from user input.
**Before:**
```javascript
const sanitizePath = (fileName) => {
const base = path.resolve(__dirname, '..');
const filePath = path.resolve(base, fileName);
if (!filePath.startsWith(base)) {
throw new Error('Invalid path');
}
return filePath;
};
```
**After:**
```javascript
const ALLOWED_FILES = {
'FUTURE.md': path.resolve(__dirname, '..', 'FUTURE.md'),
'DEVELOPMENT_LOG.md': path.resolve(__dirname, '..', 'DEVELOPMENT_LOG.md'),
};
```
**Impact:** Path traversal attacks prevented entirely via allowlist approach.
### 🟠 #2: Admin Route Bypass Fix
**Files:** `client/App.jsx`, `client/pages/AboutPage.jsx`
**Fix:** `/admin/about` route passes `<AboutPage admin />` prop. AboutPage conditionally calls `api.aboutAdmin()` when `admin=true`. Public `/about` continues to call `api.about()`.
**Before:** Single AboutPage component served both public and admin content via same endpoint.
**After:**
```javascript
// App.jsx
<Route
path="/admin/about"
element={
<RequireAuth role="admin">
<AdminShell>
<AboutPage admin />
</AdminShell>
</RequireAuth>
}
/>
// AboutPage.jsx
const load = useCallback(async () => {
setLoading(true);
try {
setAbout(admin ? await api.aboutAdmin() : await api.about());
} finally {
setLoading(false);
}
}, [admin]);
```
**Impact:** Admin-only content isolated from public endpoint. No information leakage.
### 🟠 #3: Sensitive Info Redaction
**File:** `routes/aboutAdmin.js`
**Fix:** Expanded `redactSensitiveContent()` with additional patterns:
- File paths (`.home`, `.etc`, `.var`, etc.)
- Connection strings (mongodb://, postgres://, mysql://, redis://, amqp://)
- Environment variable secrets (SECRET_, KEY_, TOKEN_, PASS_, etc.)
- Internal URLs (localhost, 127.0.0.1, 0.0.0.0)
- Security-related keywords (CRITICAL, vulnerability, exploit)
**Before:** Only internal IPs and basic password/api_key patterns.
**After:** Comprehensive redaction covering:
- Internal IPs (10.x, 172.16-31.x, 192.168.x)
- Connection strings (all major protocols)
- File paths (common system directories)
- Environment variable secrets (various naming patterns)
- Internal URLs
- Security-sensitive content lines
**Impact:** Sensitive data in documentation files completely redacted before serving.
### 🟠 #4: Error Message Leaks
**File:** `routes/aboutAdmin.js`
**Fix:** Removed `err.message` from `console.error`. HTTP 500 response returns generic message only.
**Before:**
```javascript
} catch (err) {
console.error('[aboutAdmin] Error reading files:', err.message);
res.status(500).json({
error: 'Failed to read files',
details: err.message
});
}
```
**After:**
```javascript
} catch (err) {
console.error('[aboutAdmin] Error reading files');
res.status(500).json({
error: 'Failed to read project documentation files',
code: 'FILE_READ_ERROR'
});
}
```
**Impact:** No path disclosure or internal error details exposed to clients.
### 🟡 #5: Race Condition Fix
**File:** `server.js`
**Fix:** Regular user creation wrapped in `db.transaction()` to ensure atomic check-and-insert.
**Before:**
```javascript
const existingRegular = db.prepare('SELECT id FROM users WHERE username = ?').get(regularUser);
if (!existingRegular) {
db.prepare('INSERT INTO users ...').run(...);
}
```
**After:**
```javascript
const createRegularUser = db.transaction(() => {
const existingRegular = db.prepare('SELECT id FROM users WHERE username = ?').get(regularUser);
if (!existingRegular) {
db.prepare('INSERT INTO users ...').run(...);
return true;
}
return false;
});
createRegularUser();
```
**Impact:** Prevents duplicate user creation under concurrent requests.
### 🟡 #6: Password Validation
**File:** `server.js`
**Fix:** `INIT_REGULAR_PASS` validated for minimum 8 characters. `process.exit(1)` on validation failure.
**Before:** No validation; any password length accepted.
**After:**
```javascript
if (regularPass && regularPass.length < 8) {
console.error('[seed] INIT_REGULAR_PASS must be at least 8 characters');
process.exit(1);
}
```
**Impact:** Enforces minimum password strength for seeded regular users.
---
### Added (2026-05-09)
- **Regular User Seed Environment Variables** — `INIT_REGULAR_USER` and `INIT_REGULAR_PASS` create a non-admin user on first run for role-based testing
- **Database Migration v0.42** — `bill_history_ranges` table creation moved into versioned migration system
- **Admin-only `/about` endpoint** — `/api/about-admin` serves FUTURE.md and DEVELOPMENT_LOG.md to admins only
### Security (2026-05-09)
- **Admin-only `/admin/about` route guard** — React `RequireAuth` middleware protects `/admin/about` route
- **Rate limiting on `/api/about-admin`** — `adminActionLimiter` (30 req/15min per IP) applied to prevent brute-force attempts
- **XSS prevention** — `rehype-sanitize` added to ReactMarkdown component in AboutPage.jsx
- **Content redaction** — `routes/aboutAdmin.js` sanitizes paths, redacts internal IPs, passwords, API keys
- **Error sanitization** — Error messages exclude paths to prevent path disclosure
- **Non-admin test user** — Added `INIT_REGULAR_USER` and `INIT_REGULAR_PASS` env vars for role-based testing
**Changes:**
- `client/App.jsx``/admin/about` route protected with `RequireAuth role="admin"`
- `server.js``adminActionLimiter` applied to `/api/about-admin` (30 req/15min IP)
- `client/pages/AboutPage.jsx``rehypeSanitize` added to `ReactMarkdown` component
- `client/api.js``aboutAdmin: () => get('/about-admin')` endpoint function added
---
## Session Token Expiry Cleanup (v0.43)
**Added:** Automatic session token cleanup on startup and periodic background cleanup.
**Changes:**
- `db/database.js` — Added `cleanupExpiredSessions()` function, v0.43 migration (`sessions.created_at` column), COLUMN_WHITELIST entry for `created_at`
- `server.js` — Calls cleanup on startup, sets up periodic cleanup every 24h (configurable via `SESSION_CLEANUP_INTERVAL_MS`)
- `services/authService.js` — Purges user's expired sessions on login and `createSession`, added logging to `pruneExpiredSessions`
### Components
#### 1. `cleanupExpiredSessions()` — `db/database.js`
```javascript
function cleanupExpiredSessions() {
const result = db.prepare("DELETE FROM sessions WHERE expires_at < datetime('now')").run();
console.log(`[cleanup] Purged ${result.changes} expired sessions`);
return result;
}
```
**Purpose**: Remove sessions that have exceeded their 7-day expiration window.
**Usage**:
- Called on server startup
- Called every 24 hours via `setInterval`
**Returns**: `{ changes: number }` — Number of rows deleted
#### 2. Periodic Cleanup — `server.js`
```javascript
const CLEANUP_INTERVAL_MS = parseInt(process.env.SESSION_CLEANUP_INTERVAL_MS) || 86400000; // 24 hours default
setInterval(() => {
try {
console.log('[cleanup] Running periodic session cleanup');
cleanupExpiredSessions();
} catch (err) {
console.error('[cleanup-error] Failed to run periodic session cleanup:', err.message);
}
}, CLEANUP_INTERVAL_MS);
```
**Environment Variable**: `SESSION_CLEANUP_INTERVAL_MS` (default: `86400000` = 24 hours)
**Startup Cleanup**:
```javascript
async function main() {
const db = getDb();
// Run session cleanup on startup
const { cleanupExpiredSessions } = require('./db/database');
try {
console.log('[cleanup] Running session cleanup on startup');
cleanupExpiredSessions();
} catch (err) {
console.error('[cleanup-error] Failed to run startup session cleanup:', err.message);
}
// ...
}
```
#### 3. Per-User Cleanup — `services/authService.js`
Both `login()` and `createSession()` now clean up expired sessions for the specific user before creating a new session:
```javascript
// Clean up expired sessions for this user before creating new session
try {
db.prepare("DELETE FROM sessions WHERE user_id = ? AND expires_at < datetime('now')").run(user.id);
} catch (err) {
console.error('[cleanup-error] Failed to cleanup user expired sessions:', err.message);
}
```
**Benefits**:
- Prevents accumulation of expired sessions per user
- Reduces database size
- Improves session query performance
### Database Migration v0.43
**Migration**: `sessions: add created_at column`
**Purpose**: Track when sessions are created (for future analytics/debugging).
**SQL**:
```sql
ALTER TABLE sessions ADD COLUMN created_at TEXT DEFAULT (datetime('now'));
```
**Column Whitelist Entry**:
```javascript
const COLUMN_WHITELIST = new Set([
// ... other columns ...
'created_at', // sessions table
]);
```
**Safety**: Column name validated against whitelist before executing ALTER statement.
### Log Output
**Startup**:
```
[cleanup] Running session cleanup on startup
[cleanup] Purged 0 expired sessions
[cleanup] Scheduled periodic cleanup every 86400000ms
```
**Periodic Cleanup** (every 24 hours by default):
```
[cleanup] Running periodic session cleanup
[cleanup] Purged 0 expired sessions
```
**Per-User Cleanup** (on login):
```
[cleanup-error] Failed to cleanup user expired sessions: [error details if fails]
```
### Testing
**Test 1: Fresh DB — Cleanup on Startup** ✅
- Container starts with empty data volume
- Migration v0.43 applied (`sessions.created_at` column added)
- Startup cleanup runs, purges 0 expired sessions
- Logs confirm: `[cleanup] Scheduled periodic cleanup every 86400000ms`
**Test 2: Login — Per-User Cleanup** ✅
- Login creates new session
- Old expired sessions for that user purged
- New session created with fresh `expires_at`
**Test 3: Periodic Cleanup Interval** ✅
- `SESSION_CLEANUP_INTERVAL_MS` env var overrides default 24h
- Custom value logged on startup
- Cleanup runs at specified interval
### Error Handling
- Startup cleanup failures: Logged, app continues
- Periodic cleanup failures: Logged, retry on next interval
- Per-user cleanup failures: Logged, new session still created
**No crash on cleanup failure.** The app continues regardless of cleanup status.
### Environment Variables
| Variable | Default | Description |
|----------|---------|-------------|
| `SESSION_CLEANUP_INTERVAL_MS` | `86400000` (24h) | Interval between periodic cleanup runs |
---
## Test Report — Session Token Expiry Cleanup Verification
**Date**: 2026-05-09 20:12 CDT
| Test | Status | Details |
|------|--------|---------|
| **Docker build** | ✅ PASS | Fresh build with `--no-cache`, image tagged `bill-tracker:local` |
| **Fresh DB test** | ✅ PASS | Container started, data volume mounted |
| **Startup cleanup log** | ✅ PASS | `[cleanup] Running session cleanup on startup`, `[cleanup] Purged 0 expired sessions` |
| **Periodic cleanup log** | ✅ PASS | `[cleanup] Scheduled periodic cleanup every 86400000ms` |
| **Migration v0.43** | ✅ PASS | `[migration] sessions.created_at column added`, column verified in database |
| **Login test** | ✅ PASS | Login returns valid user object with `must_change_password=true` |
| **Code review: `cleanupExpiredSessions()`** | ✅ PASS | Function exported from `db/database.js` |
| **Code review: Periodic interval** | ✅ PASS | `SESSION_CLEANUP_INTERVAL_MS` env var supported |
| **Code review: Per-user cleanup** | ✅ PASS | `login()` and `createSession()` both purge expired sessions for target user |
| **Code review: Error handling** | ✅ PASS | Cleanup failures logged, app continues running |
| **Engineering manual update** | ✅ PASS | Added Session Token Expiry Cleanup section |
**Verdict: ALL TESTS PASSED** ✅
---
## Version 0.19.2 Update
### 🔴 Migration System Fix (2026-05-09)
**Added:** Legacy database detection and reconciliation for existing deployments that preddate the migration tracking system.
**Problem:** Existing deployments created before v0.19.0 had a populated database without the `schema_migrations` table tracking applied migrations. On upgrade, the app would start but fail to reconcile existing migrations, potentially causing:
- Duplicate column errors on migrations that already ran
- Missing index errors on indexes that already existed
- Inconsistent migration state
**Solution:** Added `handleLegacyDatabase()` and `reconcileLegacyMigrations()` functions to detect legacy databases and safely reconcile migration state.
**Changes:**
- `db/database.js` — Added `handleLegacyDatabase()` function
- `db/database.js` — Added `reconcileLegacyMigrations()` function
- `db/database.js` — Modified `initSchema()` to call `handleLegacyDatabase()` before `runMigrations()`
**Detection Logic:**
Legacy database is detected when:
1. `schema_migrations` table exists and has zero rows, OR table doesn't exist (but tables do)
2. Core tables exist (`users`, `bills`, `payments`, `categories`, `settings`)
3. The database has data but no migration tracking
**Reconciliation Process:**
1. **Check for notification columns migration:** If old notification columns exist, mark as applied
2. **Iterate through all migrations:** For each migration definition:
- Run the migration's `check()` function to see if its changes are already present
- If already present, call `recordMigration()` to mark it as applied in `schema_migrations`
- Log: `[migration] Recorded legacy migration {version}: {description}`
3. **Complete reconciliation:** Log `[migration] Legacy database reconciliation complete`
4. **Apply remaining migrations:** `runMigrations()` proceeds as normal, applying only truly pending migrations
**Example Log Output:**
```
[migration] Detected legacy database, reconciling schema migrations...
[migration] Applied v0.4: monthly_bill_state: per-bill per-month overrides
[migration] Recorded legacy migration v0.4: monthly_bill_state: per-bill per-month overrides
[migration] Applied v0.14.4: bills: optional credit-card APR / interest rate
[migration] Recorded legacy migration v0.14.4: bills: optional credit-card APR / interest rate
[migration] Applied v0.38: import_history: per-user audit log
[migration] Recorded legacy migration v0.38: import_history: per-user audit log
[migration] Applied v0.40: ownership: user-scoped bills/categories
[migration] Recorded legacy migration v0.40: ownership: user-scoped bills/categories
[migration] Legacy database reconciliation complete
[migration] Applying v0.2: payments: soft-delete column
[migration] payments.deleted_at column added
[migration] Applied v0.2: payments: soft-delete column
[migration] Applying v0.3: payments: compound index for tracker query
[migration] Applied v0.3: payments: compound index for tracker query
```
**Benefits:**
- Existing deployments can upgrade to v0.19.2 without manual intervention
- No data loss during migration reconciliation
- No duplicate column/index errors
- Seamless upgrade path from any pre-v0.19.0 version
**Testing:**
**Test 1: Fresh Database (v0.19.2)** ✅
- Container starts with empty data volume
- Migrations applied in order (v0.2 through v0.42)
- Admin and regular users created successfully
- Login functional
**Test 2: Simulated Legacy Database (pre-v0.19.0)** ✅
- Created database with tables but NO `schema_migrations` table
- Container detected legacy database and logged detection
- All existing migrations recorded in `schema_migrations`
- Remaining migrations applied correctly
- Login functional
**Files Modified:**
- `db/database.js` — Legacy database detection and reconciliation added
**Impact:**
- Existing users can safely upgrade from any version to v0.19.2
- No manual database intervention required
- Migration state remains consistent and auditable
---
## Version 0.19.0 Update
### Security Fixes (2026-05-09)
**Added:** Admin-only `/admin/about` route guard, rate limiting on `/api/about-admin`, content sanitization with `rehype-sanitize`, and new environment variables for non-admin user creation.
**Changes:**
- `client/App.jsx``/admin/about` route protected with `RequireAuth role="admin"`
- `server.js``adminActionLimiter` applied to `/api/about-admin` (30 req/15min IP)
- `client/pages/AboutPage.jsx``rehypeSanitize` added to `ReactMarkdown` component
- `client/api.js``aboutAdmin: () => get('/about-admin')` endpoint function added
---
### Migration System Note (v0.19.1)
Database migrations now use explicit version tracking via the `schema_migrations` table. All migrations are idempotent and safe to re-run after `git pull`. The `runMigrations()` function queries `schema_migrations` before applying each migration, skipping already-applied versions.
**Key Changes**:
- New `schema_migrations` table with `version`, `description`, `applied_at` columns
- Helper functions: `hasMigrationBeenApplied()`, `recordMigration()`
- Migrations are defined as versioned objects with explicit `version`/`description`/`run()`
- Safe `git pull && npm start` upgrades without migration state issues
2026-05-09 13:03:36 -05:00
---
## Table of Contents
1. [High Level Overview](#1-high-level-overview)
2. [Frontend Documentation](#2-frontend-documentation)
3. [Backend Documentation](#3-backend-documentation)
4. [Authentication & Authorization](#4-authentication--authorization)
5. [API Documentation](#5-api-documentation)
6. [Database Documentation](#6-database-documentation)
7. [Error Handling & Troubleshooting](#7-error-handling--troubleshooting)
8. [Code Navigation Index](#8-code-navigation-index)
9. [Infrastructure & Deployment](#9-infrastructure--deployment)
10. [Sequence Flows](#10-sequence-flows)
---
## 1. High Level Overview
### App Purpose
BillTracker is a self-hosted monthly bill tracking system for households and small setups. It manages:
- **Recurring bills**: Track due dates, expected amounts, categories, autopay, interest rates, website login info
- **Monthly tracker**: Record actual payments, skip bills, view spending vs expectations
- **Calendar view**: Visual grid showing due dates and payments
- **Analytics**: Charts, category spend, payment history
- **User management**: Admin creates users, sets roles, manages authentication
- **Notifications**: Email alerts for due bills (3d, 1d, today, overdue)
- **Data management**: Import/Export bills, full database backup/restore
### Architecture Summary
**Stack**: Node.js + Express (backend) + React + Vite (frontend) + SQLite (database)
**Layered Architecture**:
```
┌─────────────────────────────────────────────────────────────┐
│ Frontend (React) │
│ Pages (client/pages/) • Components (client/components/) │
│ Router (client/App.jsx) • API Client (client/api.js) │
└─────────────────────────────────────────────────────────────┘
HTTP/JSON
┌─────────────────────────────────────────────────────────────┐
│ Backend (Express) │
│ Routes (routes/) • Services (services/) • Middleware │
│ Auth (authService.js) • OIDC (oidcService.js) │
└─────────────────────────────────────────────────────────────┘
SQL
┌─────────────────────────────────────────────────────────────┐
│ Database (SQLite) │
│ Schema (db/schema.sql) • Migrations (db/database.js) │
│ Users • Sessions • Bills • Payments • Categories • etc. │
└─────────────────────────────────────────────────────────────┘
```
### Tech Stack
| Layer | Component | Version | Purpose |
|-------|-----------|---------|---------|
| **Runtime** | Node.js | v20+ | Backend server |
| **Framework** | Express | ^4.18.2 | HTTP server, routing, middleware |
| **Frontend** | React | ^18.3.1 | UI components |
| **Build** | Vite | ^5.4.10 | Bundler, dev server |
| **Router** | react-router-dom | ^6.26.2 | Client-side routing |
| **Database** | better-sqlite3 | ^12.9.0 | SQLite wrapper | Migration tracking |
2026-05-09 13:03:36 -05:00
| **Auth** | bcryptjs | ^2.4.3 | Password hashing |
| **OIDC** | openid-client | ^5.7.1 | Authentik integration |
| **Email** | nodemailer | ^6.9.14 | SMTP email sending |
| **Scheduler** | node-cron | ^3.0.3 | Background jobs |
| **UI Libs** | shadcn/ui | - | Component primitives |
| **Styling** | TailwindCSS | ^3.4.14 | Utility-first CSS |
### Major Components
| Component | Location | Purpose |
|-----------|----------|---------|
| **server.js** | Root | Express entry, middleware setup, route mounting |
| **db/database.js** | `db/` | SQLite connection, migrations, settings |
| **services/authService.js** | `services/` | Session management, login/logout |
| **services/oidcService.js** | `services/` | Authentik OIDC integration |
| **services/backupService.js** | `services/` | Database backup/restore |
| **middleware/requireAuth.js** | `middleware/` | Auth guard middleware |
| **middleware/csrf.js** | `middleware/` | CSRF token generation/validation |
| **workers/dailyWorker.js** | `workers/` | Daily background tasks |
### Database Migration System
The database migration system provides explicit version tracking to ensure safe upgrades via `git pull && npm start`.
**Migration Tracking Table** (`schema_migrations`):
| Column | Type | Purpose |
|--------|------|---------|
| `id` | INTEGER PRIMARY KEY AUTOINCREMENT | Internal tracking ID |
| `version` | TEXT NOT NULL UNIQUE | Migration version identifier (e.g., `v0.2`, `v0.3`) |
| `description` | TEXT NOT NULL | Human-readable migration description |
| `applied_at` | TEXT NOT NULL DEFAULT (datetime('now')) | Timestamp when migration was applied |
**Migration Functions**:
- `hasMigrationBeenApplied(version)`: Check if a migration version has been applied
- `recordMigration(version, description)`: Record a migration as applied
- `runMigrations()`: Execute pending migrations with version tracking
**Migration Format**:
Migrations are defined as versioned objects with explicit `version`, `description`, and `run()` function:
```javascript
{
version: 'v0.2',
description: 'payments: soft-delete column',
run: function() { /* migration logic */ }
}
```
**Idempotent Migration Execution**:
1. Query `schema_migrations` table for applied versions
2. Skip migrations that have already been applied
3. Apply pending migrations in order
4. Log each migration status (applied vs skipped)
**Benefits**:
- Users can safely `git pull && npm start` without migration state issues
- Migrations are repeatable and trackable
- Clear audit trail of which migrations have been applied
- No risk of re-applying migrations that modify data
2026-05-09 13:03:36 -05:00
### Request Lifecycle
```
┌─────────────────────────────────────────────────────────────┐
│ 1. Request arrives at Express (server.js) │
│ • Security headers applied (securityHeaders) │
│ • CSRF token set on response (csrfTokenProvider) │
└─────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────┐
│ 2. Route-level middleware chain │
│ • CSRF validation (csrfMiddleware) │
│ • Auth check (requireAuth) │
│ • User role check (requireUser/requireAdmin) │
│ • Rate limiting (loginLimiter, exportLimiter, etc.) │
└─────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────┐
│ 3. Route handler (routes/*.js) │
│ • Input validation │
│ • Service layer calls │
│ • Database queries │
└─────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────┐
│ 4. Service layer (services/*.js) │
│ • Business logic │
│ • External service calls (SMTP, OIDC) │
│ • Data transformation │
└─────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────┐
│ 5. Database layer (db/database.js) │
│ • SQL query execution │
│ • Schema validation (PRAGMA foreign_keys=ON) │
│ • Migration runs (if schema changes detected) │
└─────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────┐
│ 6. Response sent to frontend │
│ • JSON format │
│ • CSRF token in cookie │
│ • Error formatted via errorFormatter │
└─────────────────────────────────────────────────────────────┘
```
---
## 2. Frontend Documentation
### Route Mapping
| Route | Page Component | Auth Required | Purpose |
|-------|----------------|---------------|---------|
| `/` | Redirect | No | Redirects to login or app based on auth |
| `/login` | LoginPage.jsx | No | Username/password login form |
| `/tracker` | TrackerPage.jsx | Yes | Monthly bill tracking view |
| `/bills` | BillsPage.jsx | Yes | Bill CRUD interface |
| `/categories` | CategoriesPage.jsx | Yes | Category management |
| `/calendar` | CalendarPage.jsx | Yes | Monthly calendar view |
| `/summary` | SummaryPage.jsx | Yes | Monthly summary/spending view |
| `/analytics` | AnalyticsPage.jsx | Yes | Charts and analytics |
| `/profile` | ProfilePage.jsx | Yes | User profile, password change |
| `/settings` | SettingsPage.jsx | Yes | App settings (theme, format) |
| `/data` | DataPage.jsx | Yes | Import/export data tools |
| `/admin` | AdminPage.jsx | Yes (admin) | User management, backups, OIDC config |
| `/about` | AboutPage.jsx | No | Version info, changelog |
| `/status` | StatusPage.jsx | Yes (admin) | System status, worker health |
### Key Frontend Files
#### Core Files
| File | Purpose | Key Functions |
|------|---------|---------------|
| `client/main.jsx` | React entry point | Creates root, renders App |
| `client/App.jsx` | Router config | Defines all routes, layout wrapper |
| `client/api.js` | API client | `get`, `post`, `put`, `delete`, auth, CSRF |
| `client/hooks/useAuth.jsx` | Auth state | `login`, `logout`, `user`, `loading` |
#### Query Hooks (TanStack Query v0.22.0+)
| Hook | File | Query Key | Stale Time | Purpose |
|------|------|-----------|------------|---------|
| `useTracker(year, month)` | `client/hooks/useQueries.js` | `['tracker', year, month]` | 5 min | Monthly tracker data |
| `useBills()` | `client/hooks/useQueries.js` | `['bills']` | 5 min | List all bills |
| `useCategories()` | `client/hooks/useQueries.js` | `['categories']` | 1 hour | List categories |
2026-05-09 13:03:36 -05:00
#### Layout Components
| File | Purpose | Key Features |
|------|---------|--------------|
| `client/components/layout/Layout.jsx` | Main layout wrapper | Sidebar, top bar, content area |
| `client/components/layout/Sidebar.jsx` | Navigation sidebar | Links to all pages, collapse |
| `client/components/layout/BrandBlock.jsx` | App branding | Logo, title, version |
| `client/components/layout/NavPill.jsx` | Nav item | Active state, icon |
#### UI Components (shadcn/ui)
| Component | File | Purpose |
|-----------|------|---------|
| Button | `client/components/ui/button.jsx` | Primary, secondary, ghost |
| Input | `client/components/ui/input.jsx` | Form input |
| Card | `client/components/ui/card.jsx` | Content container |
| Table | `client/components/ui/table.jsx` | Data table |
| Tabs | `client/components/ui/tabs.jsx` | Tab navigation |
| Dialog | `client/components/ui/dialog.jsx` | Modal dialogs |
| Badge | `client/components/ui/badge.jsx` | Status badges |
| Switch | `client/components/ui/switch.jsx` | Toggle switch |
| Alert Dialog | `client/components/ui/alert-dialog.jsx` | Confirm action dialog |
#### Page Components
| Page | Route | API Calls | State |
|------|-------|-----------|-------|
| LoginPage | `/login` | `POST /api/auth/login` | `user`, `error` |
| TrackerPage | `/tracker` | `GET /api/tracker`, `GET /api/tracker/upcoming`, `POST /api/auth/logout-all` | `data`, `year`, `month`, `activeBillId`, `refetch` |
2026-05-09 13:03:36 -05:00
| BillsPage | `/bills` | `GET /api/bills`, `POST /api/bills`, `PUT /api/bills/:id`, `DELETE /api/bills/:id` | `bills`, `categories`, `modalState` |
| CategoriesPage | `/categories` | `GET /api/categories`, `POST /api/categories` | `categories` |
| CalendarPage | `/calendar` | `GET /api/bills`, `GET /api/tracker` | `year`, `month`, `dates` |
| SummaryPage | `/summary` | `GET /api/summary` | `data`, `year`, `month` |
| AnalyticsPage | `/analytics` | `GET /api/analytics` | `filters`, `data` |
| ProfilePage | `/profile` | `GET /api/user`, `POST /api/profile` | `user`, `notifications` |
| SettingsPage | `/settings` | `GET /api/settings`, `PUT /api/settings` | `settings` |
| DataPage | `/data` | `GET /api/import`, `POST /api/import`, `POST /api/export` | `importState`, `exportState` |
| AdminPage | `/admin` | Multiple admin endpoints | `users`, `backups`, `oidc` |
| AboutPage | `/about`, `/admin/about` | `GET /api/about`, `GET /api/about-admin` | `about` |
2026-05-09 13:03:36 -05:00
### State Management
**Approach**: Component-local state + React Context (ThemeContext)
**Key State**:
| State | Location | Purpose |
|-------|----------|---------|
| `theme` | ThemeContext | Light/dark mode |
| `user` | useAuth hook | Current user object |
| `loading` | useAuth hook | Auth state |
| `error` | useAuth hook | Auth errors |
| `bills`, `categories`, etc. | Page components | API response data |
### Validation
**Frontend validation occurs before API calls**:
```javascript
// client/api.js - validation wrapper
function validateInput(schema, data) {
// Check required fields
// Type validation
// Range validation (numbers, dates)
}
```
**Common validations**:
| Field | Validation |
|-------|-----------|
| `due_day` | Integer 1-31 |
| `expected_amount` | Number ≥ 0 |
| `interest_rate` | Number 0-100 or null |
| `password` | Min 8 characters (admin) |
| `username` | Min 3 characters (admin) |
| `year/month` | Valid date range |
### API Client (`client/api.js`)
**Key Functions**:
```javascript
// GET request with CSRF token
await apiGet('/api/bills', { year, month })
// POST with CSRF
await apiPost('/api/bills', { name, due_day, ... })
// PUT for updates
await apiPut('/api/bills/:id', { name, ... })
// DELETE
await apiDelete('/api/bills/:id')
```
**CSRF Handling**:
- Token stored in `bt_csrf_token` cookie (httpOnly)
- Sent in `x-csrf-token` header for POST/PUT/DELETE
- Auto-retrieved from cookie on each request
---
## 3. Backend Documentation
### Core Backend Files
| File | Purpose | Key Functions |
|------|---------|---------------|
| `server.js` | Express entry | Middleware setup, route mounting, error handling |
| `db/database.js` | DB connection | SQLite init, migrations, settings |
| `db/schema.sql` | Schema definition | All table definitions |
| `services/authService.js` | Auth service | Login, logout, session management |
| `services/oidcService.js` | OIDC service | Authentik integration |
| `services/backupService.js` | Backup service | SQLite backup/restore |
| `middleware/requireAuth.js` | Auth guards | `requireAuth`, `requireUser`, `requireAdmin` |
| `middleware/csrf.js` | CSRF protection | Token generation/validation |
| `middleware/rateLimiter.js` | Rate limiting | Per-endpoint limits |
| `middleware/securityHeaders.js` | Security headers | CSP, HSTS, XSS protection |
| `middleware/errorFormatter.js` | Error formatting | JSON error responses |
### Route Handlers (routes/*.js)
| Route File | API Prefix | Auth | Purpose |
|------------|------------|------|---------|
| `authLogin.js` | `/api/auth/login` | None | Local login |
| `auth.js` | `/api/auth` | CSRF | Logout, password change |
| `authOidc.js` | `/api/auth/oidc` | CSRF | OIDC login/callback |
| `tracker.js` | `/api/tracker` | Auth+User | Monthly tracking data |
| `bills.js` | `/api/bills` | Auth+User | Bill CRUD |
| `payments.js` | `/api/payments` | Auth+User | Payment CRUD |
| `categories.js` | `/api/categories` | Auth+User | Category CRUD |
| `settings.js` | `/api/settings` | Auth+User | Settings CRUD |
| `user.js` | `/api/user` | Auth+User | User profile |
| `calendar.js` | `/api/calendar` | Auth+User | Calendar data |
| `summary.js` | `/api/summary` | Auth+User | Monthly summary |
| `monthly-starting-amounts.js` | `/api/monthly-starting-amounts` | Auth+User | Starting balance |
| `analytics.js` | `/api/analytics` | Auth+User | Analytics data |
| `notifications.js` | `/api/notifications` | Auth+User | Notification settings |
| `admin.js` | `/api/admin` | Auth+Admin | Admin functions |
| `export.js` | `/api/export` | Auth+User | Data export |
| `import.js` | `/api/import` | Auth+User | Data import |
| `status.js` | `/api/status` | Auth+Admin | System status |
| `about.js` | `/api/about` | None | Version info |
| `version.js` | `/api/version` | None | Version string |
### Service Layer Functions
#### db/database.js
| Function | Purpose | Parameters | Returns |
|----------|---------|------------|---------|
| `initSchema()` | Initialize database schema | None | void (calls `runMigrations()`) |
| `runMigrations()` | Execute pending migrations | None | void (skips already-applied) |
| `hasMigrationBeenApplied(version)` | Check migration status | `version` (string) | `true` or `false` |
| `recordMigration(version, description)` | Record applied migration | `version`, `description` | void |
| `seedDefaults()` | Seed settings and categories | None | void |
| `ensureUserDefaultCategories(userId)` | Seed default categories per user | `userId` | void |
| `getSetting(key)` | Read single setting | `key` (string) | value or `null` |
| `setSetting(key, value)` | Write single setting | `key`, `value` | void |
| `getDbPath()` | Get database file path | None | string (absolute path) |
| `closeDb()` | Close database connection | None | void |
**Migration System Flow**:
1. `initSchema()` reads `db/schema.sql` and executes all table definitions
2. Creates `schema_migrations` table if not exists
3. `runMigrations()` iterates through versioned migrations
4. For each migration, `hasMigrationBeenApplied()` checks if already done
5. If not applied: executes `run()` then `recordMigration()`
6. If already applied: logs skip message, moves to next migration
**Migration Format**:
```javascript
{
version: 'v0.2',
description: 'payments: soft-delete column',
run: function() { /* migration logic */ }
}
```
2026-05-09 13:03:36 -05:00
#### authService.js
| Function | Purpose | Parameters | Returns |
|----------|---------|------------|---------|
| `login(username, password)` | Authenticate user | `username`, `password` | `{sessionId, user}` or null |
| `logout(sessionId)` | Destroy session | `sessionId` | void |
| `createSession(userId)` | Create session for OIDC | `userId` | `{sessionId, user}` |
| `getSessionUser(sessionId)` | Validate session | `sessionId` | `user` or null |
| `hashPassword(password)` | Hash password | `password` | `Promise<hash>` |
| `publicUser(user)` | Strip sensitive data | `user` object | Public user object |
| `cookieOpts(req)` | Get cookie options | `req` | `{httpOnly, sameSite, secure, maxAge}` |
| `pruneExpiredSessions()` | Clean expired sessions | None | void |
| `rotateSessionId(oldId, userId)` | Security rotation | `oldId`, `userId` | `newId` or null |
#### oidcService.js
| Function | Purpose | Parameters | Returns |
|----------|---------|------------|---------|
| `getOidcConfig()` | Get effective config | None | Config object or null |
| `isOidcLoginActive()` | Check if enabled | None | boolean |
| `createLoginState(redirectTo)` | Create PKCE state | `redirectTo` | `{id, nonce, codeVerifier}` |
| `consumeLoginState(stateId)` | Validate state | `stateId` | State or null |
| `buildAuthorizationUrl(config, state)` | Build redirect URL | `config`, `state` | `Promise<URL>` |
| `exchangeAndVerifyTokens(config, code, stateId, savedState)` | Exchange code for tokens | `config`, `code`, `stateId`, `savedState` | Verified claims |
| `findOrProvisionUser(claims, config)` | Find or create user | `claims`, `config` | User object |
| `mapRoleFromClaims(claims, config)` | Map groups to role | `claims`, `config` | `'admin'` or `'user'` |
| `testOidcConfiguration(config)` | Test OIDC setup | `config` | `{ok, error, ...}` |
| `getAdminOidcSettings()` | Admin settings | None | Settings object |
| `getPublicOidcInfo()` | Public info | None | `{oidc_enabled, oidc_provider_name}` |
#### backupService.js
| Function | Purpose | Parameters | Returns |
|----------|---------|------------|---------|
| `createBackup(prefix)` | Create SQLite backup | `prefix` | `{id, filename, size_bytes, checksum}` |
| `restoreBackup(id)` | Restore from backup | `backupId` | `{restored_from, pre_restore_backup}` |
| `deleteBackup(id)` | Delete backup | `backupId` | `{deleted: true, id, deleted_at}` |
| `listBackups()` | List backups | None | Array of backup metadata |
| `getBackupFile(id)` | Get backup path | `backupId` | `{path, metadata}` |
| `importBackupBuffer(buffer, options)` | Import backup | `buffer`, `{expectedChecksum}` | Backup metadata |
| `validateSqliteDatabase(filePath)` | Validate DB file | `filePath` | void or throws |
| `checksumFile(filePath)` | SHA-256 checksum | `filePath` | `hex string` |
#### notificationService.js
| Function | Purpose | Parameters | Returns |
|----------|---------|------------|---------|
| `runNotifications()` | Send due bill emails | None | void |
| `sendTestEmail(to)` | Test SMTP config | `email` | void |
| `createTransport()` | Create SMTP transport | None | Nodemailer transport |
#### cleanupService.js
| Function | Purpose | Parameters | Returns |
|----------|---------|------------|---------|
| `runAllCleanup()` | Run all cleanup tasks | None | `{import_sessions, temp_exports, ...}` |
| `validateAndApplySettings(settings)` | Update cleanup config | `settings` | Updated config |
| `getCleanupStatus()` | Get cleanup status | None | `{settings, last_run, last_result}` |
#### statusService.js
| Function | Purpose | Parameters | Returns |
|----------|---------|------------|---------|
| `buildTrackerRow(bill, payments, year, month, today)` | Build tracker row | `bill`, `payments`, `year`, `month`, `today` | Row object |
| `resolveDueDate(bill, year, month)` | Calculate due date | `bill`, `year`, `month` | `YYYY-MM-DD` |
| `getCycleRange(year, month)` | Get date range | `year`, `month` | `{start, end}` |
### Middleware Chain
#### requireAuth.js
| Middleware | Purpose | Check |
|------------|---------|-------|
| `requireAuth` | General auth | Session valid, user active |
| `requireUser` | User role | Role is 'user' or 'admin', not default admin |
| `requireAdmin` | Admin role | Role is 'admin' |
#### CSRF Protection
| Setting | Default | Purpose |
|---------|---------|---------|
| `CSRF_HTTP_ONLY` | `true` | Cookie not accessible via JS |
| `CSRF_SAME_SITE` | `'strict'` | Same-site cookie |
| `CSRF_SECURE` | `true` | HTTPS only |
| `CSRF_COOKIE_NAME` | `'bt_csrf_token'` | Cookie name |
**Flow**:
1. `csrfTokenProvider` sets cookie on every response
2. `csrfMiddleware` validates token on POST/PUT/DELETE
3. Token can be in header, query, or body
#### Rate Limiters
| Limiter | Max | Window | Endpoints |
|---------|-----|--------|-----------|
| `loginLimiter` | 10 | 15 min | `/api/auth/login` |
| `passwordLimiter` | 5 | 15 min | `/api/profile`, `/api/admin/users/:id/password` |
| `importLimiter` | 20 | 15 min | `/api/import/*` |
| `exportLimiter` | 30 | 15 min | `/api/export/*` |
| `adminActionLimiter` | 30 | 15 min | `/api/admin/*` |
| `oidcLimiter` | 20 | 15 min | `/api/auth/oidc/*` |
| `backupOperationLimiter` | 5 | 60 min | `/api/admin/backups/*` |
### Error Handling
#### errorFormatter.js
| Error Type | Status Code | Response Format |
|------------|-------------|-----------------|
| Validation | 400 | `{error: 'Validation failed', field: 'field_name'}` |
| Auth | 401 | `{error: 'Not authenticated', code: 'AUTH_ERROR'}` |
| Forbidden | 403 | `{error: 'Access denied', code: 'FORBIDDEN'}` |
| Not Found | 404 | `{error: 'Not found', code: 'NOT_FOUND'}` |
| Conflict | 409 | `{error: 'Already exists', code: 'CONFLICT'}` |
| Rate Limit | 429 | `{error: 'Too many requests'}` |
| Server | 500 | `{error: 'Internal server error'}` |
**Standard Error Format**:
```javascript
{
error: 'Error message',
code: 'ERROR_CODE',
field: 'optional_field_name'
}
```
---
## 4. Authentication & Authorization
### Login Flow
```
┌─────────────────────────────────────────────────────────────┐
│ 1. User submits login form │
│ • Username, password │
└─────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────┐
│ 2. POST /api/auth/login │
│ • rateLimiter (loginLimiter) │
│ • Body: {username, password} │
└─────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────┐
│ 3. authService.login() │
│ • Query user by username │
│ • Check active flag │
│ • Check auth_provider === 'local' │
│ • bcrypt.compare(password, password_hash) │
└─────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────┐
│ 4. Create session │
│ • Generate UUID for sessionId │
│ • Insert into sessions table (expires in 7 days) │
│ • Update last_login_at │
│ • Return {sessionId, user} │
└─────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────┐
│ 5. Set cookie │
│ • Cookie: bt_session=<sessionId>
│ • httpOnly: true, sameSite: strict, secure: depends │
│ • Max-Age: 7 days │
└─────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────┐
│ 6. Return response │
│ • JSON: {user: {id, username, display_name, role, ...}} │
│ • CSRF token set on cookie │
└─────────────────────────────────────────────────────────────┘
```
### Session/JWT Handling
**Session Storage**: SQLite `sessions` table
| Column | Type | Purpose |
|--------|------|---------|
| `id` | TEXT (UUID) | Session identifier |
| `user_id` | INTEGER | Reference to users.id |
| `expires_at` | TEXT (ISO) | Expiration timestamp |
| `created_at` | TEXT (ISO) | Session creation time |
**Session Validation**:
```sql
SELECT u.id, u.username, u.display_name, u.role, u.must_change_password,
u.active, u.is_default_admin
FROM sessions s
JOIN users u ON u.id = s.user_id
WHERE s.id = ? AND s.expires_at > datetime('now') AND u.active = 1
```
**Session Duration**: 7 days
**Session Destruction**:
- Explicit logout: `DELETE FROM sessions WHERE id = ?`
- Session expiry: Daily worker pruning
- User deactivation: `DELETE FROM sessions WHERE user_id = ?`
- Role change: `DELETE FROM sessions WHERE user_id = ?`
### RBAC (Role-Based Access Control)
| Role | Capabilities |
|------|--------------|
| `user` | View/modify own bills, categories, payments, settings, profile |
| `admin` | All user capabilities + user management, backups, OIDC config, system settings |
**Admin Guard**:
```javascript
function requireAdmin(req, res, next) {
if (req.user?.role !== 'admin') {
return res.status(403).json({error: 'Access denied: admin account required'});
}
next();
}
```
### Middleware Chain
**Route Protection Example**:
```javascript
// Admin routes
app.use('/api/admin', csrfMiddleware, requireAuth, requireAdmin,
adminActionLimiter, require('./routes/admin'));
// User routes
app.use('/api/bills', csrfMiddleware, requireAuth, requireUser,
require('./routes/bills'));
// Public routes (no auth)
app.use('/api/about', require('./routes/about'));
app.use('/api/version', require('./routes/version'));
```
### Cookie Handling
| Cookie | Name | Type | Secure | SameSite | Purpose |
|--------|------|------|--------|----------|---------|
| Session | `bt_session` | httpOnly | Configurable | `strict` | Auth session |
| CSRF | `bt_csrf_token` | httpOnly | Configurable | `strict` | CSRF token |
**Cookie Options** (determined at runtime):
```javascript
function cookieOpts(req) {
const cookieSecure = envFlag('COOKIE_SECURE');
const httpsSecure = envFlag('HTTPS');
const secure = cookieSecure !== null
? cookieSecure
: httpsSecure !== null
? httpsSecure
: requestLooksHttps(req); // Check X-Forwarded-Proto
return {
httpOnly: true,
sameSite: 'strict',
secure,
maxAge: SESSION_DAYS * 86400 * 1000, // 7 days
path: '/',
};
}
```
### OIDC/Authentik Flow
```
┌─────────────────────────────────────────────────────────────┐
│ 1. User clicks OIDC login button │
│ • Redirects to frontend login page │
└─────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────┐
│ 2. Frontend calls /api/auth/oidc/login │
│ • Query params: ?redirect_to=/tracker │
└─────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────┐
│ 3. Create login state │
│ • Generate PKCE code_verifier (32 bytes base64url) │
│ • Generate nonce (16 bytes hex) │
│ • Store in oidc_states table (expires 5 min) │
│ • Code challenge = SHA256(code_verifier) base64url │
└─────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────┐
│ 4. Redirect to OIDC provider │
│ • Authorization URL built with: │
│ • client_id, redirect_uri, response_type=code │
│ • state (login state ID), nonce │
│ • code_challenge, code_challenge_method=S256 │
│ • scopes: openid, email, profile, groups │
└─────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────┐
│ 5. User authenticates with OIDC provider │
│ • Authentik validates credentials │
│ • Provider sends user back to redirect_uri │
└─────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────┐
│ 6. Callback: /api/auth/oidc/callback │
│ • Query params: code, state │
│ • Rate limited (oidcLimiter) │
└─────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────┐
│ 7. Exchange code for tokens │
│ • POST to OIDC token endpoint │
│ • client_id + client_secret + code + redirect_uri │
│ • code_verifier for PKCE │
└─────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────┐
│ 8. Verify ID token │
│ • JWT signature via JWKS │
│ • Issuer validation (iss claim) │
│ • Audience validation (aud claim) │
│ • Expiry validation (exp claim) │
│ • Nonce validation (replay protection) │
│ • State validation (replay protection) │
└─────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────┐
│ 9. Find or provision user │
│ • Look up by sub (external_subject) │
│ • Look up by email if email_verified=true │
│ • Auto-provision if OIDC_AUTO_PROVISION=true │
│ • Map groups to role (admin if in admin_group) │
└─────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────┐
│ 10. Create local session │
│ • Same mechanism as local login │
│ • Set bt_session cookie │
│ • Redirect to redirect_to (or /) │
└─────────────────────────────────────────────────────────────┘
```
**OIDC Configuration**:
```bash
# Environment variables (fallback if DB settings blank)
OIDC_ISSUER_URL=https://auth.example.com/application/o/bills/.well-known/openid-configuration
OIDC_CLIENT_ID=<client-id>
OIDC_CLIENT_SECRET=<client-secret>
OIDC_REDIRECT_URI=https://bills.example.com/api/auth/oidc/callback
OIDC_SCOPES=openid email profile groups
OIDC_ADMIN_GROUP=bill-tracker-admins
OIDC_AUTO_PROVISION=true
```
### Failure Scenarios
| Scenario | Status Code | Response | Recovery |
|----------|-------------|----------|----------|
| Invalid credentials | 401 | `{error: 'Invalid username or password', code: 'AUTH_ERROR'}` | Retry with correct credentials |
| Session expired | 401 | `{error: 'Not authenticated', code: 'AUTH_ERROR'}` | Re-login |
| Role insufficient | 403 | `{error: 'Access denied: admin account required', code: 'FORBIDDEN'}` | Login as admin |
| Rate limited | 429 | `{error: 'Too many login attempts...'}` | Wait 15 minutes |
| CSRF invalid | 403 | `{error: 'CSRF token validation failed', code: 'CSRF_INVALID'}` | Refresh page |
| OIDC config missing | 501 | `{error: 'OIDC authentication is not configured...'}` | Configure OIDC in Admin |
| OIDC provider error | 502 | `{error: 'Failed to reach the identity provider...'}` | Check OIDC provider status |
| OIDC callback expired state | Redirect + query param | `/?oidc_error=invalid_or_expired_state` | Start login flow again |
### Code Locations
| Component | File |
|-----------|------|
| Login endpoint | `routes/authLogin.js` |
| Auth middleware | `middleware/requireAuth.js` |
| CSRF middleware | `middleware/csrf.js` |
| Session management | `services/authService.js` |
| OIDC login | `routes/authOidc.js` |
| OIDC service | `services/oidcService.js` |
| OIDC tables | `db/schema.sql` (sessions, oidc_states) |
---
## 5. API Documentation
### Authentication Endpoints
#### POST /api/auth/login
**Purpose**: Local username/password login
**Request**:
```json
{
"username": "string (required)",
"password": "string (required)"
}
```
**Response**:
```json
{
"user": {
"id": 1,
"username": "admin",
"display_name": "Administrator",
"role": "admin",
"active": true,
"is_default_admin": true,
"must_change_password": false,
"first_login": false
}
}
```
**Rate Limit**: 10 per 15 minutes per IP (bypassed if no users exist)
**Errors**:
| Status | Code | Message |
|--------|------|---------|
| 400 | `VALIDATION_ERROR` | Username/password missing |
| 401 | `AUTH_ERROR` | Invalid credentials |
| 403 | `FORBIDDEN` | Local login disabled |
| 429 | `RATE_LIMITED` | Too many attempts |
#### GET /api/auth/oidc/login
**Purpose**: Initiate OIDC login flow
**Query Parameters**:
- `redirect_to` (optional): URL to redirect after login
**Response**: HTTP 302 redirect to OIDC provider
**Errors**:
| Status | Redirect | Reason |
|--------|----------|--------|
| 501 | - | OIDC not configured |
| 502 | - | Provider unreachable |
#### GET /api/auth/oidc/callback
**Purpose**: OIDC callback handler
**Query Parameters**:
- `code`: Authorization code
- `state`: Login state ID
- `error` (optional): Provider error
**Response**: HTTP 302 redirect to frontend or error page
**Errors**:
| Status | Redirect | Reason |
|--------|----------|--------|
| 302 | `/?oidc_error=not_configured` | OIDC disabled |
| 302 | `/?oidc_error=authorization_failed` | Provider signalled error |
| 302 | `/?oidc_error=invalid_callback` | Missing code or state |
| 302 | `/?oidc_error=invalid_or_expired_state` | State invalid/expired |
| 302 | `/?oidc_error=authentication_failed` | Token validation failure |
| 302 | `/?oidc_error=access_denied` | User not in admin group (if required) |
#### GET /api/auth/logout
**Purpose**: Logout (session invalidation)
**Response**:
```json
{
"success": true
}
```
### Tracker Endpoints
#### GET /api/tracker
**Purpose**: Monthly tracker data
**Query Parameters**:
- `year` (optional, default: current year)
- `month` (optional, default: current month)
**Response**:
```json
{
"year": 2026,
"month": 5,
"today": "2026-05-09",
"summary": {
"total_expected": 450.00,
"total_starting": 500.00,
"has_starting_amounts": true,
"total_paid": 320.00,
"remaining": 180.00,
"overdue": 75.00,
"count_paid": 5,
"count_upcoming": 3,
"count_late": 2,
"count_autodraft": 1
},
"rows": [
{
"id": 1,
"name": "Rent",
"category_name": "Housing",
"due_date": "2026-05-01",
"expected_amount": 1200.00,
"actual_amount": 1200.00,
"total_paid": 1200.00,
"balance": 0,
"status": "paid",
"autopay_enabled": false,
"autodraft_status": "none"
},
...
]
}
```
#### GET /api/tracker/upcoming
**Purpose**: Bills due in next N days
**Query Parameters**:
- `days` (optional, default: 30, max: 365)
**Response**:
```json
{
"days": 30,
"today": "2026-05-09",
"upcoming": [
{
"id": 2,
"name": "Internet",
"category_name": "Phone & Internet",
"due_date": "2026-05-15",
"expected_amount": 60.00,
"status": "due_soon",
"days_until_due": 6
}
]
}
```
### Bills Endpoints
#### GET /api/bills
**Purpose**: List all bills (active or all)
**Query Parameters**:
- `inactive` (optional): If "true", include inactive bills
**Response**: Array of bill objects
#### GET /api/bills/:id
**Purpose**: Get single bill by ID
**Response**: Bill object
#### POST /api/bills
**Purpose**: Create new bill
**Request**:
```json
{
"name": "Internet",
"category_id": 5,
"due_day": 15,
"override_due_date": null,
"expected_amount": 60.00,
"interest_rate": null,
"billing_cycle": "monthly",
"autopay_enabled": false,
"autodraft_status": "none",
"website": null,
"username": null,
"account_info": null,
"has_2fa": false,
"notes": "Fiber optic internet",
"history_visibility": "default"
}
```
**Response**: Created bill with ID
#### PUT /api/bills/:id
**Purpose**: Update bill
**Request**: Partial bill object
**Response**: Updated bill
#### DELETE /api/bills/:id
**Purpose**: Hard-delete bill (irreversible)
**Response**:
```json
{
"success": true,
"deleted_bill_id": 1,
"deleted_bill_name": "Rent",
"warning": "Bill and all associated payments, monthly state, and history ranges were permanently deleted."
}
```
#### GET /api/bills/:id/payments
**Purpose**: List payments for a bill
**Query Parameters**:
- `page` (optional, default: 1)
- `limit` (optional, default: 20, max: 100)
**Response**:
```json
{
"bill_id": 1,
"bill_name": "Rent",
"total": 5,
"page": 1,
"limit": 20,
"pages": 1,
"payments": [...]
}
```
#### POST /api/bills/:id/toggle-paid
**Purpose**: Toggle bill as paid/unpaid
**Request**:
```json
{
"amount": 1200.00,
"paid_date": "2026-05-01",
"method": "ACH",
"notes": "Rent payment"
}
```
**Response**:
```json
{
"success": true,
"isPaid": true,
"action": "created_payment",
"payment": { ... }
}
```
#### GET /api/bills/:id/monthly-state
**Purpose**: Get monthly state override for a bill
**Query Parameters**:
- `year` (required)
- `month` (required)
**Response**:
```json
{
"bill_id": 1,
"year": 2026,
"month": 5,
"actual_amount": 1200.00,
"notes": null,
"is_skipped": false
}
```
#### PUT /api/bills/:id/monthly-state
**Purpose**: Set monthly state override
**Request**:
```json
{
"year": 2026,
"month": 5,
"actual_amount": 1250.00,
"notes": "Rent increased",
"is_skipped": false
}
```
**Response**: Saved state
### Categories Endpoints
#### GET /api/categories
**Purpose**: List all categories for user
**Response**: Array of category objects
#### POST /api/categories
**Purpose**: Create new category
**Request**:
```json
{
"name": "Entertainment"
}
```
**Response**: Created category
### Settings Endpoints
#### GET /api/settings
**Purpose**: Get all settings
**Response**:
```json
{
"currency": "USD",
"date_format": "MM/DD/YYYY",
"grace_period_days": "5",
"notify_days_before": "3",
"backup_enabled": "false",
...
}
```
#### PUT /api/settings
**Purpose**: Update settings
**Request**: Partial settings object
**Response**: Updated settings
### User Endpoints
#### GET /api/user
**Purpose**: Get current user profile
**Response**: User object (without password)
### Calendar Endpoints
#### GET /api/calendar
**Purpose**: Calendar data for a month
**Query Parameters**:
- `year` (optional)
- `month` (optional)
**Response**: Calendar data with bill due dates
### Summary Endpoints
#### GET /api/summary
**Purpose**: Monthly spending summary
**Query Parameters**:
- `year` (optional)
- `month` (optional)
**Response**:
```json
{
"year": 2026,
"month": 5,
"total_expected": 450.00,
"total_actual": 425.00,
"total_paid": 425.00,
"total_starting": 500.00,
"remaining": 75.00,
"by_category": [...],
"by_bill": [...]
}
```
### Analytics Endpoints
#### GET /api/analytics
**Purpose**: Analytics data with filters
**Query Parameters**:
- `start_date` (optional, default: 30 days ago)
- `end_date` (optional, default: today)
- `category_id` (optional)
- `bill_id` (optional)
**Response**:
```json
{
"start_date": "2026-04-09",
"end_date": "2026-05-09",
"total_spent": 425.00,
"expected_vs_actual": {
"expected": 450.00,
"actual": 425.00,
"difference": -25.00
},
"by_category": [...],
"payment_history": [...]
}
```
### Profile Endpoints
#### GET /api/profile
**Purpose**: Get user profile
**Response**: User profile
#### POST /api/profile
**Purpose**: Update user profile
**Request**:
```json
{
"display_name": "John Doe",
"notification_email": "john@example.com",
"notifications_enabled": true,
"notify_3d": true,
"notify_1d": true,
"notify_due": true,
"notify_overdue": true,
"current_password": "oldpass123",
"new_password": "newpass456"
}
```
**Response**: Updated user
**Rate Limit**: 5 per 15 minutes per IP
### Admin Endpoints
**All admin routes require `requireAuth` + `requireAdmin` + `csrfMiddleware` + `adminActionLimiter`**
#### GET /api/admin/has-users
**Purpose**: Check if other users exist (lockout protection)
**Response**:
```json
{
"has_users": true
}
```
#### GET /api/admin/users
**Purpose**: List all users
**Response**: Array of user objects with admin fields
#### POST /api/admin/users
**Purpose**: Create new user
**Request**:
```json
{
"username": "newuser",
"password": "password123"
}
```
**Response**: Created user
**Errors**:
| Status | Message |
|--------|---------|
| 400 | Username/password too short |
| 409 | Username already taken |
#### PUT /api/admin/users/:id/password
**Purpose**: Reset user password
**Request**:
```json
{
"password": "newpassword123"
}
```
**Response**:
```json
{
"success": true
}
```
**Effects**:
- Updates password hash
- Sets `must_change_password = 1`
- Invalidates all user sessions
#### PUT /api/admin/users/:id/role
**Purpose**: Promote/demote user
**Request**:
```json
{
"role": "admin"
}
```
**Response**: Updated user
**Validations**:
- Cannot change own role
- Cannot remove last admin
- Deletes all sessions for target user
#### PUT /api/admin/users/:id/active
**Purpose**: Deactivate/reactivate user
**Request**:
```json
{
"active": false
}
```
**Response**: Updated user
**Effects**:
- Sets `active = 0/1`
- Invalidates all sessions if deactivated
#### DELETE /api/admin/users/:id
**Purpose**: Delete user (irreversible)
**Response**:
```json
{
"success": true,
"deleted_user_id": 2
}
```
**Effects**:
- Deletes all user data (sessions, imports, exports, bills, categories)
#### GET /api/about-admin
**Purpose**: Retrieve FUTURE.md and DEVELOPMENT_LOG.md content for admin-only About page
**Route**: `/api/about-admin` (admin-only, rate-limited)
**Authentication**: Requires `requireAuth` + `requireAdmin` + `csrfMiddleware` + `adminActionLimiter`
**Rate Limit**: 30 requests per 15 minutes per IP
**Response**:
```json
{
"future": "# Future Roadmap\n\n## Planned Features\n\n- [ ] Feature A\n- [ ] Feature B\n",
"devlog": "# Development Log\n\n## v0.19.1 (2026-05-09)\n\n### Added\n\n- Regular User Seed Environment Variables\n"
}
```
**Errors**:
| Status | Message |
|--------|---------|
| 401 | Unauthorized (not logged in) |
| 403 | Forbidden (not admin) |
| 500 | Internal Server Error (file read failure) |
**Security Measures**:
1. **Path Traversal Protection**`sanitizePath()` validates file paths before access
2. **Content Redaction** — Internal IPs, passwords, and API keys are redacted from content
3. **Error Sanitization** — Error messages exclude file paths to prevent path disclosure
4. **XSS Prevention**`rehype-sanitize` applied to ReactMarkdown rendering in `AboutPage.jsx`
**Implementation**:
- Route: `routes/aboutAdmin.js`
- Server entry: `server.js` (mounted at `/api/about-admin`)
- Client: `client/api.js` (`aboutAdmin()` function)
- UI: `client/pages/AboutPage.jsx` (rendered via `ReactMarkdown` with `rehypeSanitize`)
---
### Backup Endpoints
2026-05-09 13:03:36 -05:00
| Method | Endpoint | Purpose | Rate Limit |
|--------|----------|---------|------------|
| POST | `/api/admin/backups` | Create backup | 5/60min |
| GET | `/api/admin/backups` | List backups | 5/60min |
| GET | `/api/admin/backups/:id/download` | Download backup | - |
| POST | `/api/admin/backups/:id/restore` | Restore backup | 5/60min |
| DELETE | `/api/admin/backups/:id` | Delete backup | 5/60min |
| POST | `/api/admin/backups/import` | Import backup | 5/60min |
| GET | `/api/admin/backups/settings` | Get backup schedule | 5/60min |
| PUT | `/api/admin/backups/settings` | Update backup schedule | 5/60min |
| POST | `/api/admin/backups/run-scheduled-now` | Run scheduled backup now | 5/60min |
**Backup Request/Response**:
```json
// POST /api/admin/backups
// Response (201 Created)
{
"id": "bill-tracker-backup-2026-05-09-03-45-32-456Z-abcd1234.sqlite",
"filename": "bill-tracker-backup-2026-05-09-03-45-32-456Z-abcd1234.sqlite",
"type": "manual",
"created_at": "2026-05-09T03:45:32.456Z",
"modified_at": "2026-05-09T03:45:32.456Z",
"size_bytes": 200704,
"checksum": "abc123def456..."
}
```
**Restore Request/Response**:
```json
// POST /api/admin/backups/:id/restore
// Response
{
"restored_from": "bill-tracker-backup-2026-05-08-02-00-00-000Z-1234abcd.sqlite",
"pre_restore_backup": "pre-restore-2026-05-09-03-46-00-000Z-5678efgh.sqlite",
"restored_at": "2026-05-09T03:46:00.000Z",
"restart_required": false
}
```
**Import Backup**:
- Body: Binary SQLite file
- Header: `X-Checksum-SHA256: <hash>` (optional, if provided, validated)
- Max size: 100MB
#### Cleanup Endpoints
| Method | Endpoint | Purpose |
|--------|----------|---------|
| GET | `/api/admin/cleanup` | Get cleanup settings and status |
| PUT | `/api/admin/cleanup` | Update cleanup settings |
| POST | `/api/admin/cleanup/run` | Run cleanup immediately |
**Cleanup Settings**:
```json
{
"import_sessions_enabled": true,
"temp_exports_enabled": true,
"temp_export_max_age_hours": 2,
"backup_partials_enabled": true,
"import_history_enabled": false,
"import_history_max_age_days": 365
}
```
#### Auth Mode Endpoints
| Method | Endpoint | Purpose |
|--------|----------|---------|
| GET | `/api/admin/auth-mode` | Get auth configuration |
| PUT | `/api/admin/auth-mode` | Update auth configuration |
| POST | `/api/admin/auth-mode/oidc-test` | Test OIDC configuration |
**Auth Mode Settings**:
```json
{
"auth_mode": "multi",
"local_login_enabled": true,
"oidc_login_enabled": false,
"oidc_configured": false,
"oidc_issuer_url_set": false,
"oidc_client_id_set": false,
"oidc_client_secret_set": false,
"oidc_redirect_uri_set": false,
"oidc_missing_fields": ["issuer URL", "client ID", "client secret", "redirect URI"],
"can_disable_local": false,
"warnings": []
}
```
### Export Endpoints
#### GET /api/export
**Purpose**: Export user data as XLSX
**Query Parameters**:
- `export_type` (optional): 'bills', 'payments', 'categories', 'full'
- `start_date`, `end_date` (optional): Date range
**Response**: XLSX file download
**Rate Limit**: 30 per 15 minutes per IP
### Import Endpoints
#### GET /api/import
**Purpose**: Get import history and preview settings
#### POST /api/import
**Purpose**: Preview or apply import
**Form Data**:
- `file`: XLSX file
- `preview`: 'true' or 'false'
**Response** (preview):
```json
{
"preview": true,
"rows_parsed": 10,
"rows_created": 8,
"rows_updated": 2,
"rows_skipped": 0,
"rows_errored": 0,
"data": [...]
}
```
**Response** (apply):
```json
{
"preview": false,
"imported_at": "2026-05-09T03:50:00.000Z",
"rows_parsed": 10,
"rows_created": 8,
"rows_updated": 2,
"rows_skipped": 0,
"rows_errored": 0
}
```
**Rate Limit**: 20 per 15 minutes per IP
### Status Endpoints
#### GET /api/status
**Purpose**: System status (admin only)
**Response**:
```json
{
"version": "0.19.0",
"node_env": "production",
"db_path": "/data/bills.db",
"backup_path": "/data/backups",
"sqlite_version": "3.45.0",
"users_count": 2,
"bills_count": 15,
"last_worker_run": "2026-05-09T06:00:00.000Z",
"last_worker_status": "success",
"uptime_seconds": 86400,
"last_error": null
}
```
#### GET /api/about-admin
2026-05-09 13:03:36 -05:00
**Purpose**: Admin-only access to FUTURE.md and DEVELOPMENT_LOG.md content
**Auth**: `requireAuth` + `requireAdmin` middleware required
**Rate Limit**: 30 per 15 minutes per IP (adminActionLimiter)
**CSRF**: Required (csrfMiddleware)
**Request**: None
2026-05-09 13:03:36 -05:00
**Response**:
```json
{
"future": "# Bill Tracker — Future Improvements...",
"developmentLog": "# Bill Tracker — Development Log..."
2026-05-09 13:03:36 -05:00
}
```
**Errors**:
| Status | Code | Message |
|--------|------|---------|
| 401 | `AUTH_ERROR` | Not authenticated |
| 403 | `FORBIDDEN` | Access denied: admin account required |
| 400 | `INVALID_FILE_PATH` | Path traversal attempt detected |
| 429 | `RATE_LIMITED` | Too many admin actions |
| 500 | `FILE_READ_ERROR` | Failed to read documentation files |
**Security Measures**:
- Path traversal protection via `sanitizePath()` function
- Content redaction of internal IPs, passwords, API keys
- Error message sanitization to prevent path disclosure
---
## Environment Variables
### Initialization Variables
| Variable | Default | Description |
|----------|---------|-------------|
| `INIT_ADMIN_USER` | `admin` | Username for initial admin account |
| `INIT_ADMIN_PASS` | *required* | Password for initial admin account |
| `INIT_TEST_USER` | `testuser` | Username for initial test admin account |
| `INIT_TEST_PASS` | `testpass123` | Password for initial test admin account |
| `INIT_REGULAR_USER` | `regularuser` | Username for initial non-admin user (for testing) |
| `INIT_REGULAR_PASS` | `regularpass123` | Password for initial non-admin user |
| `DB_PATH` | `./data/bills.db` | SQLite database file path |
| `PORT` | `3000` | Server port |
| `SESSION_DAYS` | `7` | Session duration in days |
| `COOKIE_SECURE` | *auto-detect* | Force HTTPS-only cookies |
| `HTTPS` | *auto-detect* | Server running behind HTTPS proxy |
| `CSRF_HTTP_ONLY` | `true` | CSRF cookie httpOnly flag |
| `CSRF_SAME_SITE` | `strict` | CSRF cookie SameSite policy |
| `CSRF_SECURE` | *auto-detect* | CSRF cookie HTTPS-only |
| `INIT_REGULAR_USER` | *optional* | Create non-admin user on first run if set |
| `INIT_REGULAR_PASS` | *optional* | Password for regular user (if INIT_REGULAR_USER set) |
**First-Run User Creation Flow:**
On first startup, if no users exist:
1. If `INIT_ADMIN_USER` and `INIT_ADMIN_PASS` are set:
- Create admin user with `role='admin'` and `is_default_admin=1`
- If `INIT_TEST_USER` and `INIT_TEST_PASS` are also set:
- Create test admin user with `role='admin'` and `is_default_admin=0`
2. If `INIT_REGULAR_USER` and `INIT_REGULAR_PASS` are set:
- Create regular user with `role='user'` and `is_default_admin=0`
Regular users are created with `bcryptjs` password hashing and default notification settings.
| `DB_PATH` | `./data/bills.db` | SQLite database file path |
| `PORT` | `3000` | Server port |
| `SESSION_DAYS` | `7` | Session duration in days |
| `COOKIE_SECURE` | *auto-detect* | Force HTTPS-only cookies |
| `HTTPS` | *auto-detect* | Server running behind HTTPS proxy |
| `CSRF_HTTP_ONLY` | `true` | CSRF cookie httpOnly flag |
| `CSRF_SAME_SITE` | `strict` | CSRF cookie SameSite policy |
| `CSRF_SECURE` | *auto-detect* | CSRF cookie HTTPS-only |
### OIDC Variables
| Variable | Description |
|----------|-------------|
| `OIDC_ISSUER_URL` | Authentik discovery URL |
| `OIDC_CLIENT_ID` | OIDC client ID |
| `OIDC_CLIENT_SECRET` | OIDC client secret |
| `OIDC_REDIRECT_URI` | Callback URL |
| `OIDC_SCOPES` | Space-separated scopes |
| `OIDC_ADMIN_GROUP` | Group requiring admin access |
| `OIDC_AUTO_PROVISION` | Auto-create users from OIDC |
---
## Security Measures
### Rate Limiting
| Limiter | Max | Window | Endpoints |
|---------|-----|--------|-----------|
| `loginLimiter` | 10 | 15 min | `/api/auth/login` |
| `passwordLimiter` | 5 | 15 min | `/api/profile`, `/api/admin/users/:id/password` |
| `importLimiter` | 20 | 15 min | `/api/import/*` |
| `exportLimiter` | 30 | 15 min | `/api/export/*` |
| `adminActionLimiter` | 30 | 15 min | `/api/admin/*`, `/api/about-admin` |
| `oidcLimiter` | 20 | 15 min | `/api/auth/oidc/*` |
| `backupOperationLimiter` | 5 | 60 min | `/api/admin/backups/*` |
### Authentication & Authorization
| Feature | Implementation |
|---------|----------------|
| Session duration | 7 days |
| Password hashing | bcryptjs (salt rounds: 10) |
| CSRF protection | Configurable via env vars |
| Admin guard | `requireAdmin` middleware |
### Content Security
| Measure | Implementation |
|---------|----------------|
| XSS prevention | `rehype-sanitize` on markdown content |
| Path traversal | `sanitizePath()` in `routes/aboutAdmin.js` |
| Content redaction | Internal IPs, passwords, API keys redacted |
| Error sanitization | Stack traces excluded, paths obscured |
### Input Validation
| Field | Validation |
|-------|-----------|
| `due_day` | Integer 1-31 |
| `expected_amount` | Number ≥ 0 |
| `interest_rate` | Number 0-100 or null |
| `password` | Min 8 characters (admin) |
| `username` | Min 3 characters (admin) |
2026-05-09 13:03:36 -05:00
---
## 6. Database Documentation
### Schema Overview
**Database**: SQLite (better-sqlite3)
**Schema Location**: `db/schema.sql`
**Migration Logic**: `db/database.js` (runMigrations function)
### Migration System
#### schema_migrations Table
The `schema_migrations` table tracks applied database migrations to ensure idempotent, repeatable deployments.
| Column | Type | Constraints | Description |
|--------|------|-------------|-------------|
| `id` | INTEGER | PRIMARY KEY AUTOINCREMENT | Internal tracking ID |
| `version` | TEXT | NOT NULL, UNIQUE | Migration version (e.g., `v0.2`, `v0.41`) |
| `description` | TEXT | NOT NULL | Human-readable migration description |
| `applied_at` | TEXT | NOT NULL DEFAULT (datetime('now')) | Timestamp of application |
**Purpose**: Prevent duplicate migration execution and provide audit trail of applied migrations.
**Helper Functions**:
- `hasMigrationBeenApplied(version)`: Returns `true` if version exists in `schema_migrations`
- `recordMigration(version, description)`: Inserts record into `schema_migrations`
- `runMigrations()`: Iterates through all migrations, skipping already-applied versions
#### Migration Process
1. On startup, `initSchema()` creates `schema_migrations` table if it doesn't exist
2. `runMigrations()` queries each migration version in order
3. Applied migrations are skipped with a log message
4. Pending migrations are executed and recorded
5. All migrations are logged with status (`[migration] Applying X` vs `[migration] Skipping already applied X`)
#### Migration Format
Migrations are defined as versioned objects in `db/database.js`:
```javascript
const migrations = [
{
version: 'v0.2',
description: 'payments: soft-delete column',
run: function() { /* SQL logic */ }
},
// ... additional migrations
];
```
**Key Properties**:
- `version`: Unique version string (e.g., `v0.2`, `v0.41`)
- `description`: Brief description of what changed
- `run()`: Function containing migration SQL logic
**Idempotency**: Each migration can safely be re-run after a `git pull`; skipped migrations log but don't error.
2026-05-09 13:03:36 -05:00
### Table Definitions
#### users
| Column | Type | Constraints | Description |
|--------|------|-------------|-------------|
| `id` | INTEGER | PRIMARY KEY, AUTOINCREMENT | User ID |
| `username` | TEXT | NOT NULL, UNIQUE (CASE-insensitive) | Login username |
| `password_hash` | TEXT | NOT NULL | bcrypt hash of password |
| `role` | TEXT | NOT NULL, CHECK('admin', 'user') | Role: admin or user |
| `active` | INTEGER | NOT NULL, DEFAULT 1 | 1=active, 0=deactivated |
| `is_default_admin` | INTEGER | NOT NULL, DEFAULT 0 | 1=initial admin account |
| `must_change_password` | INTEGER | NOT NULL, DEFAULT 0 | 1=force password change on next login |
| `first_login` | INTEGER | NOT NULL, DEFAULT 1 | 1=user has never logged in |
| `created_at` | TEXT | DEFAULT (datetime('now')) | Account creation time |
| `updated_at` | TEXT | DEFAULT (datetime('now')) | Last update time |
| `notification_email` | TEXT | | User's email for notifications |
| `notifications_enabled` | INTEGER | NOT NULL, DEFAULT 0 | 1=receive email notifications |
| `notify_3d` | INTEGER | NOT NULL, DEFAULT 1 | Notify 3 days before due |
| `notify_1d` | INTEGER | NOT NULL, DEFAULT 1 | Notify 1 day before due |
| `notify_due` | INTEGER | NOT NULL, DEFAULT 1 | Notify on due date |
| `notify_overdue` | INTEGER | NOT NULL, DEFAULT 1 | Notify for overdue bills |
| `display_name` | TEXT | | Display name (OIDC) |
| `last_password_change_at` | TEXT | | Last password change timestamp |
| `auth_provider` | TEXT | NOT NULL, DEFAULT 'local' | 'local' or 'oidc' |
| `external_subject` | TEXT | | OIDC sub claim |
| `email` | TEXT | | OIDC email claim |
| `last_login_at` | TEXT | | Last login timestamp |
**Indexes**:
- `idx_sessions_user_id` (on sessions.user_id)
- `idx_sessions_expires` (on sessions.expires_at)
#### sessions
| Column | Type | Constraints | Description |
|--------|------|-------------|-------------|
| `id` | TEXT | PRIMARY KEY | Session UUID |
| `user_id` | INTEGER | NOT NULL, REFERENCES users(id) ON DELETE CASCADE | User ID |
| `expires_at` | TEXT | NOT NULL | Expiration timestamp |
| `created_at` | TEXT | DEFAULT (datetime('now')) | Session creation time |
**Indexes**:
- `idx_sessions_user_id` on `user_id`
- `idx_sessions_expires` on `expires_at`
#### bills
| Column | Type | Constraints | Description |
|--------|------|-------------|-------------|
| `id` | INTEGER | PRIMARY KEY, AUTOINCREMENT | Bill ID |
| `user_id` | INTEGER | NOT NULL, REFERENCES users(id) ON DELETE CASCADE | Owner user ID |
| `name` | TEXT | NOT NULL | Bill name |
| `category_id` | INTEGER | REFERENCES categories(id) ON DELETE SET NULL | Category reference |
| `due_day` | INTEGER | NOT NULL, CHECK(1-31) | Due day of month |
| `override_due_date` | TEXT | | Custom due date override |
| `bucket` | TEXT | CHECK('1st', '15th') | Payment bucket |
| `expected_amount` | REAL | NOT NULL, DEFAULT 0 | Expected monthly amount |
| `interest_rate` | REAL | CHECK(0-100) | APR or interest rate |
| `billing_cycle` | TEXT | DEFAULT 'monthly', CHECK | 'monthly', 'quarterly', 'annually', 'irregular' |
| `autopay_enabled` | INTEGER | NOT NULL, DEFAULT 0 | 1=autopay enabled |
| `autodraft_status` | TEXT | NOT NULL, DEFAULT 'none' | 'none', 'pending', 'assumed_paid', 'confirmed' |
| `website` | TEXT | | Bill provider website |
| `username` | TEXT | | Bill provider username |
| `account_info` | TEXT | | Bill account info |
| `has_2fa` | INTEGER | NOT NULL, DEFAULT 0 | 1=2FA enabled |
| `history_visibility` | TEXT | NOT NULL, DEFAULT 'default' | 'default', 'all', 'ranges', 'none' |
| `active` | INTEGER | NOT NULL, DEFAULT 1 | 1=active, 0=inactive |
| `notes` | TEXT | | User notes |
| `is_seeded` | INTEGER | NOT NULL, DEFAULT 0 | 1=demo data |
| `created_at` | TEXT | DEFAULT (datetime('now')) | Creation time |
| `updated_at` | TEXT | DEFAULT (datetime('now')) | Last update time |
**Indexes**:
- `idx_bills_active` on `active`
- `idx_bills_user_active` on `user_id, active`
- `idx_bills_user_id` on `user_id`
#### categories
| Column | Type | Constraints | Description |
|--------|------|-------------|-------------|
| `id` | INTEGER | PRIMARY KEY, AUTOINCREMENT | Category ID |
| `user_id` | INTEGER | NOT NULL, REFERENCES users(id) ON DELETE CASCADE | Owner user ID |
| `name` | TEXT | NOT NULL | Category name |
| `created_at` | TEXT | DEFAULT (datetime('now')) | Creation time |
| `updated_at` | TEXT | DEFAULT (datetime('now')) | Last update time |
**Constraints**:
- Unique constraint: `UNIQUE(user_id, name COLLATE NOCASE)`
**Indexes**:
- `idx_categories_user_name` on `user_id, name`
- `idx_categories_user_name_unique` on `user_id, name COLLATE NOCASE`
#### payments
| Column | Type | Constraints | Description |
|--------|------|-------------|-------------|
| `id` | INTEGER | PRIMARY KEY, AUTOINCREMENT | Payment ID |
| `bill_id` | INTEGER | NOT NULL, REFERENCES bills(id) ON DELETE CASCADE | Bill reference |
| `amount` | REAL | NOT NULL | Payment amount |
| `paid_date` | TEXT | NOT NULL | Payment date (YYYY-MM-DD) |
| `method` | TEXT | | Payment method (cash, check, card, ACH) |
| `notes` | TEXT | | Payment notes |
| `deleted_at` | TEXT | | Soft-delete timestamp |
| `created_at` | TEXT | DEFAULT (datetime('now')) | Creation time |
| `updated_at` | TEXT | DEFAULT (datetime('now')) | Last update time |
**Indexes**:
- `idx_payments_bill_id` on `bill_id`
- `idx_payments_paid_date` on `paid_date`
- `idx_payments_bill_date_del` on `bill_id, paid_date, deleted_at`
- `idx_payments_deleted` on `deleted_at`
#### monthly_bill_state
| Column | Type | Constraints | Description |
|--------|------|-------------|-------------|
| `id` | INTEGER | PRIMARY KEY, AUTOINCREMENT | State ID |
| `bill_id` | INTEGER | NOT NULL, REFERENCES bills(id) ON DELETE CASCADE | Bill reference |
| `year` | INTEGER | NOT NULL, CHECK(2000-2100) | Year |
| `month` | INTEGER | NOT NULL, CHECK(1-12) | Month |
| `actual_amount` | REAL | | Actual amount paid (override expected) |
| `notes` | TEXT | | Month-specific notes |
| `is_skipped` | INTEGER | NOT NULL, DEFAULT 0 | 1=skip this month |
| `created_at` | TEXT | DEFAULT (datetime('now')) | Creation time |
| `updated_at` | TEXT | DEFAULT (datetime('now')) | Last update time |
**Constraints**:
- `UNIQUE(bill_id, year, month)`
**Indexes**:
- `idx_monthly_bill_state_lookup` on `bill_id, year, month`
#### settings
| Column | Type | Constraints | Description |
|--------|------|-------------|-------------|
| `key` | TEXT | PRIMARY KEY | Setting key |
| `value` | TEXT | NOT NULL | Setting value |
| `updated_at` | TEXT | DEFAULT (datetime('now')) | Last update time |
#### notifications
| Column | Type | Constraints | Description |
|--------|------|-------------|-------------|
| `id` | INTEGER | PRIMARY KEY, AUTOINCREMENT | Notification ID |
| `bill_id` | INTEGER | NOT NULL, REFERENCES bills(id) ON DELETE CASCADE | Bill reference |
| `user_id` | INTEGER | NOT NULL, REFERENCES users(id) ON DELETE CASCADE | User reference |
| `year` | INTEGER | NOT NULL | Year |
| `month` | INTEGER | NOT NULL | Month |
| `type` | TEXT | NOT NULL | Notification type |
| `sent_date` | TEXT | NOT NULL, DEFAULT (date('now')) | Date sent |
**Constraints**:
- `UNIQUE(bill_id, user_id, year, month, type, sent_date)`
**Indexes**:
- `idx_notifications_lookup` on `bill_id, user_id, year, month`
#### oidc_states
| Column | Type | Constraints | Description |
|--------|------|-------------|-------------|
| `id` | TEXT | PRIMARY KEY | State UUID |
| `nonce` | TEXT | NOT NULL | Nonce for replay protection |
| `code_verifier` | TEXT | NOT NULL | PKCE code verifier |
| `redirect_to` | TEXT | | Redirect URL after login |
| `created_at` | TEXT | NOT NULL | Creation time |
| `expires_at` | TEXT | NOT NULL | Expiration time |
**Indexes**:
- `idx_oidc_states_expires` on `expires_at`
#### bill_history_ranges
| Column | Type | Constraints | Description |
|--------|------|-------------|-------------|
| `id` | INTEGER | PRIMARY KEY, AUTOINCREMENT | Range ID |
| `bill_id` | INTEGER | NOT NULL, REFERENCES bills(id) ON DELETE CASCADE | Bill reference |
| `start_year` | INTEGER | NOT NULL | Start year |
| `start_month` | INTEGER | NOT NULL | Start month |
| `end_year` | INTEGER | | End year (NULL = open-ended) |
| `end_month` | INTEGER | | End month (NULL = open-ended) |
| `label` | TEXT | | Range label |
| `created_at` | TEXT | DEFAULT (datetime('now')) | Creation time |
| `updated_at` | TEXT | DEFAULT (datetime('now')) | Last update time |
**Indexes**:
- `idx_bill_history_ranges_bill` on `bill_id`
#### monthly_income
| Column | Type | Constraints | Description |
|--------|------|-------------|-------------|
| `id` | INTEGER | PRIMARY KEY, AUTOINCREMENT | Income record ID |
| `user_id` | INTEGER | NOT NULL, REFERENCES users(id) ON DELETE CASCADE | User reference |
| `year` | INTEGER | NOT NULL, CHECK(2000-2100) | Year |
| `month` | INTEGER | NOT NULL, CHECK(1-12) | Month |
| `label` | TEXT | NOT NULL, DEFAULT 'Salary' | Income source |
| `amount` | REAL | NOT NULL, DEFAULT 0 | Income amount |
| `created_at` | TEXT | DEFAULT (datetime('now')) | Creation time |
| `updated_at` | TEXT | DEFAULT (datetime('now')) | Last update time |
**Constraints**:
- `UNIQUE(user_id, year, month)`
**Indexes**:
- `idx_monthly_income_user_month` on `user_id, year, month`
#### monthly_starting_amounts
| Column | Type | Constraints | Description |
|--------|------|-------------|-------------|
| `id` | INTEGER | PRIMARY KEY, AUTOINCREMENT | Amount record ID |
| `user_id` | INTEGER | NOT NULL, REFERENCES users(id) ON DELETE CASCADE | User reference |
| `year` | INTEGER | NOT NULL, CHECK(2000-2100) | Year |
| `month` | INTEGER | NOT NULL, CHECK(1-12) | Month |
| `first_amount` | REAL | NOT NULL, DEFAULT 0, CHECK ≥ 0 | Amount for 1st of month |
| `fifteenth_amount` | REAL | NOT NULL, DEFAULT 0, CHECK ≥ 0 | Amount for 15th of month |
| `other_amount` | REAL | NOT NULL, DEFAULT 0, CHECK ≥ 0 | Other amount |
| `notes` | TEXT | | Notes |
| `created_at` | TEXT | DEFAULT (datetime('now')) | Creation time |
| `updated_at` | TEXT | DEFAULT (datetime('now')) | Last update time |
**Constraints**:
- `UNIQUE(user_id, year, month)`
**Indexes**:
- `idx_monthly_starting_amounts_user_month` on `user_id, year, month`
#### import_sessions
| Column | Type | Constraints | Description |
|--------|------|-------------|-------------|
| `id` | TEXT | PRIMARY KEY | Session UUID |
| `user_id` | INTEGER | NOT NULL | User reference |
| `created_at` | TEXT | NOT NULL | Creation time |
| `expires_at` | TEXT | NOT NULL | Expiration time |
| `preview_json` | TEXT | NOT NULL | JSON preview data |
**Indexes**:
- `idx_import_sessions_user` on `user_id`
- `idx_import_sessions_expires` on `expires_at`
#### import_history
| Column | Type | Constraints | Description |
|--------|------|-------------|-------------|
| `id` | INTEGER | PRIMARY KEY, AUTOINCREMENT | History ID |
| `user_id` | INTEGER | NOT NULL | User reference |
| `imported_at` | TEXT | NOT NULL | Import timestamp |
| `source_filename` | TEXT | | Source filename |
| `file_type` | TEXT | DEFAULT 'xlsx' | File type |
| `sheet_name` | TEXT | | Sheet name |
| `rows_parsed` | INTEGER | DEFAULT 0 | Parsed rows |
| `rows_created` | INTEGER | DEFAULT 0 | New rows created |
| `rows_updated` | INTEGER | DEFAULT 0 | Existing rows updated |
| `rows_skipped` | INTEGER | DEFAULT 0 | Skipped rows |
| `rows_ambiguous` | INTEGER | DEFAULT 0 | Ambiguous rows |
| `rows_errored` | INTEGER | DEFAULT 0 | Errored rows |
| `options_json` | TEXT | | Import options |
| `summary_json` | TEXT | | Summary JSON |
**Indexes**:
- `idx_import_history_user` on `user_id`
### Data Flow
#### User-scoped Data
All user-modifiable data is scoped to `user_id`:
| Table | user_id Reference | onDelete |
|-------|------------------|----------|
| bills | `user_id` | CASCADE |
| categories | `user_id` | CASCADE |
| payments | bills.user_id | CASCADE |
| monthly_bill_state | bills.user_id (via bill_id) | CASCADE |
| monthly_income | `user_id` | CASCADE |
| monthly_starting_amounts | `user_id` | CASCADE |
| import_sessions | `user_id` | N/A |
| import_history | `user_id` | N/A |
#### Data Access Pattern
```sql
-- User bill list
SELECT * FROM bills WHERE user_id = ? AND active = 1 ORDER BY due_day;
-- User categories
SELECT * FROM categories WHERE user_id = ? ORDER BY name;
-- User payments (with bill info)
SELECT p.*, b.name AS bill_name, b.due_day
FROM payments p
JOIN bills b ON b.id = p.bill_id
WHERE b.user_id = ? AND p.deleted_at IS NULL;
-- User monthly state
SELECT * FROM monthly_bill_state WHERE bill_id IN (
SELECT id FROM bills WHERE user_id = ?
) AND year = ? AND month = ?;
```
### Migration System
**Migration Location**: `db/database.js` (runMigrations function)
**Migration Types**:
1. **Column additions**: `ALTER TABLE users ADD COLUMN column_name type`
2. **Table creations**: `CREATE TABLE IF NOT EXISTS`
3. **Index additions**: `CREATE INDEX IF NOT EXISTS`
4. **Schema rewrites**: Table rename + recreate (for breaking changes)
**Security Features**:
- Column name whitelist validation
- SQL definition string validation
- No user input in ALTER statements
**Migration Log** (from `db/schema.sql` comments):
- v0.2: `payments.deleted_at` column
- v0.3: `idx_payments_bill_date_del` index
- v0.4: `monthly_bill_state` table
- v0.13: Profile columns (`display_name`, `last_password_change_at`)
- v0.14: `bills.history_visibility`, `bill_history_ranges` table, `bills.interest_rate`
- v0.14.4: `bills.interest_rate` column
- v0.15: Cleanup worker settings
- v0.17: OIDC columns (`auth_provider`, `external_subject`, `email`, `last_login_at`)
- v0.17: `oidc_states` table
- v0.18: Monthly income, monthly starting amounts tables
- v0.18.2: Monthly starting amounts table
- v0.18.3: `monthly_starting_amounts.other_amount` column
- v0.18.1: Monthly income table
- v0.38: Import history table
- v0.39: Import sessions table
- v0.40: User-scoped bills/categories
- v0.41: Seeded flags (`is_seeded`)
### Entity Relationship Diagram
```
┌─────────────────┐
│ users │
├─────────────────┤
│ id (PK) │
│ username (U) │
│ password_hash │
│ role │
│ active │
│ ... │
└────────┬────────┘
│ 1:N
┌────────▼────────┐ ┌─────────────────┐
│ sessions │ │ bills │
├─────────────────┤ ├─────────────────┤
│ id (PK) │ │ id (PK) │
│ user_id (FK) │ │ user_id (FK) │
│ expires_at │ │ ... │
│ created_at │ └────────┬────────┘
└─────────────────┘ │
│ 1:N
┌────────────┴────────────┐
│ │
┌──────────▼──────────┐ ┌──────────▼──────────┐
│ categories │ │ payments │
├─────────────────────┤ ├─────────────────────┤
│ id (PK) │ │ id (PK) │
│ user_id (FK) │ │ bill_id (FK) │
│ ... │ │ ... │
└─────────────────────┘ └──────────┬──────────┘
┌────────────┴────────────┐
│ │
┌──────────▼──────────┐ ┌──────────▼──────────┐
│ monthly_bill_state│ │ monthly_income │
├─────────────────────┤ ├─────────────────────┤
│ id (PK) │ │ id (PK) │
│ bill_id (FK) │ │ user_id (FK) │
│ year, month (U) │ │ year, month (U) │
│ ... │ │ ... │
└─────────────────────┘ └─────────────────────┘
```
---
## 7. Error Handling & Troubleshooting
### Troubleshooting Matrix
| Symptom | Likely Cause | Logs to Inspect | Files to Inspect | Services Involved | Recovery Steps |
|---------|--------------|-----------------|------------------|-------------------|----------------|
| **Login fails** | | | | | |
| Invalid credentials | Wrong username/password | `server.js` console | `routes/authLogin.js`, `services/authService.js` | authService | Verify credentials, check for typos |
| Session expired | Session deleted or expired | `server.js` console | `services/authService.js` | authService, session pruning worker | Re-login |
| Account locked | `active = 0` | `server.js` console | `db/schema.sql` (users.active) | AuthService | Admin sets `active = 1` |
| Password mismatch | Hash changed | `server.js` console | `services/authService.js` | authService | Reset password via admin |
| Local login disabled | Admin disabled it | `server.js` console | `db/schema.sql` (settings) | Settings | Enable local login in Admin |
| **Auth issues** | | | | | |
| CSRF token invalid | Token mismatch or expired | `server.js` console | `middleware/csrf.js` | CSRF middleware | Refresh page |
| Role insufficient | User role check failed | `server.js` console | `middleware/requireAuth.js` | Auth middleware | Login as admin/user |
| OIDC callback fails | Provider error or config issue | `server.js` console | `routes/authOidc.js`, `services/oidcService.js` | oidcService | Check OIDC config in Admin |
| **API failures** | | | | | |
| 404 Not Found | Endpoint or resource missing | `server.js` console | `routes/*.js` | Route handlers | Verify endpoint, check resource ID |
| 400 Bad Request | Validation error | `server.js` console | `routes/*.js`, `services/*.js` | Validation logic | Check request body, query params |
| 429 Rate Limited | Too many requests | `server.js` console | `middleware/rateLimiter.js` | Rate limiter | Wait, reduce request frequency |
| 500 Server Error | Unhandled exception | `server.js` console, `NODE_ENV=production` | Any | All services | Check server logs, reproduce |
### Database issues
| Symptom | Likely Cause | Logs to Inspect | Files to Inspect | Services Involved | Recovery Steps |
|---------|--------------|-----------------|------------------|-------------------|----------------|
2026-05-09 13:03:36 -05:00
| Database locked | WAL checkpoint or long transaction | `server.js` console | `db/database.js` | Database connection | Wait, restart app if needed |
| Schema mismatch | Migration failed | `server.js` console | `db/database.js` (runMigrations) | Database init | Check migrations, fix manually |
| Migration runs repeatedly | Version not recorded | `server.js` console | `db/database.js` | Migration system | Manually record version in `schema_migrations` table |
| Migration conflicts | Concurrent startup | `server.js` console | `db/database.js` | Migration system | Restart once, migrations are idempotent |
2026-05-09 13:03:36 -05:00
| Foreign key violation | Related record deleted | `server.js` console | `db/schema.sql` | Schema | Check related data, adjust onDelete |
| **Performance issues** | | | | | |
| Slow queries | Missing index, complex join | `server.js` console | `db/database.js` (schema) | Database | Add indexes, optimize queries |
| Memory leak | Session or cache buildup | `server.js` console | `services/*.js` | Service cleanup | Restart app, check cleanup workers |
| High CPU | Cron job, import, export | `server.js` console | `workers/*.js`, `routes/*.js` | Background jobs | Schedule jobs off-peak |
| **Notification issues** | | | | | |
| Emails not sent | SMTP not configured or failing | `server.js` console | `services/notificationService.js` | notificationService | Configure SMTP in Admin, test |
| Notification not recorded | Unique constraint violation | `server.js` console | `db/schema.sql` (notifications) | notificationService | Check notification table |
| **Backup/restore issues** | | | | | |
| Backup fails | Disk full, permission denied | `server.js` console | `services/backupService.js` | backupService | Check disk space, permissions |
| Restore fails | Backup corrupted, wrong DB | `server.js` console | `services/backupService.js` | backupService | Verify backup integrity, restore from good backup |
| Import/Export fails | Invalid file, permission | `server.js` console | `routes/export.js`, `routes/import.js` | import/export | Check file format, permissions |
| **Worker failures** | | | | | |
| Daily worker not running | Cron not scheduled or errored | `server.js` console | `workers/dailyWorker.js` | dailyWorker | Check cron, restart app |
| Cleanup failing | Disk space, permission | `server.js` console | `services/cleanupService.js` | cleanupService | Check disk space, permissions |
### Common Error Codes and Solutions
| Error Code | Status | Message | Solution |
|------------|--------|---------|----------|
| `AUTH_ERROR` | 401 | Not authenticated | Re-login |
| `VALIDATION_ERROR` | 400 | Input validation failed | Check request body |
| `FORBIDDEN` | 403 | Access denied | Login as correct role |
| `NOT_FOUND` | 404 | Resource not found | Check ID |
| `CONFLICT` | 409 | Duplicate entry | Check for existing record |
| `RATE_LIMITED` | 429 | Too many requests | Wait or reduce requests |
| `CSRF_INVALID` | 403 | CSRF token validation failed | Refresh page |
| `IMPORT_REQUEST_ERROR` | 400 | Import request failed | Smaller/valid file |
| `IMPORT_SERVER_ERROR` | 500 | Import server error | Check server logs |
### Migration Troubleshooting
| Issue | Symptom | Cause | Resolution |
|-------|---------|-------|------------|
| Migration runs on every startup | `[migration] Applying v0.2`, repeated logs | Version not recorded in `schema_migrations` | Manually insert version: `INSERT INTO schema_migrations (version, description) VALUES ('v0.2', 'payments: soft-delete column')` |
| Migration fails with duplicate column | `SQLITE_ERROR: duplicate column name` | Migration re-ran after partial success | Verify column exists, manually record version, restart |
| Concurrent startup conflicts | `[migration] Applying`, then error | Multiple instances start simultaneously | Migrations are idempotent; restart once after all instances stable |
| Version mismatch | Expected migration not applied | Version string changed between runs | Ensure version strings match exactly; update `schema_migrations` if needed |
2026-05-09 13:03:36 -05:00
### Log Files and Locations
| Log Source | Location | Purpose |
|------------|----------|---------|
| **Server console** | stdout/stderr | All console.log/error output |
| **Debug logs** | Set `NODE_DEBUG=express` | Express internals |
| **SQL logs** | `process.env.DEBUG=better-sqlite3` | SQL queries |
| **Migration logs** | `[migration] Applying X`, `[migration] Skipping already applied X` | Migration execution status |
2026-05-09 13:03:36 -05:00
### Recovery Procedures
#### 1. Admin Locked Out
**Symptom**: Can't login, no users with admin role
**Recovery**:
```bash
# Reset default admin password (if INIT_ADMIN_USER/PASS set)
docker-compose exec app node -e "
const bcrypt = require('bcryptjs');
const hash = bcrypt.hashSync('newpassword123', 12);
console.log(hash);
"
# Then run SQL: UPDATE users SET password_hash='newhash' WHERE is_default_admin=1;
```
---
#### 2. Legacy Database Password Reset (2026-05-09)
**Added**: Automatic password reset for default admin user when `INIT_ADMIN_PASS` is set, including on legacy databases that predate the migration system.
**Problem**: Existing deployments created before v0.19.2 had legacy databases where the admin password was unknown or set during initial setup. On upgrade, there was no way to reset the default admin password without manually accessing the database.
**Solution**: When `INIT_ADMIN_PASS` environment variable is set, the app now automatically resets the password for the user with `is_default_admin = 1` and sets `must_change_password = 1` to force a password change on next login.
**Implementation**: Added logic in `db/database.js` in the `initSchema()` function, running after `seedDefaults()` but before `runMigrations()`:
```javascript
// After seedDefaults() - reset default admin password if INIT_ADMIN_PASS is set
if (process.env.INIT_ADMIN_PASS) {
const initUser = process.env.INIT_ADMIN_USER || 'admin';
const initPass = process.env.INIT_ADMIN_PASS;
const bcrypt = require('bcryptjs');
const newPasswordHash = bcrypt.hashSync(initPass, 12);
// Reset password for the default admin user if INIT_ADMIN_PASS is set
const result = db.prepare(`
UPDATE users SET password_hash = ?, must_change_password = 1
WHERE username = ? AND is_default_admin = 1
`).run(newPasswordHash, initUser);
if (result.changes > 0) {
console.log(`[init] Reset password for default admin user: ${initUser}`);
}
}
```
**Key Features**:
- Only resets password for users with `is_default_admin = 1`
- Always sets `must_change_password = 1` to enforce change on next login
- Only runs when `INIT_ADMIN_PASS` is set (explicit opt-in)
- Logs when password is reset for audit trail
**Testing Verification**:
| Test | Scenario | Expected Result | Status |
|------|----------|-----------------|--------|
| Fresh DB | `INIT_ADMIN_PASS=admin123` set | Admin created with `must_change_password=1`, login works | ✅ |
| Legacy DB | Existing DB with unknown admin password, `INIT_ADMIN_PASS=admin123` | Admin password reset, `must_change_password=1`, login works | ✅ |
| Non-default admin | User `kaspa` with `is_default_admin=0` | Password NOT reset (login with admin123 fails) | ✅ |
| No INIT_ADMIN_PASS | Legacy DB without env var | No password reset, original passwords preserved | ✅ |
**Example Log Output**:
```
[seed] Created initial admin user: admin
DB initialized successfully
Database migrations complete for /data/db/bills.db
Opening DB at: /data/db/bills.db
[init] Reset password for default admin user: admin
[migration] Skipping already applied v0.2: payments: soft-delete column
...
DB initialized successfully
```
**Migration System Enhancement**:
The legacy database reconciliation also ensures that `run` functions exist for all migration entries. Previously, some migrations in `reconcileLegacyMigrations()` only had `check()` functions, which meant they'd be recorded as applied but their SQL changes wouldn't actually be executed if they weren't already present. Now every migration has a `run()` function:
```javascript
const migrations = [
{
version: 'v0.2',
description: 'payments: soft-delete column',
check: function() { /* check if column exists */ },
run: function() { /* actually add column if missing */ } // Added
},
// ... all other migrations have run() functions
];
```
This ensures migrations actually execute their SQL changes when needed, not just record themselves as "already done."
**Security Considerations**:
- Password reset only occurs when `INIT_ADMIN_PASS` is explicitly set (opt-in)
- Only affects users with `is_default_admin = 1` (not other admins)
- Forces password change on next login via `must_change_password = 1`
- No password reset occurs if `INIT_ADMIN_PASS` is not set (preserves existing passwords)
**Files Modified**:
- `db/database.js` — Added password reset logic in `initSchema()`
**Impact**:
- Existing deployments can safely upgrade without manual password reset intervention
- Legacy databases with unknown admin passwords can be recovered by setting `INIT_ADMIN_PASS`
- Password reset is auditable via logs
- Non-default admin users are protected from unintended password changes
2026-05-09 13:03:36 -05:00
#### 2. Corrupted Database
**Symptom**: App fails to start, database errors in logs
**Recovery**:
```bash
# Restore from backup
cp /path/to/backup.sqlite /data/bills.db
# Or rebuild from export (if available)
```
#### 3. Session Starvation
**Symptom**: All users logged out, can't re-login
**Recovery**:
```bash
# Prune all sessions
docker-compose exec app node -e "
const { getDb } = require('./db/database');
const db = getDb();
db.prepare('DELETE FROM sessions').run();
console.log('Sessions pruned');
"
```
#### 4. Rate Limiter Stuck
**Symptom**: All requests return 429, even after waiting
**Recovery**:
```bash
# Restart the app (in-memory rate limiter)
docker-compose restart app
# Or in Node REPL:
node -e "const { resetStores } = require('./middleware/rateLimiter'); resetStores(); console.log('Limiters reset');"
```
#### 5. OIDC Configuration Broken
**Symptom**: OIDC login failing, discovery errors
**Recovery**:
```bash
# Clear OIDC client cache (forces re-discovery)
node -e "
const { invalidateClientCache } = require('./services/oidcService');
invalidateClientCache();
console.log('Client cache invalidated');
"
```
### Debug Commands
#### Check Database Integrity
```bash
docker-compose exec app sqlite3 /data/bills.db "PRAGMA integrity_check;"
```
#### View Active Sessions
```bash
docker-compose exec app node -e "
const { getDb } = require('./db/database');
const db = getDb();
const sessions = db.prepare('SELECT s.id, u.username, s.expires_at FROM sessions s JOIN users u ON u.id = s.user_id WHERE s.expires_at > datetime(\"now\")').all();
console.log(sessions);
"
```
#### List All Users
```bash
docker-compose exec app node -e "
const { getDb } = require('./db/database');
const db = getDb();
const users = db.prepare('SELECT id, username, role, active, auth_provider FROM users').all();
console.log(users);
"
```
#### View Bill Counts by Category
```bash
docker-compose exec app node -e "
const { getDb } = require('./db/database');
const db = getDb();
const counts = db.prepare('SELECT c.name, COUNT(b.id) as count FROM categories c LEFT JOIN bills b ON b.category_id = c.id GROUP BY c.id').all();
console.table(counts);
"
```
---
## 8. Code Navigation Index
### Feature-to-File Mapping
| Feature | Frontend Files | Backend Files | Services | Middleware | Tests |
|---------|----------------|---------------|----------|------------|-------|
| **User Authentication** | `client/pages/LoginPage.jsx`, `client/hooks/useAuth.jsx`, `client/api.js` | `routes/authLogin.js`, `routes/auth.js`, `routes/authOidc.js` | `authService.js`, `oidcService.js` | `requireAuth.js`, `csrf.js` | `test-functional.js`, `scripts/test-oidc-smoke.js` |
| **Monthly Tracker** | `client/pages/TrackerPage.jsx`, `client/components/MobileBillRow.jsx` | `routes/tracker.js` | `statusService.js`, `statusRuntime.js` | `requireAuth.js`, `requireUser.js` | `test-functional.js` |
| **Bill CRUD** | `client/pages/BillsPage.jsx`, `client/components/BillModal.jsx` | `routes/bills.js` | - | `requireAuth.js`, `requireUser.js`, `csrf.js` | `test-functional.js` |
| **Payment Recording** | `client/components/StatusBadge.jsx`, `client/pages/TrackerPage.jsx` | `routes/payments.js`, `routes/bills.js` (toggle-paid) | - | `requireAuth.js`, `requireUser.js`, `csrf.js` | `test-functional.js` |
| **Categories** | `client/pages/CategoriesPage.jsx` | `routes/categories.js` | - | `requireAuth.js`, `requireUser.js`, `csrf.js` | - |
| **Monthly State Overrides** | `client/pages/BillsPage.jsx`, `client/components/TrackerPage.jsx` | `routes/bills.js` (monthly-state) | - | `requireAuth.js`, `requireUser.js`, `csrf.js` | - |
| **Calendar View** | `client/pages/CalendarPage.jsx` | `routes/calendar.js` | `statusService.js` | `requireAuth.js`, `requireUser.js` | - |
| **Summary View** | `client/pages/SummaryPage.jsx` | `routes/summary.js` | - | `requireAuth.js`, `requireUser.js` | - |
| **Analytics** | `client/pages/AnalyticsPage.jsx` | `routes/analytics.js` | - | `requireAuth.js`, `requireUser.js` | - |
| **User Profile** | `client/pages/ProfilePage.jsx` | `routes/user.js`, `routes/profile.js` | `authService.js` | `requireAuth.js`, `requireUser.js`, `passwordLimiter` | - |
| **App Settings** | `client/pages/SettingsPage.jsx` | `routes/settings.js` | - | `requireAuth.js`, `requireUser.js` | - |
| **Notifications** | `client/pages/ProfilePage.jsx` (settings) | `routes/notifications.js` | `notificationService.js`, `statusRuntime.js` | `requireAuth.js` | - |
| **Data Import** | `client/pages/DataPage.jsx` | `routes/import.js` | `spreadsheetImportService.js`, `userDbImportService.js` | `requireAuth.js`, `requireUser.js`, `importLimiter` | `scripts/test-import.js` |
| **Data Export** | `client/pages/DataPage.jsx` | `routes/export.js` | - | `requireAuth.js`, `requireUser.js`, `exportLimiter` | - |
| **Admin User Management** | `client/pages/AdminPage.jsx` | `routes/admin.js` (users) | `authService.js` | `requireAuth.js`, `requireAdmin.js`, `adminActionLimiter` | - |
| **Admin Backups** | `client/pages/AdminPage.jsx` (backups tab) | `routes/admin.js` (backups) | `backupService.js`, `backupScheduler.js` | `requireAuth.js`, `requireAdmin.js`, `backupOperationLimiter` | - |
| **Admin OIDC Config** | `client/pages/AdminPage.jsx` (auth tab) | `routes/admin.js` (auth-mode) | `oidcService.js` | `requireAuth.js`, `requireAdmin.js` | `scripts/test-oidc-smoke.js` |
| **Admin Cleanup** | `client/pages/AdminPage.jsx` (cleanup tab) | `routes/admin.js` (cleanup) | `cleanupService.js` | `requireAuth.js`, `requireAdmin.js` | - |
| **System Status** | `client/pages/StatusPage.jsx` | `routes/status.js` | `statusRuntime.js`, `statusService.js` | `requireAuth.js`, `requireAdmin.js` | - |
### Component File Tree
```
client/
├── components/
│ ├── layout/
│ │ ├── Layout.jsx # Main layout wrapper
│ │ ├── Sidebar.jsx # Navigation sidebar
│ │ ├── BrandBlock.jsx # App branding (logo, title, version)
│ │ └── NavPill.jsx # Nav link item
│ ├── ui/
│ │ ├── button.jsx # shadcn button
│ │ ├── input.jsx # shadcn input
│ │ ├── card.jsx # shadcn card
│ │ ├── table.jsx # shadcn table
│ │ ├── tabs.jsx # shadcn tabs
│ │ ├── dialog.jsx # shadcn dialog
│ │ ├── badge.jsx # shadcn badge
│ │ ├── switch.jsx # shadcn switch
│ │ ├── select.jsx # shadcn select
│ │ ├── dropdown-menu.jsx # shadcn dropdown
│ │ ├── label.jsx # shadcn label
│ │ ├── input-dialog.jsx # Custom dialog with input
│ │ ├── confirm-dialog.jsx # Confirmation dialog
│ │ ├── alert-dialog.jsx # shadcn alert dialog
│ │ ├── separator.jsx # shadcn separator
│ │ ├── tooltip.jsx # shadcn tooltip
│ │ ├── checkbox.jsx # shadcn checkbox
│ │ └── theme-toggle.jsx # Theme switcher
│ ├── BillsTableInner.jsx # Bills table component
│ ├── MobileBillRow.jsx # Mobile bill row
│ ├── MobileTrackerRow.jsx # Mobile tracker row
│ ├── StatusBadge.jsx # Payment status badge
│ ├── SummaryCard.jsx # Summary statistics card
│ ├── MarkdownText.jsx # Markdown renderer
│ ├── ReleaseNotesDialog.jsx # Release notes modal
│ └── ...
├── pages/
│ ├── LoginPage.jsx # Login page
│ ├── TrackerPage.jsx # Monthly tracker
│ ├── BillsPage.jsx # Bill CRUD
│ ├── CategoriesPage.jsx # Category management
│ ├── CalendarPage.jsx # Calendar view
│ ├── SummaryPage.jsx # Monthly summary
│ ├── AnalyticsPage.jsx # Analytics charts
│ ├── ProfilePage.jsx # User profile
│ ├── SettingsPage.jsx # App settings
│ ├── DataPage.jsx # Import/export
│ ├── AdminPage.jsx # Admin panel
│ ├── StatusPage.jsx # System status
│ ├── AboutPage.jsx # Version/info
│ └── ReleaseNotesPage.jsx # Release notes
├── hooks/
│ └── useAuth.jsx # Auth state hook
├── contexts/
│ └── ThemeContext.jsx # Theme state
├── api.js # API client
├── App.jsx # Router config
├── main.jsx # React entry
└── lib/
├── utils.js # Utility functions
└── version.js # Version constants
```
### Service Layer Dependencies
```
services/
├── authService.js # Session management, login/logout
├── oidcService.js # Authentik OIDC integration
├── backupService.js # SQLite backup/restore
├── backupScheduler.js # Scheduled backups
├── notificationService.js # Email notifications
├── cleanupService.js # Cleanup tasks
├── spreadsheetImportService.js # XLSX import
├── userDbImportService.js # SQLite user import
├── statusRuntime.js # Worker/runtime status
└── statusService.js # Tracker status calculations
```
### Middleware Chain by Route
| Route Prefix | Middleware Chain |
|--------------|------------------|
| `/api/auth/login` | `loginLimiter` |
| `/api/auth` | `csrfMiddleware` |
| `/api/auth/oidc` | `csrfMiddleware`, `oidcLimiter` |
| `/api/tracker` | `csrfMiddleware`, `requireAuth`, `requireUser` |
| `/api/bills` | `csrfMiddleware`, `requireAuth`, `requireUser` |
| `/api/payments` | `csrfMiddleware`, `requireAuth`, `requireUser` |
| `/api/categories` | `csrfMiddleware`, `requireAuth`, `requireUser` |
| `/api/settings` | `csrfMiddleware`, `requireAuth`, `requireUser` |
| `/api/user` | `csrfMiddleware`, `requireAuth`, `requireUser` |
| `/api/calendar` | `csrfMiddleware`, `requireAuth`, `requireUser` |
| `/api/summary` | `csrfMiddleware`, `requireAuth`, `requireUser` |
| `/api/monthly-starting-amounts` | `csrfMiddleware`, `requireAuth`, `requireUser` |
| `/api/analytics` | `csrfMiddleware`, `requireAuth`, `requireUser` |
| `/api/notifications` | `csrfMiddleware`, `requireAuth` |
| `/api/profile` | `csrfMiddleware`, `requireAuth`, `requireUser`, `passwordLimiter` |
| `/api/admin` | `csrfMiddleware`, `requireAuth`, `requireAdmin`, `adminActionLimiter` |
| `/api/export` | `csrfMiddleware`, `requireAuth`, `requireUser`, `exportLimiter` |
| `/api/import` | `csrfMiddleware`, `requireAuth`, `requireUser`, `importLimiter` |
| `/api/status` | `csrfMiddleware`, `requireAuth`, `requireAdmin` |
| `/api/about` | (none) |
| `/api/version` | (none) |
### Test Files
| Test File | Purpose |
|-----------|---------|
| `test-functional.js` | Functional tests for all features |
| `run-functional-test.js` | Test runner |
| `scripts/test-import.js` | Import functionality test |
| `scripts/test-oidc-smoke.js` | OIDC configuration smoke test |
| `scripts/test-cookie-options.js` | Cookie options test |
---
## 9. Infrastructure & Deployment
### Docker Setup
#### Dockerfile
```dockerfile
# Base image
FROM node:20-alpine AS base
# Install dependencies
FROM base AS deps
RUN apk add --no-cache libc6-compat
WORKDIR /app
COPY package.json package-lock.json ./
RUN npm ci --only=production
# Build stage
FROM base AS builder
WORKDIR /app
COPY --from=deps /app/node_modules ./node_modules
COPY . .
RUN npm run build
# Production
FROM base AS production
WORKDIR /app
COPY --from=deps /app/node_modules ./node_modules
COPY --from=builder /app/dist ./dist
COPY --from=builder /app/*.js ./
COPY --from=builder /app/scripts ./scripts
COPY --from=builder /app/db ./db
COPY --from=builder /app/middleware ./middleware
COPY --from=builder /app/routes ./routes
COPY --from=builder /app/services ./services
COPY --from=builder /app/workers ./workers
COPY --from=builder /app/client ./client
COPY --from=builder /app/.npmrc ./
COPY --from=builder /app/postcss.config.js ./
COPY --from=builder /app/tailwind.config.js ./
COPY --from=builder /app/vite.config.js ./
COPY --from=builder /app/index.html ./
COPY --from=builder /app/.env.example ./
# Environment variables
ENV NODE_ENV=production
ENV PORT=3000
# Data directories
VOLUME ["/data", "/backups"]
# Entry point
ENTRYPOINT ["./docker-entrypoint.sh"]
CMD ["node", "server.js"]
```
#### docker-compose.yml
```yaml
version: '3.8'
services:
app:
image: bill-tracker:latest
container_name: bill-tracker
ports:
- "3030:3000"
volumes:
- ./data:/data
- ./backups:/backups
environment:
- NODE_ENV=production
- PORT=3000
- DB_PATH=/data/bills.db
- BACKUP_PATH=/data/backups
- HTTPS=true
- COOKIE_SECURE=true
# OIDC (optional)
# - OIDC_ISSUER_URL=
# - OIDC_CLIENT_ID=
# - OIDC_CLIENT_SECRET=
# - OIDC_REDIRECT_URI=
restart: unless-stopped
healthcheck:
test: ["CMD", "wget", "--no-verbose", "--tries=1", "--spider", "http://localhost:3000/api/version"]
interval: 30s
timeout: 10s
retries: 3
start_period: 30s
```
### Environment Variables
| Variable | Default | Required | Description |
|----------|---------|----------|-------------|
| `PORT` | `3000` | No | API server port |
| `NODE_ENV` | `production` | No | Environment mode |
| `DB_PATH` | `db/bills.db` | No | SQLite database path |
| `BACKUP_PATH` | `backups/` | No | Backup directory path |
| `HTTPS` | `false` | No | Enable HTTPS (for HSTS, secure cookies) |
| `COOKIE_SECURE` | `false` | No | Force secure cookies |
| `CORS_ORIGIN` | (disabled) | No | CORS allowed origins (comma-separated) |
| `INIT_ADMIN_USER` | `admin` | No | Initial admin username (first run only) |
| `INIT_ADMIN_PASS` | `admin123` | No | Initial admin password (first run only) |
| `OIDC_ISSUER_URL` | - | No | OIDC issuer URL (fallback) |
| `OIDC_CLIENT_ID` | - | No | OIDC client ID (fallback) |
| `OIDC_CLIENT_SECRET` | - | No | OIDC client secret (fallback) |
| `OIDC_REDIRECT_URI` | - | No | OIDC redirect URI (fallback) |
| `OIDC_SCOPES` | `openid email profile groups` | No | OIDC scopes (fallback) |
| `OIDC_ADMIN_GROUP` | - | No | OIDC admin group name (fallback) |
| `OIDC_AUTO_PROVISION` | `true` | No | Auto-create users from OIDC |
| `OIDC_PROVIDER_NAME` | `authentik` | No | Provider name (fallback) |
### Ports and Services
| Port | Service | Purpose |
|------|---------|---------|
| `3000` | Express | Main API server |
| `3030` | Host (Docker) | Exposed port for app |
### Monitoring & Logging
#### Runtime Status (`statusRuntime.js`)
| Status Type | Key | Description |
|-------------|-----|-------------|
| Worker | `last_worker_run_at` | Last daily worker execution |
| Worker | `last_worker_status` | `success` or `error` |
| Worker | `last_worker_error` | Error message if failed |
| Notification | `last_notification_send_at` | Last email send |
| Notification | `last_notification_error` | Last email error |
| Runtime | `last_error_at` | Last error timestamp |
| Runtime | `last_error_message` | Last error message |
#### Health Check
```bash
# Health endpoint (public)
GET /api/version
# Docker healthcheck
wget --no-verbose --tries=1 --spider http://localhost:3000/api/version
```
### CI/CD Pipeline
**Current**: Manual deployment via `deploy.sh`
**Deploy Script**:
```bash
#!/bin/bash
# deploy.sh
# Build frontend
npm run build
# Sync to server (rsync)
rsync -avz dist/ user@server:/var/www/bill-tracker/
rsync -avz node_modules/ user@server:/var/www/bill-tracker/
# Restart server (migrations run automatically on startup)
2026-05-09 13:03:36 -05:00
ssh user@server "pm2 restart bill-tracker"
```
**Deployment Notes**:
- Database migrations run automatically on startup (no manual intervention required)
- Migrations are idempotent: safe to `git pull` and restart repeatedly
- After `git pull`, migrations are applied in version order during startup
- Check `[migration]` logs for applied/skipped migration status
2026-05-09 13:03:36 -05:00
### Security Considerations
| Feature | Implementation |
|---------|----------------|
| **Password Hashing** | bcrypt with cost factor 12 |
| **Session Storage** | SQLite with 7-day expiry |
| **CSRF Protection** | Double-submit cookie pattern |
| **Rate Limiting** | In-memory express-rate-limit |
| **SQL Injection** | Parameterized queries (prepared statements) |
| **XSS Protection** | CSP with nonce, no inline scripts in HTML emails |
| **CORS** | Disabled by default, configurable via env |
| **HSTS** | Only when HTTPS=true |
| **Secure Cookies** | httpOnly, sameSite=strict, secure when HTTPS |
| **OIDC Security** | PKCE, state/nonce, JWKS signature verification |
| **Backup Security** | chmod 600, SHA-256 checksums, restrictive dir (0o700) |
### Database Backups
**Manual**:
```bash
# Create backup
curl -X POST http://localhost:3000/api/admin/backups
# List backups
curl http://localhost:3000/api/admin/backups
# Download backup
curl -o backup.sqlite http://localhost:3000/api/admin/backups/backup-id.sqlite
```
**Scheduled** (Admin panel):
- Enable: `backup_schedule_enabled`
- Frequency: daily/weekly
- Time: HH:MM
- Retention: N backups
### Troubleshooting Commands
#### View Logs (Docker)
```bash
docker-compose logs -f app
```
#### Restart App (Docker)
```bash
docker-compose restart app
```
#### Rebuild Image
```bash
docker-compose build --no-cache
docker-compose up -d
```
#### Enter Container Shell
```bash
docker-compose exec app sh
```
#### Check Database Size
```bash
docker-compose exec app ls -lh /data/bills.db
```
---
## 10. Sequence Flows
### Database Initialization & Migration Flow
```
┌─────────────────────────────────────────────────────────────┐
│ Application Startup (server.js) │
│ 1. require('./db/database.js') │
│ 2. getDb() called │
└─────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────┐
│ db/database.js - getDb() │
│ 3. Check if db already initialized (db variable) │
│ 4. If not: call initSchema() │
└─────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────┐
│ db/database.js - initSchema() │
│ 5. Read db/schema.sql and execute │
│ 6. Create schema_migrations table if not exists │
│ CREATE TABLE IF NOT EXISTS schema_migrations ( │
│ id INTEGER PRIMARY KEY AUTOINCREMENT, │
│ version TEXT NOT NULL UNIQUE, │
│ description TEXT NOT NULL, │
│ applied_at TEXT NOT NULL DEFAULT (datetime('now')) │
│ ) │
│ 7. Call runMigrations() │
└─────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────┐
│ db/database.js - runMigrations() │
│ 8. Define all migrations with version/description/run │
│ 9. For each migration: │
│ a. Check hasMigrationBeenApplied(migration.version) │
│ b. If NOT applied: │
│ i. Log: "Applying {version}: {description}" │
│ ii. Execute migration.run() │
│ iii. Call recordMigration(version, description) │
│ c. If already applied: │
│ i. Log: "Skipping already applied {version}" │
└─────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────┐
│ Application continues │
│ 10. Seed defaults (seedDefaults()) │
│ 11. Server starts listening on port │
└─────────────────────────────────────────────────────────────┘
```
**Migration Flow Details**:
1. **First startup after upgrade**: All pending migrations execute in version order
2. **Subsequent startups**: Skipped migrations logged but no SQL executed
3. **Idempotent**: Safe to run `git pull && npm start` repeatedly
4. **Audit trail**: Every applied migration recorded in `schema_migrations`
2026-05-09 13:03:36 -05:00
### Login Flow (Local)
```
┌─────────────────────────────────────────────────────────────┐
│ User │
│ 1. Opens /login.html │
│ 2. Enters username + password │
│ 3. Clicks "Login" button │
└─────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────┐
│ Frontend (LoginPage.jsx) │
│ 4. Calls apiPost('/api/auth/login', {username, password}) │
└─────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────┐
│ Backend (server.js) │
│ 5. POST /api/auth/login │
│ 6. loginLimiter checks IP (skip if no users exist) │
│ 7. authLogin.js handler runs │
└─────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────┐
│ Service (authService.login) │
│ 8. Query user by username │
│ 9. Check user.active === 1 │
│ 10. Check auth_provider === 'local' │
│ 11. bcrypt.compare(password, password_hash) │
│ 12. If match: create session, return {sessionId, user} │
└─────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────┐
│ Response │
│ 13. Set cookie: bt_session=<sessionId>
│ 14. JSON: {user: {id, username, role, ...}} │
│ 15. Set CSRF token cookie (bt_csrf_token) │
└─────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────┐
│ Frontend (useAuth hook) │
│ 16. Stores user in context │
│ 17. Redirects to /tracker or / │
└─────────────────────────────────────────────────────────────┘
```
### OIDC Login Flow
```
┌─────────────────────────────────────────────────────────────┐
│ User │
│ 1. Clicks "Login with Authentik" button │
└─────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────┐
│ Frontend (LoginPage.jsx) │
│ 2. Calls apiGet('/api/auth/oidc/login?redirect_to=/tracker')│
└─────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────┐
│ Backend (authOidc.js) │
│ 3. GET /api/auth/oidc/login │
│ 4. isOidcLoginActive() check │
│ 5. createLoginState(redirect_to) │
│ • Generates PKCE code_verifier │
│ • Generates nonce │
│ • Stores in oidc_states (expires 5 min) │
│ 6. buildAuthorizationUrl(config, state) │
└─────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────┐
│ Response │
│ 7. HTTP 302 redirect to OIDC provider authorization URL │
│ • Includes: client_id, redirect_uri, response_type=code │
│ • Includes: state (login state ID), nonce │
│ • Includes: code_challenge, code_challenge_method=S256 │
│ • Includes: scopes (openid, email, profile, groups) │
└─────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────┐
│ OIDC Provider (Authentik) │
│ 8. User authenticates (credentials) │
│ 9. Provider creates ID token │
│ 10. Redirects to redirect_uri with code + state │
└─────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────┐
│ Backend (authOidc.js) │
│ 11. GET /api/auth/oidc/callback │
│ 12. oidcLimiter checks IP │
│ 13. consumeLoginState(state) │
│ • Validates expiry │
│ • Returns: {nonce, code_verifier, redirect_to} │
│ 14. exchangeAndVerifyTokens(config, code, stateId, savedState) │
│ • POST to token endpoint with code + client_secret │
│ • Verifies JWT signature via JWKS │
│ • Validates: iss, aud, exp, nonce, state │
│ 15. findOrProvisionUser(claims, config) │
│ • Look up by sub (external_subject) │
│ • Look up by email if email_verified=true │
│ • Auto-provision if enabled │
│ • Map groups to role (admin if in admin_group) │
└─────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────┐
│ Service (authService.createSession) │
│ 16. Create session for user │
│ 17. Set cookie: bt_session=<sessionId>
└─────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────┐
│ Response │
│ 18. HTTP 302 redirect to redirect_to (or /) │
└─────────────────────────────────────────────────────────────┘
```
### Authenticated API Request Flow
```
┌─────────────────────────────────────────────────────────────┐
│ Frontend (api.js) │
│ 1. apiGet('/api/tracker', {year, month}) │
│ 2. Retrieve CSRF token from bt_csrf_token cookie │
│ 3. Set header: x-csrf-token=<token>
│ 4. Set header: cookie: bt_session=<sessionId>
│ 5. Send request to backend │
└─────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────┐
│ Backend (server.js) │
│ 6. requestLooksHttps(req) │
│ • Checks: req.secure, x-forwarded-proto │
│ 7. csrfTokenProvider sets cookie (if not present) │
│ 8. route middleware chain runs │
└─────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────┐
│ Middleware (requireAuth) │
│ 9. getSessionUser(cookie.bt_session) │
│ • Query sessions + users table │
│ • Check: expires_at > now, active = 1 │
│ 10. If valid: attach req.user, next() │
│ If invalid: return 401 {error: 'Not authenticated'} │
└─────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────┐
│ Middleware (requireUser) │
│ 11. Check role is 'user' or 'admin' │
│ 12. Check not default admin (no tracker access) │
│ 13. next() │
└─────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────┐
│ Middleware (csrfMiddleware) │
│ 14. validateCsrfToken(req) │
│ • Check header x-csrf-token matches cookie │
│ • Check query.csrf_token │
│ • Check body.csrf_token │
│ 15. If valid: next() │
│ If invalid: return 403 {error: 'CSRF token validation failed'} │
└─────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────┐
│ Route Handler (tracker.js) │
│ 16. ensureUserDefaultCategories(req.user.id) │
│ 17. db.prepare('SELECT * FROM bills WHERE user_id = ?') │
│ 18. Build tracker rows │
│ 19. Return JSON response │
└─────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────┐
│ Frontend (useAuth hook) │
│ 20. Update state with response │
│ 21. Re-render component │
└─────────────────────────────────────────────────────────────┘
```
### Background Worker Flow (Daily)
```
┌─────────────────────────────────────────────────────────────┐
│ Server Start (server.js) │
│ 1. main() │
│ 2. Check if users exist, create admin if needed │
│ 3. workers/dailyWorker.js start() │
└─────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────┐
│ Worker (dailyWorker.js) │
│ 4. markWorkerStarted(nextDailyRunIso()) │
│ 5. runDailyTasks() │
│ • Prune expired sessions │
│ • Run notifications (email) │
│ • Run cleanup (temp files, import sessions) │
│ • Auto-mark autopay bills as assumed_paid │
│ 6. markWorkerSuccess(nextDailyRunIso()) │
└─────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────┐
│ Cron Job (node-cron) │
│ 7. cron.schedule('0 6 * * *', ...) │
│ Runs daily at 6:00 AM │
│ 8. Same runDailyTasks() as above │
└─────────────────────────────────────────────────────────────┘
```
### Notification Flow
```
┌─────────────────────────────────────────────────────────────┐
│ Daily Worker (dailyWorker.js) │
│ 1. runNotifications() │
└─────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────┐
│ Notification Service (notificationService.js) │
│ 2. Check SMTP enabled, host configured │
│ 3. Get recipients (users with notifications_enabled=1 OR │
│ global recipient configured) │
│ 4. For each bill: │
│ • Calculate due date for this month │
│ • Determine notification type (due_3d, due_1d, due_today,│
│ overdue) │
│ • Check if already sent today (notifications table) │
│ • Check user notification preferences │
│ • Build HTML email template │
│ • Send email via nodemailer │
│ • Record in notifications table (to prevent duplicates) │
└─────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────┐
│ Email (Nodemailer) │
│ 5. SMTP transport sendMail() │
│ 6. Return success/error │
└─────────────────────────────────────────────────────────────┘
```
---
## Version History
| Version | Date | Changes |
|---------|------|---------|
| **0.19.0** | 2026-05-09 | Complete Engineering Reference Manual |
| **0.18.3** | 2026-05-08 | Added `monthly_starting_amounts.other_amount` column |
| **0.18.2** | 2026-05-08 | Added `monthly_starting_amounts` table |
| **0.18.1** | 2026-05-08 | Added `monthly_income` table |
| **0.17** | 2026-05-08 | OIDC columns (`auth_provider`, `external_subject`, `email`, `last_login_at`), `oidc_states` table |
| **0.14** | 2026-05-08 | Added `history_visibility`, `bill_history_ranges`, `interest_rate` |
| **0.13** | 2026-05-08 | Profile columns (`display_name`, `last_password_change_at`) |
| **0.12** | 2026-05-08 | Bill history ranges table |
| **0.11** | 2026-05-08 | Import history table |
| **0.10** | 2026-05-08 | Import sessions table |
| **0.4** | 2026-05-08 | `monthly_bill_state` table |
| **0.2** | 2026-05-08 | `payments.deleted_at` column |
---
## Quick Reference
### Critical Endpoints
| Method | Endpoint | Auth | Purpose |
|--------|----------|------|---------|
| POST | `/api/auth/login` | None | Local login |
| GET | `/api/auth/oidc/login` | None | OIDC login start |
| GET | `/api/tracker` | User | Monthly tracker data |
| GET | `/api/bills` | User | List bills |
| GET | `/api/admin/users` | Admin | List users |
| POST | `/api/admin/backups` | Admin | Create backup |
| GET | `/api/version` | None | Version info |
| GET | `/api/about` | None | About info |
| POST | `/api/auth/logout-all` | User | Invalidate all sessions (v0.22.2) |
| POST | `/api/auth/change-password` | User | Change password with session rotation (v0.22.2) |
2026-05-09 13:03:36 -05:00
### Critical Settings
| Key | Default | Description |
|-----|---------|-------------|
| `local_login_enabled` | `true` | Enable username/password login |
| `oidc_login_enabled` | `false` | Enable OIDC login |
| `oidc_issuer_url` | - | OIDC provider URL |
| `oidc_client_id` | - | OIDC client ID |
| `oidc_client_secret` | - | OIDC client secret |
| `oidc_redirect_uri` | - | OIDC redirect URI |
| `oidc_admin_group` | - | OIDC group for admin access |
| `oidc_auto_provision` | `true` | Auto-create users from OIDC |
| `notify_smtp_enabled` | `false` | Enable email notifications |
| `notify_smtp_host` | - | SMTP host |
| `notify_smtp_port` | `587` | SMTP port |
| `backup_enabled` | `false` | Enable manual backups |
| `backup_schedule_enabled` | `false` | Enable scheduled backups |
### Database Tables Reference
| Table | Purpose |
|-------|---------|
| `users` | User accounts |
| `sessions` | Active sessions |
| `bills` | Bill records |
| `categories` | Bill categories |
| `payments` | Payment records |
| `monthly_bill_state` | Monthly bill overrides |
| `settings` | Application settings |
| `notifications` | Notification history |
| `oidc_states` | OIDC login state |
| `bill_history_ranges` | History visibility ranges |
| `monthly_income` | Monthly income records |
| `monthly_starting_amounts` | Starting balance records |
| `import_sessions` | Import preview sessions |
| `import_history` | Import history |
| `audit_log` | Security event tracking (v0.22.0) |
2026-05-09 13:03:36 -05:00
---
**This document is the canonical reference for the Bill Tracker system.**
*Last updated: 2026-05-10*
2026-05-09 13:03:36 -05:00
*Author: Bishop (code reviewer and architecture validator)*
---
## Version 0.22.x Update (2026-05-10)
### React Query Migration (v0.22.0)
**Added:** TanStack Query (React Query) for data fetching and caching.
**Changes:**
- `client/hooks/useQueries.js` — New custom hooks (`useTracker`, `useBills`, `useCategories`)
- `client/App.jsx` — Added `QueryClientProvider` and `ReactQueryDevtools`
- `client/pages/TrackerPage.jsx` — Migrated to `useTracker` hook
**Hooks:**
| Hook | Query Key | Stale Time | Cache Time |
|------|-----------|------------|------------|
| `useTracker(year, month)` | `['tracker', year, month]` | 5 minutes | 30 minutes |
| `useBills()` | `['bills']` | 5 minutes | 30 minutes |
| `useCategories()` | `['categories']` | 1 hour | 2 hours |
**DevTools:**
- `ReactQueryDevtools` is included but disabled by default
- Open via browser console: `ReactQueryDevtools.openDevTools()`
**Benefits:**
- Automatic caching and stale-while-revalidate
- Background refetching on window focus
- Request deduplication
- Optimistic updates
### N+1 Query Optimization (v0.22.1)
**Added:** Batch query execution to eliminate N+1 problems in tracker and analytics.
**Changes:**
- `routes/tracker.js` — Batch payments, monthly state, and prevpayments queries with `billIds.map(() => '?').join(',')`
- `routes/analytics.js` — Batch payments queries
- `db/database.js` — Parameterized IN clauses, empty billIds guards
**Pattern:**
```javascript
// Before (N queries)
for (const bill of bills) {
const payments = db.prepare('SELECT * FROM payments WHERE bill_id = ?').all(bill.id);
}
// After (1 query)
const billIds = bills.map(b => b.id);
if (billIds.length > 0) {
const placeholders = billIds.map(() => '?').join(',');
const payments = db.prepare(`SELECT * FROM payments WHERE bill_id IN (${placeholders})`).all(...billIds);
}
```
**Impact:** Single tracker page load reduced from 50+ queries to ~5.
### Session Token Rotation (v0.22.2)
**Added:** Session rotation on password change and logout-all endpoint.
**Changes:**
- `services/authService.js``rotateSessionId()`, `invalidateOtherSessions()`
- `routes/auth.js``POST /api/auth/logout-all`, password change with session rotation
- `routes/admin.js` — Audit logging for `password.change` and `logout.all`
- `db/database.js``sessions.created_at` column
**Functions:**
| Function | Purpose |
|----------|---------|
| `rotateSessionId(oldSessionId, userId)` | Regenerate session ID on privilege escalation |
| `invalidateOtherSessions(userId, keepSessionId)` | Invalidate all sessions except specified one |
**Endpoints:**
| Endpoint | Auth | Purpose |
|----------|------|---------|
| `POST /api/auth/logout-all` | User | Invalidate all sessions and current session |
| `POST /api/auth/change-password` | User | Change password with session rotation |
**Audit Events:**
| Action | Details |
|--------|---------|
| `password.change` | User password changed, session rotated |
| `logout.all` | User logged out from all sessions |
| `seed.flag_reset` | ENV-seeded user flags reset |
### ENV-Seeded User First-Login Fix (v0.22.3)
**Added:** Skip first-login and must-change-password flags for ENV-seeded users.
**Changes:**
- `setup/firstRun.js``runFromEnv()` resets `first_login=0`, `must_change_password=0`
- `server.js` — Regular user seeding resets flags, logs `seed.flag_reset` audit event
- `db/database.js` — Init code resets flags for default admin user
**Audit Events:**
| Source | Action | Details |
|--------|--------|---------|
| `first-run-env` | `seed.flag_reset` | ENV vars: username, flags: [first_login, must_change_password] |
| `server-seed` | `seed.flag_reset` | Regular user seeding |
---