null/BillTracker
null
/
BillTracker
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases 1 Wiki Activity
BillTracker/docs/Engineering_Reference_Manua...

2746 lines
114 KiB
Markdown
Raw Blame History

# Engineering Reference Manual — Bill Tracker
**Status:** Complete
**Last Updated:** 2026-05-09
**Owner:** Bishop
**Version:** 0.19.0
---
## Table of Contents
1. [High Level Overview](#1-high-level-overview)
2. [Frontend Documentation](#2-frontend-documentation)
3. [Backend Documentation](#3-backend-documentation)
4. [Authentication & Authorization](#4-authentication--authorization)
5. [API Documentation](#5-api-documentation)
6. [Database Documentation](#6-database-documentation)
7. [Error Handling & Troubleshooting](#7-error-handling--troubleshooting)
8. [Code Navigation Index](#8-code-navigation-index)
9. [Infrastructure & Deployment](#9-infrastructure--deployment)
10. [Sequence Flows](#10-sequence-flows)
---
## 1. High Level Overview
### App Purpose
BillTracker is a self-hosted monthly bill tracking system for households and small setups. It manages:
- **Recurring bills**: Track due dates, expected amounts, categories, autopay, interest rates, website login info
- **Monthly tracker**: Record actual payments, skip bills, view spending vs expectations
- **Calendar view**: Visual grid showing due dates and payments
- **Analytics**: Charts, category spend, payment history
- **User management**: Admin creates users, sets roles, manages authentication
- **Notifications**: Email alerts for due bills (3d, 1d, today, overdue)
- **Data management**: Import/Export bills, full database backup/restore
### Architecture Summary
**Stack**: Node.js + Express (backend) + React + Vite (frontend) + SQLite (database)
**Layered Architecture**:
```
┌─────────────────────────────────────────────────────────────┐
│ Frontend (React) │
│ Pages (client/pages/) • Components (client/components/) │
│ Router (client/App.jsx) • API Client (client/api.js) │
└─────────────────────────────────────────────────────────────┘
HTTP/JSON
┌─────────────────────────────────────────────────────────────┐
│ Backend (Express) │
│ Routes (routes/) • Services (services/) • Middleware │
│ Auth (authService.js) • OIDC (oidcService.js) │
└─────────────────────────────────────────────────────────────┘
SQL
┌─────────────────────────────────────────────────────────────┐
│ Database (SQLite) │
│ Schema (db/schema.sql) • Migrations (db/database.js) │
│ Users • Sessions • Bills • Payments • Categories • etc. │
└─────────────────────────────────────────────────────────────┘
```
### Tech Stack
| Layer | Component | Version | Purpose |
|-------|-----------|---------|---------|
| **Runtime** | Node.js | v20+ | Backend server |
| **Framework** | Express | ^4.18.2 | HTTP server, routing, middleware |
| **Frontend** | React | ^18.3.1 | UI components |
| **Build** | Vite | ^5.4.10 | Bundler, dev server |
| **Router** | react-router-dom | ^6.26.2 | Client-side routing |
| **Database** | better-sqlite3 | ^12.9.0 | SQLite wrapper |
| **Auth** | bcryptjs | ^2.4.3 | Password hashing |
| **OIDC** | openid-client | ^5.7.1 | Authentik integration |
| **Email** | nodemailer | ^6.9.14 | SMTP email sending |
| **Scheduler** | node-cron | ^3.0.3 | Background jobs |
| **UI Libs** | shadcn/ui | - | Component primitives |
| **Styling** | TailwindCSS | ^3.4.14 | Utility-first CSS |
### Major Components
| Component | Location | Purpose |
|-----------|----------|---------|
| **server.js** | Root | Express entry, middleware setup, route mounting |
| **db/database.js** | `db/` | SQLite connection, migrations, settings |
| **services/authService.js** | `services/` | Session management, login/logout |
| **services/oidcService.js** | `services/` | Authentik OIDC integration |
| **services/backupService.js** | `services/` | Database backup/restore |
| **middleware/requireAuth.js** | `middleware/` | Auth guard middleware |
| **middleware/csrf.js** | `middleware/` | CSRF token generation/validation |
| **workers/dailyWorker.js** | `workers/` | Daily background tasks |
### Request Lifecycle
```
┌─────────────────────────────────────────────────────────────┐
│ 1. Request arrives at Express (server.js) │
│ • Security headers applied (securityHeaders) │
│ • CSRF token set on response (csrfTokenProvider) │
└─────────────────────────────────────────────────────────────┘
│
┌─────────────────────────────────────────────────────────────┐
│ 2. Route-level middleware chain │
│ • CSRF validation (csrfMiddleware) │
│ • Auth check (requireAuth) │
│ • User role check (requireUser/requireAdmin) │
│ • Rate limiting (loginLimiter, exportLimiter, etc.) │
└─────────────────────────────────────────────────────────────┘
│
┌─────────────────────────────────────────────────────────────┐
│ 3. Route handler (routes/*.js) │
│ • Input validation │
│ • Service layer calls │
│ • Database queries │
└─────────────────────────────────────────────────────────────┘
│
┌─────────────────────────────────────────────────────────────┐
│ 4. Service layer (services/*.js) │
│ • Business logic │
│ • External service calls (SMTP, OIDC) │
│ • Data transformation │
└─────────────────────────────────────────────────────────────┘
│
┌─────────────────────────────────────────────────────────────┐
│ 5. Database layer (db/database.js) │
│ • SQL query execution │
│ • Schema validation (PRAGMA foreign_keys=ON) │
│ • Migration runs (if schema changes detected) │
└─────────────────────────────────────────────────────────────┘
│
┌─────────────────────────────────────────────────────────────┐
│ 6. Response sent to frontend │
│ • JSON format │
│ • CSRF token in cookie │
│ • Error formatted via errorFormatter │
└─────────────────────────────────────────────────────────────┘
```
---
## 2. Frontend Documentation
### Route Mapping
| Route | Page Component | Auth Required | Purpose |
|-------|----------------|---------------|---------|
| `/` | Redirect | No | Redirects to login or app based on auth |
| `/login` | LoginPage.jsx | No | Username/password login form |
| `/tracker` | TrackerPage.jsx | Yes | Monthly bill tracking view |
| `/bills` | BillsPage.jsx | Yes | Bill CRUD interface |
| `/categories` | CategoriesPage.jsx | Yes | Category management |
| `/calendar` | CalendarPage.jsx | Yes | Monthly calendar view |
| `/summary` | SummaryPage.jsx | Yes | Monthly summary/spending view |
| `/analytics` | AnalyticsPage.jsx | Yes | Charts and analytics |
| `/profile` | ProfilePage.jsx | Yes | User profile, password change |
| `/settings` | SettingsPage.jsx | Yes | App settings (theme, format) |
| `/data` | DataPage.jsx | Yes | Import/export data tools |
| `/admin` | AdminPage.jsx | Yes (admin) | User management, backups, OIDC config |
| `/about` | AboutPage.jsx | No | Version info, changelog |
| `/status` | StatusPage.jsx | Yes (admin) | System status, worker health |
### Key Frontend Files
#### Core Files
| File | Purpose | Key Functions |
|------|---------|---------------|
| `client/main.jsx` | React entry point | Creates root, renders App |
| `client/App.jsx` | Router config | Defines all routes, layout wrapper |
| `client/api.js` | API client | `get`, `post`, `put`, `delete`, auth, CSRF |
| `client/hooks/useAuth.jsx` | Auth state | `login`, `logout`, `user`, `loading` |
#### Layout Components
| File | Purpose | Key Features |
|------|---------|--------------|
| `client/components/layout/Layout.jsx` | Main layout wrapper | Sidebar, top bar, content area |
| `client/components/layout/Sidebar.jsx` | Navigation sidebar | Links to all pages, collapse |
| `client/components/layout/BrandBlock.jsx` | App branding | Logo, title, version |
| `client/components/layout/NavPill.jsx` | Nav item | Active state, icon |
#### UI Components (shadcn/ui)
| Component | File | Purpose |
|-----------|------|---------|
| Button | `client/components/ui/button.jsx` | Primary, secondary, ghost |
| Input | `client/components/ui/input.jsx` | Form input |
| Card | `client/components/ui/card.jsx` | Content container |
| Table | `client/components/ui/table.jsx` | Data table |
| Tabs | `client/components/ui/tabs.jsx` | Tab navigation |
| Dialog | `client/components/ui/dialog.jsx` | Modal dialogs |
| Badge | `client/components/ui/badge.jsx` | Status badges |
| Switch | `client/components/ui/switch.jsx` | Toggle switch |
| Alert Dialog | `client/components/ui/alert-dialog.jsx` | Confirm action dialog |
#### Page Components
| Page | Route | API Calls | State |
|------|-------|-----------|-------|
| LoginPage | `/login` | `POST /api/auth/login` | `user`, `error` |
| TrackerPage | `/tracker` | `GET /api/tracker`, `GET /api/tracker/upcoming` | `data`, `year`, `month`, `activeBillId` |
| BillsPage | `/bills` | `GET /api/bills`, `POST /api/bills`, `PUT /api/bills/:id`, `DELETE /api/bills/:id` | `bills`, `categories`, `modalState` |
| CategoriesPage | `/categories` | `GET /api/categories`, `POST /api/categories` | `categories` |
| CalendarPage | `/calendar` | `GET /api/bills`, `GET /api/tracker` | `year`, `month`, `dates` |
| SummaryPage | `/summary` | `GET /api/summary` | `data`, `year`, `month` |
| AnalyticsPage | `/analytics` | `GET /api/analytics` | `filters`, `data` |
| ProfilePage | `/profile` | `GET /api/user`, `POST /api/profile` | `user`, `notifications` |
| SettingsPage | `/settings` | `GET /api/settings`, `PUT /api/settings` | `settings` |
| DataPage | `/data` | `GET /api/import`, `POST /api/import`, `POST /api/export` | `importState`, `exportState` |
| AdminPage | `/admin` | Multiple admin endpoints | `users`, `backups`, `oidc` |
### State Management
**Approach**: Component-local state + React Context (ThemeContext)
**Key State**:
| State | Location | Purpose |
|-------|----------|---------|
| `theme` | ThemeContext | Light/dark mode |
| `user` | useAuth hook | Current user object |
| `loading` | useAuth hook | Auth state |
| `error` | useAuth hook | Auth errors |
| `bills`, `categories`, etc. | Page components | API response data |
### Validation
**Frontend validation occurs before API calls**:
```javascript
// client/api.js - validation wrapper
function validateInput(schema, data) {
// Check required fields
// Type validation
// Range validation (numbers, dates)
}
```
**Common validations**:
| Field | Validation |
|-------|-----------|
| `due_day` | Integer 1-31 |
| `expected_amount` | Number ≥ 0 |
| `interest_rate` | Number 0-100 or null |
| `password` | Min 8 characters (admin) |
| `username` | Min 3 characters (admin) |
| `year/month` | Valid date range |
### API Client (`client/api.js`)
**Key Functions**:
```javascript
// GET request with CSRF token
await apiGet('/api/bills', { year, month })
// POST with CSRF
await apiPost('/api/bills', { name, due_day, ... })
// PUT for updates
await apiPut('/api/bills/:id', { name, ... })
// DELETE
await apiDelete('/api/bills/:id')
```
**CSRF Handling**:
- Token stored in `bt_csrf_token` cookie (httpOnly)
- Sent in `x-csrf-token` header for POST/PUT/DELETE
- Auto-retrieved from cookie on each request
---
## 3. Backend Documentation
### Core Backend Files
| File | Purpose | Key Functions |
|------|---------|---------------|
| `server.js` | Express entry | Middleware setup, route mounting, error handling |
| `db/database.js` | DB connection | SQLite init, migrations, settings |
| `db/schema.sql` | Schema definition | All table definitions |
| `services/authService.js` | Auth service | Login, logout, session management |
| `services/oidcService.js` | OIDC service | Authentik integration |
| `services/backupService.js` | Backup service | SQLite backup/restore |
| `middleware/requireAuth.js` | Auth guards | `requireAuth`, `requireUser`, `requireAdmin` |
| `middleware/csrf.js` | CSRF protection | Token generation/validation |
| `middleware/rateLimiter.js` | Rate limiting | Per-endpoint limits |
| `middleware/securityHeaders.js` | Security headers | CSP, HSTS, XSS protection |
| `middleware/errorFormatter.js` | Error formatting | JSON error responses |
### Route Handlers (routes/*.js)
| Route File | API Prefix | Auth | Purpose |
|------------|------------|------|---------|
| `authLogin.js` | `/api/auth/login` | None | Local login |
| `auth.js` | `/api/auth` | CSRF | Logout, password change |
| `authOidc.js` | `/api/auth/oidc` | CSRF | OIDC login/callback |
| `tracker.js` | `/api/tracker` | Auth+User | Monthly tracking data |
| `bills.js` | `/api/bills` | Auth+User | Bill CRUD |
| `payments.js` | `/api/payments` | Auth+User | Payment CRUD |
| `categories.js` | `/api/categories` | Auth+User | Category CRUD |
| `settings.js` | `/api/settings` | Auth+User | Settings CRUD |
| `user.js` | `/api/user` | Auth+User | User profile |
| `calendar.js` | `/api/calendar` | Auth+User | Calendar data |
| `summary.js` | `/api/summary` | Auth+User | Monthly summary |
| `monthly-starting-amounts.js` | `/api/monthly-starting-amounts` | Auth+User | Starting balance |
| `analytics.js` | `/api/analytics` | Auth+User | Analytics data |
| `notifications.js` | `/api/notifications` | Auth+User | Notification settings |
| `admin.js` | `/api/admin` | Auth+Admin | Admin functions |
| `export.js` | `/api/export` | Auth+User | Data export |
| `import.js` | `/api/import` | Auth+User | Data import |
| `status.js` | `/api/status` | Auth+Admin | System status |
| `about.js` | `/api/about` | None | Version info |
| `version.js` | `/api/version` | None | Version string |
### Service Layer Functions
#### authService.js
| Function | Purpose | Parameters | Returns |
|----------|---------|------------|---------|
| `login(username, password)` | Authenticate user | `username`, `password` | `{sessionId, user}` or null |
| `logout(sessionId)` | Destroy session | `sessionId` | void |
| `createSession(userId)` | Create session for OIDC | `userId` | `{sessionId, user}` |
| `getSessionUser(sessionId)` | Validate session | `sessionId` | `user` or null |
| `hashPassword(password)` | Hash password | `password` | `Promise<hash>` |
| `publicUser(user)` | Strip sensitive data | `user` object | Public user object |
| `cookieOpts(req)` | Get cookie options | `req` | `{httpOnly, sameSite, secure, maxAge}` |
| `pruneExpiredSessions()` | Clean expired sessions | None | void |
| `rotateSessionId(oldId, userId)` | Security rotation | `oldId`, `userId` | `newId` or null |
#### oidcService.js
| Function | Purpose | Parameters | Returns |
|----------|---------|------------|---------|
| `getOidcConfig()` | Get effective config | None | Config object or null |
| `isOidcLoginActive()` | Check if enabled | None | boolean |
| `createLoginState(redirectTo)` | Create PKCE state | `redirectTo` | `{id, nonce, codeVerifier}` |
| `consumeLoginState(stateId)` | Validate state | `stateId` | State or null |
| `buildAuthorizationUrl(config, state)` | Build redirect URL | `config`, `state` | `Promise<URL>` |
| `exchangeAndVerifyTokens(config, code, stateId, savedState)` | Exchange code for tokens | `config`, `code`, `stateId`, `savedState` | Verified claims |
| `findOrProvisionUser(claims, config)` | Find or create user | `claims`, `config` | User object |
| `mapRoleFromClaims(claims, config)` | Map groups to role | `claims`, `config` | `'admin'` or `'user'` |
| `testOidcConfiguration(config)` | Test OIDC setup | `config` | `{ok, error, ...}` |
| `getAdminOidcSettings()` | Admin settings | None | Settings object |
| `getPublicOidcInfo()` | Public info | None | `{oidc_enabled, oidc_provider_name}` |
#### backupService.js
| Function | Purpose | Parameters | Returns |
|----------|---------|------------|---------|
| `createBackup(prefix)` | Create SQLite backup | `prefix` | `{id, filename, size_bytes, checksum}` |
| `restoreBackup(id)` | Restore from backup | `backupId` | `{restored_from, pre_restore_backup}` |
| `deleteBackup(id)` | Delete backup | `backupId` | `{deleted: true, id, deleted_at}` |
| `listBackups()` | List backups | None | Array of backup metadata |
| `getBackupFile(id)` | Get backup path | `backupId` | `{path, metadata}` |
| `importBackupBuffer(buffer, options)` | Import backup | `buffer`, `{expectedChecksum}` | Backup metadata |
| `validateSqliteDatabase(filePath)` | Validate DB file | `filePath` | void or throws |
| `checksumFile(filePath)` | SHA-256 checksum | `filePath` | `hex string` |
#### notificationService.js
| Function | Purpose | Parameters | Returns |
|----------|---------|------------|---------|
| `runNotifications()` | Send due bill emails | None | void |
| `sendTestEmail(to)` | Test SMTP config | `email` | void |
| `createTransport()` | Create SMTP transport | None | Nodemailer transport |
#### cleanupService.js
| Function | Purpose | Parameters | Returns |
|----------|---------|------------|---------|
| `runAllCleanup()` | Run all cleanup tasks | None | `{import_sessions, temp_exports, ...}` |
| `validateAndApplySettings(settings)` | Update cleanup config | `settings` | Updated config |
| `getCleanupStatus()` | Get cleanup status | None | `{settings, last_run, last_result}` |
#### statusService.js
| Function | Purpose | Parameters | Returns |
|----------|---------|------------|---------|
| `buildTrackerRow(bill, payments, year, month, today)` | Build tracker row | `bill`, `payments`, `year`, `month`, `today` | Row object |
| `resolveDueDate(bill, year, month)` | Calculate due date | `bill`, `year`, `month` | `YYYY-MM-DD` |
| `getCycleRange(year, month)` | Get date range | `year`, `month` | `{start, end}` |
### Middleware Chain
#### requireAuth.js
| Middleware | Purpose | Check |
|------------|---------|-------|
| `requireAuth` | General auth | Session valid, user active |
| `requireUser` | User role | Role is 'user' or 'admin', not default admin |
| `requireAdmin` | Admin role | Role is 'admin' |
#### CSRF Protection
| Setting | Default | Purpose |
|---------|---------|---------|
| `CSRF_HTTP_ONLY` | `true` | Cookie not accessible via JS |
| `CSRF_SAME_SITE` | `'strict'` | Same-site cookie |
| `CSRF_SECURE` | `true` | HTTPS only |
| `CSRF_COOKIE_NAME` | `'bt_csrf_token'` | Cookie name |
**Flow**:
1. `csrfTokenProvider` sets cookie on every response
2. `csrfMiddleware` validates token on POST/PUT/DELETE
3. Token can be in header, query, or body
#### Rate Limiters
| Limiter | Max | Window | Endpoints |
|---------|-----|--------|-----------|
| `loginLimiter` | 10 | 15 min | `/api/auth/login` |
| `passwordLimiter` | 5 | 15 min | `/api/profile`, `/api/admin/users/:id/password` |
| `importLimiter` | 20 | 15 min | `/api/import/*` |
| `exportLimiter` | 30 | 15 min | `/api/export/*` |
| `adminActionLimiter` | 30 | 15 min | `/api/admin/*` |
| `oidcLimiter` | 20 | 15 min | `/api/auth/oidc/*` |
| `backupOperationLimiter` | 5 | 60 min | `/api/admin/backups/*` |
### Error Handling
#### errorFormatter.js
| Error Type | Status Code | Response Format |
|------------|-------------|-----------------|
| Validation | 400 | `{error: 'Validation failed', field: 'field_name'}` |
| Auth | 401 | `{error: 'Not authenticated', code: 'AUTH_ERROR'}` |
| Forbidden | 403 | `{error: 'Access denied', code: 'FORBIDDEN'}` |
| Not Found | 404 | `{error: 'Not found', code: 'NOT_FOUND'}` |
| Conflict | 409 | `{error: 'Already exists', code: 'CONFLICT'}` |
| Rate Limit | 429 | `{error: 'Too many requests'}` |
| Server | 500 | `{error: 'Internal server error'}` |
**Standard Error Format**:
```javascript
{
error: 'Error message',
code: 'ERROR_CODE',
field: 'optional_field_name'
}
```
---
## 4. Authentication & Authorization
### Login Flow
```
┌─────────────────────────────────────────────────────────────┐
│ 1. User submits login form │
│ • Username, password │
└─────────────────────────────────────────────────────────────┘
│
┌─────────────────────────────────────────────────────────────┐
│ 2. POST /api/auth/login │
│ • rateLimiter (loginLimiter) │
│ • Body: {username, password} │
└─────────────────────────────────────────────────────────────┘
│
┌─────────────────────────────────────────────────────────────┐
│ 3. authService.login() │
│ • Query user by username │
│ • Check active flag │
│ • Check auth_provider === 'local' │
│ • bcrypt.compare(password, password_hash) │
└─────────────────────────────────────────────────────────────┘
│
┌─────────────────────────────────────────────────────────────┐
│ 4. Create session │
│ • Generate UUID for sessionId │
│ • Insert into sessions table (expires in 7 days) │
│ • Update last_login_at │
│ • Return {sessionId, user} │
└─────────────────────────────────────────────────────────────┘
│
┌─────────────────────────────────────────────────────────────┐
│ 5. Set cookie │
│ • Cookie: bt_session=<sessionId> │
│ • httpOnly: true, sameSite: strict, secure: depends │
│ • Max-Age: 7 days │
└─────────────────────────────────────────────────────────────┘
│
┌─────────────────────────────────────────────────────────────┐
│ 6. Return response │
│ • JSON: {user: {id, username, display_name, role, ...}} │
│ • CSRF token set on cookie │
└─────────────────────────────────────────────────────────────┘
```
### Session/JWT Handling
**Session Storage**: SQLite `sessions` table
| Column | Type | Purpose |
|--------|------|---------|
| `id` | TEXT (UUID) | Session identifier |
| `user_id` | INTEGER | Reference to users.id |
| `expires_at` | TEXT (ISO) | Expiration timestamp |
| `created_at` | TEXT (ISO) | Session creation time |
**Session Validation**:
```sql
SELECT u.id, u.username, u.display_name, u.role, u.must_change_password,
u.active, u.is_default_admin
FROM sessions s
JOIN users u ON u.id = s.user_id
WHERE s.id = ? AND s.expires_at > datetime('now') AND u.active = 1
```
**Session Duration**: 7 days
**Session Destruction**:
- Explicit logout: `DELETE FROM sessions WHERE id = ?`
- Session expiry: Daily worker pruning
- User deactivation: `DELETE FROM sessions WHERE user_id = ?`
- Role change: `DELETE FROM sessions WHERE user_id = ?`
### RBAC (Role-Based Access Control)
| Role | Capabilities |
|------|--------------|
| `user` | View/modify own bills, categories, payments, settings, profile |
| `admin` | All user capabilities + user management, backups, OIDC config, system settings |
**Admin Guard**:
```javascript
function requireAdmin(req, res, next) {
if (req.user?.role !== 'admin') {
return res.status(403).json({error: 'Access denied: admin account required'});
}
next();
}
```
### Middleware Chain
**Route Protection Example**:
```javascript
// Admin routes
app.use('/api/admin', csrfMiddleware, requireAuth, requireAdmin,
adminActionLimiter, require('./routes/admin'));
// User routes
app.use('/api/bills', csrfMiddleware, requireAuth, requireUser,
require('./routes/bills'));
// Public routes (no auth)
app.use('/api/about', require('./routes/about'));
app.use('/api/version', require('./routes/version'));
```
### Cookie Handling
| Cookie | Name | Type | Secure | SameSite | Purpose |
|--------|------|------|--------|----------|---------|
| Session | `bt_session` | httpOnly | Configurable | `strict` | Auth session |
| CSRF | `bt_csrf_token` | httpOnly | Configurable | `strict` | CSRF token |
**Cookie Options** (determined at runtime):
```javascript
function cookieOpts(req) {
const cookieSecure = envFlag('COOKIE_SECURE');
const httpsSecure = envFlag('HTTPS');
const secure = cookieSecure !== null
? cookieSecure
: httpsSecure !== null
? httpsSecure
: requestLooksHttps(req); // Check X-Forwarded-Proto
return {
httpOnly: true,
sameSite: 'strict',
secure,
maxAge: SESSION_DAYS * 86400 * 1000, // 7 days
path: '/',
};
}
```
### OIDC/Authentik Flow
```
┌─────────────────────────────────────────────────────────────┐
│ 1. User clicks OIDC login button │
│ • Redirects to frontend login page │
└─────────────────────────────────────────────────────────────┘
│
┌─────────────────────────────────────────────────────────────┐
│ 2. Frontend calls /api/auth/oidc/login │
│ • Query params: ?redirect_to=/tracker │
└─────────────────────────────────────────────────────────────┘
│
┌─────────────────────────────────────────────────────────────┐
│ 3. Create login state │
│ • Generate PKCE code_verifier (32 bytes base64url) │
│ • Generate nonce (16 bytes hex) │
│ • Store in oidc_states table (expires 5 min) │
│ • Code challenge = SHA256(code_verifier) base64url │
└─────────────────────────────────────────────────────────────┘
│
┌─────────────────────────────────────────────────────────────┐
│ 4. Redirect to OIDC provider │
│ • Authorization URL built with: │
│ • client_id, redirect_uri, response_type=code │
│ • state (login state ID), nonce │
│ • code_challenge, code_challenge_method=S256 │
│ • scopes: openid, email, profile, groups │
└─────────────────────────────────────────────────────────────┘
│
┌─────────────────────────────────────────────────────────────┐
│ 5. User authenticates with OIDC provider │
│ • Authentik validates credentials │
│ • Provider sends user back to redirect_uri │
└─────────────────────────────────────────────────────────────┘
│
┌─────────────────────────────────────────────────────────────┐
│ 6. Callback: /api/auth/oidc/callback │
│ • Query params: code, state │
│ • Rate limited (oidcLimiter) │
└─────────────────────────────────────────────────────────────┘
│
┌─────────────────────────────────────────────────────────────┐
│ 7. Exchange code for tokens │
│ • POST to OIDC token endpoint │
│ • client_id + client_secret + code + redirect_uri │
│ • code_verifier for PKCE │
└─────────────────────────────────────────────────────────────┘
│
┌─────────────────────────────────────────────────────────────┐
│ 8. Verify ID token │
│ • JWT signature via JWKS │
│ • Issuer validation (iss claim) │
│ • Audience validation (aud claim) │
│ • Expiry validation (exp claim) │
│ • Nonce validation (replay protection) │
│ • State validation (replay protection) │
└─────────────────────────────────────────────────────────────┘
│
┌─────────────────────────────────────────────────────────────┐
│ 9. Find or provision user │
│ • Look up by sub (external_subject) │
│ • Look up by email if email_verified=true │
│ • Auto-provision if OIDC_AUTO_PROVISION=true │
│ • Map groups to role (admin if in admin_group) │
└─────────────────────────────────────────────────────────────┘
│
┌─────────────────────────────────────────────────────────────┐
│ 10. Create local session │
│ • Same mechanism as local login │
│ • Set bt_session cookie │
│ • Redirect to redirect_to (or /) │
└─────────────────────────────────────────────────────────────┘
```
**OIDC Configuration**:
```bash
# Environment variables (fallback if DB settings blank)
OIDC_ISSUER_URL=https://auth.example.com/application/o/bills/.well-known/openid-configuration
OIDC_CLIENT_ID=<client-id>
OIDC_CLIENT_SECRET=<client-secret>
OIDC_REDIRECT_URI=https://bills.example.com/api/auth/oidc/callback
OIDC_SCOPES=openid email profile groups
OIDC_ADMIN_GROUP=bill-tracker-admins
OIDC_AUTO_PROVISION=true
```
### Failure Scenarios
| Scenario | Status Code | Response | Recovery |
|----------|-------------|----------|----------|
| Invalid credentials | 401 | `{error: 'Invalid username or password', code: 'AUTH_ERROR'}` | Retry with correct credentials |
| Session expired | 401 | `{error: 'Not authenticated', code: 'AUTH_ERROR'}` | Re-login |
| Role insufficient | 403 | `{error: 'Access denied: admin account required', code: 'FORBIDDEN'}` | Login as admin |
| Rate limited | 429 | `{error: 'Too many login attempts...'}` | Wait 15 minutes |
| CSRF invalid | 403 | `{error: 'CSRF token validation failed', code: 'CSRF_INVALID'}` | Refresh page |
| OIDC config missing | 501 | `{error: 'OIDC authentication is not configured...'}` | Configure OIDC in Admin |
| OIDC provider error | 502 | `{error: 'Failed to reach the identity provider...'}` | Check OIDC provider status |
| OIDC callback expired state | Redirect + query param | `/?oidc_error=invalid_or_expired_state` | Start login flow again |
### Code Locations
| Component | File |
|-----------|------|
| Login endpoint | `routes/authLogin.js` |
| Auth middleware | `middleware/requireAuth.js` |
| CSRF middleware | `middleware/csrf.js` |
| Session management | `services/authService.js` |
| OIDC login | `routes/authOidc.js` |
| OIDC service | `services/oidcService.js` |
| OIDC tables | `db/schema.sql` (sessions, oidc_states) |
---
## 5. API Documentation
### Authentication Endpoints
#### POST /api/auth/login
**Purpose**: Local username/password login
**Request**:
```json
{
"username": "string (required)",
"password": "string (required)"
}
```
**Response**:
```json
{
"user": {
"id": 1,
"username": "admin",
"display_name": "Administrator",
"role": "admin",
"active": true,
"is_default_admin": true,
"must_change_password": false,
"first_login": false
}
}
```
**Rate Limit**: 10 per 15 minutes per IP (bypassed if no users exist)
**Errors**:
| Status | Code | Message |
|--------|------|---------|
| 400 | `VALIDATION_ERROR` | Username/password missing |
| 401 | `AUTH_ERROR` | Invalid credentials |
| 403 | `FORBIDDEN` | Local login disabled |
| 429 | `RATE_LIMITED` | Too many attempts |
#### GET /api/auth/oidc/login
**Purpose**: Initiate OIDC login flow
**Query Parameters**:
- `redirect_to` (optional): URL to redirect after login
**Response**: HTTP 302 redirect to OIDC provider
**Errors**:
| Status | Redirect | Reason |
|--------|----------|--------|
| 501 | - | OIDC not configured |
| 502 | - | Provider unreachable |
#### GET /api/auth/oidc/callback
**Purpose**: OIDC callback handler
**Query Parameters**:
- `code`: Authorization code
- `state`: Login state ID
- `error` (optional): Provider error
**Response**: HTTP 302 redirect to frontend or error page
**Errors**:
| Status | Redirect | Reason |
|--------|----------|--------|
| 302 | `/?oidc_error=not_configured` | OIDC disabled |
| 302 | `/?oidc_error=authorization_failed` | Provider signalled error |
| 302 | `/?oidc_error=invalid_callback` | Missing code or state |
| 302 | `/?oidc_error=invalid_or_expired_state` | State invalid/expired |
| 302 | `/?oidc_error=authentication_failed` | Token validation failure |
| 302 | `/?oidc_error=access_denied` | User not in admin group (if required) |
#### GET /api/auth/logout
**Purpose**: Logout (session invalidation)
**Response**:
```json
{
"success": true
}
```
### Tracker Endpoints
#### GET /api/tracker
**Purpose**: Monthly tracker data
**Query Parameters**:
- `year` (optional, default: current year)
- `month` (optional, default: current month)
**Response**:
```json
{
"year": 2026,
"month": 5,
"today": "2026-05-09",
"summary": {
"total_expected": 450.00,
"total_starting": 500.00,
"has_starting_amounts": true,
"total_paid": 320.00,
"remaining": 180.00,
"overdue": 75.00,
"count_paid": 5,
"count_upcoming": 3,
"count_late": 2,
"count_autodraft": 1
},
"rows": [
{
"id": 1,
"name": "Rent",
"category_name": "Housing",
"due_date": "2026-05-01",
"expected_amount": 1200.00,
"actual_amount": 1200.00,
"total_paid": 1200.00,
"balance": 0,
"status": "paid",
"autopay_enabled": false,
"autodraft_status": "none"
},
...
]
}
```
#### GET /api/tracker/upcoming
**Purpose**: Bills due in next N days
**Query Parameters**:
- `days` (optional, default: 30, max: 365)
**Response**:
```json
{
"days": 30,
"today": "2026-05-09",
"upcoming": [
{
"id": 2,
"name": "Internet",
"category_name": "Phone & Internet",
"due_date": "2026-05-15",
"expected_amount": 60.00,
"status": "due_soon",
"days_until_due": 6
}
]
}
```
### Bills Endpoints
#### GET /api/bills
**Purpose**: List all bills (active or all)
**Query Parameters**:
- `inactive` (optional): If "true", include inactive bills
**Response**: Array of bill objects
#### GET /api/bills/:id
**Purpose**: Get single bill by ID
**Response**: Bill object
#### POST /api/bills
**Purpose**: Create new bill
**Request**:
```json
{
"name": "Internet",
"category_id": 5,
"due_day": 15,
"override_due_date": null,
"expected_amount": 60.00,
"interest_rate": null,
"billing_cycle": "monthly",
"autopay_enabled": false,
"autodraft_status": "none",
"website": null,
"username": null,
"account_info": null,
"has_2fa": false,
"notes": "Fiber optic internet",
"history_visibility": "default"
}
```
**Response**: Created bill with ID
#### PUT /api/bills/:id
**Purpose**: Update bill
**Request**: Partial bill object
**Response**: Updated bill
#### DELETE /api/bills/:id
**Purpose**: Hard-delete bill (irreversible)
**Response**:
```json
{
"success": true,
"deleted_bill_id": 1,
"deleted_bill_name": "Rent",
"warning": "Bill and all associated payments, monthly state, and history ranges were permanently deleted."
}
```
#### GET /api/bills/:id/payments
**Purpose**: List payments for a bill
**Query Parameters**:
- `page` (optional, default: 1)
- `limit` (optional, default: 20, max: 100)
**Response**:
```json
{
"bill_id": 1,
"bill_name": "Rent",
"total": 5,
"page": 1,
"limit": 20,
"pages": 1,
"payments": [...]
}
```
#### POST /api/bills/:id/toggle-paid
**Purpose**: Toggle bill as paid/unpaid
**Request**:
```json
{
"amount": 1200.00,
"paid_date": "2026-05-01",
"method": "ACH",
"notes": "Rent payment"
}
```
**Response**:
```json
{
"success": true,
"isPaid": true,
"action": "created_payment",
"payment": { ... }
}
```
#### GET /api/bills/:id/monthly-state
**Purpose**: Get monthly state override for a bill
**Query Parameters**:
- `year` (required)
- `month` (required)
**Response**:
```json
{
"bill_id": 1,
"year": 2026,
"month": 5,
"actual_amount": 1200.00,
"notes": null,
"is_skipped": false
}
```
#### PUT /api/bills/:id/monthly-state
**Purpose**: Set monthly state override
**Request**:
```json
{
"year": 2026,
"month": 5,
"actual_amount": 1250.00,
"notes": "Rent increased",
"is_skipped": false
}
```
**Response**: Saved state
### Categories Endpoints
#### GET /api/categories
**Purpose**: List all categories for user
**Response**: Array of category objects
#### POST /api/categories
**Purpose**: Create new category
**Request**:
```json
{
"name": "Entertainment"
}
```
**Response**: Created category
### Settings Endpoints
#### GET /api/settings
**Purpose**: Get all settings
**Response**:
```json
{
"currency": "USD",
"date_format": "MM/DD/YYYY",
"grace_period_days": "5",
"notify_days_before": "3",
"backup_enabled": "false",
...
}
```
#### PUT /api/settings
**Purpose**: Update settings
**Request**: Partial settings object
**Response**: Updated settings
### User Endpoints
#### GET /api/user
**Purpose**: Get current user profile
**Response**: User object (without password)
### Calendar Endpoints
#### GET /api/calendar
**Purpose**: Calendar data for a month
**Query Parameters**:
- `year` (optional)
- `month` (optional)
**Response**: Calendar data with bill due dates
### Summary Endpoints
#### GET /api/summary
**Purpose**: Monthly spending summary
**Query Parameters**:
- `year` (optional)
- `month` (optional)
**Response**:
```json
{
"year": 2026,
"month": 5,
"total_expected": 450.00,
"total_actual": 425.00,
"total_paid": 425.00,
"total_starting": 500.00,
"remaining": 75.00,
"by_category": [...],
"by_bill": [...]
}
```
### Analytics Endpoints
#### GET /api/analytics
**Purpose**: Analytics data with filters
**Query Parameters**:
- `start_date` (optional, default: 30 days ago)
- `end_date` (optional, default: today)
- `category_id` (optional)
- `bill_id` (optional)
**Response**:
```json
{
"start_date": "2026-04-09",
"end_date": "2026-05-09",
"total_spent": 425.00,
"expected_vs_actual": {
"expected": 450.00,
"actual": 425.00,
"difference": -25.00
},
"by_category": [...],
"payment_history": [...]
}
```
### Profile Endpoints
#### GET /api/profile
**Purpose**: Get user profile
**Response**: User profile
#### POST /api/profile
**Purpose**: Update user profile
**Request**:
```json
{
"display_name": "John Doe",
"notification_email": "john@example.com",
"notifications_enabled": true,
"notify_3d": true,
"notify_1d": true,
"notify_due": true,
"notify_overdue": true,
"current_password": "oldpass123",
"new_password": "newpass456"
}
```
**Response**: Updated user
**Rate Limit**: 5 per 15 minutes per IP
### Admin Endpoints
**All admin routes require `requireAuth` + `requireAdmin` + `csrfMiddleware` + `adminActionLimiter`**
#### GET /api/admin/has-users
**Purpose**: Check if other users exist (lockout protection)
**Response**:
```json
{
"has_users": true
}
```
#### GET /api/admin/users
**Purpose**: List all users
**Response**: Array of user objects with admin fields
#### POST /api/admin/users
**Purpose**: Create new user
**Request**:
```json
{
"username": "newuser",
"password": "password123"
}
```
**Response**: Created user
**Errors**:
| Status | Message |
|--------|---------|
| 400 | Username/password too short |
| 409 | Username already taken |
#### PUT /api/admin/users/:id/password
**Purpose**: Reset user password
**Request**:
```json
{
"password": "newpassword123"
}
```
**Response**:
```json
{
"success": true
}
```
**Effects**:
- Updates password hash
- Sets `must_change_password = 1`
- Invalidates all user sessions
#### PUT /api/admin/users/:id/role
**Purpose**: Promote/demote user
**Request**:
```json
{
"role": "admin"
}
```
**Response**: Updated user
**Validations**:
- Cannot change own role
- Cannot remove last admin
- Deletes all sessions for target user
#### PUT /api/admin/users/:id/active
**Purpose**: Deactivate/reactivate user
**Request**:
```json
{
"active": false
}
```
**Response**: Updated user
**Effects**:
- Sets `active = 0/1`
- Invalidates all sessions if deactivated
#### DELETE /api/admin/users/:id
**Purpose**: Delete user (irreversible)
**Response**:
```json
{
"success": true,
"deleted_user_id": 2
}
```
**Effects**:
- Deletes all user data (sessions, imports, exports, bills, categories)
#### Backup Endpoints
| Method | Endpoint | Purpose | Rate Limit |
|--------|----------|---------|------------|
| POST | `/api/admin/backups` | Create backup | 5/60min |
| GET | `/api/admin/backups` | List backups | 5/60min |
| GET | `/api/admin/backups/:id/download` | Download backup | - |
| POST | `/api/admin/backups/:id/restore` | Restore backup | 5/60min |
| DELETE | `/api/admin/backups/:id` | Delete backup | 5/60min |
| POST | `/api/admin/backups/import` | Import backup | 5/60min |
| GET | `/api/admin/backups/settings` | Get backup schedule | 5/60min |
| PUT | `/api/admin/backups/settings` | Update backup schedule | 5/60min |
| POST | `/api/admin/backups/run-scheduled-now` | Run scheduled backup now | 5/60min |
**Backup Request/Response**:
```json
// POST /api/admin/backups
// Response (201 Created)
{
"id": "bill-tracker-backup-2026-05-09-03-45-32-456Z-abcd1234.sqlite",
"filename": "bill-tracker-backup-2026-05-09-03-45-32-456Z-abcd1234.sqlite",
"type": "manual",
"created_at": "2026-05-09T03:45:32.456Z",
"modified_at": "2026-05-09T03:45:32.456Z",
"size_bytes": 200704,
"checksum": "abc123def456..."
}
```
**Restore Request/Response**:
```json
// POST /api/admin/backups/:id/restore
// Response
{
"restored_from": "bill-tracker-backup-2026-05-08-02-00-00-000Z-1234abcd.sqlite",
"pre_restore_backup": "pre-restore-2026-05-09-03-46-00-000Z-5678efgh.sqlite",
"restored_at": "2026-05-09T03:46:00.000Z",
"restart_required": false
}
```
**Import Backup**:
- Body: Binary SQLite file
- Header: `X-Checksum-SHA256: <hash>` (optional, if provided, validated)
- Max size: 100MB
#### Cleanup Endpoints
| Method | Endpoint | Purpose |
|--------|----------|---------|
| GET | `/api/admin/cleanup` | Get cleanup settings and status |
| PUT | `/api/admin/cleanup` | Update cleanup settings |
| POST | `/api/admin/cleanup/run` | Run cleanup immediately |
**Cleanup Settings**:
```json
{
"import_sessions_enabled": true,
"temp_exports_enabled": true,
"temp_export_max_age_hours": 2,
"backup_partials_enabled": true,
"import_history_enabled": false,
"import_history_max_age_days": 365
}
```
#### Auth Mode Endpoints
| Method | Endpoint | Purpose |
|--------|----------|---------|
| GET | `/api/admin/auth-mode` | Get auth configuration |
| PUT | `/api/admin/auth-mode` | Update auth configuration |
| POST | `/api/admin/auth-mode/oidc-test` | Test OIDC configuration |
**Auth Mode Settings**:
```json
{
"auth_mode": "multi",
"local_login_enabled": true,
"oidc_login_enabled": false,
"oidc_configured": false,
"oidc_issuer_url_set": false,
"oidc_client_id_set": false,
"oidc_client_secret_set": false,
"oidc_redirect_uri_set": false,
"oidc_missing_fields": ["issuer URL", "client ID", "client secret", "redirect URI"],
"can_disable_local": false,
"warnings": []
}
```
### Export Endpoints
#### GET /api/export
**Purpose**: Export user data as XLSX
**Query Parameters**:
- `export_type` (optional): 'bills', 'payments', 'categories', 'full'
- `start_date`, `end_date` (optional): Date range
**Response**: XLSX file download
**Rate Limit**: 30 per 15 minutes per IP
### Import Endpoints
#### GET /api/import
**Purpose**: Get import history and preview settings
#### POST /api/import
**Purpose**: Preview or apply import
**Form Data**:
- `file`: XLSX file
- `preview`: 'true' or 'false'
**Response** (preview):
```json
{
"preview": true,
"rows_parsed": 10,
"rows_created": 8,
"rows_updated": 2,
"rows_skipped": 0,
"rows_errored": 0,
"data": [...]
}
```
**Response** (apply):
```json
{
"preview": false,
"imported_at": "2026-05-09T03:50:00.000Z",
"rows_parsed": 10,
"rows_created": 8,
"rows_updated": 2,
"rows_skipped": 0,
"rows_errored": 0
}
```
**Rate Limit**: 20 per 15 minutes per IP
### Status Endpoints
#### GET /api/status
**Purpose**: System status (admin only)
**Response**:
```json
{
"version": "0.19.0",
"node_env": "production",
"db_path": "/data/bills.db",
"backup_path": "/data/backups",
"sqlite_version": "3.45.0",
"users_count": 2,
"bills_count": 15,
"last_worker_run": "2026-05-09T06:00:00.000Z",
"last_worker_status": "success",
"uptime_seconds": 86400,
"last_error": null
}
```
#### GET /api/about
**Purpose**: Public version info
**Response**:
```json
{
"version": "0.19.0",
"name": "Bill Tracker",
"build_time": "2026-05-01T00:00:00.000Z"
}
```
---
## 6. Database Documentation
### Schema Overview
**Database**: SQLite (better-sqlite3)
**Schema Location**: `db/schema.sql`
**Migration Logic**: `db/database.js` (runMigrations function)
### Table Definitions
#### users
| Column | Type | Constraints | Description |
|--------|------|-------------|-------------|
| `id` | INTEGER | PRIMARY KEY, AUTOINCREMENT | User ID |
| `username` | TEXT | NOT NULL, UNIQUE (CASE-insensitive) | Login username |
| `password_hash` | TEXT | NOT NULL | bcrypt hash of password |
| `role` | TEXT | NOT NULL, CHECK('admin', 'user') | Role: admin or user |
| `active` | INTEGER | NOT NULL, DEFAULT 1 | 1=active, 0=deactivated |
| `is_default_admin` | INTEGER | NOT NULL, DEFAULT 0 | 1=initial admin account |
| `must_change_password` | INTEGER | NOT NULL, DEFAULT 0 | 1=force password change on next login |
| `first_login` | INTEGER | NOT NULL, DEFAULT 1 | 1=user has never logged in |
| `created_at` | TEXT | DEFAULT (datetime('now')) | Account creation time |
| `updated_at` | TEXT | DEFAULT (datetime('now')) | Last update time |
| `notification_email` | TEXT | | User's email for notifications |
| `notifications_enabled` | INTEGER | NOT NULL, DEFAULT 0 | 1=receive email notifications |
| `notify_3d` | INTEGER | NOT NULL, DEFAULT 1 | Notify 3 days before due |
| `notify_1d` | INTEGER | NOT NULL, DEFAULT 1 | Notify 1 day before due |
| `notify_due` | INTEGER | NOT NULL, DEFAULT 1 | Notify on due date |
| `notify_overdue` | INTEGER | NOT NULL, DEFAULT 1 | Notify for overdue bills |
| `display_name` | TEXT | | Display name (OIDC) |
| `last_password_change_at` | TEXT | | Last password change timestamp |
| `auth_provider` | TEXT | NOT NULL, DEFAULT 'local' | 'local' or 'oidc' |
| `external_subject` | TEXT | | OIDC sub claim |
| `email` | TEXT | | OIDC email claim |
| `last_login_at` | TEXT | | Last login timestamp |
**Indexes**:
- `idx_sessions_user_id` (on sessions.user_id)
- `idx_sessions_expires` (on sessions.expires_at)
#### sessions
| Column | Type | Constraints | Description |
|--------|------|-------------|-------------|
| `id` | TEXT | PRIMARY KEY | Session UUID |
| `user_id` | INTEGER | NOT NULL, REFERENCES users(id) ON DELETE CASCADE | User ID |
| `expires_at` | TEXT | NOT NULL | Expiration timestamp |
| `created_at` | TEXT | DEFAULT (datetime('now')) | Session creation time |
**Indexes**:
- `idx_sessions_user_id` on `user_id`
- `idx_sessions_expires` on `expires_at`
#### bills
| Column | Type | Constraints | Description |
|--------|------|-------------|-------------|
| `id` | INTEGER | PRIMARY KEY, AUTOINCREMENT | Bill ID |
| `user_id` | INTEGER | NOT NULL, REFERENCES users(id) ON DELETE CASCADE | Owner user ID |
| `name` | TEXT | NOT NULL | Bill name |
| `category_id` | INTEGER | REFERENCES categories(id) ON DELETE SET NULL | Category reference |
| `due_day` | INTEGER | NOT NULL, CHECK(1-31) | Due day of month |
| `override_due_date` | TEXT | | Custom due date override |
| `bucket` | TEXT | CHECK('1st', '15th') | Payment bucket |
| `expected_amount` | REAL | NOT NULL, DEFAULT 0 | Expected monthly amount |
| `interest_rate` | REAL | CHECK(0-100) | APR or interest rate |
| `billing_cycle` | TEXT | DEFAULT 'monthly', CHECK | 'monthly', 'quarterly', 'annually', 'irregular' |
| `autopay_enabled` | INTEGER | NOT NULL, DEFAULT 0 | 1=autopay enabled |
| `autodraft_status` | TEXT | NOT NULL, DEFAULT 'none' | 'none', 'pending', 'assumed_paid', 'confirmed' |
| `website` | TEXT | | Bill provider website |
| `username` | TEXT | | Bill provider username |
| `account_info` | TEXT | | Bill account info |
| `has_2fa` | INTEGER | NOT NULL, DEFAULT 0 | 1=2FA enabled |
| `history_visibility` | TEXT | NOT NULL, DEFAULT 'default' | 'default', 'all', 'ranges', 'none' |
| `active` | INTEGER | NOT NULL, DEFAULT 1 | 1=active, 0=inactive |
| `notes` | TEXT | | User notes |
| `is_seeded` | INTEGER | NOT NULL, DEFAULT 0 | 1=demo data |
| `created_at` | TEXT | DEFAULT (datetime('now')) | Creation time |
| `updated_at` | TEXT | DEFAULT (datetime('now')) | Last update time |
**Indexes**:
- `idx_bills_active` on `active`
- `idx_bills_user_active` on `user_id, active`
- `idx_bills_user_id` on `user_id`
#### categories
| Column | Type | Constraints | Description |
|--------|------|-------------|-------------|
| `id` | INTEGER | PRIMARY KEY, AUTOINCREMENT | Category ID |
| `user_id` | INTEGER | NOT NULL, REFERENCES users(id) ON DELETE CASCADE | Owner user ID |
| `name` | TEXT | NOT NULL | Category name |
| `created_at` | TEXT | DEFAULT (datetime('now')) | Creation time |
| `updated_at` | TEXT | DEFAULT (datetime('now')) | Last update time |
**Constraints**:
- Unique constraint: `UNIQUE(user_id, name COLLATE NOCASE)`
**Indexes**:
- `idx_categories_user_name` on `user_id, name`
- `idx_categories_user_name_unique` on `user_id, name COLLATE NOCASE`
#### payments
| Column | Type | Constraints | Description |
|--------|------|-------------|-------------|
| `id` | INTEGER | PRIMARY KEY, AUTOINCREMENT | Payment ID |
| `bill_id` | INTEGER | NOT NULL, REFERENCES bills(id) ON DELETE CASCADE | Bill reference |
| `amount` | REAL | NOT NULL | Payment amount |
| `paid_date` | TEXT | NOT NULL | Payment date (YYYY-MM-DD) |
| `method` | TEXT | | Payment method (cash, check, card, ACH) |
| `notes` | TEXT | | Payment notes |
| `deleted_at` | TEXT | | Soft-delete timestamp |
| `created_at` | TEXT | DEFAULT (datetime('now')) | Creation time |
| `updated_at` | TEXT | DEFAULT (datetime('now')) | Last update time |
**Indexes**:
- `idx_payments_bill_id` on `bill_id`
- `idx_payments_paid_date` on `paid_date`
- `idx_payments_bill_date_del` on `bill_id, paid_date, deleted_at`
- `idx_payments_deleted` on `deleted_at`
#### monthly_bill_state
| Column | Type | Constraints | Description |
|--------|------|-------------|-------------|
| `id` | INTEGER | PRIMARY KEY, AUTOINCREMENT | State ID |
| `bill_id` | INTEGER | NOT NULL, REFERENCES bills(id) ON DELETE CASCADE | Bill reference |
| `year` | INTEGER | NOT NULL, CHECK(2000-2100) | Year |
| `month` | INTEGER | NOT NULL, CHECK(1-12) | Month |
| `actual_amount` | REAL | | Actual amount paid (override expected) |
| `notes` | TEXT | | Month-specific notes |
| `is_skipped` | INTEGER | NOT NULL, DEFAULT 0 | 1=skip this month |
| `created_at` | TEXT | DEFAULT (datetime('now')) | Creation time |
| `updated_at` | TEXT | DEFAULT (datetime('now')) | Last update time |
**Constraints**:
- `UNIQUE(bill_id, year, month)`
**Indexes**:
- `idx_monthly_bill_state_lookup` on `bill_id, year, month`
#### settings
| Column | Type | Constraints | Description |
|--------|------|-------------|-------------|
| `key` | TEXT | PRIMARY KEY | Setting key |
| `value` | TEXT | NOT NULL | Setting value |
| `updated_at` | TEXT | DEFAULT (datetime('now')) | Last update time |
#### notifications
| Column | Type | Constraints | Description |
|--------|------|-------------|-------------|
| `id` | INTEGER | PRIMARY KEY, AUTOINCREMENT | Notification ID |
| `bill_id` | INTEGER | NOT NULL, REFERENCES bills(id) ON DELETE CASCADE | Bill reference |
| `user_id` | INTEGER | NOT NULL, REFERENCES users(id) ON DELETE CASCADE | User reference |
| `year` | INTEGER | NOT NULL | Year |
| `month` | INTEGER | NOT NULL | Month |
| `type` | TEXT | NOT NULL | Notification type |
| `sent_date` | TEXT | NOT NULL, DEFAULT (date('now')) | Date sent |
**Constraints**:
- `UNIQUE(bill_id, user_id, year, month, type, sent_date)`
**Indexes**:
- `idx_notifications_lookup` on `bill_id, user_id, year, month`
#### oidc_states
| Column | Type | Constraints | Description |
|--------|------|-------------|-------------|
| `id` | TEXT | PRIMARY KEY | State UUID |
| `nonce` | TEXT | NOT NULL | Nonce for replay protection |
| `code_verifier` | TEXT | NOT NULL | PKCE code verifier |
| `redirect_to` | TEXT | | Redirect URL after login |
| `created_at` | TEXT | NOT NULL | Creation time |
| `expires_at` | TEXT | NOT NULL | Expiration time |
**Indexes**:
- `idx_oidc_states_expires` on `expires_at`
#### bill_history_ranges
| Column | Type | Constraints | Description |
|--------|------|-------------|-------------|
| `id` | INTEGER | PRIMARY KEY, AUTOINCREMENT | Range ID |
| `bill_id` | INTEGER | NOT NULL, REFERENCES bills(id) ON DELETE CASCADE | Bill reference |
| `start_year` | INTEGER | NOT NULL | Start year |
| `start_month` | INTEGER | NOT NULL | Start month |
| `end_year` | INTEGER | | End year (NULL = open-ended) |
| `end_month` | INTEGER | | End month (NULL = open-ended) |
| `label` | TEXT | | Range label |
| `created_at` | TEXT | DEFAULT (datetime('now')) | Creation time |
| `updated_at` | TEXT | DEFAULT (datetime('now')) | Last update time |
**Indexes**:
- `idx_bill_history_ranges_bill` on `bill_id`
#### monthly_income
| Column | Type | Constraints | Description |
|--------|------|-------------|-------------|
| `id` | INTEGER | PRIMARY KEY, AUTOINCREMENT | Income record ID |
| `user_id` | INTEGER | NOT NULL, REFERENCES users(id) ON DELETE CASCADE | User reference |
| `year` | INTEGER | NOT NULL, CHECK(2000-2100) | Year |
| `month` | INTEGER | NOT NULL, CHECK(1-12) | Month |
| `label` | TEXT | NOT NULL, DEFAULT 'Salary' | Income source |
| `amount` | REAL | NOT NULL, DEFAULT 0 | Income amount |
| `created_at` | TEXT | DEFAULT (datetime('now')) | Creation time |
| `updated_at` | TEXT | DEFAULT (datetime('now')) | Last update time |
**Constraints**:
- `UNIQUE(user_id, year, month)`
**Indexes**:
- `idx_monthly_income_user_month` on `user_id, year, month`
#### monthly_starting_amounts
| Column | Type | Constraints | Description |
|--------|------|-------------|-------------|
| `id` | INTEGER | PRIMARY KEY, AUTOINCREMENT | Amount record ID |
| `user_id` | INTEGER | NOT NULL, REFERENCES users(id) ON DELETE CASCADE | User reference |
| `year` | INTEGER | NOT NULL, CHECK(2000-2100) | Year |
| `month` | INTEGER | NOT NULL, CHECK(1-12) | Month |
| `first_amount` | REAL | NOT NULL, DEFAULT 0, CHECK ≥ 0 | Amount for 1st of month |
| `fifteenth_amount` | REAL | NOT NULL, DEFAULT 0, CHECK ≥ 0 | Amount for 15th of month |
| `other_amount` | REAL | NOT NULL, DEFAULT 0, CHECK ≥ 0 | Other amount |
| `notes` | TEXT | | Notes |
| `created_at` | TEXT | DEFAULT (datetime('now')) | Creation time |
| `updated_at` | TEXT | DEFAULT (datetime('now')) | Last update time |
**Constraints**:
- `UNIQUE(user_id, year, month)`
**Indexes**:
- `idx_monthly_starting_amounts_user_month` on `user_id, year, month`
#### import_sessions
| Column | Type | Constraints | Description |
|--------|------|-------------|-------------|
| `id` | TEXT | PRIMARY KEY | Session UUID |
| `user_id` | INTEGER | NOT NULL | User reference |
| `created_at` | TEXT | NOT NULL | Creation time |
| `expires_at` | TEXT | NOT NULL | Expiration time |
| `preview_json` | TEXT | NOT NULL | JSON preview data |
**Indexes**:
- `idx_import_sessions_user` on `user_id`
- `idx_import_sessions_expires` on `expires_at`
#### import_history
| Column | Type | Constraints | Description |
|--------|------|-------------|-------------|
| `id` | INTEGER | PRIMARY KEY, AUTOINCREMENT | History ID |
| `user_id` | INTEGER | NOT NULL | User reference |
| `imported_at` | TEXT | NOT NULL | Import timestamp |
| `source_filename` | TEXT | | Source filename |
| `file_type` | TEXT | DEFAULT 'xlsx' | File type |
| `sheet_name` | TEXT | | Sheet name |
| `rows_parsed` | INTEGER | DEFAULT 0 | Parsed rows |
| `rows_created` | INTEGER | DEFAULT 0 | New rows created |
| `rows_updated` | INTEGER | DEFAULT 0 | Existing rows updated |
| `rows_skipped` | INTEGER | DEFAULT 0 | Skipped rows |
| `rows_ambiguous` | INTEGER | DEFAULT 0 | Ambiguous rows |
| `rows_errored` | INTEGER | DEFAULT 0 | Errored rows |
| `options_json` | TEXT | | Import options |
| `summary_json` | TEXT | | Summary JSON |
**Indexes**:
- `idx_import_history_user` on `user_id`
### Data Flow
#### User-scoped Data
All user-modifiable data is scoped to `user_id`:
| Table | user_id Reference | onDelete |
|-------|------------------|----------|
| bills | `user_id` | CASCADE |
| categories | `user_id` | CASCADE |
| payments | bills.user_id | CASCADE |
| monthly_bill_state | bills.user_id (via bill_id) | CASCADE |
| monthly_income | `user_id` | CASCADE |
| monthly_starting_amounts | `user_id` | CASCADE |
| import_sessions | `user_id` | N/A |
| import_history | `user_id` | N/A |
#### Data Access Pattern
```sql
-- User bill list
SELECT * FROM bills WHERE user_id = ? AND active = 1 ORDER BY due_day;
-- User categories
SELECT * FROM categories WHERE user_id = ? ORDER BY name;
-- User payments (with bill info)
SELECT p.*, b.name AS bill_name, b.due_day
FROM payments p
JOIN bills b ON b.id = p.bill_id
WHERE b.user_id = ? AND p.deleted_at IS NULL;
-- User monthly state
SELECT * FROM monthly_bill_state WHERE bill_id IN (
SELECT id FROM bills WHERE user_id = ?
) AND year = ? AND month = ?;
```
### Migration System
**Migration Location**: `db/database.js` (runMigrations function)
**Migration Types**:
1. **Column additions**: `ALTER TABLE users ADD COLUMN column_name type`
2. **Table creations**: `CREATE TABLE IF NOT EXISTS`
3. **Index additions**: `CREATE INDEX IF NOT EXISTS`
4. **Schema rewrites**: Table rename + recreate (for breaking changes)
**Security Features**:
- Column name whitelist validation
- SQL definition string validation
- No user input in ALTER statements
**Migration Log** (from `db/schema.sql` comments):
- v0.2: `payments.deleted_at` column
- v0.3: `idx_payments_bill_date_del` index
- v0.4: `monthly_bill_state` table
- v0.13: Profile columns (`display_name`, `last_password_change_at`)
- v0.14: `bills.history_visibility`, `bill_history_ranges` table, `bills.interest_rate`
- v0.14.4: `bills.interest_rate` column
- v0.15: Cleanup worker settings
- v0.17: OIDC columns (`auth_provider`, `external_subject`, `email`, `last_login_at`)
- v0.17: `oidc_states` table
- v0.18: Monthly income, monthly starting amounts tables
- v0.18.2: Monthly starting amounts table
- v0.18.3: `monthly_starting_amounts.other_amount` column
- v0.18.1: Monthly income table
- v0.38: Import history table
- v0.39: Import sessions table
- v0.40: User-scoped bills/categories
- v0.41: Seeded flags (`is_seeded`)
### Entity Relationship Diagram
```
┌─────────────────┐
│ users │
├─────────────────┤
│ id (PK) │
│ username (U) │
│ password_hash │
│ role │
│ active │
│ ... │
└────────┬────────┘
│
│ 1:N
│
│
┌────────▼────────┐ ┌─────────────────┐
│ sessions │ │ bills │
├─────────────────┤ ├─────────────────┤
│ id (PK) │ │ id (PK) │
│ user_id (FK) │ │ user_id (FK) │
│ expires_at │ │ ... │
│ created_at │ └────────┬────────┘
└─────────────────┘ │
│ 1:N
│
┌────────────┴────────────┐
│ │
┌──────────▼──────────┐ ┌──────────▼──────────┐
│ categories │ │ payments │
├─────────────────────┤ ├─────────────────────┤
│ id (PK) │ │ id (PK) │
│ user_id (FK) │ │ bill_id (FK) │
│ ... │ │ ... │
└─────────────────────┘ └──────────┬──────────┘
│
┌────────────┴────────────┐
│ │
┌──────────▼──────────┐ ┌──────────▼──────────┐
│ monthly_bill_state│ │ monthly_income │
├─────────────────────┤ ├─────────────────────┤
│ id (PK) │ │ id (PK) │
│ bill_id (FK) │ │ user_id (FK) │
│ year, month (U) │ │ year, month (U) │
│ ... │ │ ... │
└─────────────────────┘ └─────────────────────┘
```
---
## 7. Error Handling & Troubleshooting
### Troubleshooting Matrix
| Symptom | Likely Cause | Logs to Inspect | Files to Inspect | Services Involved | Recovery Steps |
|---------|--------------|-----------------|------------------|-------------------|----------------|
| **Login fails** | | | | | |
| Invalid credentials | Wrong username/password | `server.js` console | `routes/authLogin.js`, `services/authService.js` | authService | Verify credentials, check for typos |
| Session expired | Session deleted or expired | `server.js` console | `services/authService.js` | authService, session pruning worker | Re-login |
| Account locked | `active = 0` | `server.js` console | `db/schema.sql` (users.active) | AuthService | Admin sets `active = 1` |
| Password mismatch | Hash changed | `server.js` console | `services/authService.js` | authService | Reset password via admin |
| Local login disabled | Admin disabled it | `server.js` console | `db/schema.sql` (settings) | Settings | Enable local login in Admin |
| **Auth issues** | | | | | |
| CSRF token invalid | Token mismatch or expired | `server.js` console | `middleware/csrf.js` | CSRF middleware | Refresh page |
| Role insufficient | User role check failed | `server.js` console | `middleware/requireAuth.js` | Auth middleware | Login as admin/user |
| OIDC callback fails | Provider error or config issue | `server.js` console | `routes/authOidc.js`, `services/oidcService.js` | oidcService | Check OIDC config in Admin |
| **API failures** | | | | | |
| 404 Not Found | Endpoint or resource missing | `server.js` console | `routes/*.js` | Route handlers | Verify endpoint, check resource ID |
| 400 Bad Request | Validation error | `server.js` console | `routes/*.js`, `services/*.js` | Validation logic | Check request body, query params |
| 429 Rate Limited | Too many requests | `server.js` console | `middleware/rateLimiter.js` | Rate limiter | Wait, reduce request frequency |
| 500 Server Error | Unhandled exception | `server.js` console, `NODE_ENV=production` | Any | All services | Check server logs, reproduce |
| **Database issues** | | | | | |
| Database locked | WAL checkpoint or long transaction | `server.js` console | `db/database.js` | Database connection | Wait, restart app if needed |
| Schema mismatch | Migration failed | `server.js` console | `db/database.js` (runMigrations) | Database init | Check migrations, fix manually |
| Foreign key violation | Related record deleted | `server.js` console | `db/schema.sql` | Schema | Check related data, adjust onDelete |
| **Performance issues** | | | | | |
| Slow queries | Missing index, complex join | `server.js` console | `db/database.js` (schema) | Database | Add indexes, optimize queries |
| Memory leak | Session or cache buildup | `server.js` console | `services/*.js` | Service cleanup | Restart app, check cleanup workers |
| High CPU | Cron job, import, export | `server.js` console | `workers/*.js`, `routes/*.js` | Background jobs | Schedule jobs off-peak |
| **Notification issues** | | | | | |
| Emails not sent | SMTP not configured or failing | `server.js` console | `services/notificationService.js` | notificationService | Configure SMTP in Admin, test |
| Notification not recorded | Unique constraint violation | `server.js` console | `db/schema.sql` (notifications) | notificationService | Check notification table |
| **Backup/restore issues** | | | | | |
| Backup fails | Disk full, permission denied | `server.js` console | `services/backupService.js` | backupService | Check disk space, permissions |
| Restore fails | Backup corrupted, wrong DB | `server.js` console | `services/backupService.js` | backupService | Verify backup integrity, restore from good backup |
| Import/Export fails | Invalid file, permission | `server.js` console | `routes/export.js`, `routes/import.js` | import/export | Check file format, permissions |
| **Worker failures** | | | | | |
| Daily worker not running | Cron not scheduled or errored | `server.js` console | `workers/dailyWorker.js` | dailyWorker | Check cron, restart app |
| Cleanup failing | Disk space, permission | `server.js` console | `services/cleanupService.js` | cleanupService | Check disk space, permissions |
### Common Error Codes and Solutions
| Error Code | Status | Message | Solution |
|------------|--------|---------|----------|
| `AUTH_ERROR` | 401 | Not authenticated | Re-login |
| `VALIDATION_ERROR` | 400 | Input validation failed | Check request body |
| `FORBIDDEN` | 403 | Access denied | Login as correct role |
| `NOT_FOUND` | 404 | Resource not found | Check ID |
| `CONFLICT` | 409 | Duplicate entry | Check for existing record |
| `RATE_LIMITED` | 429 | Too many requests | Wait or reduce requests |
| `CSRF_INVALID` | 403 | CSRF token validation failed | Refresh page |
| `IMPORT_REQUEST_ERROR` | 400 | Import request failed | Smaller/valid file |
| `IMPORT_SERVER_ERROR` | 500 | Import server error | Check server logs |
### Log Files and Locations
| Log Source | Location | Purpose |
|------------|----------|---------|
| **Server console** | stdout/stderr | All console.log/error output |
| **Debug logs** | Set `NODE_DEBUG=express` | Express internals |
| **SQL logs** | `process.env.DEBUG=better-sqlite3` | SQL queries |
### Recovery Procedures
#### 1. Admin Locked Out
**Symptom**: Can't login, no users with admin role
**Recovery**:
```bash
# Reset default admin password (if INIT_ADMIN_USER/PASS set)
docker-compose exec app node -e "
const bcrypt = require('bcryptjs');
const hash = bcrypt.hashSync('newpassword123', 12);
console.log(hash);
"
# Then run SQL: UPDATE users SET password_hash='newhash' WHERE is_default_admin=1;
```
#### 2. Corrupted Database
**Symptom**: App fails to start, database errors in logs
**Recovery**:
```bash
# Restore from backup
cp /path/to/backup.sqlite /data/bills.db
# Or rebuild from export (if available)
```
#### 3. Session Starvation
**Symptom**: All users logged out, can't re-login
**Recovery**:
```bash
# Prune all sessions
docker-compose exec app node -e "
const { getDb } = require('./db/database');
const db = getDb();
db.prepare('DELETE FROM sessions').run();
console.log('Sessions pruned');
"
```
#### 4. Rate Limiter Stuck
**Symptom**: All requests return 429, even after waiting
**Recovery**:
```bash
# Restart the app (in-memory rate limiter)
docker-compose restart app
# Or in Node REPL:
node -e "const { resetStores } = require('./middleware/rateLimiter'); resetStores(); console.log('Limiters reset');"
```
#### 5. OIDC Configuration Broken
**Symptom**: OIDC login failing, discovery errors
**Recovery**:
```bash
# Clear OIDC client cache (forces re-discovery)
node -e "
const { invalidateClientCache } = require('./services/oidcService');
invalidateClientCache();
console.log('Client cache invalidated');
"
```
### Debug Commands
#### Check Database Integrity
```bash
docker-compose exec app sqlite3 /data/bills.db "PRAGMA integrity_check;"
```
#### View Active Sessions
```bash
docker-compose exec app node -e "
const { getDb } = require('./db/database');
const db = getDb();
const sessions = db.prepare('SELECT s.id, u.username, s.expires_at FROM sessions s JOIN users u ON u.id = s.user_id WHERE s.expires_at > datetime(\"now\")').all();
console.log(sessions);
"
```
#### List All Users
```bash
docker-compose exec app node -e "
const { getDb } = require('./db/database');
const db = getDb();
const users = db.prepare('SELECT id, username, role, active, auth_provider FROM users').all();
console.log(users);
"
```
#### View Bill Counts by Category
```bash
docker-compose exec app node -e "
const { getDb } = require('./db/database');
const db = getDb();
const counts = db.prepare('SELECT c.name, COUNT(b.id) as count FROM categories c LEFT JOIN bills b ON b.category_id = c.id GROUP BY c.id').all();
console.table(counts);
"
```
---
## 8. Code Navigation Index
### Feature-to-File Mapping
| Feature | Frontend Files | Backend Files | Services | Middleware | Tests |
|---------|----------------|---------------|----------|------------|-------|
| **User Authentication** | `client/pages/LoginPage.jsx`, `client/hooks/useAuth.jsx`, `client/api.js` | `routes/authLogin.js`, `routes/auth.js`, `routes/authOidc.js` | `authService.js`, `oidcService.js` | `requireAuth.js`, `csrf.js` | `test-functional.js`, `scripts/test-oidc-smoke.js` |
| **Monthly Tracker** | `client/pages/TrackerPage.jsx`, `client/components/MobileBillRow.jsx` | `routes/tracker.js` | `statusService.js`, `statusRuntime.js` | `requireAuth.js`, `requireUser.js` | `test-functional.js` |
| **Bill CRUD** | `client/pages/BillsPage.jsx`, `client/components/BillModal.jsx` | `routes/bills.js` | - | `requireAuth.js`, `requireUser.js`, `csrf.js` | `test-functional.js` |
| **Payment Recording** | `client/components/StatusBadge.jsx`, `client/pages/TrackerPage.jsx` | `routes/payments.js`, `routes/bills.js` (toggle-paid) | - | `requireAuth.js`, `requireUser.js`, `csrf.js` | `test-functional.js` |
| **Categories** | `client/pages/CategoriesPage.jsx` | `routes/categories.js` | - | `requireAuth.js`, `requireUser.js`, `csrf.js` | - |
| **Monthly State Overrides** | `client/pages/BillsPage.jsx`, `client/components/TrackerPage.jsx` | `routes/bills.js` (monthly-state) | - | `requireAuth.js`, `requireUser.js`, `csrf.js` | - |
| **Calendar View** | `client/pages/CalendarPage.jsx` | `routes/calendar.js` | `statusService.js` | `requireAuth.js`, `requireUser.js` | - |
| **Summary View** | `client/pages/SummaryPage.jsx` | `routes/summary.js` | - | `requireAuth.js`, `requireUser.js` | - |
| **Analytics** | `client/pages/AnalyticsPage.jsx` | `routes/analytics.js` | - | `requireAuth.js`, `requireUser.js` | - |
| **User Profile** | `client/pages/ProfilePage.jsx` | `routes/user.js`, `routes/profile.js` | `authService.js` | `requireAuth.js`, `requireUser.js`, `passwordLimiter` | - |
| **App Settings** | `client/pages/SettingsPage.jsx` | `routes/settings.js` | - | `requireAuth.js`, `requireUser.js` | - |
| **Notifications** | `client/pages/ProfilePage.jsx` (settings) | `routes/notifications.js` | `notificationService.js`, `statusRuntime.js` | `requireAuth.js` | - |
| **Data Import** | `client/pages/DataPage.jsx` | `routes/import.js` | `spreadsheetImportService.js`, `userDbImportService.js` | `requireAuth.js`, `requireUser.js`, `importLimiter` | `scripts/test-import.js` |
| **Data Export** | `client/pages/DataPage.jsx` | `routes/export.js` | - | `requireAuth.js`, `requireUser.js`, `exportLimiter` | - |
| **Admin User Management** | `client/pages/AdminPage.jsx` | `routes/admin.js` (users) | `authService.js` | `requireAuth.js`, `requireAdmin.js`, `adminActionLimiter` | - |
| **Admin Backups** | `client/pages/AdminPage.jsx` (backups tab) | `routes/admin.js` (backups) | `backupService.js`, `backupScheduler.js` | `requireAuth.js`, `requireAdmin.js`, `backupOperationLimiter` | - |
| **Admin OIDC Config** | `client/pages/AdminPage.jsx` (auth tab) | `routes/admin.js` (auth-mode) | `oidcService.js` | `requireAuth.js`, `requireAdmin.js` | `scripts/test-oidc-smoke.js` |
| **Admin Cleanup** | `client/pages/AdminPage.jsx` (cleanup tab) | `routes/admin.js` (cleanup) | `cleanupService.js` | `requireAuth.js`, `requireAdmin.js` | - |
| **System Status** | `client/pages/StatusPage.jsx` | `routes/status.js` | `statusRuntime.js`, `statusService.js` | `requireAuth.js`, `requireAdmin.js` | - |
### Component File Tree
```
client/
├── components/
│ ├── layout/
│ │ ├── Layout.jsx # Main layout wrapper
│ │ ├── Sidebar.jsx # Navigation sidebar
│ │ ├── BrandBlock.jsx # App branding (logo, title, version)
│ │ └── NavPill.jsx # Nav link item
│ ├── ui/
│ │ ├── button.jsx # shadcn button
│ │ ├── input.jsx # shadcn input
│ │ ├── card.jsx # shadcn card
│ │ ├── table.jsx # shadcn table
│ │ ├── tabs.jsx # shadcn tabs
│ │ ├── dialog.jsx # shadcn dialog
│ │ ├── badge.jsx # shadcn badge
│ │ ├── switch.jsx # shadcn switch
│ │ ├── select.jsx # shadcn select
│ │ ├── dropdown-menu.jsx # shadcn dropdown
│ │ ├── label.jsx # shadcn label
│ │ ├── input-dialog.jsx # Custom dialog with input
│ │ ├── confirm-dialog.jsx # Confirmation dialog
│ │ ├── alert-dialog.jsx # shadcn alert dialog
│ │ ├── separator.jsx # shadcn separator
│ │ ├── tooltip.jsx # shadcn tooltip
│ │ ├── checkbox.jsx # shadcn checkbox
│ │ └── theme-toggle.jsx # Theme switcher
│ ├── BillsTableInner.jsx # Bills table component
│ ├── MobileBillRow.jsx # Mobile bill row
│ ├── MobileTrackerRow.jsx # Mobile tracker row
│ ├── StatusBadge.jsx # Payment status badge
│ ├── SummaryCard.jsx # Summary statistics card
│ ├── MarkdownText.jsx # Markdown renderer
│ ├── ReleaseNotesDialog.jsx # Release notes modal
│ └── ...
├── pages/
│ ├── LoginPage.jsx # Login page
│ ├── TrackerPage.jsx # Monthly tracker
│ ├── BillsPage.jsx # Bill CRUD
│ ├── CategoriesPage.jsx # Category management
│ ├── CalendarPage.jsx # Calendar view
│ ├── SummaryPage.jsx # Monthly summary
│ ├── AnalyticsPage.jsx # Analytics charts
│ ├── ProfilePage.jsx # User profile
│ ├── SettingsPage.jsx # App settings
│ ├── DataPage.jsx # Import/export
│ ├── AdminPage.jsx # Admin panel
│ ├── StatusPage.jsx # System status
│ ├── AboutPage.jsx # Version/info
│ └── ReleaseNotesPage.jsx # Release notes
├── hooks/
│ └── useAuth.jsx # Auth state hook
├── contexts/
│ └── ThemeContext.jsx # Theme state
├── api.js # API client
├── App.jsx # Router config
├── main.jsx # React entry
└── lib/
├── utils.js # Utility functions
└── version.js # Version constants
```
### Service Layer Dependencies
```
services/
├── authService.js # Session management, login/logout
├── oidcService.js # Authentik OIDC integration
├── backupService.js # SQLite backup/restore
├── backupScheduler.js # Scheduled backups
├── notificationService.js # Email notifications
├── cleanupService.js # Cleanup tasks
├── spreadsheetImportService.js # XLSX import
├── userDbImportService.js # SQLite user import
├── statusRuntime.js # Worker/runtime status
└── statusService.js # Tracker status calculations
```
### Middleware Chain by Route
| Route Prefix | Middleware Chain |
|--------------|------------------|
| `/api/auth/login` | `loginLimiter` |
| `/api/auth` | `csrfMiddleware` |
| `/api/auth/oidc` | `csrfMiddleware`, `oidcLimiter` |
| `/api/tracker` | `csrfMiddleware`, `requireAuth`, `requireUser` |
| `/api/bills` | `csrfMiddleware`, `requireAuth`, `requireUser` |
| `/api/payments` | `csrfMiddleware`, `requireAuth`, `requireUser` |
| `/api/categories` | `csrfMiddleware`, `requireAuth`, `requireUser` |
| `/api/settings` | `csrfMiddleware`, `requireAuth`, `requireUser` |
| `/api/user` | `csrfMiddleware`, `requireAuth`, `requireUser` |
| `/api/calendar` | `csrfMiddleware`, `requireAuth`, `requireUser` |
| `/api/summary` | `csrfMiddleware`, `requireAuth`, `requireUser` |
| `/api/monthly-starting-amounts` | `csrfMiddleware`, `requireAuth`, `requireUser` |
| `/api/analytics` | `csrfMiddleware`, `requireAuth`, `requireUser` |
| `/api/notifications` | `csrfMiddleware`, `requireAuth` |
| `/api/profile` | `csrfMiddleware`, `requireAuth`, `requireUser`, `passwordLimiter` |
| `/api/admin` | `csrfMiddleware`, `requireAuth`, `requireAdmin`, `adminActionLimiter` |
| `/api/export` | `csrfMiddleware`, `requireAuth`, `requireUser`, `exportLimiter` |
| `/api/import` | `csrfMiddleware`, `requireAuth`, `requireUser`, `importLimiter` |
| `/api/status` | `csrfMiddleware`, `requireAuth`, `requireAdmin` |
| `/api/about` | (none) |
| `/api/version` | (none) |
### Test Files
| Test File | Purpose |
|-----------|---------|
| `test-functional.js` | Functional tests for all features |
| `run-functional-test.js` | Test runner |
| `scripts/test-import.js` | Import functionality test |
| `scripts/test-oidc-smoke.js` | OIDC configuration smoke test |
| `scripts/test-cookie-options.js` | Cookie options test |
---
## 9. Infrastructure & Deployment
### Docker Setup
#### Dockerfile
```dockerfile
# Base image
FROM node:20-alpine AS base
# Install dependencies
FROM base AS deps
RUN apk add --no-cache libc6-compat
WORKDIR /app
COPY package.json package-lock.json ./
RUN npm ci --only=production
# Build stage
FROM base AS builder
WORKDIR /app
COPY --from=deps /app/node_modules ./node_modules
COPY . .
RUN npm run build
# Production
FROM base AS production
WORKDIR /app
COPY --from=deps /app/node_modules ./node_modules
COPY --from=builder /app/dist ./dist
COPY --from=builder /app/*.js ./
COPY --from=builder /app/scripts ./scripts
COPY --from=builder /app/db ./db
COPY --from=builder /app/middleware ./middleware
COPY --from=builder /app/routes ./routes
COPY --from=builder /app/services ./services
COPY --from=builder /app/workers ./workers
COPY --from=builder /app/client ./client
COPY --from=builder /app/.npmrc ./
COPY --from=builder /app/postcss.config.js ./
COPY --from=builder /app/tailwind.config.js ./
COPY --from=builder /app/vite.config.js ./
COPY --from=builder /app/index.html ./
COPY --from=builder /app/.env.example ./
# Environment variables
ENV NODE_ENV=production
ENV PORT=3000
# Data directories
VOLUME ["/data", "/backups"]
# Entry point
ENTRYPOINT ["./docker-entrypoint.sh"]
CMD ["node", "server.js"]
```
#### docker-compose.yml
```yaml
version: '3.8'
services:
app:
image: bill-tracker:latest
container_name: bill-tracker
ports:
- "3030:3000"
volumes:
- ./data:/data
- ./backups:/backups
environment:
- NODE_ENV=production
- PORT=3000
- DB_PATH=/data/bills.db
- BACKUP_PATH=/data/backups
- HTTPS=true
- COOKIE_SECURE=true
# OIDC (optional)
# - OIDC_ISSUER_URL=
# - OIDC_CLIENT_ID=
# - OIDC_CLIENT_SECRET=
# - OIDC_REDIRECT_URI=
restart: unless-stopped
healthcheck:
test: ["CMD", "wget", "--no-verbose", "--tries=1", "--spider", "http://localhost:3000/api/version"]
interval: 30s
timeout: 10s
retries: 3
start_period: 30s
```
### Environment Variables
| Variable | Default | Required | Description |
|----------|---------|----------|-------------|
| `PORT` | `3000` | No | API server port |
| `NODE_ENV` | `production` | No | Environment mode |
| `DB_PATH` | `db/bills.db` | No | SQLite database path |
| `BACKUP_PATH` | `backups/` | No | Backup directory path |
| `HTTPS` | `false` | No | Enable HTTPS (for HSTS, secure cookies) |
| `COOKIE_SECURE` | `false` | No | Force secure cookies |
| `CORS_ORIGIN` | (disabled) | No | CORS allowed origins (comma-separated) |
| `INIT_ADMIN_USER` | `admin` | No | Initial admin username (first run only) |
| `INIT_ADMIN_PASS` | `admin123` | No | Initial admin password (first run only) |
| `OIDC_ISSUER_URL` | - | No | OIDC issuer URL (fallback) |
| `OIDC_CLIENT_ID` | - | No | OIDC client ID (fallback) |
| `OIDC_CLIENT_SECRET` | - | No | OIDC client secret (fallback) |
| `OIDC_REDIRECT_URI` | - | No | OIDC redirect URI (fallback) |
| `OIDC_SCOPES` | `openid email profile groups` | No | OIDC scopes (fallback) |
| `OIDC_ADMIN_GROUP` | - | No | OIDC admin group name (fallback) |
| `OIDC_AUTO_PROVISION` | `true` | No | Auto-create users from OIDC |
| `OIDC_PROVIDER_NAME` | `authentik` | No | Provider name (fallback) |
### Ports and Services
| Port | Service | Purpose |
|------|---------|---------|
| `3000` | Express | Main API server |
| `3030` | Host (Docker) | Exposed port for app |
### Monitoring & Logging
#### Runtime Status (`statusRuntime.js`)
| Status Type | Key | Description |
|-------------|-----|-------------|
| Worker | `last_worker_run_at` | Last daily worker execution |
| Worker | `last_worker_status` | `success` or `error` |
| Worker | `last_worker_error` | Error message if failed |
| Notification | `last_notification_send_at` | Last email send |
| Notification | `last_notification_error` | Last email error |
| Runtime | `last_error_at` | Last error timestamp |
| Runtime | `last_error_message` | Last error message |
#### Health Check
```bash
# Health endpoint (public)
GET /api/version
# Docker healthcheck
wget --no-verbose --tries=1 --spider http://localhost:3000/api/version
```
### CI/CD Pipeline
**Current**: Manual deployment via `deploy.sh`
**Deploy Script**:
```bash
#!/bin/bash
# deploy.sh
# Build frontend
npm run build
# Sync to server (rsync)
rsync -avz dist/ user@server:/var/www/bill-tracker/
rsync -avz node_modules/ user@server:/var/www/bill-tracker/
# Restart server
ssh user@server "pm2 restart bill-tracker"
```
### Security Considerations
| Feature | Implementation |
|---------|----------------|
| **Password Hashing** | bcrypt with cost factor 12 |
| **Session Storage** | SQLite with 7-day expiry |
| **CSRF Protection** | Double-submit cookie pattern |
| **Rate Limiting** | In-memory express-rate-limit |
| **SQL Injection** | Parameterized queries (prepared statements) |
| **XSS Protection** | CSP with nonce, no inline scripts in HTML emails |
| **CORS** | Disabled by default, configurable via env |
| **HSTS** | Only when HTTPS=true |
| **Secure Cookies** | httpOnly, sameSite=strict, secure when HTTPS |
| **OIDC Security** | PKCE, state/nonce, JWKS signature verification |
| **Backup Security** | chmod 600, SHA-256 checksums, restrictive dir (0o700) |
### Database Backups
**Manual**:
```bash
# Create backup
curl -X POST http://localhost:3000/api/admin/backups
# List backups
curl http://localhost:3000/api/admin/backups
# Download backup
curl -o backup.sqlite http://localhost:3000/api/admin/backups/backup-id.sqlite
```
**Scheduled** (Admin panel):
- Enable: `backup_schedule_enabled`
- Frequency: daily/weekly
- Time: HH:MM
- Retention: N backups
### Troubleshooting Commands
#### View Logs (Docker)
```bash
docker-compose logs -f app
```
#### Restart App (Docker)
```bash
docker-compose restart app
```
#### Rebuild Image
```bash
docker-compose build --no-cache
docker-compose up -d
```
#### Enter Container Shell
```bash
docker-compose exec app sh
```
#### Check Database Size
```bash
docker-compose exec app ls -lh /data/bills.db
```
---
## 10. Sequence Flows
### Login Flow (Local)
```
┌─────────────────────────────────────────────────────────────┐
│ User │
│ 1. Opens /login.html │
│ 2. Enters username + password │
│ 3. Clicks "Login" button │
└─────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────┐
│ Frontend (LoginPage.jsx) │
│ 4. Calls apiPost('/api/auth/login', {username, password}) │
└─────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────┐
│ Backend (server.js) │
│ 5. POST /api/auth/login │
│ 6. loginLimiter checks IP (skip if no users exist) │
│ 7. authLogin.js handler runs │
└─────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────┐
│ Service (authService.login) │
│ 8. Query user by username │
│ 9. Check user.active === 1 │
│ 10. Check auth_provider === 'local' │
│ 11. bcrypt.compare(password, password_hash) │
│ 12. If match: create session, return {sessionId, user} │
└─────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────┐
│ Response │
│ 13. Set cookie: bt_session=<sessionId> │
│ 14. JSON: {user: {id, username, role, ...}} │
│ 15. Set CSRF token cookie (bt_csrf_token) │
└─────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────┐
│ Frontend (useAuth hook) │
│ 16. Stores user in context │
│ 17. Redirects to /tracker or / │
└─────────────────────────────────────────────────────────────┘
```
### OIDC Login Flow
```
┌─────────────────────────────────────────────────────────────┐
│ User │
│ 1. Clicks "Login with Authentik" button │
└─────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────┐
│ Frontend (LoginPage.jsx) │
│ 2. Calls apiGet('/api/auth/oidc/login?redirect_to=/tracker')│
└─────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────┐
│ Backend (authOidc.js) │
│ 3. GET /api/auth/oidc/login │
│ 4. isOidcLoginActive() check │
│ 5. createLoginState(redirect_to) │
│ • Generates PKCE code_verifier │
│ • Generates nonce │
│ • Stores in oidc_states (expires 5 min) │
│ 6. buildAuthorizationUrl(config, state) │
└─────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────┐
│ Response │
│ 7. HTTP 302 redirect to OIDC provider authorization URL │
│ • Includes: client_id, redirect_uri, response_type=code │
│ • Includes: state (login state ID), nonce │
│ • Includes: code_challenge, code_challenge_method=S256 │
│ • Includes: scopes (openid, email, profile, groups) │
└─────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────┐
│ OIDC Provider (Authentik) │
│ 8. User authenticates (credentials) │
│ 9. Provider creates ID token │
│ 10. Redirects to redirect_uri with code + state │
└─────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────┐
│ Backend (authOidc.js) │
│ 11. GET /api/auth/oidc/callback │
│ 12. oidcLimiter checks IP │
│ 13. consumeLoginState(state) │
│ • Validates expiry │
│ • Returns: {nonce, code_verifier, redirect_to} │
│ 14. exchangeAndVerifyTokens(config, code, stateId, savedState) │
│ • POST to token endpoint with code + client_secret │
│ • Verifies JWT signature via JWKS │
│ • Validates: iss, aud, exp, nonce, state │
│ 15. findOrProvisionUser(claims, config) │
│ • Look up by sub (external_subject) │
│ • Look up by email if email_verified=true │
│ • Auto-provision if enabled │
│ • Map groups to role (admin if in admin_group) │
└─────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────┐
│ Service (authService.createSession) │
│ 16. Create session for user │
│ 17. Set cookie: bt_session=<sessionId> │
└─────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────┐
│ Response │
│ 18. HTTP 302 redirect to redirect_to (or /) │
└─────────────────────────────────────────────────────────────┘
```
### Authenticated API Request Flow
```
┌─────────────────────────────────────────────────────────────┐
│ Frontend (api.js) │
│ 1. apiGet('/api/tracker', {year, month}) │
│ 2. Retrieve CSRF token from bt_csrf_token cookie │
│ 3. Set header: x-csrf-token=<token> │
│ 4. Set header: cookie: bt_session=<sessionId> │
│ 5. Send request to backend │
└─────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────┐
│ Backend (server.js) │
│ 6. requestLooksHttps(req) │
│ • Checks: req.secure, x-forwarded-proto │
│ 7. csrfTokenProvider sets cookie (if not present) │
│ 8. route middleware chain runs │
└─────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────┐
│ Middleware (requireAuth) │
│ 9. getSessionUser(cookie.bt_session) │
│ • Query sessions + users table │
│ • Check: expires_at > now, active = 1 │
│ 10. If valid: attach req.user, next() │
│ If invalid: return 401 {error: 'Not authenticated'} │
└─────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────┐
│ Middleware (requireUser) │
│ 11. Check role is 'user' or 'admin' │
│ 12. Check not default admin (no tracker access) │
│ 13. next() │
└─────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────┐
│ Middleware (csrfMiddleware) │
│ 14. validateCsrfToken(req) │
│ • Check header x-csrf-token matches cookie │
│ • Check query.csrf_token │
│ • Check body.csrf_token │
│ 15. If valid: next() │
│ If invalid: return 403 {error: 'CSRF token validation failed'} │
└─────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────┐
│ Route Handler (tracker.js) │
│ 16. ensureUserDefaultCategories(req.user.id) │
│ 17. db.prepare('SELECT * FROM bills WHERE user_id = ?') │
│ 18. Build tracker rows │
│ 19. Return JSON response │
└─────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────┐
│ Frontend (useAuth hook) │
│ 20. Update state with response │
│ 21. Re-render component │
└─────────────────────────────────────────────────────────────┘
```
### Background Worker Flow (Daily)
```
┌─────────────────────────────────────────────────────────────┐
│ Server Start (server.js) │
│ 1. main() │
│ 2. Check if users exist, create admin if needed │
│ 3. workers/dailyWorker.js start() │
└─────────────────────────────────────────────────────────────┘
│
▒
┌─────────────────────────────────────────────────────────────┐
│ Worker (dailyWorker.js) │
│ 4. markWorkerStarted(nextDailyRunIso()) │
│ 5. runDailyTasks() │
│ • Prune expired sessions │
│ • Run notifications (email) │
│ • Run cleanup (temp files, import sessions) │
│ • Auto-mark autopay bills as assumed_paid │
│ 6. markWorkerSuccess(nextDailyRunIso()) │
└─────────────────────────────────────────────────────────────┘
│
▒
┌─────────────────────────────────────────────────────────────┐
│ Cron Job (node-cron) │
│ 7. cron.schedule('0 6 * * *', ...) │
│ Runs daily at 6:00 AM │
│ 8. Same runDailyTasks() as above │
└─────────────────────────────────────────────────────────────┘
```
### Notification Flow
```
┌─────────────────────────────────────────────────────────────┐
│ Daily Worker (dailyWorker.js) │
│ 1. runNotifications() │
└─────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────┐
│ Notification Service (notificationService.js) │
│ 2. Check SMTP enabled, host configured │
│ 3. Get recipients (users with notifications_enabled=1 OR │
│ global recipient configured) │
│ 4. For each bill: │
│ • Calculate due date for this month │
│ • Determine notification type (due_3d, due_1d, due_today,│
│ overdue) │
│ • Check if already sent today (notifications table) │
│ • Check user notification preferences │
│ • Build HTML email template │
│ • Send email via nodemailer │
│ • Record in notifications table (to prevent duplicates) │
└─────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────┐
│ Email (Nodemailer) │
│ 5. SMTP transport sendMail() │
│ 6. Return success/error │
└─────────────────────────────────────────────────────────────┘
```
---
## Version History
| Version | Date | Changes |
|---------|------|---------|
| **0.19.0** | 2026-05-09 | Complete Engineering Reference Manual |
| **0.18.3** | 2026-05-08 | Added `monthly_starting_amounts.other_amount` column |
| **0.18.2** | 2026-05-08 | Added `monthly_starting_amounts` table |
| **0.18.1** | 2026-05-08 | Added `monthly_income` table |
| **0.17** | 2026-05-08 | OIDC columns (`auth_provider`, `external_subject`, `email`, `last_login_at`), `oidc_states` table |
| **0.14** | 2026-05-08 | Added `history_visibility`, `bill_history_ranges`, `interest_rate` |
| **0.13** | 2026-05-08 | Profile columns (`display_name`, `last_password_change_at`) |
| **0.12** | 2026-05-08 | Bill history ranges table |
| **0.11** | 2026-05-08 | Import history table |
| **0.10** | 2026-05-08 | Import sessions table |
| **0.4** | 2026-05-08 | `monthly_bill_state` table |
| **0.2** | 2026-05-08 | `payments.deleted_at` column |
---
## Quick Reference
### Critical Endpoints
| Method | Endpoint | Auth | Purpose |
|--------|----------|------|---------|
| POST | `/api/auth/login` | None | Local login |
| GET | `/api/auth/oidc/login` | None | OIDC login start |
| GET | `/api/tracker` | User | Monthly tracker data |
| GET | `/api/bills` | User | List bills |
| GET | `/api/admin/users` | Admin | List users |
| POST | `/api/admin/backups` | Admin | Create backup |
| GET | `/api/version` | None | Version info |
| GET | `/api/about` | None | About info |
### Critical Settings
| Key | Default | Description |
|-----|---------|-------------|
| `local_login_enabled` | `true` | Enable username/password login |
| `oidc_login_enabled` | `false` | Enable OIDC login |
| `oidc_issuer_url` | - | OIDC provider URL |
| `oidc_client_id` | - | OIDC client ID |
| `oidc_client_secret` | - | OIDC client secret |
| `oidc_redirect_uri` | - | OIDC redirect URI |
| `oidc_admin_group` | - | OIDC group for admin access |
| `oidc_auto_provision` | `true` | Auto-create users from OIDC |
| `notify_smtp_enabled` | `false` | Enable email notifications |
| `notify_smtp_host` | - | SMTP host |
| `notify_smtp_port` | `587` | SMTP port |
| `backup_enabled` | `false` | Enable manual backups |
| `backup_schedule_enabled` | `false` | Enable scheduled backups |
### Database Tables Reference
| Table | Purpose |
|-------|---------|
| `users` | User accounts |
| `sessions` | Active sessions |
| `bills` | Bill records |
| `categories` | Bill categories |
| `payments` | Payment records |
| `monthly_bill_state` | Monthly bill overrides |
| `settings` | Application settings |
| `notifications` | Notification history |
| `oidc_states` | OIDC login state |
| `bill_history_ranges` | History visibility ranges |
| `monthly_income` | Monthly income records |
| `monthly_starting_amounts` | Starting balance records |
| `import_sessions` | Import preview sessions |
| `import_history` | Import history |
---
**This document is the canonical reference for the Bill Tracker system.**
*Last updated: 2026-05-09*
*Author: Bishop (code reviewer and architecture validator)*
Reference in New Issue View Git Blame Copy Permalink