52 KiB
Engineering Reference Manual — Bill Tracker
Status: Current code reference
Last Updated: 2026-05-10
Version: 0.23.2
Primary stack: Node.js + Express, React + Vite, SQLite via better-sqlite3
This manual reflects the current application code in server.js, routes/, services/, middleware/, db/, client/, package.json, Dockerfile, and docker-compose.yml. It is written as a current-state reference, not a changelog.
1. System Overview
Bill Tracker is a self-hosted bill management application. It supports:
- Local username/password authentication, optional single-user mode, and optional Authentik/OIDC login.
- User-scoped bills, categories, payments, monthly bill overrides, monthly income, and starting cash buckets.
- Admin user management, backup/restore, cleanup, auth-mode/OIDC configuration, status checks, and migration rollback.
- Spreadsheet and user-SQLite import workflows.
- CSV, Excel, and user-SQLite export workflows.
- SMTP-based bill due notifications.
- React SPA frontend with protected user/admin routes and CSRF-protected JSON APIs.
Runtime flow:
server.jsinitializes SQLite throughdb/database.js, runs schema/migrations, seeds defaults/admin user, cleans expired sessions, then starts Express.- Express applies security headers, JSON body parsing, cookies, CSRF token provisioning, route-level CSRF/auth/rate-limit middleware, static
dist/serving, and JSON error formatting. - Route files under
routes/validate input, enforce ownership throughreq.user.id, and use service modules for business logic. - React under
client/calls/api/*throughclient/api.jswithcredentials: includeand CSRF headers on mutating methods.
2. Project Layout
server.js— Express entry point and route mounting.routes/— HTTP API handlers.services/— auth, OIDC, backup, cleanup, notification, import, status, audit business logic.middleware/— auth guards, CSRF, rate limits, security headers, error formatting.db/schema.sql— base SQLite schema.db/database.js— DB connection, migrations, defaults, settings, rollback support.workers/dailyWorker.js— scheduled notification/cleanup worker entry.client/— React SPA.dist/— generated Vite build output served by Express.Dockerfile,docker-compose.yml,docker-entrypoint.sh— container deployment.
3. Backend Entry Point and Middleware
server.js
Server defaults:
- Port:
PORT || 3000. - Static frontend:
dist/. - Optional CORS: enabled when
CORS_ORIGINis set; credentials allowed. - Session cleanup: runs on startup and every
SESSION_CLEANUP_INTERVAL_MS || 86400000ms. - Admin seed:
INIT_ADMIN_USER || admin;INIT_ADMIN_PASSor generated/default behavior indb/database.js. - Environment variables INIT_ADMIN_USER and INIT_ADMIN_PASS (or INIT_REGULAR_USER + INIT_REGULAR_PASS) skip the first-login flow entirely by pre-seeding users with first_login=0 flags via setup/firstRun.js.
- Optional regular-user seed:
INIT_REGULAR_USER+INIT_REGULAR_PASS; password must be at least 8 chars.
Global middleware order:
securityHeaders- optional
cors express.json()cookieParser()csrfTokenProvider- mounted API routers with route-level rate-limit/auth/CSRF middleware
- static
legacy/, redirect/login.htmlto/login, staticdist/ - SPA fallback
GET *servingdist/index.htmlafter ensuring a CSRF token cookie errorFormatter- final JSON error handler for malformed JSON/body size/runtime errors
Authentication middleware
middleware/requireAuth.js exports:
requireAuth: attachesreq.userfrombt_session; in single-user mode attaches the configured active regular user without a session.requireUser: permitsuserandadminroles but blocks the default admin account from tracker access.requireAdmin: requiresreq.user.role === 'admin'.
CSRF middleware
middleware/csrf.js:
- Cookie name:
CSRF_COOKIE_NAME || bt_csrf_token. - Header name:
x-csrf-token. - Defaults:
CSRF_HTTP_ONLY !== false,CSRF_SAME_SITE || strict,CSRF_SECURE !== false. csrfTokenProvidersets a token cookie on responses.csrfMiddlewarevalidates mutating requests unlessreq.csrfSkipis set. Token may come from header, query, or body and must match cookie.- Failures return 403 and audit
csrf.failure.
Rate limits
Defined in middleware/rateLimiter.js:
- Login: 10 / 15 min.
- Password changes: 5 / 15 min.
- Import: 20 / 15 min.
- Export: 30 / 15 min.
- Admin actions: 30 / 15 min.
- OIDC: 20 / 15 min.
- Backup operations: 5 / 60 min.
- Demo-data clear: 3 / 15 min.
Rate-limit responses are JSON: { "error": "..." }.
Security headers
middleware/securityHeaders.js sets CSP with a per-request nonce plus common hardening headers. Static SPA scripts/styles must comply with the generated CSP.
Error formatting
middleware/errorFormatter.js standardizes route responses into JSON with error, code, and optional field. Common statuses: 400 validation, 401 auth, 403 forbidden, 404 not found, 409 conflict, 429 rate limit, 500 server error.
4. Services
services/authService.js
- Cookie:
bt_session. - Session lifetime: 7 days.
- Password hashing: bcrypt, cost 12.
- Functions:
login(username, password)— verifies active local user, rejects OIDC-only accounts, cleans expired sessions for that user, creates a new session, updateslast_login_at, and returns{sessionId, user}ornull.createSession(userId)— creates a session for an OIDC-provisioned or existing local user after validating that the user is active.logout(sessionId)— deletes one session.getSessionUser(sessionId)— returns public user fields when the session exists, is unexpired, and belongs to an active user.rotateSessionId(oldSessionId, userId)— validates the current session, deletes it, and inserts a replacement session in a transaction. Password changes use this to prevent session fixation.invalidateOtherSessions(userId, keepSessionId)— deletes all sessions for a user except the supplied current session; whenkeepSessionIdisnull, deletes every session for that user.pruneExpiredSessions()— deletes expired sessions.publicUser(user)— strips password/session secrets and normalizes booleans.
Password changes through /api/auth/change-password update the password hash, clear must_change_password, stamp last_password_change_at, invalidate all other sessions, rotate the current session ID when a valid session cookie is present, set a replacement bt_session cookie, and audit password.change.
/api/auth/logout-all calls invalidateOtherSessions(req.user.id, null), deletes the current cookie session if present, audits logout.all, clears bt_session, and returns {success:true}.
services/oidcService.js
Handles Authentik/OIDC login with openid-client:
- Reads settings first, then env fallbacks: issuer URL, client ID/secret, redirect URI, token auth method, scopes, provider name, admin group, auto-provision flag.
- Builds PKCE login state in
oidc_states. - Discovers provider and caches OIDC client for 1 hour.
- Exchanges callback code, verifies nonce/state, maps claims to local user.
- Role mapping uses configured admin group; default role is
user. - Client secret is write-only through admin settings.
services/backupService.js
- Backup directory:
BACKUP_PATH || backups/. - Valid backup IDs match managed prefixes only:
bill-tracker-backup,pre-restore,imported-backup,scheduled-backup. - Creates SQLite backups with checksum and metadata.
- Validates uploads by SQLite
integrity_checkand optional SHA-256 checksum. - Restore creates a pre-restore backup before swapping DB.
- Path traversal is prevented by ID regex and
path.relativechecks.
services/backupScheduler.js
- Validates daily/weekly schedule,
HH:MMtime, retention count 1-365. - Stores schedule settings in
settings. - Uses
node-cron, supports run-now, reload, next-run status, and retention cleanup.
services/cleanupService.js
Cleanup tasks:
- Expired import sessions.
- Stale temp SQLite export files in OS temp dir.
- Orphan
.partial/.uploadbackup files. - Optional old import-history pruning.
Settings are stored in settings; run results are stored as JSON.
services/notificationService.js
- Builds SMTP transport from global notification settings.
- Sends test email to an admin-provided address.
- Runs due-bill notifications for 3 days, 1 day, due today, and overdue.
- De-duplicates sends via
notificationsunique key. - Recipients come from user notification settings when enabled/allowed or global recipient settings.
services/spreadsheetImportService.js
- Accepts XLSX buffers only, max 10 MB, max 5,000 rows.
- Detects monthly sheets, headers, categories, bills, amounts, and ambiguous rows.
- Creates
import_sessionspreview records with 24-hour TTL. - Apply step creates/updates user-owned categories, bills, payments, monthly state, and import history.
services/userDbImportService.js
- Accepts user SQLite export files up to 50 MB.
- Requires export metadata and known tables.
- Sanitizes categories, bills, payments, monthly state, and starting amounts.
- Preview stores an import session; apply maps export IDs to current user-owned IDs.
services/statusService.js
Shared tracker/calendar logic:
resolveDueDate(bill, year, month)clamps due day to month length.resolveBucket(bill)uses bucket or due-day threshold.getCycleRange(year, month)returns first/last day of month.calculateStatus(...)returns paid/autodraft/upcoming/due/overdue-style status.buildTrackerRow(...)returns row data for the monthly tracker.
services/auditService.js
Writes audit_log rows for security-sensitive events such as login, logout, password changes, role changes, CSRF failures, user seed flag resets, migration runs, and migration rollback attempts.
db/database.js does not import logAudit at module load time. It uses a lazy getLogAudit() helper so migration code can write audit rows without creating a circular dependency: auditService imports database.js, and database.js needs audit logging during migrations. If the lazy require fails, the helper degrades to a no-op audit function while console logging still records migration progress/errors.
5. API Reference
All routes are prefixed with /api unless stated otherwise. Most mutating endpoints require CSRF token header x-csrf-token. Auth is cookie-based. User routes are scoped to the authenticated user unless noted.
Response conventions:
- Success: JSON object/array unless endpoint downloads a file.
- Validation:
400 {error, code?, field?}. - Unauthenticated:
401. - Forbidden:
403. - Missing resource:
404. - Conflict:
409. - Rate-limited:
429.
5.1 Auth
Mounted under /api/auth.
-
POST /auth/login- Body:
{username, password}. - Validation: both required; local login must be enabled.
- Response: sets
bt_session;{user}.
- Body:
-
POST /auth/logout- Auth: required.
- Body: none.
- Response: clears cookie;
{success:true}.
-
POST /auth/logout-all- Auth: required; CSRF skip is set before the router mount, matching other auth/session mutation routes.
- Body: none.
- Behavior: deletes every session for the current user by calling
invalidateOtherSessions(userId, null), also deletes the current cookie session, auditslogout.all, clearsbt_session, and returns{success:true}.
-
GET /auth/me- Auth: required unless single-user mode supplies user.
- Response: public user object.
-
GET /auth/mode- Public.
- Response: auth mode, local-login flag, OIDC public info, single-user status.
-
POST /auth/restore-multi-user-mode- Auth: required.
- Body: none.
- Response: restores multi-user mode where allowed.
-
POST /auth/acknowledge-privacy- Auth: required.
- Body: none.
- Response: updates first-login/privacy acknowledgement flags.
-
POST /auth/change-password- Auth: required; password limiter; CSRF skip is set before the router mount.
- Body:
{current_password, new_password}. - Validation: current password required unless
must_change_passwordis set; new password min 8. - Behavior: updates password hash, clears
must_change_password, updateslast_password_change_at, invalidates all other sessions, rotates the current session ID when a validbt_sessionexists, sets the new cookie, auditspassword.change, and returns{success:true}.
-
GET /auth/has-users- Response: whether non-default users exist.
-
GET /auth/users- Auth: admin.
- Response: safe user list.
-
POST /auth/users- Auth: admin.
- Body:
{username, password}. - Validation: username min 3, password min 8, unique username.
- Response: created safe user.
5.2 OIDC Auth
Mounted under /api/auth/oidc; OIDC rate limiter applies.
-
GET /auth/oidc/login?redirect_to=/path- Public when OIDC active.
- Creates PKCE state and redirects to provider authorization URL.
-
GET /auth/oidc/callback?code=&state=- Public callback.
- Validates state/nonce, exchanges code, provisions/fetches user, creates session, redirects to saved path or app root.
- Error redirects use query errors such as
access_deniedorauthentication_failed.
5.3 Admin
Mounted under /api/admin; all require requireAuth + requireAdmin + adminActionLimiter at the mount level. Backup subroutes also use backupOperationLimiter.
-
GET /admin/has-users- Response:
{has_users:boolean}for users other than current admin.
- Response:
-
GET /admin/users- Response: safe users ordered by default-admin, role, username.
-
POST /admin/users- Body:
{username, password}. - Validation: username min 3, password min 8, unique.
- Response 201: created user.
- Body:
-
PUT /admin/users/:id/password- Body:
{password}. - Validation: password min 8; user exists.
- Response:
{success:true}; invalidates target sessions and requires password change.
- Body:
-
PUT /admin/users/:id/role- Body:
{role:"admin"|"user"}. - Validation: cannot change own role; cannot remove last admin.
- Response: updated safe user; invalidates target sessions; audits role change.
- Body:
-
PUT /admin/users/:id/active- Body:
{active:boolean}. - Validation: user exists; cannot deactivate self.
- Response: updated safe user; deactivation invalidates sessions.
- Body:
-
DELETE /admin/users/:id- Validation: user exists; cannot delete self.
- Response:
{success:true, deleted_user_id}; transaction deletes import sessions/history, sessions, and user.
-
POST /admin/backups- Body: none.
- Response 201: backup metadata
{id, filename, size_bytes, checksum, ...}.
-
GET /admin/backups- Response:
{backups:[metadata...]}.
- Response:
-
GET /admin/backups/settings- Response: backup schedule status/settings.
-
PUT /admin/backups/settings- Body:
{enabled, frequency:"daily"|"weekly", time:"HH:MM", retention_count}. - Validation: frequency, valid time, retention 1-365.
- Response: saved schedule status.
- Body:
-
POST /admin/backups/run-scheduled-now- Body: none.
- Response 201: scheduled backup result.
-
POST /admin/backups/import- Content-Type:
application/octet-stream,application/x-sqlite3, orapplication/vnd.sqlite3. - Body: raw SQLite backup, max 100 MB.
- Optional checksum:
X-Checksum-Sha256header or?checksum=. - Response 201: imported backup metadata.
- Content-Type:
-
GET /admin/backups/:id/download- Response: file download. ID must be a managed backup filename.
-
POST /admin/backups/:id/restore- Response:
{restored_from, pre_restore_backup}; validates and restores managed backup.
- Response:
-
DELETE /admin/backups/:id- Response:
{deleted:true, id, deleted_at}.
- Response:
-
GET /admin/cleanup- Response: cleanup settings and last result.
-
PUT /admin/cleanup- Body: any of
{import_sessions_enabled, temp_exports_enabled, temp_export_max_age_hours, backup_partials_enabled, import_history_enabled, import_history_max_age_days}. - Validation: booleans; temp export age 1-72 hours; import history age 30-3650 days.
- Response: updated cleanup status.
- Body: any of
-
POST /admin/cleanup/run- Response: cleanup run result by task.
-
GET /admin/auth-mode- Response: local/single-user/OIDC settings and lockout warnings. Client secret is not returned.
-
POST /admin/auth-mode/oidc-test- Body: submitted or saved OIDC config fields.
- Response:
{ok:true,...}or 400 with test error. Never returns secret/token material.
-
PUT /admin/auth-mode- Body: legacy
{auth_mode, default_user_id}plus OIDC/local settings such aslocal_login_enabled,oidc_login_enabled,oidc_issuer_url,oidc_client_id,oidc_client_secret,oidc_redirect_uri,oidc_scopes,oidc_admin_group,oidc_auto_provision. - Validation: cannot disable all login methods; cannot disable local login until OIDC is configured, enabled, and has an admin group; cannot enable incomplete OIDC.
- Response:
{success:true, ...authModeStatus}.
- Body: legacy
-
POST /admin/migrations/rollback- Body:
{version:"v0.44"|"v0.45"|"v0.46"}. - Validation: version required; migration must be present in
schema_migrations;ROLLBACK_SQL_MAPmust define rollback SQL for that version. - Behavior: calls
rollbackMigration(version)fromdb/database.js, audits success asmigration.rollback, and returns{success:true, version, description, elapsed_ms}. - Error mapping:
NOT_APPLIEDbecomes HTTP 404 with{error};ROLLBACK_NOT_SUPPORTEDbecomes HTTP 422 with{error}; other rollback failures become HTTP 500 with{error:"Rollback failed", details}and are audited asmigration.rollback.failure.
- Body:
5.4 Bills
Mounted under /api/bills; auth: user/admin tracker access.
Bill object fields include id, user_id, name, category_id, category_name, due_day, override_due_date, bucket, expected_amount, interest_rate, billing_cycle, autopay_enabled, autodraft_status, website, username, account_info, has_2fa, active, notes, history_visibility, is_seeded, cycle_type, cycle_day, timestamps, and has_history_ranges on list/detail queries.
Validation shared by create/update:
namerequired for create.due_day: integer 1-31.expected_amount: numeric, defaults to 0.interest_rate: null/empty or number 0-100.category_id: must belong to current user.history_visibility:default,all,ranges, ornone.cycle_type:monthly,weekly,biweekly,quarterly,annual.cycle_day: monthly 1-31; weekly/biweekly day name; quarterly/annual text up to 50 chars.
Endpoints:
-
GET /bills?inactive=true- Response: current user's bills; inactive excluded unless
inactive=true.
- Response: current user's bills; inactive excluded unless
-
GET /bills/:id- Response: one owned bill or 404.
-
POST /bills- Body: bill fields.
- Response 201: created bill.
-
PUT /bills/:id- Body: partial bill fields.
- Response: updated bill.
-
DELETE /bills/:id- Hard delete; cascades payments, monthly state, history ranges.
- Response:
{success:true, deleted_bill_id, deleted_bill_name, warning}.
-
GET /bills/:id/monthly-state?year=&month=- Validation: year 2000-2100; month 1-12.
- Response:
{bill_id, year, month, actual_amount, notes, is_skipped}.
-
PUT /bills/:id/monthly-state- Body:
{year, month, actual_amount, notes, is_skipped}. - Validation: year/month; actual_amount null or non-negative.
- Response: saved monthly state with timestamps.
- Body:
-
GET /bills/:id/payments?page=1&limit=20- Validation: limit capped at 100.
- Response:
{bill_id, bill_name, total, page, limit, pages, payments}.
-
POST /bills/:id/toggle-paid- Body optional:
{amount, paid_date, method, notes}. - If latest live payment exists, soft-deletes it. Otherwise creates a payment for amount or bill expected amount.
- Response:
{success, isPaid, action, payment?}.
- Body optional:
-
GET /bills/:id/history-ranges- Response:
{bill_id, history_visibility, ranges}.
- Response:
-
POST /bills/:id/history-ranges- Body:
{start_year, start_month, end_year?, end_month?, label?}. - Validation: years 2000-2100, months 1-12, end both present/absent and not before start.
- Response 201: created range.
- Body:
-
PUT /bills/:id/history-ranges/:rangeId- Body: partial range fields.
- Response: updated range.
-
DELETE /bills/:id/history-ranges/:rangeId- Response:
{success:true}.
- Response:
5.5 Payments
Mounted under /api/payments; auth: user/admin tracker access. All queries are user-owned through joined bill ownership. Delete is soft delete via deleted_at.
-
GET /payments?bill_id=&year=&month=- Validation: year and month must be supplied together; year 2000-2100; month 1-12.
- Response: live payments ordered descending by
paid_date.
-
GET /payments/:id- Response: live payment or 404.
-
POST /payments- Body:
{bill_id, amount, paid_date, method?, notes?}. - Validation: bill exists and owned; amount > 0; required fields present.
- Response 201: created payment.
- Body:
-
POST /payments/quick- Body:
{bill_id, amount?, paid_date?, method?, notes?}. - Defaults amount to bill expected amount and date to today; confirms autodraft status for autopay bills.
- Response 201: created payment.
- Body:
-
POST /payments/bulk- Body:
{payments:[{bill_id, amount, paid_date, method?, notes?}]}. - Validation: array required; max 50; bill_id integer;
paid_dateYYYY-MM-DD; amount finite >= 0. - Duplicate live payments by user/bill/date/amount are skipped.
- Response 201:
{created:[...], skipped:[...], errors:[...]}.
- Body:
-
PUT /payments/:id- Body: partial
{amount, paid_date, method, notes}. - Response: updated payment. Current code preserves existing fields when omitted.
- Body: partial
-
DELETE /payments/:id- Response:
{success:true}after settingdeleted_at.
- Response:
-
POST /payments/:id/restore- Response: restored payment with
deleted_at:null.
- Response: restored payment with
5.6 Categories
Mounted under /api/categories; auth: user/admin tracker access.
-
GET /categories- Seeds default categories for user if needed.
- Response: categories with bill counts/payment counts and bill summaries.
-
POST /categories- Body:
{name}. - Validation: non-empty name; unique per user case-insensitive.
- Response 201: created category.
- Body:
-
PUT /categories/:id- Body:
{name}. - Validation: category belongs to user; non-empty unique name.
- Response: updated category.
- Body:
-
DELETE /categories/:id- Validation: category belongs to user.
- Behavior: transaction nulls category on owned bills, then deletes category.
- Response: deletion summary.
5.7 Tracker and Calendar
-
GET /tracker?year=&month=- Auth: user/admin tracker access.
- Defaults to current year/month.
- Response includes
year,month, trackerrows, totals, starting amount info, previous month paid total, three-month averages/trends, and generated timestamp. - Row fields come from
buildTrackerRowplus monthly override state and previous-month payment data.
-
GET /tracker/upcoming?days=30- Auth: user/admin tracker access.
- Response: upcoming active bills in the requested horizon.
-
GET /calendar?year=&month=- Auth: user/admin tracker access.
- Defaults current year/month.
- Response: month days with payment entries, bills/due-status entries, and totals
{expectedTotal, paidTotal, remainingTotal, paidPercent}.
5.8 Summary and Starting Amounts
-
GET /summary?year=&month=- Auth: user/admin tracker access.
- Validation: valid year/month.
- Response:
{year, month, income, expenses, starting_amounts, previous_month, summary, chart, generated_at}.
-
PUT /summary/income- Body:
{year, month, amount, label?}. - Validation: valid year/month; amount 0-1,000,000,000; label trimmed to 80 chars.
- Response:
{year, month, income}after upsert intomonthly_income.
- Body:
-
GET /monthly-starting-amounts?year=&month=- Response:
{year, month, first_amount, fifteenth_amount, other_amount, combined_amount, paid deductions, remaining values, notes}.
- Response:
-
PUT /monthly-starting-amounts- Body:
{year, month, first_amount, fifteenth_amount, other_amount, notes?}. - Validation: valid year/month; numeric amounts.
- Response: recomputed starting-amount response after upsert.
- Body:
5.9 Analytics
GET /analytics/summary?year=&month=&months=&category_id=&bill_id=&include_inactive=true&include_skipped=false- Auth: user/admin tracker access.
- Validation: year/month valid; months clamped by route validation; IDs parsed as integers.
- Response includes monthly spending, expected vs actual, category totals, bill totals, filters, and generated timestamp.
5.10 Settings
-
GET /settings- Auth: user/admin tracker access.
- Response: user-visible settings from the allowed settings key list.
-
PUT /settings- Auth: user/admin tracker access.
- Body: key/value object for allowed user setting keys.
- Response: updated settings object.
-
POST /settings/seed-demo-data- Auth: user/admin tracker access.
- Response: demo seed result.
5.11 Notifications
Mounted under /api/notifications. Server mount requires requireAuth; route-level guards further restrict.
-
GET /notifications/admin- Auth: admin.
- Response: global SMTP/notification settings.
-
PUT /notifications/admin- Auth: admin.
- Body: allowed global SMTP and notification settings.
- Response: saved settings.
-
POST /notifications/test- Auth: admin.
- Body:
{to}. - Validation: recipient required.
- Response: send result or SMTP error.
-
GET /notifications/me- Auth: user/admin tracker access.
- Response: current user's notification email and toggle settings.
-
PUT /notifications/me- Auth: user/admin tracker access.
- Body:
{notification_email, notifications_enabled, notify_3d, notify_1d, notify_due, notify_overdue}. - Response: saved user notification settings.
5.12 Profile
Mounted under /api/profile; auth: user/admin tracker access; password limiter applies at mount.
-
GET /profile- Response: safe profile, notification settings, export URLs, import-history URL.
-
PATCH /profile- Body:
{display_name}. - Validation: string, max 100 chars.
- Response:
{success:true, profile}.
- Body:
-
GET /profile/settings- Response: user notification preferences only.
-
PATCH /profile/settings- Body:
{notification_email|email, notifications_enabled, notify_3d, notify_1d, notify_due, notify_overdue}. - Validation: email value string/null, max 255 chars.
- Response:
{success:true}.
- Body:
-
POST /profile/change-password- Body:
{current_password, new_password, confirm_new_password}. - Validation: all required; confirmation matches; new password min 8; current password must verify.
- Response: rotates current session, invalidates others,
{success:true}.
- Body:
-
GET /profile/exports- Response: metadata for
user_dbanduser_excelexport URLs.
- Response: metadata for
-
GET /profile/import-history- Response:
{history:[...]}.
- Response:
5.13 User Demo Data
Mounted under /api/user; auth: user/admin tracker access.
-
POST /user/seed-demo-data- Response: seed result for current user.
-
POST /user/clear-demo-data- Rate-limited by demo-data limiter.
- Behavior: deletes
is_seeded=1bills/categories for current user and records import history. - Response: deletion counts.
5.14 Import
Mounted under /api/import; auth: user/admin tracker access; import limiter applies.
-
POST /import/spreadsheet/preview?parse_all_sheets=true&year=&month=- Content-Type:
application/octet-stream. - Headers: optional
X-Filename. - Body: XLSX file buffer, max 10 MB.
- Response: preview session, parsed rows, categories/bills/payment candidates, ambiguous decisions, errors.
- Content-Type:
-
POST /import/spreadsheet/apply- Body:
{import_session_id, decisions, options}. - Validation: session exists, not expired, belongs to user.
- Response: created/updated/skipped/error counts and import history summary.
- Body:
-
POST /import/user-db/preview- Content-Type:
application/octet-stream. - Headers: optional
X-Filename. - Body: user SQLite export file, max 50 MB.
- Response: preview session and sanitized import plan.
- Content-Type:
-
POST /import/user-db/apply- Body:
{import_session_id, options}. - Validation: session exists, not expired, belongs to user.
- Response: apply result with ID mappings/counts.
- Body:
-
GET /import/history- Response: current user's import history.
5.15 Export
Mounted under /api/export; auth: user/admin tracker access; export limiter applies.
-
GET /export?year=YYYY&format=csv|xlsx- Response: file download of payment/bill history for the requested year. CSV includes date, bill, category, expected, paid, method, notes, actual amount, monthly notes. XLSX includes enriched rows.
-
GET /export/user-excel- Response: Excel workbook with user categories, bills, payments, monthly state, monthly starting amounts, and notes/metadata.
-
GET /export/user-db- Response: portable SQLite file with export metadata and user-owned categories, bills, payments, monthly state, monthly starting amounts, and notes.
5.16 Status
GET /status- Auth: admin.
- Response: app version, uptime, runtime worker state, DB health/counts/path/size, SMTP configuration status, backup status/schedule, current-month tracker health, recent errors.
5.17 About and Version
-
GET /about- Public.
- Response: package version and public project/about metadata.
-
GET /about-admin- Auth: admin; admin action limiter; CSRF middleware.
- Response: package version plus sanitized/redacted
FUTURE.mdandDEVELOPMENT_LOG.mdcontent. - File allowlist only:
FUTURE.md,DEVELOPMENT_LOG.md.
-
GET /version- Public.
- Response: current package version and latest structured notes from
HISTORY.md.
-
GET /version/history- Public.
- Response: package version and raw history text, or error if unavailable.
6. Database Reference
SQLite uses WAL mode and foreign keys. Base schema is in db/schema.sql; db/database.js applies migrations to reach the current schema.
Tables and columns
users
id INTEGER PRIMARY KEYusername TEXT NOT NULL UNIQUE COLLATE NOCASEpassword_hash TEXT NOT NULLrole TEXT NOT NULL DEFAULT 'user'(adminoruser)must_change_password INTEGER NOT NULL DEFAULT 0first_login INTEGER NOT NULL DEFAULT 1created_at TEXT DEFAULT datetime('now')updated_at TEXT DEFAULT datetime('now')notification_email TEXTnotifications_enabled INTEGER NOT NULL DEFAULT 0notify_3d INTEGER NOT NULL DEFAULT 1notify_1d INTEGER NOT NULL DEFAULT 1notify_due INTEGER NOT NULL DEFAULT 1notify_overdue INTEGER NOT NULL DEFAULT 1display_name TEXTlast_password_change_at TEXTauth_provider TEXT NOT NULL DEFAULT 'local'external_subject TEXTemail TEXTlast_login_at TEXTactive INTEGER NOT NULL DEFAULT 1is_default_admin INTEGER NOT NULL DEFAULT 0
sessions
id TEXT PRIMARY KEYuser_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADEexpires_at TEXT NOT NULLcreated_at TEXT DEFAULT datetime('now')
categories
id INTEGER PRIMARY KEYuser_id INTEGER REFERENCES users(id) ON DELETE CASCADEname TEXT NOT NULLcreated_at TEXT DEFAULT datetime('now')updated_at TEXT DEFAULT datetime('now')is_seeded INTEGER NOT NULL DEFAULT 0
bills
id INTEGER PRIMARY KEYuser_id INTEGER REFERENCES users(id) ON DELETE CASCADEname TEXT NOT NULLcategory_id INTEGER REFERENCES categories(id) ON DELETE SET NULLdue_day INTEGER NOT NULL CHECK 1-31override_due_date TEXTbucket TEXT CHECK ('1st','15th')expected_amount REAL NOT NULL DEFAULT 0interest_rate REAL CHECK null or 0-100billing_cycle TEXT DEFAULT 'monthly' CHECK ('monthly','quarterly','annually','irregular')autopay_enabled INTEGER NOT NULL DEFAULT 0autodraft_status TEXT NOT NULL DEFAULT 'none' CHECK ('none','pending','assumed_paid','confirmed')website TEXTusername TEXTaccount_info TEXThas_2fa INTEGER NOT NULL DEFAULT 0active INTEGER NOT NULL DEFAULT 1notes TEXTcreated_at TEXT DEFAULT datetime('now')updated_at TEXT DEFAULT datetime('now')history_visibility TEXT NOT NULL DEFAULT 'default'is_seeded INTEGER NOT NULL DEFAULT 0cycle_type TEXT NOT NULL DEFAULT 'monthly'cycle_day TEXT
payments
id INTEGER PRIMARY KEYbill_id INTEGER NOT NULL REFERENCES bills(id) ON DELETE CASCADEamount REAL NOT NULLpaid_date TEXT NOT NULLmethod TEXTnotes TEXTcreated_at TEXT DEFAULT datetime('now')updated_at TEXT DEFAULT datetime('now')deleted_at TEXT
monthly_bill_state
id INTEGER PRIMARY KEYbill_id INTEGER NOT NULL REFERENCES bills(id) ON DELETE CASCADEyear INTEGER NOT NULL CHECK 2000-2100month INTEGER NOT NULL CHECK 1-12actual_amount REALnotes TEXTis_skipped INTEGER NOT NULL DEFAULT 0created_at TEXT DEFAULT datetime('now')updated_at TEXT DEFAULT datetime('now')- Unique:
(bill_id, year, month)
monthly_income
id INTEGER PRIMARY KEYuser_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADEyear INTEGER NOT NULLmonth INTEGER NOT NULLlabel TEXT NOT NULL DEFAULT 'Salary'amount REAL NOT NULL DEFAULT 0created_at TEXT DEFAULT datetime('now')updated_at TEXT DEFAULT datetime('now')- Unique intended/current logic:
(user_id, year, month)via migration/index.
monthly_starting_amounts
id INTEGER PRIMARY KEYuser_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADEyear INTEGER NOT NULLmonth INTEGER NOT NULLfirst_amount REAL NOT NULL DEFAULT 0fifteenth_amount REAL NOT NULL DEFAULT 0other_amount REAL NOT NULL DEFAULT 0notes TEXTcreated_at TEXT DEFAULT datetime('now')updated_at TEXT DEFAULT datetime('now')- Unique intended/current logic:
(user_id, year, month)via migration/index.
bill_history_ranges
id INTEGER PRIMARY KEYbill_id INTEGER NOT NULL REFERENCES bills(id) ON DELETE CASCADEstart_year INTEGER NOT NULLstart_month INTEGER NOT NULLend_year INTEGERend_month INTEGERlabel TEXTcreated_at TEXT DEFAULT datetime('now')updated_at TEXT DEFAULT datetime('now')
settings
key TEXT PRIMARY KEYvalue TEXT NOT NULLupdated_at TEXT DEFAULT datetime('now')
Used for app settings, auth mode, OIDC settings, SMTP settings, backup schedule, cleanup settings, and worker state.
notifications
id INTEGER PRIMARY KEYbill_id INTEGER NOT NULL REFERENCES bills(id) ON DELETE CASCADEuser_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADEyear INTEGER NOT NULLmonth INTEGER NOT NULLtype TEXT NOT NULL(due_3d,due_1d,due_today,overdue)sent_date TEXT NOT NULL DEFAULT date('now')- Unique:
(bill_id, user_id, year, month, type, sent_date)
import_sessions
id TEXT PRIMARY KEYuser_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADEcreated_at TEXT NOT NULLexpires_at TEXT NOT NULLpreview_json TEXT NOT NULL
import_history
id INTEGER PRIMARY KEYuser_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADEimported_at TEXT NOT NULLsource_filename TEXTfile_type TEXT DEFAULT 'xlsx'sheet_name TEXTrows_parsed INTEGER DEFAULT 0rows_created INTEGER DEFAULT 0rows_updated INTEGER DEFAULT 0rows_skipped INTEGER DEFAULT 0rows_ambiguous INTEGER DEFAULT 0rows_errored INTEGER DEFAULT 0options_json TEXTsummary_json TEXT
oidc_states
id TEXT PRIMARY KEYnonce TEXT NOT NULLcode_verifier TEXT NOT NULLredirect_to TEXTcreated_at TEXT NOT NULLexpires_at TEXT NOT NULL
audit_log
id INTEGER PRIMARY KEYuser_id INTEGERaction TEXT NOT NULLentity_type TEXTentity_id INTEGERdetails_json TEXTip_address TEXTuser_agent TEXTcreated_at TEXT DEFAULT datetime('now')
schema_migrations
id INTEGER PRIMARY KEYversion TEXT NOT NULL UNIQUEdescription TEXT NOT NULLapplied_at TEXT NOT NULL DEFAULT datetime('now')
Indexes
Important indexes include:
idx_bills_active(active)idx_bills_user_active(user_id, active)idx_bills_user_name(user_id, name)idx_categories_user_name(user_id, name)idx_categories_user_name_unique(user_id, name COLLATE NOCASE)idx_payments_bill_id(bill_id)idx_payments_paid_date(paid_date)idx_payments_bill_date_del(bill_id, paid_date, deleted_at)idx_payments_deleted(deleted_at)idx_payments_method(method)idx_sessions_user_id(user_id)idx_sessions_expires(expires_at)idx_monthly_bill_state_lookup(bill_id, year, month)idx_monthly_income_user_month(user_id, year, month)idx_monthly_starting_amounts_user(user_id)idx_monthly_starting_amounts_user_month(user_id, year, month)idx_notifications_lookup(bill_id, user_id, year, month)idx_import_sessions_user(user_id),idx_import_sessions_expires(expires_at)idx_import_history_user(user_id),idx_import_history_imported_at(imported_at)idx_oidc_states_expires(expires_at)idx_bill_history_ranges_bill(bill_id)idx_audit_log_user(user_id, created_at),idx_audit_log_action(action, created_at)
Migration system
db/database.js:
- Reads
db/schema.sqlininitSchema(). - Creates
schema_migrations. - Detects and reconciles legacy databases.
- Applies ordered migrations only when not already recorded.
- Validates dependency chains before applying dependent migrations.
- Uses a column whitelist for dynamic
ALTER TABLEstatements. - Wraps versioned migrations in transactions, with special handling for
v0.40because it uses PRAGMA-driven table rebuild work. - Supports rollback SQL for selected migrations through
ROLLBACK_SQL_MAPandrollbackMigration(version).
Current migration set:
v0.2payments soft delete.v0.3tracker payment compound index.v0.4monthly bill state.v0.13user profile columns.v0.14bill history visibility.v0.14.4bill interest rate.v0.15import sessions/history.v0.17OIDC/external identity columns and state table.v0.18.1monthly income.v0.18.2monthly starting amounts.v0.18.3other starting amount bucket.v0.38per-user import audit history.v0.40ownership for bills/categories.v0.41seeded demo-data flags.v0.42bill history ranges.v0.43sessioncreated_at.v0.44performance indexes.v0.45audit log.v0.46billcycle_typeandcycle_day.- Unversioned user notification columns are also reconciled.
Migration logging is both console-based and audit-backed:
runMigrations()logs start, dependency status, transaction begin/commit, per-migration elapsed time, skips for already-applied migrations, failures with rollback messages, and total elapsed time.- Audit events use the lazy
getLogAudit()helper to avoid theauditService -> database.js -> auditServicecircular dependency. - Audit actions include
migration.start,migration.complete, andmigration.failure. - Rollback paths audit
migration.rollbackandmigration.rollback.failure.
Rollback support is defined by ROLLBACK_SQL_MAP:
v0.44— drops selected performance indexes:idx_bills_user_name,idx_payments_method,idx_monthly_starting_amounts_user, andidx_import_history_imported_at.v0.45— dropsidx_audit_log_user,idx_audit_log_action, and theaudit_logtable.v0.46— dropsbills.cycle_dayandbills.cycle_type.
rollbackMigration(version) requires an initialized database, verifies the version exists in schema_migrations, looks up rollback SQL in ROLLBACK_SQL_MAP, executes all rollback statements inside a transaction, deletes the migration record, logs elapsed time, audits success, and returns {success:true, version, description, elapsed_ms}. If the migration is not recorded, it throws NOT_APPLIED. If no rollback definition exists, it throws ROLLBACK_NOT_SUPPORTED. Execution failures roll back the transaction and are audited as migration.rollback.failure.
The admin API exposes rollback through POST /api/admin/migrations/rollback. The route requires admin auth through the /api/admin mount. It maps NOT_APPLIED to HTTP 404, ROLLBACK_NOT_SUPPORTED to HTTP 422, and unexpected rollback failures to HTTP 500.
The lazy audit helper in db/database.js is:
let _logAudit = null;
function getLogAudit() {
if (!_logAudit) {
try { _logAudit = require('../services/auditService').logAudit; } catch { _logAudit = () => {}; }
}
return _logAudit;
}
Use this pattern for database-layer audit calls instead of a top-level require('../services/auditService').
7. Frontend Reference
Frontend stack
- React
^18.3.1 - Vite
^5.4.10 - React Router
^6.26.2 - TanStack Query
^5.100.9 - Tailwind CSS
^3.4.14 - Radix/shadcn-style UI primitives
sonnerfor toastsreact-markdown,remark-gfm,rehype-sanitizefor markdown rendering
client/main.jsx
Creates the React root and wraps App with global providers including auth/theme where defined.
client/App.jsx
- Creates a TanStack
QueryClientwithstaleTime2 minutes, retry 1, no refetch on focus. - Uses lazy loading and
SuspensewithPageLoaderfor most pages. - Wraps route elements in
ErrorBoundary. - Exposes
ReactQueryDevtools. - Provides skip link for keyboard users.
Routes:
/login→LoginPagepublic./about→AboutPagepublic./release-notes→ReleaseNotesPagepublic./admin→AdminPage, admin only./admin/about→ admin shell +AboutPage admin, admin only./admin/roadmap→ admin shell +AboutPage admin, admin only./admin/status→ admin shell +StatusPage, admin only./status→ redirects admin to/admin/status./→ authenticated user layout index,TrackerPage./calendar→CalendarPage./summary→SummaryPage./bills→BillsPage./categories→CategoriesPage./analytics→AnalyticsPage./settings→SettingsPage./data→DataPage./profile→ProfilePage.*under user layout → redirect/.
RequireAuth behavior:
- Shows loading while auth state is
undefined. - Redirects unauthenticated users to
/login. - Allows admins to access user routes except default admin is redirected to
/admin. - Allows single-user mode only for user routes.
- Redirects role mismatches to
/adminor/.
API client
client/api.js:
_fetch(method, path, body)calls/api${path}with JSON headers andcredentials: include.- Mutating methods read
bt_csrf_tokencookie and sendx-csrf-token. - Non-OK responses throw
Errorwithstatus,data,details, andcode. - Exposes grouped functions for auth, admin, notifications, profile, tracker, calendar, summary, bills, payments, categories, settings, analytics, status, version/about, import, and export.
- File download/upload endpoints use raw
fetchbecause responses/bodies are blobs or octet streams.
Auth state
client/hooks/useAuth.jsx:
- Maintains
user,singleUserMode, and loading state. - Calls
api.authMode()andapi.me()on startup. - Exposes
logout()andrefresh().
Query hooks
client/hooks/useQueries.js:
useTracker(year, month)→api.tracker(year, month).useBills()→api.allBills().useCategories()→api.categories().
These use TanStack Query keys and cache server data for common pages.
Pages
LoginPage.jsx— local login plus OIDC login availability based on/auth/mode.TrackerPage.jsx— monthly tracker, payment interactions, upcoming bills, starting amount awareness, bulk/session logout action.CalendarPage.jsx— calendar grid backed by/calendar.SummaryPage.jsx— monthly plan, income, starting amounts, expenses, chart data.BillsPage.jsx— bill CRUD, categories, monthly state/history range controls.CategoriesPage.jsx— category list/create/update/delete and related bill info.AnalyticsPage.jsx— analytics summary filters and charts.SettingsPage.jsx— user/app settings and demo data seed.DataPage.jsx— export, spreadsheet import, user DB import, import history.ProfilePage.jsx— display name, notification preferences, password change, export/import-history links.AdminPage.jsx— users, auth mode/OIDC, backups, cleanup, notifications, migrations, admin settings.StatusPage.jsx— admin system status.AboutPage.jsx— public or admin markdown/about view; admin mode uses/about-admin; markdown is sanitized.ReleaseNotesPage.jsx— release history display.
Components
- Layout:
Layout,Sidebar,BrandBlock,NavPill. - Domain UI:
AdminDashboard,BillModal,BillsTableInner,MobileBillRow,MobileTrackerRow,StatusBadge,SummaryCard,MarkdownText,ReleaseNotesDialog. - Reliability:
ErrorBoundary,PageLoader. - UI primitives: alert dialog, badge, button, card, checkbox, confirm dialog, dialog, dropdown menu, input dialog, input, label, select, separator, skeleton, switch, table, tabs, theme toggle, tooltip.
Frontend security notes
- CSRF header is sent on POST/PUT/PATCH/DELETE.
- Auth is cookie/session based; no tokens are stored in localStorage.
- Admin routes are client-guarded and server-guarded.
- Markdown rendering uses
rehype-sanitize. - Error boundaries prevent route crashes from taking down the whole SPA.
8. Infrastructure and Deployment
package.json
Version: 0.23.2.
Scripts:
npm run dev:api—node --watch server.js.npm run dev:ui— Vite dev server.npm run dev— concurrently runs API and UI.npm run build— Vite production build.npm start—node server.js.
Key runtime dependencies:
- Express, cookie-parser, cors, express-rate-limit.
- better-sqlite3.
- bcryptjs.
- openid-client.
- nodemailer.
- node-cron.
- React, React DOM, React Router, TanStack Query.
- Radix UI primitives, lucide-react, Tailwind utilities.
- xlsx for spreadsheet import/export.
Dockerfile
Multi-stage build:
- Builder:
node:18-alpine, installs build depspython3 make g++, runsnpm install, copies source, runsnpm run build. - Runtime:
node:18-alpine, installsbash nano su-exec, creates non-rootbilluser, copies built app, creates/data/db,/data/backups,/app/backups, sets ownership and restrictive permissions.
Runtime environment:
NODE_ENV=productionPORT=3000DB_PATH=/data/db/bills.dbBACKUP_PATH=/data/backups
Exposes port 3000, declares volume /data, entrypoint docker-entrypoint.sh, command node server.js.
docker-compose.yml
Service: bill-tracker.
- Image:
dream.scheller.ltd/null/billtracker:latest. - Container name:
bill-tracker. - Ports: host
3030to container3000. - Volume:
/portainer/hosting/bill-tracker/data:/data. - Restart:
unless-stopped. - Environment includes
INIT_ADMIN_USER,INIT_ADMIN_PASS, and CSRF cookie settings.
Important deployment note: the compose file currently sets CSRF_SECURE: "true"; for plain HTTP development this prevents CSRF cookies from being sent by browsers. Use HTTPS or override to false only in local/dev.
9. Auth and Security Flows
Local login
- Client calls
GET /auth/modeto determine local/OIDC visibility. - Client submits
POST /auth/login. - Server checks
local_login_enabled, validates credentials through bcrypt, rejects inactive users, cleans expired sessions, creates a 7-day session. - Server sets
bt_sessioncookie usingcookieOpts(req). - Client calls
GET /auth/meto populate auth state.
OIDC login
- Client navigates to
/api/auth/oidc/login. - Server verifies active OIDC config, creates PKCE state, redirects to provider.
- Provider returns to
/api/auth/oidc/callback. - Server validates state/nonce, exchanges code, maps/provisions user, creates local session cookie, redirects back into SPA.
Password change
- Current password and matching new password are required.
- New password must be at least 8 chars.
- Server updates hash, clears
must_change_password, setslast_password_change_at. - Other sessions are invalidated and current session is rotated when possible.
Authorization
- All user data routes enforce owner scope by
req.user.idin SQL. - Admin-only routes require
requireAdminon server. - Default admin cannot use tracker routes.
- Role changes invalidate target sessions.
- Deactivation invalidates target sessions.
Backup safety
- Managed filename regex and path checks prevent traversal.
- Uploads are written to temp paths first, validated, then moved.
- Restore creates a pre-restore backup.
Import safety
- Spreadsheet import accepts only XLSX and validates size, sheets, rows, cells, headers, and decisions.
- User DB import validates SQLite magic, size, required metadata/tables, and maps all data to current user ownership.
10. Operational Notes
Startup behavior
- DB path is
DB_PATHordb/bills.db. - DB open logging prints only
path.basename(DB_PATH)instead of the full database path, so startup logs identify the file without exposing the full filesystem location. - SQLite WAL and foreign keys are enabled.
- Schema and migrations run automatically.
- Default categories/settings are seeded.
- Expired sessions are purged at startup.
- A periodic expired-session cleanup interval is scheduled.
- Backup scheduler and daily worker are started where server code imports/starts them.
Environment-seeded regular users use INIT_REGULAR_USER and INIT_REGULAR_PASS. New seeded users are inserted with first_login = 0 and must_change_password = 0. Existing seeded regular users have their password hash updated and both flags reset to 0, then the server audits seed.flag_reset with the username, reset flags, and source: "server-seed". This lets ENV-managed users skip first-login/privacy/password-change gates after seed refreshes.
Environment variables
Common variables used by current code:
PORTDB_PATHBACKUP_PATHINIT_ADMIN_USERINIT_ADMIN_PASSINIT_REGULAR_USERINIT_REGULAR_PASSSESSION_CLEANUP_INTERVAL_MSCORS_ORIGINCOOKIE_SECUREHTTPSCSRF_HTTP_ONLYCSRF_SAME_SITECSRF_SECURECSRF_COOKIE_NAMEOIDC_ISSUER_URLOIDC_CLIENT_IDOIDC_CLIENT_SECRETOIDC_REDIRECT_URIOIDC_TOKEN_AUTH_METHOD
Most notification, OIDC, backup, cleanup, and auth-mode settings are also stored in the settings table and managed from Admin UI.
Known code characteristics to preserve
- Use transactions for multi-step destructive or bulk DB changes.
- Keep user-owned SQL scoped by
req.user.id. - Keep admin lockout protection before changing login methods.
- Do not expose
password_hash, session IDs, OIDC client secret, or internal backup paths in API responses. - Keep import preview/apply separated so users can resolve ambiguous spreadsheet data before DB writes.
- Prefer soft delete for payments; bill deletion is intentionally hard delete and returns an explicit warning.
- DB path support: db/database.js uses path.basename(DB_PATH) in logging to anonymize the DB path while still providing useful diagnostic information. Absolute and relative paths are both supported.
11. Verification Checklist Used for This Reference
Reviewed current code sources:
server.js- all route files under
routes/ - service files under
services/ - middleware files under
middleware/ db/schema.sqlanddb/database.js- actual initialized SQLite schema via
better-sqlite3introspection client/App.jsx,client/api.js,client/hooks/useAuth.jsx,client/hooks/useQueries.js- page/component inventory under
client/pages/andclient/components/ package.jsonDockerfiledocker-compose.yml
The previous manual contained large historical update sections and stale route/page descriptions. This version replaces those with a current-state engineering reference.