Initial import of zappier-edge from zapier monorepo

This commit is contained in:
George Lambert 2026-09-11 13:50:46 -04:00
commit f0d193b221
120 changed files with 19867 additions and 0 deletions

BIN
docs/.DS_Store vendored Normal file

Binary file not shown.

126
docs/ACCOUNTING.md Normal file
View file

@ -0,0 +1,126 @@
# Company Accounting Walkthrough
This guide covers the company-side accounting surface: invoicing, purchase-order
billing, reports, trends, and CSV export. Everything here runs in the **admin
console** at `/admin`.
> Audience: operations and accounting staff. For pricing and customer setup see
> `USER-MANAGEMENT.md`; for the end-user view see `CUSTOMER-PORTAL.md`.
---
## 1. Sign in
Open `http://<host>:<port>/admin` and sign in. The development credentials are
`demo` / `$$$Adm1n###` (override with `DEMO_ADMIN_USER` / `DEMO_ADMIN_PASSWORD`;
the primary account is `ADMIN_USER` / `ADMIN_KEY`).
![Admin login](screenshots/admin-login.png)
---
## 2. Billing identity per customer
Before invoicing, each customer needs a **billing type** and an **email** — set
on the **Customers** tab:
- **Stripe** — metered usage is reported to Stripe by the daily billing job.
- **Purchase order** — invoiced manually with a PO number; issued PO invoices
get a 30-day due date automatically.
![Customers with billing types](screenshots/admin-customers.png)
Customers can also carry a **prepaid balance** (funded from the customer
portal). When an invoice is issued and the balance fully covers the billable
amount, the balance is drawn down and the invoice goes straight to **paid**.
Partial coverage is left untouched — there are no partial payments.
---
## 3. Generate invoices
On the **Invoices** tab, pick a **period** (month), optionally narrow to one
customer, optionally set a **PO number**, and click **Generate**.
![Invoices tab](screenshots/admin-invoices.png)
Generation rules:
- One invoice per customer with usage in the period, grouped by endpoint.
- The tier **monthly credit** is applied; only the remainder is billable.
- Regenerating a period **replaces drafts** (e.g. after late-arriving usage)
and **skips issued/paid invoices** — the result panel lists who was skipped
and why.
- Invoice ids are `INV-<period>-<sequence>`, e.g. `INV-2026-07-0002`.
## 4. Issue, view, collect
Each row shows status, billing type, totals, and due amount:
- **Issue** (draft → issued) — finalizes the invoice; PO invoices get a due
date 30 days out. Balance-covered invoices skip straight to **paid**.
- **View** — opens the print-ready invoice in a new tab. Use the browser's
**Print → Save as PDF** to produce a PDF for the customer.
- **Mark paid** (issued → paid) — record collection for PO invoices.
![Print-ready invoice](screenshots/admin-invoice-html.png)
Filter the table by customer, period, or status with the filter bar.
---
## 5. Reports and trends
The **Reports** tab answers "who owes what, and how is usage trending?"
![Reports tab](screenshots/admin-reports.png)
- **Billing report** — per-customer calls, gross usage, credits applied, and
billable amount for a date range. Filter by customer or billing type (all
Stripe customers, all PO customers, or one customer). Summary cards total
the selection.
- **Download CSV** — the same rows as `billing-report.csv` with the active
filters, ready for the accounting package.
- **Usage trend** — daily or weekly buckets as a bar chart; hover a bar for
exact calls and amount.
The same data is available as JSON/CSV from the API:
`GET /admin/api/reports/billing?from=…&to=…&customerId=…&billingType=…&format=csv`
and `GET /admin/api/reports/usage-trend?bucket=day|week`.
---
## 6. System snapshot
The **System** tab shows integration health (Zapier app directory, version,
triggers, creates) and the current period at a glance: calls, billable amount,
open invoice count, and open amount.
![System tab](screenshots/admin-system.png)
---
## 7. Automated Stripe reporting
A daily job (`src/jobs/report-usage.ts`, scheduled separately) reports the
billable delta of every Stripe-billed customer to Stripe Billing meter events.
It is idempotent: a ledger records the cumulative reported cents per customer
per period, and only the delta since the last successful run is sent. PO
customers are excluded by having no `stripeCustomerId`.
Environment (`.env` at the project root):
```
STRIPE_SECRET_KEY=sk_live_or_test_...
ZAPPIER_DB=/absolute/path/to/zappier.db # optional
```
---
## Data notes
- Money is integer **cents** everywhere internally; the UI formats dollars.
- All accounting data lives in the SQLite database (`zappier.db` by default):
`invoices`, `invoice_lines`, `billing_ledger`, `customers`.
- The Stripe billing job and the admin console can run from any working
directory — all paths resolve from the installation root.

BIN
docs/ACCOUNTING.pdf Normal file

Binary file not shown.

98
docs/CUSTOMER-PORTAL.md Normal file
View file

@ -0,0 +1,98 @@
# Customer Portal Walkthrough
The self-service portal for end-user customers at `/portal`: account setup,
two-factor authentication, usage and balance, invoices, reloads, and API
documentation.
> Audience: your API customers. For the company side see `ACCOUNTING.md` and
> `USER-MANAGEMENT.md`.
---
## 1. Create your account
Open `http://<host>:<port>/portal` and choose **Create an account**.
![Sign up](screenshots/portal-signup.png)
- Name, email, and a password of at least 8 characters.
- You start on the **Free** plan and get an **API key immediately**.
- If the company already created an account for you (you received an API key
by email), sign up with the **same email address** — your existing account,
plan, and API key are kept and the password you choose is attached to it.
Sign-in afterwards is email + password on the same screen.
![Sign in](screenshots/portal-login.png)
---
## 2. Dashboard
![Dashboard](screenshots/portal-dashboard.png)
- **Usage this month** — gross usage since the 1st (UTC).
- **Included credit** — how much of your plan's monthly credit was consumed.
- **Billable** — what exceeds the credit (what you'd be invoiced today).
- **Prepaid balance** — funds available for automatic invoice payment.
- **Your API key** — copy it, or **regenerate** it. Regenerating invalidates
the old key immediately; use it as the `x-api-key` header.
## 3. Usage & pricing
The **API & pricing** tab shows the live rate card (free / fixed / variable
per endpoint, with size-based pricing for storage) and every plan's multiplier
and monthly credit — the same numbers the server bills from.
![API & pricing](screenshots/portal-docs.png)
The interactive API reference (Swagger UI) is linked at the top (`/docs`).
## 4. Invoices
![Invoices](screenshots/portal-invoices.png)
Your invoice history with status (`draft`, `issued`, `paid`), totals, credit,
amount due, and due date. **View / print** opens a print-ready invoice — use
the browser's **Print → Save as PDF** for a copy.
![Printable invoice](screenshots/portal-invoice-html.png)
Only your own invoices are visible; other customers' ids return "not found".
## 5. Billing: reloads & email invoicing
![Billing](screenshots/portal-billing.png)
- **Reload balance** — add $1$10,000. Your prepaid balance is **drawn down
automatically** when an invoice is issued: if it fully covers the amount
due, the invoice is paid instantly.
- **Email invoicing** — receive a copy of each new invoice by email.
## 6. Security: two-factor authentication
On the **Security** tab, click **Set up 2FA**:
![2FA setup](screenshots/portal-2fa-setup.png)
1. Scan the QR code with any authenticator app (or type the secret manually).
2. Enter the 6-digit code it shows to enable 2FA.
From then on, sign-in requires the password **and** the current code. You can
disable 2FA with a valid code from the same tab. Sessions expire after 7 days.
---
## Portal API reference
Session-based (`Authorization: Bearer <token>` from signup/login):
- `POST /portal/api/signup` · `POST /portal/api/login` · `POST /portal/api/logout`
- `GET /portal/api/me` · `POST /portal/api/api-key`
- `GET /portal/api/usage` · `GET /portal/api/pricing`
- `GET /portal/api/invoices` · `GET /portal/api/invoices/:id[?format=html]`
- `POST /portal/api/2fa/setup|enable|disable`
- `POST /portal/api/reload` · `PUT /portal/api/email-invoicing`
Security notes: passwords are stored as scrypt hashes; TOTP secrets are only
shown during setup; no hash or secret is ever returned by the API.

BIN
docs/CUSTOMER-PORTAL.pdf Normal file

Binary file not shown.

406
docs/DEVELOPER.md Normal file
View file

@ -0,0 +1,406 @@
# Zappier — Developer Documentation
**Version:** 0.2.0 · **Last updated:** 2026-07-28
Metered API platform: per-endpoint pricing, per-tier multipliers, usage
ledger, Stripe metered billing, purchase-order invoicing, a company admin
console, a customer portal with TOTP 2FA, and a Zapier integration. This
document is the full technical reference. For operations, see
[USER-MANUAL.md](USER-MANUAL.md).
---
## Table of contents
1. [Architecture](#1-architecture)
2. [Module reference](#2-module-reference)
3. [Pricing engine](#3-pricing-engine)
4. [Request lifecycle](#4-request-lifecycle)
5. [Persistence (SQLite)](#5-persistence-sqlite)
6. [Billing pipeline](#6-billing-pipeline)
7. [API reference](#7-api-reference)
8. [Zapier app](#8-zapier-app)
9. [Testing](#9-testing)
10. [Configuration](#10-configuration)
11. [Extending the system](#11-extending-the-system)
---
## 1. Architecture
**Stack:** Node 20 · TypeScript (strict) · Express 4 · better-sqlite3 ·
express-openapi-validator · swagger-ui-express · stripe SDK · Zapier Platform
(core ^19) · jest + supertest (root) and mocha (zapier-app).
```
┌────────────────────────────────────────────┐
│ Express app │
│ (src/app.ts) │
│ │
Browser ── /admin ──────┼── static admin UI (admin/) │
Browser ── /admin/api ──┼── adminAuth → adminRouter (src/admin.ts) │
│ pricing · customers · invoices · │
│ reports · zapier status │
Browser ── /portal ─────┼── static portal SPA (portal/) │
Browser ── /portal/api ─┼── portalRouter (src/portal.ts) │
│ signup/login+2FA · me · usage · │
│ invoices · reload · prefs │
Browser ── /docs ───────┼── swagger-ui (openapi.yaml) │
│ │
Client ──── /v1/* ──────┼── apiKeyAuth (src/auth.ts) │
│ └─ OpenAPI request validation │
│ └─ meter() (src/meter.ts) │
│ ├─ quoteCall() (src/pricing.ts) │
│ └─ UsageRepo.record() │
└──────────────┬─────────────────────────────┘
┌───────────────┬───────────────┼───────────────┬────────────┐
│ │ │ │ │
SqliteUsageRepo SqliteCustomerRepo SqlitePricingStore SqliteInvoiceRepo
(usage_entries) (customers) (price_endpoints, (invoices,
│ │ tiers) invoice_lines)
│ SqliteSessionRepo (portal_sessions) │
└───────────────┴───────────────┴───────────────┴────────────┘
│ zappier.db (SQLite)
Cron (daily 06:17 ET) │
"Zappier billing · report usage" │
│ │
└─► src/jobs/report-usage.ts ── reportMonthlyUsage()
├─ computeBillableCents / computeDelta (src/billing/stripe.ts)
├─ SqliteBillingReportRepo (billing_reports, job_locks)
└─ stripe.billing.meterEvents.create ──► Stripe
```
Design rules the codebase follows:
- **Ports & adapters:** `UsageRepo`, `CustomerRepo`, `PricingStore`,
`MeterEventClient` are interfaces with in-memory adapters (tests) and SQLite
/ Stripe adapters (production). Nothing outside `src/db/` and the job's
`main()` touches SQL or the Stripe SDK.
- **Live pricing reads:** the `PricingContext` getters in `buildApp` read the
store on every quote, so admin edits apply without a restart.
- **Money in integer cents everywhere.** No floats cross a boundary except
multipliers, which are applied once and rounded (`Math.round`).
## 2. Module reference
| File | Responsibility | Key exports |
|---|---|---|
| `src/index.ts` | Production entry: opens SQLite, wires repos, listens | — |
| `src/app.ts` | `buildApp(deps)` — full Express wiring; `StoredItem`; `DEFAULT_CUSTOMERS` | `buildApp`, `AppDeps` |
| `src/auth.ts` | Customer model + `x-api-key` middleware | `Customer`, `CustomerRepo`, `apiKeyAuth`, `InMemoryCustomerRepo` |
| `src/pricing.ts` | Pricing domain: rules, tiers, quote algorithm, store port | `PriceRule`, `TierConfig`, `Quote`, `quoteCall`, `PricingStore`, `DEFAULT_RATE_CARD`, `DEFAULT_TIERS` |
| `src/meter.ts` | Per-request metering middleware | `meter(endpointId, repo, pricing)` |
| `src/usage.ts` | Usage ledger domain + summaries | `UsageEntry`, `UsageSummary`, `summarize`, `UsageRepo`, `InMemoryUsageRepo` |
| `src/admin.ts` | Admin API (`/admin/api`) + admin-key guard | `adminAuth`, `adminRouter` |
| `src/billing/credit.ts` | Monthly credit application | `applyMonthlyCredit`, `BilledSummary` |
| `src/billing/stripe.ts` | Stripe-facing math + client port | `METER_EVENT_NAME`, `MeterEventClient`, `computeBillableCents`, `computeDelta`, `reportUsage` |
| `src/billing/reload.ts` | Portal reloads via Stripe PaymentIntents | `stripePaymentClient`, `hasRealStripeKey` |
| `src/accounts.ts` | Portal identity: scrypt passwords, RFC 6238 TOTP, sessions | `hashPassword`, `verifyPassword`, `totp`, `verifyTotp`, `generateTotpSecret`, `totpUri`, `SessionRepo`, `InMemorySessionRepo` |
| `src/invoicing.ts` | Invoice domain + generation | `Invoice`, `InvoiceLine`, `buildInvoice`, `InvoiceRepo`, `InMemoryInvoiceRepo` |
| `src/reports.ts` | Billing/usage aggregation + CSV | `billingRows`, `usageTrend`, `toCsv`, `BillingRow`, `TrendPoint` |
| `src/portal.ts` | Customer portal API (`/portal/api`) | `portalRouter`, `PortalDeps`, `PaymentClient`, `PaymentResult` |
| `src/paths.ts` | Installation-root resolution (cwd-independent) | `PROJECT_ROOT` |
| `src/jobs/report-usage.ts` | Billing job: delta reporting to Stripe | `reportMonthlyUsage`, `firstOfMonthUtc`, `ReportUsageDeps` |
| `src/db/usage-repo.ts` | SQLite adapter: `usage_entries` | `SqliteUsageRepo` |
| `src/db/customer-repo.ts` | SQLite adapter: `customers` (seeds when empty; idempotent column migrations) | `SqliteCustomerRepo` |
| `src/db/pricing-store.ts` | SQLite adapter: `price_endpoints`, `tiers` (seeds when empty) | `SqlitePricingStore` |
| `src/db/billing-repo.ts` | SQLite adapter: `billing_reports`, `job_locks` | `SqliteBillingReportRepo`, `BillingReportRepo`, `JobLockRepo` |
| `src/db/invoice-repo.ts` | SQLite adapter: `invoices`, `invoice_lines` | `SqliteInvoiceRepo` |
| `src/db/session-repo.ts` | SQLite adapter: `portal_sessions` | `SqliteSessionRepo` |
| `openapi.yaml` | Public API contract; drives validation and `/docs` | — |
| `admin/` | Dependency-free admin SPA (`index.html`, `app.js`) | — |
| `portal/` | Dependency-free customer portal SPA (`index.html`, `app.js`) | — |
## 3. Pricing engine
### Types (`src/pricing.ts`)
```ts
type PriceRule =
| { kind: 'free' }
| { kind: 'fixed'; fixedCents: number }
| { kind: 'variable'; baseCents: number; perKbCents: number; perMbCents: number };
interface TierConfig {
id: string; name: string;
multiplier: number; // e.g. 0.5 = 50% of list
monthlyCreditCents: number; // free included usage per month
defaultRule?: PriceRule; // fallback for endpoints with no rule
}
interface Quote {
endpointId: string;
listCents: number; // before multiplier
totalCents: number; // after multiplier — this is what is recorded
breakdown: { baseCents: number; metadataCents: number; attachmentCents: number };
}
```
### `quoteCall(pricing, tierId, endpointId, usage, multiplierOverride?)`
1. Resolve tier — throws `Unknown tier: <id>` (mapped to 403 by `meter`).
2. Resolve rule: rate-card rule for `endpointId`, else the tier's
`defaultRule`, else throw `No price rule for <tier>/<endpoint>` (403).
3. Compute breakdown:
- `free` → all zeros.
- `fixed``baseCents = fixedCents`.
- `variable``baseCents` + `perKbCents × ceil(metadataBytes/1024)` +
`perMbCents × ceil(attachmentBytes/1048576)`. Note the ceiling: 1 byte of
metadata bills a full KB unit; attachments bill per started MB.
4. `listCents` = sum of breakdown; `multiplier = multiplierOverride ?? tier.multiplier`;
`totalCents = Math.round(listCents × multiplier)`.
`totalCents` (never `listCents`) is what the usage ledger records and what the
billing pipeline sums.
## 4. Request lifecycle
For `POST /v1/storage`:
1. `express.json()` parses JSON bodies (multipart handled by the validator's
multer — 25 MB per-file cap).
2. `apiKeyAuth` (`src/auth.ts`) — `x-api-key``Customer` on `req.customer`,
else 401.
3. `express-openapi-validator` checks the request against `openapi.yaml`
(400 on violation). For `/v1/storage`, `parseMetadata` then JSON-parses the
`metadata` form field into `res.locals.parsedMetadata` (400 on bad JSON).
4. `meter('storage', usage, pricing)` (`src/meter.ts`):
- measures `metadataBytes` (UTF-8 length of the JSON-stringified metadata)
and `attachmentBytes` (sum of multer file sizes),
- calls `quoteCall` — pricing errors become 403,
- records a `UsageEntry` with `cents = quote.totalCents`,
- stashes the quote in `res.locals.quote`.
5. The route handler builds the `StoredItem` (in-memory list) and responds
`{ id, quote }`.
`GET /v1/usage` is **not** metered; it summarizes the caller's month-to-date
usage and applies the tier credit via `applyMonthlyCredit`.
## 5. Persistence (SQLite)
Single database file (`ZAPPIER_DB`, default `zappier.db`), WAL-agnostic
better-sqlite3, all tables created with `CREATE TABLE IF NOT EXISTS` in the
repo constructors. **Seeding rule:** `customers` and pricing tables seed from
`DEFAULT_CUSTOMERS` / `DEFAULT_RATE_CARD` / `DEFAULT_TIERS` only when empty.
| Table | Columns | Written by |
|---|---|---|
| `usage_entries` | `id`, `customer_id`, `endpoint_id`, `cents`, `metadata_bytes`, `attachment_bytes`, `timestamp_ms` | `meter()` on every priced call |
| `customers` | `id` PK, `name`, `tier_id`, `api_key` UNIQUE, `stripe_customer_id`, `multiplier_override`, `billing_type`, `email`, `password_hash`, `totp_secret`, `totp_enabled`, `balance_cents`, `email_invoicing` | Admin API, portal API |
| `price_endpoints` | `endpoint_id` PK, `rule_json` | Admin API |
| `tiers` | `id` PK, `name`, `multiplier`, `monthly_credit_cents`, `default_rule_json` | Admin API |
| `invoices` | `id` PK, `customer_id`, `period`, `status`, cents totals, `billing_type`, `po_number`, lifecycle timestamps | Admin API (generate/issue/paid + balance drawdown) |
| `invoice_lines` | `invoice_id`, `endpoint_id`, `calls`, `cents` | Invoice generation |
| `portal_sessions` | `token` PK, `customer_id`, `created_ms`, `expires_ms` | Portal auth |
| `billing_reports` | `customer_id` + `period` PK, `reported_cents` (cumulative), `reported_at_ms` | Billing job, after each successful meter event |
| `job_locks` | `name` PK, `acquired_at_ms` | Billing job run guard |
New customer columns are added by **idempotent migrations** (`PRAGMA
table_info` guard + `ALTER TABLE ADD COLUMN`) when the repo opens an older
database — no manual migration step.
In tests, every repo is constructed over `:memory:` databases.
## 6. Billing pipeline
### 6.1 Math (`src/billing/stripe.ts`)
```ts
computeBillableCents(entries, monthlyCreditCents)
= max(0, Σ entry.cents monthlyCreditCents)
computeDelta(billable, previouslyReported)
= max(0, billable previouslyReported)
```
`reportUsage(client, stripeCustomerId, entries, monthlyCreditCents)` is the
original whole-month reporter — **retained as public API**; the job uses the
delta path instead.
### 6.2 The job (`src/jobs/report-usage.ts`)
`reportMonthlyUsage(deps)` per run:
1. **Lock:** `locks.tryAcquireLock('report-usage', 1h TTL)` — a single atomic
`INSERT … ON CONFLICT … DO UPDATE … WHERE acquired_at_ms <= now ttl`.
Failure aborts the run; release happens in `finally`. A crashed run's lock
is taken over after the TTL.
2. **Window:** `since = firstOfMonthUtc(now)` (injectable via `deps.since` for
tests); `period = since.toISOString().slice(0, 7)` (`YYYY-MM`).
3. Per customer with a `stripeCustomerId` (others skipped silently; unknown
`tierId` warns and skips):
- `entries = usage.listFor(customer.id, since)`
- `billable = computeBillableCents(entries, tier.monthlyCreditCents)`
- `prior = billingRepo.getReportedCents(customer.id, period)` (0 for a new
month — periods are isolated by the composite PK)
- `delta = computeDelta(billable, prior)`; `delta <= 0` → log skip, continue
- `createMeterEvent({ eventName: 'zappier.api_cents',
customerId: stripeCustomerId, value: String(delta),
identifier: `${stripeCustomerId}:${period}:${billable}` })`
- **only on success:** `upsertReportedCents(customer.id, period, billable)`
— cumulative, not the delta.
**Idempotency guarantees (reviewed design):**
- *Re-run safety:* second run with same usage → delta 0 → no Stripe call.
- *Mid-month growth:* only the increase is sent; the identifier embeds the new
cumulative billable, so legitimate growth is never deduped away.
- *Crash between Stripe success and ledger write:* retry sends a byte-identical
event; Stripe drops it via the `identifier` (uniqueness enforced within a
rolling 24 h window).
- *Partial failure:* customer A's ledger write commits before customer B is
attempted; B's failure leaves A correctly recorded.
CLI: `npx ts-node src/jobs/report-usage.ts` (guarded by
`require.main === module`; loads `.env` via dotenv inside `main()`).
Scheduled by the Kimi cron job "Zappier billing · report usage to Stripe"
(`17 6 * * *`, America/New_York), which runs this command daily and reports
the outcome.
## 7. API reference
### Public API (`/v1`, auth: `x-api-key`)
Defined in `openapi.yaml`; interactive docs at `/docs`.
| Operation | Method & path | Price | Notes |
|---|---|---|---|
| `status` | GET `/v1/status` | free | Health + quote echo |
| `transform` | POST `/v1/transform` | fixed | `{text}``{output: TEXT, quote}` |
| `storage` | POST `/v1/storage` | variable | multipart: `metadata` (JSON string), `attachments[]` (≤25 MB/file) → `{id, quote}` |
| `storage-list` | GET `/v1/storage` | free | Caller's stored items |
| `usage` | GET `/v1/usage` | unmetered | Month-to-date summary with credit applied |
Error envelope: `{ "error": string }` with 400 (validation/metadata),
401 (bad key), 403 (unknown tier / no price rule), 413 (file over 25 MB).
### Admin API (`/admin/api`, auth: `x-admin-key` or login session)
| Route | Purpose |
|---|---|
| POST `/login` | `{username, password}` → session token (accounts: `ADMIN_USER`/`ADMIN_KEY`, `DEMO_ADMIN_USER`/`DEMO_ADMIN_PASSWORD`) |
| GET `/pricing` | `{ rateCard, tiers }` |
| PUT `/endpoints/:id` | Upsert a `PriceRule` (validated: 400 on bad shape) |
| DELETE `/endpoints/:id` | Remove a rule (endpoint becomes 403 for tiers without a default rule) |
| PUT `/tiers/:id` | Upsert a `TierConfig` |
| DELETE `/tiers/:id` | Remove a tier |
| GET `/customers` | List customers **without** API keys |
| POST `/customers` | Create `{name, tierId}` → full customer incl. generated `apiKey` (201, shown once) |
| PUT `/customers/:id` | Patch `name` / `tierId` / `multiplierOverride` / `stripeCustomerId` / `billingType` / `email` |
| POST `/invoices/generate` | `{period, customerId?, poNumber?}` — drafts per customer with usage; regenerating replaces drafts, skips issued/paid → `{generated, skipped}` |
| GET `/invoices` | Filters: `customerId`, `period`, `status` |
| GET `/invoices/:id` | JSON, or print-ready HTML with `?format=html` |
| POST `/invoices/:id/issue` | draft → issued (PO gets 30-day due date). **Prepaid drawdown:** if the customer's `balanceCents` fully covers `billableCents`, the balance is deducted and the invoice is saved as paid instead |
| POST `/invoices/:id/paid` | issued → paid |
| GET `/reports/billing` | `from`/`to`/`customerId`/`billingType` filters; JSON rows or `format=csv` |
| GET `/reports/usage-trend` | `bucket=day\|week`, same range filters |
| GET `/zapier/status` | Zapier app dir presence, version, triggers, creates |
### Portal API (`/portal/api`, auth: Bearer session)
| Route | Purpose |
|---|---|
| POST `/signup` | `{name, email, password≥8}` → 201 `{token, customer}`. New customer on `free` with instant API key; an email match on a passwordless (admin-created) customer **claims** that account; 409 when the email already has a password |
| POST `/login` | `{email, password, totpCode?}``{token, customer}`; 401 `totp_required` when 2FA is on and the code is missing/wrong |
| POST `/logout` | Deletes the session |
| GET `/me` | Public profile — never includes `passwordHash`/`totpSecret` |
| POST `/api-key` | Regenerates the API key (old key dies immediately) |
| GET `/usage` | Month-to-date summary with tier credit applied |
| GET `/pricing` | Live rate card + tiers for the pricing page |
| GET `/invoices` · GET `/invoices/:id` | Own invoices only (others 404); `?format=html` print view |
| POST `/2fa/setup` | Generates + stores a TOTP secret (not yet enabled) → `{secret, uri, qr}` (QR as PNG data URL via `qrcode`) |
| POST `/2fa/enable` · POST `/2fa/disable` | `{code}` verified against the stored secret |
| POST `/reload` | `{amountCents}` integer $1$10,000 via the injected `PaymentClient` — dev client credits instantly; Stripe client returns a `clientSecret` and credits on confirmation |
| PUT `/email-invoicing` | `{enabled}` preference |
Sessions live in `portal_sessions` (7-day TTL) and survive restarts.
`src/accounts.ts` implements scrypt hashing (`scrypt:N:r:p:salt:hash`,
timing-safe compare) and RFC 6238 TOTP (HMAC-SHA1, 30 s step, 6 digits,
±1 step window) with no external crypto dependency.
## 8. Zapier app
`zapier-app/` — Zapier Platform (core ^19), CommonJS, mocha tests.
| File | Purpose |
|---|---|
| `index.js` | App definition; wires auth, trigger, action |
| `authentication.js` | API-key auth; test call against `/v1/status` |
| `triggers/new_item.js` | Polling trigger: `GET /v1/storage`, newest first, dedupe by `id` |
| `creates/store_data.js` | Action: multipart `POST /v1/storage` (form-data), fields: metadata JSON + optional files |
| `test/` | mocha suite (4 tests): auth, trigger, action |
Publish flow: `zapier login``zapier push` → invite users / submit for
review. The app's base URL must point at a publicly reachable deployment of
the API server.
## 9. Testing
```bash
npm test # jest, repo root — 165 tests / 22 suites
npx tsc --noEmit # type gate
cd zapier-app && npm test # mocha — 4 tests
```
Conventions:
- **TDD** throughout; every module has in-memory adapters so tests never touch
disk or network.
- HTTP tests use **supertest** against `buildApp()` with in-memory repos.
- SQLite tests use `:memory:` databases.
- Stripe is faked by implementing `MeterEventClient`; the idempotency suite
(`tests/report-usage-idempotency.test.ts`) simulates growth, re-runs, month
rollover, Stripe throws, lock contention, and partial failure.
- The job is tested via the injectable `reportMonthlyUsage(deps)` — never by
executing `main()`.
## 10. Configuration
| Env var | Default | Used by |
|---|---|---|
| `PORT` | `3000` | `src/index.ts` |
| `ZAPPIER_DB` | `<root>/zappier.db` | `src/index.ts`, billing job |
| `ADMIN_KEY` | `admin-dev-key` | `adminAuth()` |
| `ADMIN_USER` | `admin` | Admin login |
| `DEMO_ADMIN_USER` / `DEMO_ADMIN_PASSWORD` | `demo` / `$$$Adm1n###` | Demo admin login |
| `STRIPE_SECRET_KEY` | — | Billing job; portal reloads when it starts with `sk_` (otherwise a dev payment client credits instantly) |
Both `src/index.ts` and the billing job load `.env` from the installation
root (`src/paths.ts` `PROJECT_ROOT`) — never from the process cwd — so the
compiled server and the job run from any working directory.
`.env` is gitignored (`chmod 600`); `.env.example` documents the shape.
Git identity is configured repo-local; `.gitignore` covers `node_modules/`,
`dist/`, `.env`, `zappier.db*`.
## 11. Extending the system
**Add an API endpoint:**
1. Add the path + `operationId` to `openapi.yaml` (validation & docs follow
automatically).
2. Add the route in `src/app.ts`, wrapping the handler with
`meter('<operationId>', usage, pricing)`.
3. Add a rate-card rule (admin UI or `PUT /admin/api/endpoints/<operationId>`)
— otherwise tiers without a `defaultRule` get 403.
4. Write the failing test first; keep `npm test` + `tsc` green.
**Add a customer type:** admin UI or `PUT /admin/api/tiers/:id`
(`{name, multiplier, monthlyCreditCents, defaultRule?}`).
**Swap the storage backend:** implement `UsageRepo` / `CustomerRepo` /
`PricingStore` against your database and pass them to `buildApp({...})` — no
other code changes. Same for `BillingReportRepo`/`JobLockRepo` in the job.
**Change the billing cadence:** the job is safe at any frequency (delta +
ledger + lock). The Kimi cron job controls scheduling; update its cron
expression to change cadence.
**Known intentional limitations:** stored items are in-memory (restart clears
them; usage ledger is unaffected); `reportUsage` is retained but superseded by
the delta path; Stripe identifier dedup covers a rolling 24 h window;
`releaseLock` is not owner-scoped (harmless at this job's runtime); admin
session tokens are in-memory (portal sessions are persisted); Stripe reloads
credit the balance only after payment confirmation (no webhook endpoint yet —
dev client credits instantly); email invoicing stores the preference but
sending requires SMTP wiring (deferred); PO invoices with partial prepaid
coverage are not partially paid by design.

BIN
docs/DEVELOPER.pdf Normal file

Binary file not shown.

111
docs/USER-MANAGEMENT.md Normal file
View file

@ -0,0 +1,111 @@
# User Management Walkthrough
How the company manages pricing, customer types, and customer accounts in the
**admin console** at `/admin`.
> Audience: operations staff. For invoicing/reports see `ACCOUNTING.md`; for
> the end-user view see `CUSTOMER-PORTAL.md`.
---
## 1. Admin accounts
Admin sign-ins live in the `admin_users` table — passwords are stored as
**scrypt hashes**, never in plain text. On an empty database the table is
seeded from the environment:
| Account | Username | Password (dev default) | Env override |
|---|---|---|---|
| Primary admin | `admin` | `admin-dev-key` | `ADMIN_USER` / `ADMIN_KEY` |
| Demo / stakeholder | `demo` | `$$$Adm1n###` | `DEMO_ADMIN_USER` / `DEMO_ADMIN_PASSWORD` |
The seed runs **only when the table is empty** — after that, accounts are
managed in the console and survive restarts (SQLite) and env changes.
Sign-in issues a session token; API access is also possible with the
`x-admin-key: $ADMIN_KEY` header (used by automation).
## 2. Users tab (admin account management)
The **Users** tab lists every admin account and manages their lifecycle:
![Admin users](screenshots/admin-users.png)
- **Create** — username (letters, digits, `.` `_` `-`) plus a password of at
least 8 characters. The new account can sign in immediately.
- **Deactivate / Activate** — a deactivated account is blocked from signing in
right away (401), and reactivation restores access. Deactivation persists in
the database.
- **Safety guard** — the console refuses to deactivate the **last active
admin**, so you can never lock everyone out.
Existing sessions stay valid until sign-out or server restart; deactivation
blocks *new* logins.
## 3. Rate card (per-endpoint pricing)
The **Rate card** tab sets list prices per API operation. Changes apply to the
**next API call** — no restart.
![Rate card](screenshots/admin-rate-card.png)
Three price kinds:
- **free** — never charged (e.g. `status`, `storage-list`).
- **fixed** — a flat `fixedCents` per call (e.g. `transform` at 4¢).
- **variable**`baseCents` per call plus size-based charges:
`perKbCents` per KB of metadata and `perMbCents` per MB of file attachments
(e.g. `storage`: 10¢ + 1¢/KB + 50¢/MB). This is how storing data with
metadata or attachments is priced by size.
Add an endpoint with its `operationId` from `openapi.yaml`. Deleting an
endpoint makes calls to it fail with 403 unless the customer's type has a
default rule.
## 4. Customer types (tiers)
The **Customer types** tab defines plans:
![Customer types](screenshots/admin-tiers.png)
- **Multiplier** — scales every list price (0.5 = 50% of list, 0.25 = 75% off).
- **Monthly credit (cents)** — free included usage per month, consumed before
anything is billable.
Defaults: `free` (1×, $1 credit), `pro` (0.5×, $10 credit, 8¢ default rule),
`business` (0.25×, $100 credit, 8¢ default rule).
## 5. Customers
The **Customers** tab manages individual accounts:
![Customers](screenshots/admin-customers.png)
- **Type** — assign any tier.
- **Multiplier override** — a per-customer deal that replaces the tier
multiplier (e.g. a strategic account at 0.2×).
- **Email** — used for portal login/claiming and email invoicing.
- **Billing**`Stripe` (metered via the billing job) or `Purchase order`
(manual invoicing with PO numbers and 30-day terms).
- **Create** — generates a customer id and API key. **The API key is shown
once** in the notification — copy it immediately.
Customers created here can **claim** their portal account: the first signup at
`/portal` with a matching email sets their password on the existing account
instead of creating a new one.
## 6. Admin API reference
Everything the UI does is available over HTTP (`x-admin-key` or session
Bearer):
- `POST /admin/api/login`
- `GET /admin/api/users` · `POST /admin/api/users`
· `POST /admin/api/users/:username/activate|deactivate`
- `GET /admin/api/pricing` · `PUT/DELETE /admin/api/endpoints/:id`
- `PUT/DELETE /admin/api/tiers/:id`
- `GET/POST /admin/api/customers` · `PUT /admin/api/customers/:id`
- `POST /admin/api/invoices/generate` · `GET /admin/api/invoices`
· `POST /admin/api/invoices/:id/issue|paid`
- `GET /admin/api/reports/billing` · `GET /admin/api/reports/usage-trend`
- `GET /admin/api/zapier/status`

BIN
docs/USER-MANAGEMENT.pdf Normal file

Binary file not shown.

348
docs/USER-MANUAL.md Normal file
View file

@ -0,0 +1,348 @@
# Zappier — Operations & Usage Manual
**Version:** 0.2.0 · **Last updated:** 2026-07-28
Zappier is a metered API platform: every API call your customers make is priced
per endpoint, adjusted by their customer type, tracked in a usage ledger, and
billed through Stripe once a day — or invoiced manually by purchase order.
Customers self-serve through the portal at `/portal`. This manual covers
running and operating the system. For internals, see
[DEVELOPER.md](DEVELOPER.md); for accounting procedures see
[ACCOUNTING.md](ACCOUNTING.md); for the end-user view see
[CUSTOMER-PORTAL.md](CUSTOMER-PORTAL.md).
---
## Table of contents
1. [Quick start](#1-quick-start)
2. [The three surfaces](#2-the-three-surfaces)
3. [How pricing works](#3-how-pricing-works)
4. [Operating the Pricing Admin UI](#4-operating-the-pricing-admin-ui)
5. [Using the public API](#5-using-the-public-api)
6. [Billing operations (Stripe)](#6-billing-operations-stripe)
7. [The Zapier integration](#7-the-zapier-integration)
8. [Day-to-day runbook](#8-day-to-day-runbook)
9. [Troubleshooting](#9-troubleshooting)
---
## 1. Quick start
```bash
cd /Users/marchon/zappier
npm install
npm run dev # starts the API on http://localhost:3000
```
Environment variables (all optional except `STRIPE_SECRET_KEY` for billing):
| Variable | Default | Purpose |
|---|---|---|
| `PORT` | `3000` | HTTP port for the API server |
| `ZAPPIER_DB` | `<install root>/zappier.db` | SQLite database file location |
| `ADMIN_KEY` | `admin-dev-key` | Key for the admin console and admin API |
| `ADMIN_USER` | `admin` | Admin console primary username |
| `DEMO_ADMIN_USER` / `DEMO_ADMIN_PASSWORD` | `demo` / `$$$Adm1n###` | Demo sign-in — override in production |
| `STRIPE_SECRET_KEY` | — (required for billing) | Billing job + portal reloads, loaded from `.env` |
All runtime paths (database default, `.env`, OpenAPI spec, static assets)
resolve from the installation root — the compiled server (`node
dist/index.js`) runs from any working directory, under systemd, Docker, or cron.
The `.env` file at the repo root holds `STRIPE_SECRET_KEY`. It is gitignored
and owner-only (`chmod 600`). A template is in `.env.example`.
On first start the database is created and seeded with:
- **Rate card:** `status` (free), `storage-list` (free), `transform` (fixed 4¢),
`storage` (variable: 10¢ base + 1¢/KB metadata + 50¢/MB attachments)
- **Customer types:** Free (×1.0, 100¢/month credit), Pro (×0.5, 1000¢ credit),
Business (×0.25, 10000¢ credit)
- **Demo customers:** `key-ada` (Free), `key-grace` (Pro), `key-linus` (Business)
> Seeding only happens into an **empty** database. Existing data is never
> overwritten on restart.
---
## 2. The three surfaces
| Surface | URL / location | Who it's for |
|---|---|---|
| **Public API** | `http://localhost:3000/v1/*` | Your API customers |
| **Interactive API docs** | `http://localhost:3000/docs` | Developers integrating with you |
| **Admin console** | `http://localhost:3000/admin` | You (operations & accounting) |
| **Customer portal** | `http://localhost:3000/portal` | End-user customers (self-service) |
| **Zapier app** | `zapier-app/` directory | No-code users via Zapier |
![Interactive API docs](screenshots/api-docs.png)
---
## 3. How pricing works
Every priced call returns its **quote** in the response, so customers always
know what a call cost:
```json
{
"quote": {
"endpointId": "storage",
"listCents": 62,
"totalCents": 31,
"breakdown": { "baseCents": 10, "metadataCents": 2, "attachmentCents": 50 }
}
}
```
The price of a call is computed in three steps:
1. **Endpoint rule** (from the rate card):
- `free` — always 0¢
- `fixed` — a flat `fixedCents` per call
- `variable``baseCents` + `perKbCents` × ceil(metadata bytes / 1024)
+ `perMbCents` × ceil(attachment bytes / 1 MB)
2. **Customer-type multiplier**`totalCents = round(listCents × multiplier)`.
A per-customer **multiplier override** (set in the Customers tab) wins over
the type multiplier — use it for negotiated enterprise deals.
3. **Monthly credit** — at billing time, each customer's type credit
(e.g. Pro = 1000¢) is subtracted from their month-to-date total. Only the
excess is billed.
**Worked example.** A Pro customer (×0.5) uploads a 1 MB file with 2 KB of
metadata to `storage`:
- list = 10¢ base + 2¢ metadata + 50¢ attachment = **62¢**
- Pro multiplier: 62 × 0.5 = **31¢** charged to their usage ledger
- If their month-to-date is 1500¢ and the Pro credit is 1000¢, the daily
billing job reports **500¢** to Stripe.
---
## 4. Operating the Pricing Admin UI
Open `http://localhost:3000/admin` and sign in. Two accounts are available:
| Username | Password | Purpose |
|---|---|---|
| `admin` | the `ADMIN_KEY` env value (default `admin-dev-key`) | Primary operator |
| `demo` | `$$$Adm1n###` (env `DEMO_ADMIN_PASSWORD`) | Demo / stakeholder access |
Sessions are token-based and remembered in browser local storage until you
click **Sign out** or the server restarts (tokens are in-memory — just sign in
again). The legacy `x-admin-key` header still works for scripts and curl.
### 4.1 Rate card tab
![Rate card](screenshots/admin-rate-card.png)
One row per API endpoint (matched by OpenAPI `operationId`).
- **Change a price:** edit the kind (`free` / `fixed` / `variable`) and the
cent fields, then click **Save**. Takes effect on the next API call — no
restart needed.
- **Add an endpoint:** enter the `operationId` (must match `openapi.yaml`),
pick a kind, click **Add**.
- **Delete** removes the rule. If an endpoint has no rule and the customer's
type has no default rule, calls to it are rejected with 403 — deletion is
how you turn an endpoint **off**.
### 4.2 Customer types tab
![Customer types](screenshots/admin-tiers.png)
Types are your pricing tiers.
- **Multiplier** scales every price for that type (0.5 = 50% of list).
- **Monthly credit (cents)** is the free included usage per month.
- **Add customer type:** id, name, multiplier. New types start with 0 credit;
edit after adding.
### 4.3 Customers tab
![Customers](screenshots/admin-customers.png)
- **Create** a customer: name + type. **The API key is shown once** in the
status line at the bottom — copy it immediately and send it to the customer.
- **Change type** with the dropdown, then **Save**.
- **Email** — used for portal sign-in/claiming and email invoicing.
- **Billing**`Stripe` (metered by the daily job) or `Purchase order`
(manual invoicing with PO numbers; see [ACCOUNTING.md](ACCOUNTING.md)).
- **Multiplier override**: a number here replaces the type multiplier for this
customer only. Leave blank to inherit from the type.
- Stripe customer IDs are attached via the admin API
(`PUT /admin/api/customers/:id` with `{"stripeCustomerId": "cus_..."}`) —
see section 6.
### 4.4 Invoices, Reports, System tabs
The **Invoices** tab generates monthly invoices from metered usage (per period,
optionally per customer, with optional PO number), walks them
draft → issued → paid, and opens print-ready invoice pages. The **Reports** tab
produces date-ranged billing reports (all customers, one customer, or one
billing type) with CSV download, plus daily/weekly usage-trend charts. The
**System** tab shows Zapier integration health and the current-period billing
snapshot. Full procedures: [ACCOUNTING.md](ACCOUNTING.md).
![Invoices](screenshots/admin-invoices.png)
---
## 4A. Customer portal
Your customers self-serve at `http://localhost:3000/portal`: signup (or
claiming an account you created, by email), sign-in with optional TOTP
two-factor authentication, month-to-date usage, invoice history with print
view, prepaid balance reloads (drawn down automatically at invoice issue),
email-invoicing preferences, API-key regeneration, and live pricing. The full
end-user guide is [CUSTOMER-PORTAL.md](CUSTOMER-PORTAL.md).
---
## 5. Using the public API
All calls need the customer's API key in the `x-api-key` header. Interactive
docs with a "Try it out" console are at `/docs`.
```bash
# Free status check
curl -H 'x-api-key: key-ada' http://localhost:3000/v1/status
# Fixed-price call (4¢ list)
curl -X POST -H 'x-api-key: key-grace' -H 'content-type: application/json' \
-d '{"text":"hello"}' http://localhost:3000/v1/transform
# Variable-price call: metadata + attachments
curl -X POST -H 'x-api-key: key-grace' \
-F 'metadata={"title":"Q3 report"}' \
-F 'attachments=@report.pdf' \
http://localhost:3000/v1/storage
# List your stored items (free)
curl -H 'x-api-key: key-grace' http://localhost:3000/v1/storage
# Your month-to-date usage, with credit applied
curl -H 'x-api-key: key-grace' http://localhost:3000/v1/usage
```
Limits & validation: requests are validated against `openapi.yaml` (bad
requests get 400); attachments are capped at **25 MB per file**; `metadata`
must be valid JSON (400 otherwise).
---
## 6. Billing operations (Stripe)
### 6.1 How it works
A scheduled job runs the billing reporter **daily at 06:17 America/New_York**
(Kimi cron job "Zappier billing · report usage to Stripe"). For each customer
with a Stripe ID it:
1. Sums their usage since the 1st of the month, subtracts their type's monthly
credit → **billable cents**.
2. Reports only the **delta** above what was already reported this month to
Stripe as a meter event (`zappier.api_cents`, value = cents).
3. Records the new cumulative total in the `billing_reports` ledger.
Re-running is always safe: the ledger makes repeats no-ops, a per-run lock
prevents overlapping executions, and a deterministic Stripe `identifier`
(`customer:period:billable`) dedupes crash retries.
### 6.2 One-time Stripe setup (test mode)
1. Dashboard (test mode ON) → **Billing → Meters → Create meter**:
event name `zappier.api_cents`, aggregation **Sum** of `value`,
customer mapping `stripe_customer_id`.
2. **Product catalog → Add product** "Zappier API usage" → price: recurring,
monthly, metered against that meter, **$0.01 per unit** (1 unit = 1 cent).
3. For each billable customer: create the Stripe Customer, attach a payment
method, and add a **subscription** with the metered price. Meter events for
customers without a metered subscription are recorded but never invoiced.
4. Put the `sk_test_...` key into `.env` (replace the placeholder).
5. Attach Stripe IDs to Zappier customers:
```bash
curl -X PUT -H 'x-admin-key: admin-dev-key' -H 'content-type: application/json' \
-d '{"stripeCustomerId":"cus_..."}' \
http://localhost:3000/admin/api/customers/cust_2
```
### 6.3 Verifying a run
```bash
npx ts-node src/jobs/report-usage.ts
```
Expected output per customer:
- `skip <id> <period> (nothing to report)` — no billable usage yet
- `skip <id> <period> (already reported Nc)` — no new usage since last run
- `<id>: reported N billable cents to Stripe` — delta sent
- `report-usage: another run holds the lock, abort run` — safe concurrent abort
Then check the meter's **Events** tab in the Stripe dashboard and the test
customer's **upcoming invoice**.
### 6.4 Going live
Repeat 6.2 steps 13 in live mode, replace `.env` with the `sk_live_...` key
from the **same Stripe account**, and keep the same cron. The live product
`prod_Uxv9SAeIOZzyx1` (currently deactivated) can be reactivated or recreated.
---
## 7. The Zapier integration
The `zapier-app/` directory contains the Zapier Platform app:
- **Authentication:** API key — the user pastes their API base URL
(e.g. `http://localhost:3000`) and their `key-...` customer key; the
connection is tested against `/v1/status`.
- **Trigger "New Item":** polls `GET /v1/storage` for newly stored items.
- **Action "Store Data":** calls `POST /v1/storage` with metadata and optional
file attachments — billed per the rate card.
To publish: create a Zapier developer account, `cd zapier-app && npm install
&& zapier login && zapier push`, then share the app or submit it to the Zapier
marketplace.
---
## 8. Day-to-day runbook
| Task | How |
|---|---|
| Change a price | Admin console → Rate card → Save |
| Add a customer | Admin console → Customers → Create → copy the one-time API key |
| Set a customer's billing type/email | Customers tab → Billing dropdown / Email field → Save |
| Invoice a period | Invoices tab → Generate → Issue → Mark paid (ACCOUNTING.md) |
| Export accounting data | Reports tab → filters → Download CSV |
| Give a customer a deal | Customers tab → multiplier override, or a new customer type |
| Turn an endpoint off | Rate card → Delete (calls get 403) |
| Check a customer's usage | Their portal dashboard, `GET /v1/usage` with their key, or query `zappier.db` |
| Check billing ran | Kimi notification after each 06:17 run; or run the job manually |
| Backup | Copy `zappier.db` (SQLite, single file) |
| Update dependencies | `npm outdated`, then `npm test` must stay green (165 tests) |
---
## 9. Troubleshooting
| Symptom | Likely cause | Fix |
|---|---|---|
| `401 invalid or missing API key` | Wrong/absent `x-api-key` | Re-issue key from Customers tab |
| `403 No price rule for ...` | Endpoint deleted from rate card and tier has no default rule | Re-add the rule, or set a tier `defaultRule` |
| `403 Unknown tier` | Customer's `tierId` doesn't exist | Fix the customer's type in the admin UI |
| `400 invalid metadata JSON` | `metadata` form field isn't valid JSON | Send e.g. `{"key":"value"}` |
| `413 ... file too large` | Attachment over 25 MB | Split or compress the file |
| Billing run: Stripe auth error | Placeholder/wrong key in `.env`, or key from a different Stripe account | Use the `sk_test`/`sk_live` key from the account that holds the meter |
| Stripe shows `METER_NOT_FOUND` invalid events | Meter missing or event name mismatch | Meter event name must be exactly `zappier.api_cents` |
| `another run holds the lock, abort run` | Overlapping runs, or a crash left a stale lock | Safe by design; stale locks expire after 1 hour |
| Admin UI: "invalid username or password" | Wrong credentials, or env overrides changed them | Check `ADMIN_KEY` / `DEMO_ADMIN_PASSWORD`; sign in again |
| Admin UI: "Session expired" | Server restarted (admin tokens are in-memory) | Sign in again |
| Portal login: `totp_required` | 2FA is enabled on the account | Enter the current 6-digit authenticator code |
| Portal: "invalid or expired session" | 7-day session expired | Sign in again |
| Portal reload didn't credit | Real `STRIPE_SECRET_KEY` configured → PaymentIntent awaits confirmation | Balance credits when the payment confirms; in dev (placeholder key) credit is instant |
| Invoice went straight to `paid` on Issue | Customer's prepaid balance fully covered the amount due | Working as designed — balance was drawn down |

BIN
docs/USER-MANUAL.pdf Normal file

Binary file not shown.

154
docs/WALKTHROUGH.md Normal file
View file

@ -0,0 +1,154 @@
# Zappier — Step-by-Step Usage Walkthrough
**Version:** 0.1.0 · **Last updated:** 2026-07-27
A full guided tour: from starting the server to pricing changes, customer
creation, and live API calls. Every step shows the real screen you should see.
Companion documents: [USER-MANUAL.md](USER-MANUAL.md) ·
[DEVELOPER.md](DEVELOPER.md).
---
## Step 1 — Start the server
```bash
cd /Users/marchon/zappier
npm install # first time only
npm run dev
```
Wait for the two "listening" lines:
![Start the server](walkthrough/01-start-server.png)
---
## Step 2 — Open the admin UI and sign in
Go to **http://localhost:3000/admin**. You'll see the sign-in screen:
![Sign in](walkthrough/02-login.png)
Two accounts are available:
| Username | Password | Purpose |
|---|---|---|
| `admin` | the `ADMIN_KEY` env value (default `admin-dev-key`) | Primary operator |
| `demo` | `$$$Adm1n###` | Demo / stakeholder access |
> Override either credential with the `ADMIN_USER`, `ADMIN_KEY`,
> `DEMO_ADMIN_USER`, and `DEMO_ADMIN_PASSWORD` environment variables.
---
## Step 3 — The rate card
After sign-in you land on the **Rate card** — one row per API endpoint with
its price in cents:
![Rate card](walkthrough/03-rate-card.png)
- `status`, `storage-list` — **free**
- `transform`**fixed** price per call
- `storage`**variable**: base + per-KB metadata + per-MB attachments
---
## Step 4 — Change a price
Edit any cent field — here `transform` is changed from **4¢ to 6¢** — and click
**Save**. The change is live on the very next API call; no restart, no deploy.
![Edit a price](walkthrough/04-edit-price.png)
After saving, the table re-reads from the server and shows the new value:
![Price saved](walkthrough/04b-saved-toast.png)
---
## Step 5 — Customer types
Click **Customer types** in the sidebar. Each type is a pricing tier:
a **multiplier** applied to every list price and a **monthly credit** of free
included usage (cents).
![Customer types](walkthrough/05-tiers.png)
---
## Step 6 — Add a customer type
Fill the **Add customer type** form — here `edu` / Education / ×0.6 — and click
**Add**. The new type appears immediately and can be assigned to customers.
![Add customer type](walkthrough/06-add-tier.png)
> New types start with 0 monthly credit — edit the row and **Save** to grant one.
---
## Step 7 — Customers
Click **Customers** in the sidebar. This is where accounts live: their type,
and an optional **multiplier override** for per-customer deals (blank =
inherit from type).
![Customers](walkthrough/07-customers.png)
---
## Step 8 — Create a customer and copy the API key
Enter a name, pick a type, click **Create**. The API key appears **once** in
the notification at the bottom-right — copy it and send it to the customer;
it is never shown again.
![API key shown once](walkthrough/08b-api-key-toast.png)
The new customer appears in the table right away:
![Customer created](walkthrough/08-create-customer.png)
---
## Step 9 — Explore the interactive API docs
Open **http://localhost:3000/docs** — full Swagger docs with a "Try it out"
console. Click **Authorize** and paste a customer API key to make live calls
from the browser.
![API docs](walkthrough/09-api-docs.png)
---
## Step 10 — Make an API call
Call the API with a customer key. Every priced response includes its **quote**,
so the cost of every call is transparent:
![API call with quote](walkthrough/10-api-call.png)
Note how the quote reflects the walkthrough itself: the 6¢ price set in step 4,
halved to 3¢ by Grace's Pro multiplier.
---
## Step 11 — Check usage and credits
Customers can check their own month-to-date usage anytime:
![Usage summary](walkthrough/11-usage.png)
`includedCents` is covered by the type's monthly credit; `billableCents` is
what the daily billing job would report to Stripe right now.
---
## Where to go next
- **Daily billing** runs automatically at 06:17 ET — see
[USER-MANUAL.md §6](USER-MANUAL.md#6-billing-operations-stripe) for the
Stripe meter/product/price setup and how to verify a run.
- **Troubleshooting:** [USER-MANUAL.md §9](USER-MANUAL.md#9-troubleshooting).
- **Internals:** [DEVELOPER.md](DEVELOPER.md).

BIN
docs/WALKTHROUGH.pdf Normal file

Binary file not shown.

115
docs/index.html Normal file
View file

@ -0,0 +1,115 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<title>Zappier — Documentation</title>
<style>
:root {
--bg: #f4f5fb;
--panel: #ffffff;
--ink: #171a26;
--muted: #6b7186;
--line: #e5e7f0;
--accent: #4f46e5;
--accent-soft: #eef0fe;
--radius: 12px;
--shadow: 0 1px 2px rgba(23, 26, 38, 0.05), 0 8px 24px rgba(23, 26, 38, 0.06);
}
* { box-sizing: border-box; }
body {
margin: 0;
font-family: -apple-system, "SF Pro Text", "Segoe UI", "PingFang SC", sans-serif;
background: var(--bg);
color: var(--ink);
font-size: 15px;
line-height: 1.5;
}
header {
background: linear-gradient(160deg, #312e81 0%, #4f46e5 55%, #7c74f0 100%);
color: #fff;
padding: 3rem 1.5rem 2.5rem;
text-align: center;
}
header .dot {
width: 44px; height: 44px; border-radius: 12px;
background: rgba(255, 255, 255, 0.16);
display: inline-grid; place-items: center;
font-weight: 800; font-size: 1.3rem; margin-bottom: 0.75rem;
}
header h1 { margin: 0 0 0.35rem; font-size: 1.7rem; }
header p { margin: 0; opacity: 0.85; }
main { max-width: 880px; margin: 0 auto; padding: 2rem 1.5rem 4rem; }
h2 { font-size: 0.8rem; text-transform: uppercase; letter-spacing: 0.06em; color: var(--muted); margin: 2rem 0 0.9rem; }
.cards { display: grid; grid-template-columns: repeat(auto-fill, minmax(260px, 1fr)); gap: 1rem; }
a.card {
display: block;
background: var(--panel);
border: 1px solid var(--line);
border-radius: var(--radius);
box-shadow: var(--shadow);
padding: 1.1rem 1.25rem;
text-decoration: none;
color: inherit;
transition: transform 0.12s ease, box-shadow 0.12s ease;
}
a.card:hover { transform: translateY(-2px); box-shadow: 0 4px 8px rgba(23,26,38,0.06), 0 16px 36px rgba(23,26,38,0.1); }
a.card h3 { margin: 0 0 0.35rem; font-size: 1.02rem; color: var(--accent); }
a.card p { margin: 0 0 0.7rem; color: var(--muted); font-size: 0.9rem; }
.fmt { display: inline-block; padding: 0.08rem 0.55rem; border-radius: 999px; font-size: 0.72rem; font-weight: 700; background: var(--accent-soft); color: var(--accent); margin-right: 0.3rem; }
.fmt.md { background: #f1f5f9; color: #475569; }
footer { text-align: center; color: var(--muted); font-size: 0.85rem; padding-bottom: 2rem; }
</style>
</head>
<body>
<header>
<span class="dot">Z</span>
<h1>Zappier Documentation</h1>
<p>Metered API platform — pricing, billing, invoicing, customer portal, and Zapier integration.</p>
</header>
<main>
<h2>Operations</h2>
<div class="cards">
<a class="card" href="USER-MANUAL.pdf">
<h3>Operations &amp; Usage Manual</h3>
<p>Running the system day to day: pricing, customers, billing operations, runbook, troubleshooting.</p>
<span class="fmt">PDF</span><span class="fmt">10 pages</span>
</a>
<a class="card" href="ACCOUNTING.pdf">
<h3>Company Accounting Walkthrough</h3>
<p>Invoices, purchase-order billing, prepaid drawdown, reports with CSV export, usage trends.</p>
<span class="fmt">PDF</span><span class="fmt">7 pages</span>
</a>
<a class="card" href="USER-MANAGEMENT.pdf">
<h3>User Management Walkthrough</h3>
<p>Admin accounts, rate card (free / fixed / variable pricing), customer types, customer accounts.</p>
<span class="fmt">PDF</span><span class="fmt">5 pages</span>
</a>
</div>
<h2>For your customers</h2>
<div class="cards">
<a class="card" href="CUSTOMER-PORTAL.pdf">
<h3>Customer Portal Walkthrough</h3>
<p>End-user guide: signup, two-factor authentication, usage dashboard, invoices, reloads, API &amp; pricing.</p>
<span class="fmt">PDF</span><span class="fmt">8 pages</span>
</a>
</div>
<h2>Technical</h2>
<div class="cards">
<a class="card" href="DEVELOPER.pdf">
<h3>Developer Documentation</h3>
<p>Architecture, module reference, pricing engine, billing pipeline, full API reference, extension guide.</p>
<span class="fmt">PDF</span><span class="fmt">9 pages</span>
</a>
<a class="card" href="WALKTHROUGH.pdf">
<h3>Step-by-Step Usage Walkthrough</h3>
<p>The original guided tour of the pricing admin UI with screenshots at every step.</p>
<span class="fmt">PDF</span><span class="fmt">13 pages</span>
</a>
</div>
</main>
<footer>Zappier · generated 2026-07-28</footer>
</body>
</html>

Binary file not shown.

After

Width:  |  Height:  |  Size: 85 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 40 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 124 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 124 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 401 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 76 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 91 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 59 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 64 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 69 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 60 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 78 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 57 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 59 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 71 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 42 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 52 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 398 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 50 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 402 KiB

BIN
docs/superpowers/.DS_Store vendored Normal file

Binary file not shown.

View file

@ -0,0 +1,163 @@
# Accounting, User Management & Customer Portal — Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [x]`) syntax for tracking.
**Goal:** Extend Zappier with three surfaces: (a) company accounting & system management, (b) internal user management, and (c) an end-user customer portal — with full documentation and screenshot walkthroughs for each.
**Architecture:** Same ports-and-adapters style as the existing codebase. New domain modules (`src/invoicing.ts`, `src/reports.ts`, `src/accounts.ts`) expose pure logic + repo interfaces; SQLite adapters live in `src/db/`; HTTP wiring goes into `src/admin.ts` (company side) and a new `src/portal.ts` (customer side). The admin SPA gains tabs; the portal is a second dependency-free SPA under `portal/`. No new runtime deps in Phase 12; Phase 3 adds `otplib`-style TOTP only if unavoidable — prefer `node:crypto` scrypt for passwords and a minimal TOTP implementation per RFC 6238.
**Tech Stack:** Node 20 · TS strict · Express 4 · better-sqlite3 · jest/supertest.
## Global Constraints
- Money is integer cents everywhere; multipliers applied once, `Math.round`.
- Money/date logic must be pure functions, unit-tested without HTTP or SQLite.
- All SQL in `src/db/` repos only; all tables `CREATE TABLE IF NOT EXISTS`; schema changes for existing tables use idempotent `ALTER TABLE ... ADD COLUMN` guarded by `PRAGMA table_info` checks.
- Existing 87 jest tests + 4 mocha tests stay green; `npx tsc --noEmit` clean after every task.
- No servers left running after any verification; walkthrough screenshots use a throwaway `ZAPPIER_DB`.
- PDFs: Phase 12 deliver **print-ready HTML report/invoice pages** (browser Print → PDF, zero deps) plus CSV; binary PDF generation is deferred unless the owner asks.
- The demo password `$$$Adm1n###` and `admin-dev-key` remain dev defaults, overridable by env.
## Surface map
| Surface | URL | Auth | Audience |
|---|---|---|---|
| Admin SPA (extended) | `/admin` | login token / `x-admin-key` | Company ops & accounting |
| Customer portal SPA (new) | `/portal` | customer session token (+TOTP 2FA) | End-user customers |
| Public API | `/v1` | `x-api-key` | Unchanged |
---
## Phase 1 — Accounting backend
### Task 1: Schema + customer billing identity
**Files:**
- Modify: `src/auth.ts``Customer` gains `billingType: 'stripe' | 'purchase_order'` (default `'stripe'`), `email?: string`
- Modify: `src/db/customer-repo.ts` — idempotent `ALTER TABLE customers ADD COLUMN billing_type TEXT NOT NULL DEFAULT 'stripe'`; same for `email TEXT`; map in `toCustomer`
- Modify: `src/admin.ts``PUT /customers/:id` accepts `billingType` (validated) and `email`
- Test: `tests/db-customer.test.ts`, `tests/admin.test.ts` (extend)
**Interfaces:**
- Consumes: existing `CustomerRepo`
- Produces: `Customer.billingType`, `Customer.email` used by all later tasks
- [x] Extend `tests/db-customer.test.ts`: migration on existing DB adds columns with `'stripe'` default; round-trip `billingType`/`email`
- [x] Extend `tests/admin.test.ts`: `PUT /customers/:id` sets `billingType: 'purchase_order'`; invalid value → 400
- [x] Implement; run suites; commit `feat(accounting): customer billing types and email`
### Task 2: Invoice engine
**Files:**
- Create: `src/invoicing.ts` — types + pure generation logic + repo interface
- Create: `src/db/invoice-repo.ts` — SQLite adapter
- Test: `tests/invoicing.test.ts`, `tests/db-invoice.test.ts`
**Interfaces:**
```ts
export interface Invoice {
id: string; // INV-2026-07-<customerSeq>
customerId: string;
period: string; // YYYY-MM
status: 'draft' | 'issued' | 'paid';
lines: InvoiceLine[]; // { endpointId, calls, cents }
totalCents: number; // sum of lines (gross usage)
creditCents: number; // monthly credit applied
billableCents: number; // totalCents - creditCents, floored at 0
billingType: 'stripe' | 'purchase_order';
poNumber?: string; // PO billing only
issuedAtMs?: number; dueAtMs?: number; paidAtMs?: number;
}
export interface InvoiceRepo {
save(invoice: Invoice): void;
get(id: string): Invoice | undefined;
list(filter: { customerId?: string; period?: string; status?: Invoice['status'] }): Invoice[];
nextSequence(period: string): number;
}
export function buildInvoice(args: {
customer: Customer; period: string; sequence: number;
entries: UsageEntry[]; tier: TierConfig; poNumber?: string;
}): Invoice; // groups entries by endpoint; status 'draft'
```
- [x] Tests: line grouping, credit math (partial/zero/excess), id format, PO fields
- [x] Repo tests: save/get/list filters, sequence increments per period
- [x] Implement; commit `feat(accounting): invoice engine`
### Task 3: Reports service (billing + trends, CSV/JSON)
**Files:**
- Create: `src/reports.ts``billingRows(entries, customers, tiers, range)` → rows `{customerId, name, billingType, calls, totalCents, creditCents, billableCents}`; `usageTrend(entries, bucket: 'day'|'week')``[{bucket, calls, cents}]`; `toCsv(rows)` with RFC-4180 escaping
- Test: `tests/reports.test.ts`
- [x] Tests: date-range filtering (inclusive from, exclusive to), per-customer vs all, per-billingType filter, trend bucketing across month boundary, CSV quoting of commas/quotes/newlines
- [x] Implement; commit `feat(accounting): reports service`
### Task 4: Admin accounting API
**Files:**
- Modify: `src/admin.ts` — new routes (all behind existing admin auth):
- `POST /invoices/generate { period, customerId?, poNumber? }` → builds draft invoices for the period (all customers or one; idempotent per customer+period — regenerating replaces the draft)
- `POST /invoices/:id/issue`, `POST /invoices/:id/paid`
- `GET /invoices?customerId&period&status`
- `GET /invoices/:id` (+ `?format=html` print-ready invoice page)
- `GET /reports/billing?from&to&customerId&billingType&format=json|csv`
- `GET /reports/usage-trend?from&to&bucket&customerId`
- `GET /zapier/status``{ published: boolean, triggerCount, actionCount, baseUrl }` read from `zapier-app/` files (static inspection, no network)
- Test: `tests/admin-accounting.test.ts`
- [x] Tests per route incl. CSV content-type, filter combos, invoice lifecycle transitions (draft→issued→paid; illegal transitions → 409)
- [x] Implement; commit `feat(accounting): admin accounting API`
---
## Phase 2 — Admin UI: Accounting, Reports, Users tabs
### Task 5: Invoices tab (generate, filter, lifecycle, print page)
### Task 6: Reports tab (date-range pickers, customer + billing-type filters, CSV download, trend charts as inline SVG)
### Task 7: Users tab (admin account management: list/create/deactivate admin users backed by a new `admin_users` table replacing the static two-account map; login endpoint reads the table; env seed preserved)
### Task 8: System tab (Zapier connection status, billing job last-run info from `billing_reports`/`job_locks`)
Each: admin UI section + `tests/` coverage for any new API + screenshot verification. Commit per task.
---
## Phase 3 — Customer portal (`/portal`)
### Task 9: Customer identity
- `customers` += `password_hash`, `totp_secret`, `totp_enabled`, `email_verified`
- `src/accounts.ts`: scrypt hash/verify (`node:crypto`), session tokens (new `portal_sessions` table), signup `POST /portal/api/signup` (creates customer on `free` tier + issues API key), login `POST /portal/api/login`
- Tests: hash round-trip, signup/login flows, session expiry
### Task 10: TOTP 2FA
- RFC 6238 TOTP (HMAC-SHA1, 30 s step, 6 digits) implemented in `src/accounts.ts` (no dep): `generateTotpSecret`, `totpUri(secret, email)`, `verifyTotp(secret, code, window=1)`
- Routes: `POST /portal/api/2fa/setup` (returns secret + otpauth URI; QR rendered client-side via a tiny inline QR lib or Google-Charts-free canvas QR — decision: render otpauth URI as text + QR via `qrcode` npm dep, portal-side only), `POST /portal/api/2fa/enable`, `POST /portal/api/2fa/verify` (login second step), `POST /portal/api/2fa/disable`
- Tests: known RFC vectors, window tolerance, login requires second factor when enabled
### Task 11: Portal dashboard API
- `GET /portal/api/me` (profile, tier, apiKey, regenerate key `POST /portal/api/api-key`)
- `GET /portal/api/usage` (month-to-date + credit)
- `GET /portal/api/invoices` (own invoices only, scoped by session customer)
- `POST /portal/api/reload { amountCents }` — prepaid balance: `customers` += `balance_cents`; Stripe PaymentIntent via existing SDK (test mode); balance drawn down at invoice issue before metered reporting
- `PUT /portal/api/email-invoicing { enabled, email }` — stored prefs; billing job emails PO invoices (send via SMTP env config; dev: log-only transport)
- Tests: session scoping (cannot read other customers' invoices), reload math, key regeneration invalidates old key
### Task 12: Portal SPA
- `portal/index.html` + `portal/app.js` in the same design language as the new admin: login/signup/2FA screens, dashboard (usage + balance), invoices (history + print), billing (reload, email invoicing), API docs + pricing info pages (rendered from `openapi.yaml` + live rate card)
- Screenshot verification of every screen
---
## Phase 4 — Documentation
### Task 13: `docs/ACCOUNTING.md` + PDF — company accounting walkthrough (invoices, PO billing, reports, trends, CSV export) with screenshots
### Task 14: `docs/USER-MANAGEMENT.md` + PDF — admin users walkthrough
### Task 15: `docs/CUSTOMER-PORTAL.md` + PDF — end-user walkthrough (signup, 2FA, reload, invoices, API docs)
### Task 16: Refresh `USER-MANUAL.md`, `DEVELOPER.md`, `README.md`; re-export all PDFs
---
## Self-review notes
- Spec coverage: Stripe **and** PO billing (Tasks 12, 11), invoicing histories (2, 4, 5), CSV/PDF + date range + all/individual/by-type (3, 4, 6), usage/billing trends (3, 6), management reports (4, 6, 8), Zapier connections (4, 8), user management (7), customer login/setup/2FA/invoices/reload/email invoicing/API docs/pricing (912), docs for all three surfaces (1316).
- Type consistency: `Invoice`, `InvoiceRepo`, `billingRows`, `usageTrend`, `toCsv` signatures are defined once above and reused in every later task.

File diff suppressed because it is too large Load diff

Binary file not shown.

After

Width:  |  Height:  |  Size: 37 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 401 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 72 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 72 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 72 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 60 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 65 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 59 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 64 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 83 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 60 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 58 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 50 KiB