master-zapier-plan-draft/packages/zappier/docs/DEVELOPER.md

Zappier — Developer Documentation

Version: 0.2.0 · Last updated: 2026-07-28

Metered API platform: per-endpoint pricing, per-tier multipliers, usage ledger, Stripe metered billing, purchase-order invoicing, a company admin console, a customer portal with TOTP 2FA, and a Zapier integration. This document is the full technical reference. For operations, see USER-MANUAL.md.

---

Table of contents

1. Architecture
2. Module reference
3. Pricing engine
4. Request lifecycle
5. Persistence (SQLite)
6. Billing pipeline
7. API reference
8. Zapier app
9. Testing
10. Configuration
11. Extending the system

---

1. Architecture

Stack: Node 20 · TypeScript (strict) · Express 4 · better-sqlite3 · express-openapi-validator · swagger-ui-express · stripe SDK · Zapier Platform (core ^19) · jest + supertest (root) and mocha (zapier-app).

                         ┌────────────────────────────────────────────┐
                         │                Express app                 │
                         │                 (src/app.ts)               │
                         │                                            │
 Browser ── /admin ──────┼── static admin UI (admin/)                 │
 Browser ── /admin/api ──┼── adminAuth → adminRouter (src/admin.ts)   │
                         │      pricing · customers · invoices ·      │
                         │      reports · zapier status               │
 Browser ── /portal ─────┼── static portal SPA (portal/)              │
 Browser ── /portal/api ─┼── portalRouter (src/portal.ts)             │
                         │      signup/login+2FA · me · usage ·       │
                         │      invoices · reload · prefs             │
 Browser ── /docs ───────┼── swagger-ui (openapi.yaml)                │
                         │                                            │
 Client ──── /v1/* ──────┼── apiKeyAuth (src/auth.ts)                 │
                         │     └─ OpenAPI request validation          │
                         │     └─ meter() (src/meter.ts)              │
                         │          ├─ quoteCall() (src/pricing.ts)   │
                         │          └─ UsageRepo.record()             │
                         └──────────────┬─────────────────────────────┘
                                        │
        ┌───────────────┬───────────────┼───────────────┬────────────┐
        │               │               │               │            │
 SqliteUsageRepo  SqliteCustomerRepo  SqlitePricingStore  SqliteInvoiceRepo
 (usage_entries)  (customers)        (price_endpoints,   (invoices,
        │               │             tiers)             invoice_lines)
        │          SqliteSessionRepo (portal_sessions)        │
        └───────────────┴───────────────┴───────────────┴────────────┘
                                        │ zappier.db (SQLite)
   Cron (daily 06:17 ET)                │
   "Zappier billing · report usage"     │
              │                         │
              └─► src/jobs/report-usage.ts ── reportMonthlyUsage()
                    ├─ computeBillableCents / computeDelta (src/billing/stripe.ts)
                    ├─ SqliteBillingReportRepo (billing_reports, job_locks)
                    └─ stripe.billing.meterEvents.create ──► Stripe

Design rules the codebase follows:

• Ports & adapters: UsageRepo, CustomerRepo, PricingStore, MeterEventClient are interfaces with in-memory adapters (tests) and SQLite / Stripe adapters (production). Nothing outside src/db/ and the job's main() touches SQL or the Stripe SDK.
• Live pricing reads: the PricingContext getters in buildApp read the store on every quote, so admin edits apply without a restart.
• Money in integer cents everywhere. No floats cross a boundary except multipliers, which are applied once and rounded (Math.round).

2. Module reference

File	Responsibility	Key exports
src/index.ts	Production entry: opens SQLite, wires repos, listens	—
src/app.ts	buildApp(deps) — full Express wiring; StoredItem; DEFAULT_CUSTOMERS	buildApp, AppDeps
src/auth.ts	Customer model + x-api-key middleware	Customer, CustomerRepo, apiKeyAuth, InMemoryCustomerRepo
src/pricing.ts	Pricing domain: rules, tiers, quote algorithm, store port	PriceRule, TierConfig, Quote, quoteCall, PricingStore, DEFAULT_RATE_CARD, DEFAULT_TIERS
src/meter.ts	Per-request metering middleware	meter(endpointId, repo, pricing)
src/usage.ts	Usage ledger domain + summaries	UsageEntry, UsageSummary, summarize, UsageRepo, InMemoryUsageRepo
src/admin.ts	Admin API (/admin/api) + admin-key guard	adminAuth, adminRouter
src/billing/credit.ts	Monthly credit application	applyMonthlyCredit, BilledSummary
src/billing/stripe.ts	Stripe-facing math + client port	METER_EVENT_NAME, MeterEventClient, computeBillableCents, computeDelta, reportUsage
src/billing/reload.ts	Portal reloads via Stripe PaymentIntents	stripePaymentClient, hasRealStripeKey
src/accounts.ts	Portal identity: scrypt passwords, RFC 6238 TOTP, sessions	hashPassword, verifyPassword, totp, verifyTotp, generateTotpSecret, totpUri, SessionRepo, InMemorySessionRepo
src/invoicing.ts	Invoice domain + generation	Invoice, InvoiceLine, buildInvoice, InvoiceRepo, InMemoryInvoiceRepo
src/reports.ts	Billing/usage aggregation + CSV	billingRows, usageTrend, toCsv, BillingRow, TrendPoint
src/portal.ts	Customer portal API (/portal/api)	portalRouter, PortalDeps, PaymentClient, PaymentResult
src/paths.ts	Installation-root resolution (cwd-independent)	PROJECT_ROOT
src/jobs/report-usage.ts	Billing job: delta reporting to Stripe	reportMonthlyUsage, firstOfMonthUtc, ReportUsageDeps
src/db/usage-repo.ts	SQLite adapter: usage_entries	SqliteUsageRepo
src/db/customer-repo.ts	SQLite adapter: customers (seeds when empty; idempotent column migrations)	SqliteCustomerRepo
src/db/pricing-store.ts	SQLite adapter: price_endpoints, tiers (seeds when empty)	SqlitePricingStore
src/db/billing-repo.ts	SQLite adapter: billing_reports, job_locks	SqliteBillingReportRepo, BillingReportRepo, JobLockRepo
src/db/invoice-repo.ts	SQLite adapter: invoices, invoice_lines	SqliteInvoiceRepo
src/db/session-repo.ts	SQLite adapter: portal_sessions	SqliteSessionRepo
openapi.yaml	Public API contract; drives validation and /docs	—
admin/	Dependency-free admin SPA (index.html, app.js)	—
portal/	Dependency-free customer portal SPA (index.html, app.js)	—

3. Pricing engine

Types (src/pricing.ts)

type PriceRule =
  | { kind: 'free' }
  | { kind: 'fixed'; fixedCents: number }
  | { kind: 'variable'; baseCents: number; perKbCents: number; perMbCents: number };

interface TierConfig {
  id: string; name: string;
  multiplier: number;           // e.g. 0.5 = 50% of list
  monthlyCreditCents: number;   // free included usage per month
  defaultRule?: PriceRule;      // fallback for endpoints with no rule
}

interface Quote {
  endpointId: string;
  listCents: number;            // before multiplier
  totalCents: number;           // after multiplier — this is what is recorded
  breakdown: { baseCents: number; metadataCents: number; attachmentCents: number };
}

quoteCall(pricing, tierId, endpointId, usage, multiplierOverride?)

1. Resolve tier — throws Unknown tier: <id> (mapped to 403 by meter).
2. Resolve rule: rate-card rule for endpointId, else the tier's defaultRule, else throw No price rule for <tier>/<endpoint> (403).
3. Compute breakdown:
   • free → all zeros.
   • fixed → baseCents = fixedCents.
   • variable → baseCents + perKbCents × ceil(metadataBytes/1024) + perMbCents × ceil(attachmentBytes/1048576). Note the ceiling: 1 byte of metadata bills a full KB unit; attachments bill per started MB.
4. listCents = sum of breakdown; multiplier = multiplierOverride ?? tier.multiplier; totalCents = Math.round(listCents × multiplier).

totalCents (never listCents) is what the usage ledger records and what the billing pipeline sums.

4. Request lifecycle

For POST /v1/storage:

1. express.json() parses JSON bodies (multipart handled by the validator's multer — 25 MB per-file cap).
2. apiKeyAuth (src/auth.ts) — x-api-key → Customer on req.customer, else 401.
3. express-openapi-validator checks the request against openapi.yaml (400 on violation). For /v1/storage, parseMetadata then JSON-parses the metadata form field into res.locals.parsedMetadata (400 on bad JSON).
4. meter('storage', usage, pricing) (src/meter.ts):
   • measures metadataBytes (UTF-8 length of the JSON-stringified metadata) and attachmentBytes (sum of multer file sizes),
   • calls quoteCall — pricing errors become 403,
   • records a UsageEntry with cents = quote.totalCents,
   • stashes the quote in res.locals.quote.
5. The route handler builds the StoredItem (in-memory list) and responds { id, quote }.

GET /v1/usage is not metered; it summarizes the caller's month-to-date usage and applies the tier credit via applyMonthlyCredit.

5. Persistence (SQLite)

Single database file (ZAPPIER_DB, default zappier.db), WAL-agnostic better-sqlite3, all tables created with CREATE TABLE IF NOT EXISTS in the repo constructors. Seeding rule: customers and pricing tables seed from DEFAULT_CUSTOMERS / DEFAULT_RATE_CARD / DEFAULT_TIERS only when empty.

Table	Columns	Written by
usage_entries	id, customer_id, endpoint_id, cents, metadata_bytes, attachment_bytes, timestamp_ms	meter() on every priced call
customers	id PK, name, tier_id, api_key UNIQUE, stripe_customer_id, multiplier_override, billing_type, email, password_hash, totp_secret, totp_enabled, balance_cents, email_invoicing	Admin API, portal API
price_endpoints	endpoint_id PK, rule_json	Admin API
tiers	id PK, name, multiplier, monthly_credit_cents, default_rule_json	Admin API
invoices	id PK, customer_id, period, status, cents totals, billing_type, po_number, lifecycle timestamps	Admin API (generate/issue/paid + balance drawdown)
invoice_lines	invoice_id, endpoint_id, calls, cents	Invoice generation
portal_sessions	token PK, customer_id, created_ms, expires_ms	Portal auth
billing_reports	customer_id + period PK, reported_cents (cumulative), reported_at_ms	Billing job, after each successful meter event
job_locks	name PK, acquired_at_ms	Billing job run guard

New customer columns are added by idempotent migrations (PRAGMA table_info guard + ALTER TABLE ADD COLUMN) when the repo opens an older database — no manual migration step.

In tests, every repo is constructed over :memory: databases.

6. Billing pipeline

6.1 Math (src/billing/stripe.ts)

computeBillableCents(entries, monthlyCreditCents)
  = max(0, Σ entry.cents − monthlyCreditCents)

computeDelta(billable, previouslyReported)
  = max(0, billable − previouslyReported)

reportUsage(client, stripeCustomerId, entries, monthlyCreditCents) is the original whole-month reporter — retained as public API; the job uses the delta path instead.

6.2 The job (src/jobs/report-usage.ts)

reportMonthlyUsage(deps) per run:

1. Lock: locks.tryAcquireLock('report-usage', 1h TTL) — a single atomic INSERT … ON CONFLICT … DO UPDATE … WHERE acquired_at_ms <= now − ttl. Failure aborts the run; release happens in finally. A crashed run's lock is taken over after the TTL.
2. Window: since = firstOfMonthUtc(now) (injectable via deps.since for tests); period = since.toISOString().slice(0, 7) (YYYY-MM).
3. Per customer with a stripeCustomerId (others skipped silently; unknown tierId warns and skips):
   • entries = usage.listFor(customer.id, since)
   • billable = computeBillableCents(entries, tier.monthlyCreditCents)
   • prior = billingRepo.getReportedCents(customer.id, period) (0 for a new month — periods are isolated by the composite PK)
   • delta = computeDelta(billable, prior); delta <= 0 → log skip, continue
   • createMeterEvent({ eventName: 'zappier.api_cents', customerId: stripeCustomerId, value: String(delta), identifier: ${stripeCustomerId}:${period}:${billable} })
   • only on success: upsertReportedCents(customer.id, period, billable) — cumulative, not the delta.

Idempotency guarantees (reviewed design):

• Re-run safety: second run with same usage → delta 0 → no Stripe call.
• Mid-month growth: only the increase is sent; the identifier embeds the new cumulative billable, so legitimate growth is never deduped away.
• Crash between Stripe success and ledger write: retry sends a byte-identical event; Stripe drops it via the identifier (uniqueness enforced within a rolling 24 h window).
• Partial failure: customer A's ledger write commits before customer B is attempted; B's failure leaves A correctly recorded.

CLI: npx ts-node src/jobs/report-usage.ts (guarded by require.main === module; loads .env via dotenv inside main()). Scheduled by the Kimi cron job "Zappier billing · report usage to Stripe" (17 6 * * *, America/New_York), which runs this command daily and reports the outcome.

7. API reference

Public API (/v1, auth: x-api-key)

Defined in openapi.yaml; interactive docs at /docs.

Operation	Method & path	Price	Notes
status	GET /v1/status	free	Health + quote echo
transform	POST /v1/transform	fixed	{text} → {output: TEXT, quote}
storage	POST /v1/storage	variable	multipart: metadata (JSON string), attachments[] (≤25 MB/file) → {id, quote}
storage-list	GET /v1/storage	free	Caller's stored items
usage	GET /v1/usage	unmetered	Month-to-date summary with credit applied

Error envelope: { "error": string } with 400 (validation/metadata), 401 (bad key), 403 (unknown tier / no price rule), 413 (file over 25 MB).

Admin API (/admin/api, auth: x-admin-key or login session)

Route	Purpose
POST /login	{username, password} → session token (accounts: ADMIN_USER/ADMIN_KEY, DEMO_ADMIN_USER/DEMO_ADMIN_PASSWORD)
GET /pricing	{ rateCard, tiers }
PUT /endpoints/:id	Upsert a PriceRule (validated: 400 on bad shape)
DELETE /endpoints/:id	Remove a rule (endpoint becomes 403 for tiers without a default rule)
PUT /tiers/:id	Upsert a TierConfig
DELETE /tiers/:id	Remove a tier
GET /customers	List customers without API keys
POST /customers	Create {name, tierId} → full customer incl. generated apiKey (201, shown once)
PUT /customers/:id	Patch name / tierId / multiplierOverride / stripeCustomerId / billingType / email
POST /invoices/generate	{period, customerId?, poNumber?} — drafts per customer with usage; regenerating replaces drafts, skips issued/paid → {generated, skipped}
GET /invoices	Filters: customerId, period, status
GET /invoices/:id	JSON, or print-ready HTML with ?format=html
POST /invoices/:id/issue	draft → issued (PO gets 30-day due date). Prepaid drawdown: if the customer's balanceCents fully covers billableCents, the balance is deducted and the invoice is saved as paid instead
POST /invoices/:id/paid	issued → paid
GET /reports/billing	from/to/customerId/billingType filters; JSON rows or format=csv
GET /reports/usage-trend	bucket=day|week, same range filters
GET /zapier/status	Zapier app dir presence, version, triggers, creates

Portal API (/portal/api, auth: Bearer session)

Route	Purpose
POST /signup	{name, email, password≥8} → 201 {token, customer}. New customer on free with instant API key; an email match on a passwordless (admin-created) customer claims that account; 409 when the email already has a password
POST /login	{email, password, totpCode?} → {token, customer}; 401 totp_required when 2FA is on and the code is missing/wrong
POST /logout	Deletes the session
GET /me	Public profile — never includes passwordHash/totpSecret
POST /api-key	Regenerates the API key (old key dies immediately)
GET /usage	Month-to-date summary with tier credit applied
GET /pricing	Live rate card + tiers for the pricing page
GET /invoices · GET /invoices/:id	Own invoices only (others 404); ?format=html print view
POST /2fa/setup	Generates + stores a TOTP secret (not yet enabled) → {secret, uri, qr} (QR as PNG data URL via qrcode)
POST /2fa/enable · POST /2fa/disable	{code} verified against the stored secret
POST /reload	{amountCents} integer $1–$10,000 via the injected PaymentClient — dev client credits instantly; Stripe client returns a clientSecret and credits on confirmation
PUT /email-invoicing	{enabled} preference

Sessions live in portal_sessions (7-day TTL) and survive restarts. src/accounts.ts implements scrypt hashing (scrypt:N:r:p:salt:hash, timing-safe compare) and RFC 6238 TOTP (HMAC-SHA1, 30 s step, 6 digits, ±1 step window) with no external crypto dependency.

8. Zapier app

zapier-app/ — Zapier Platform (core ^19), CommonJS, mocha tests.

File	Purpose
index.js	App definition; wires auth, trigger, action
authentication.js	API-key auth; test call against /v1/status
triggers/new_item.js	Polling trigger: GET /v1/storage, newest first, dedupe by id
creates/store_data.js	Action: multipart POST /v1/storage (form-data), fields: metadata JSON + optional files
test/	mocha suite (4 tests): auth, trigger, action

Publish flow: zapier login → zapier push → invite users / submit for review. The app's base URL must point at a publicly reachable deployment of the API server.

9. Testing

npm test                # jest, repo root — 165 tests / 22 suites
npx tsc --noEmit        # type gate
cd zapier-app && npm test   # mocha — 4 tests

Conventions:

• TDD throughout; every module has in-memory adapters so tests never touch disk or network.
• HTTP tests use supertest against buildApp() with in-memory repos.
• SQLite tests use :memory: databases.
• Stripe is faked by implementing MeterEventClient; the idempotency suite (tests/report-usage-idempotency.test.ts) simulates growth, re-runs, month rollover, Stripe throws, lock contention, and partial failure.
• The job is tested via the injectable reportMonthlyUsage(deps) — never by executing main().

10. Configuration

Env var	Default	Used by
PORT	3000	src/index.ts
ZAPPIER_DB	<root>/zappier.db	src/index.ts, billing job
ADMIN_KEY	admin-dev-key	adminAuth()
ADMIN_USER	admin	Admin login
DEMO_ADMIN_USER / DEMO_ADMIN_PASSWORD	demo / $$$Adm1n###	Demo admin login
STRIPE_SECRET_KEY	—	Billing job; portal reloads when it starts with sk_ (otherwise a dev payment client credits instantly)

Both src/index.ts and the billing job load .env from the installation root (src/paths.ts PROJECT_ROOT) — never from the process cwd — so the compiled server and the job run from any working directory. .env is gitignored (chmod 600); .env.example documents the shape. Git identity is configured repo-local; .gitignore covers node_modules/, dist/, .env, zappier.db*.

11. Extending the system

Add an API endpoint:

1. Add the path + operationId to openapi.yaml (validation & docs follow automatically).
2. Add the route in src/app.ts, wrapping the handler with meter('<operationId>', usage, pricing).
3. Add a rate-card rule (admin UI or PUT /admin/api/endpoints/<operationId>) — otherwise tiers without a defaultRule get 403.
4. Write the failing test first; keep npm test + tsc green.

Add a customer type: admin UI or PUT /admin/api/tiers/:id ({name, multiplier, monthlyCreditCents, defaultRule?}).

Swap the storage backend: implement UsageRepo / CustomerRepo / PricingStore against your database and pass them to buildApp({...}) — no other code changes. Same for BillingReportRepo/JobLockRepo in the job.

Change the billing cadence: the job is safe at any frequency (delta + ledger + lock). The Kimi cron job controls scheduling; update its cron expression to change cadence.

Known intentional limitations: stored items are in-memory (restart clears them; usage ledger is unaffected); reportUsage is retained but superseded by the delta path; Stripe identifier dedup covers a rolling 24 h window; releaseLock is not owner-scoped (harmless at this job's runtime); admin session tokens are in-memory (portal sessions are persisted); Stripe reloads credit the balance only after payment confirmation (no webhook endpoint yet — dev client credits instantly); email invoicing stores the preference but sending requires SMTP wiring (deferred); PO invoices with partial prepaid coverage are not partially paid by design.