Milestone 0: import zappier billing, Verae middleware, and Zapier research

Compose-ready workspace: packages/zappier (rate card, portal, Stripe), packages/verae-zapier-middleware (timestamp + NATS), packages/verae-zapier (CLI app), vendor/zapier-platform, and research/zapier vendor corpus. Gate 0 structure checks pass. Product code and research are not yet wired.
2026-09-09 02:37:36 -04:00 · 2026-09-09 02:37:36 -04:00 · b4150c8250
commit b4150c8250
1364 changed files with 6814366 additions and 0 deletions
--- a/docs/developer/modules/README.md
+++ b/docs/developer/modules/README.md
@ -0,0 +1,45 @@
+# Developer Module Index
+
+Each module document lists **every exported function**, its purpose, inputs, outputs, and side effects.
+
+Implementation status follows [TODO.md](../../../TODO.md) phases. Docs describe the **target contract** so implementers and reviewers share one API surface.
+
+## Middleware (`verae-zapier-middleware/src`)
+
+| Module doc | Source path | Phase |
+|------------|-------------|-------|
+| [debug.md](debug.md) | `debug/` | 1 |
+| [config.md](config.md) | `config.js` | 2 |
+| [errors.md](errors.md) | `errors.js` | 2 |
+| [app.md](app.md) | `app.js`, `index.js` | 2 |
+| [store-db.md](store-db.md) | `store/db.js` | 3 |
+| [store-tenants.md](store-tenants.md) | `store/tenants.js` | 3 |
+| [store-usage.md](store-usage.md) | `store/usage.js` | 3 |
+| [store-webhooks.md](store-webhooks.md) | `store/webhooks.js` | 3 |
+| [tokens.md](tokens.md) | `lib/tokens.js` | 4 |
+| [veraeClient.md](veraeClient.md) | `clients/veraeClient.js` | 4 |
+| [authService.md](authService.md) | `services/authService.js` | 5 |
+| [entitlementService.md](entitlementService.md) | `services/entitlementService.js` | 5 |
+| [middleware-http.md](middleware-http.md) | `middleware/*.js` | 5 |
+| [timestampService.md](timestampService.md) | `services/timestampService.js` | 6 |
+| [verifyService.md](verifyService.md) | `services/verifyService.js` | 6 |
+| [webhookService.md](webhookService.md) | `services/webhookService.js` | 6 |
+| [routes.md](routes.md) | `routes/*.js` | 6 |
+| [nats.md](nats.md) | `nats/*.js` | 7 |
+| [workers.md](workers.md) | `workers/*.js` | 8–9 |
+| [tenantService.md](tenantService.md) | `services/tenantService.js` | 10 |
+
+## Zapier app (`verae-zapier`)
+
+| Module doc | Source path | Phase |
+|------------|-------------|-------|
+| [zapier-app.md](zapier-app.md) | `index.js`, `authentication.js`, creates/searches/triggers | 11 |
+
+## Documentation rules
+
+When you add or change an export:
+
+1. Update JSDoc in the source file (`@param`, `@returns`, purpose sentence).
+2. Update the matching module markdown table.
+3. Add/adjust unit tests in the phase that owns the module.
+4. Instrument boundaries with `createDebugger('<namespace>')`.
--- a/docs/developer/modules/config.md
+++ b/docs/developer/modules/config.md
@ -0,0 +1,27 @@
+# Module: `config`
+
+**Path:** `verae-zapier-middleware/src/config.js`  
+**Phase:** 2  
+**Debug namespace:** `app`
+
+## Purpose
+
+Central runtime settings for HTTP, Verae upstream, NATS flags, and plan limits.
+
+## Exports
+
+### `loadEnvFile()`
+
+Loads `.env` without overriding existing env vars. Called at module load.
+
+### `config`
+
+See [function-reference.md](function-reference.md#configjs) for field table.
+
+### `PLAN_LIMITS`
+
+Default quotas per plan name (`free` | `starter` | `pro` | `enterprise`).
+
+## Environment variables
+
+Documented in `verae-zapier-middleware/.env.example` and root `README.md`.
--- a/docs/developer/modules/debug.md
+++ b/docs/developer/modules/debug.md
@ -0,0 +1,103 @@
+# Module: `debug`
+
+**Path:** `verae-zapier-middleware/src/debug/`  
+**Phase:** 1  
+**Purpose:** Runtime-toggleable failure tracing with secret redaction and correlation IDs.
+
+## Files
+
+| File | Role |
+|------|------|
+| `config.js` | Parse env into enable/namespaces/level |
+| `redact.js` | Strip secrets from log metadata |
+| `logger.js` | Namespaced `createDebugger` |
+| `trace-context.js` | ALS store (no logger import) |
+| `trace.js` | `withTrace`, Express middleware |
+| `index.js` | Public barrel export |
+
+---
+
+## `parseDebugVeraeEnv(raw)`
+
+| | |
+|--|--|
+| **For** | Interpret `DEBUG_VERAE` env string |
+| **Input** | `raw: string \| undefined` |
+| **Output** | `{ enabled: boolean, namespaces: Set<string>\|null }` |
+| **Notes** | `1`/`*` → all namespaces; `"auth,nats"` → allow-list |
+
+## `loadDebugConfig(env?)`
+
+| | |
+|--|--|
+| **For** | Build full debug config from environment |
+| **Input** | `env?: NodeJS.ProcessEnv` (default `process.env`) |
+| **Output** | `DebugConfig` `{ enabled, namespaces, level, filePath }` |
+
+## `shouldLog(config, namespace, messageLevel)`
+
+| | |
+|--|--|
+| **For** | Decide if a line should emit |
+| **Input** | config, namespace string, level |
+| **Output** | `boolean` |
+
+## `redact(input)`
+
+| | |
+|--|--|
+| **For** | Deep-redact secrets for safe logging |
+| **Input** | any JSON-like value |
+| **Output** | redacted clone |
+
+## `createDebugger(namespace)`
+
+| | |
+|--|--|
+| **For** | Get `{ debug, info, warn, error }` logger for a namespace |
+| **Input** | `namespace: string` |
+| **Output** | `Debugger` |
+| **Side effects** | Writes to stderr (or test sink) when enabled |
+
+### `Debugger.debug|info|warn|error(message, meta?)`
+
+| | |
+|--|--|
+| **Input** | `message: string`, `meta?: Record<string, unknown>` |
+| **Output** | `void` |
+
+## `withTrace(options, fn)`
+
+| | |
+|--|--|
+| **For** | Run work under a correlation id |
+| **Input** | `{ traceId?, span? }`, `fn: () => T\|Promise<T>` |
+| **Output** | `T\|Promise<T>` |
+
+## `getTraceId()`
+
+| | |
+|--|--|
+| **For** | Read active correlation id |
+| **Output** | `string\|null` |
+
+## `traceMiddleware(req, res, next)`
+
+| | |
+|--|--|
+| **For** | Per-request Express trace context |
+| **Input** | Express `(req, res, next)` |
+| **Output** | `void` |
+| **Sets** | `req.traceId`, response header `X-Trace-Id` |
+
+## `setDebugTestSink(sink)`
+
+| | |
+|--|--|
+| **For** | Capture log lines in unit tests |
+| **Input** | `((line: string) => void) \| null` |
+| **Output** | `void` |
+
+## Env reference
+
+See [../debugging.md](../debugging.md).
--- a/docs/developer/modules/function-reference.md
+++ b/docs/developer/modules/function-reference.md
@ -0,0 +1,486 @@
+# Function Reference — All Modules
+
+Canonical I/O contracts for the Verae Zapier middleware and Zapier app.  
+Source of truth for reviewers; keep in sync with JSDoc in code.
+
+Namespaces for debug: `auth`, `billing`, `http`, `nats`, `jobs`, `webhooks`, `trace`, `app`.
+
+---
+
+## config.js
+
+### `loadEnvFile()`
+
+| | |
+|--|--|
+| **For** | Load `.env` into `process.env` if keys are unset |
+| **Input** | none (reads file next to package root) |
+| **Output** | `void` |
+
+### `config` (exported object)
+
+| Field | Type | Description |
+|-------|------|-------------|
+| `port` | number | HTTP port |
+| `host` | string | Bind address |
+| `veraeApiBaseUrl` | string | Upstream Verae base URL (no trailing slash) |
+| `mockVerae` | boolean | Use mock client |
+| `natsEnabled` | boolean | Use JetStream workers |
+| `natsUrl` | string | NATS connection URL |
+| `tokenSecret` | string | HMAC secret for `zmt_` tokens |
+| `jobPollIntervalMs` | number | Poll delay |
+| `jobPollMaxAttempts` | number | Max poll attempts |
+| `storePath` | string | Absolute path to store file |
+| `upgradeUrl` | string | Billing upgrade link for 402 bodies |
+| `adminSecret` | string | Admin route secret |
+
+### `PLAN_LIMITS`
+
+Map of plan name → `{ timestamps, verifications, batch, batchMaxItems, requestsPerMinute }`.  
+`null` means unlimited.
+
+---
+
+## errors.js
+
+### `class AppError extends Error`
+
+| | |
+|--|--|
+| **For** | Structured operational errors returned as JSON |
+| **Constructor** | `(message: string, { status?, code?, details? })` |
+| **Properties** | `status: number`, `code: string`, `details: unknown` |
+
+### `asyncHandler(fn)`
+
+| | |
+|--|--|
+| **For** | Wrap async Express handlers so rejections hit error middleware |
+| **Input** | `(req, res, next) => Promise<any>` |
+| **Output** | Express middleware function |
+
+### `sendError(res, err)`
+
+| | |
+|--|--|
+| **For** | Write JSON error response; log 5xx with debug |
+| **Input** | Express `res`, `Error\|AppError` |
+| **Output** | `void` |
+
+---
+
+## app.js / index.js
+
+### `createApp()`
+
+| | |
+|--|--|
+| **For** | Build Express application (no listen) |
+| **Input** | none |
+| **Output** | `express.Application` |
+| **Mounts** | `GET /health`, `/zapier/*`, error handler, `traceMiddleware` |
+
+### `main()` (index.js)
+
+| | |
+|--|--|
+| **For** | Boot store, optional NATS, workers, listen |
+| **Input** | none |
+| **Output** | `Promise<void>` |
+
+---
+
+## store/db.js
+
+### `loadStore()`
+
+| | |
+|--|--|
+| **For** | Load JSON store into memory |
+| **Output** | store object |
+
+### `getStore()`
+
+| | |
+|--|--|
+| **For** | Access in-memory store |
+| **Output** | `{ tenants, apiKeys, usage, webhooks, jobWatchers }` |
+
+### `persist()`
+
+| | |
+|--|--|
+| **For** | Flush store to disk |
+| **Output** | `void` |
+
+---
+
+## store/tenants.js
+
+### `getTenant(tenantId)`
+
+| Input | `tenantId: string` |
+| Output | `Tenant\|null` |
+
+### `getTenantByApiKey(apiKey)`
+
+| Input | `apiKey: string` |
+| Output | `Tenant\|null` |
+
+### `listTenants()`
+
+| Output | `Tenant[]` |
+
+### `createTenant({ id, name, plan, veraeUsername, veraePassword, contract?, apiKey?, metadata? })`
+
+| Output | `{ tenant: Tenant, apiKey: string }` |
+
+### `resolveLimits(tenant)`
+
+| Output | Effective limits including enterprise contract overrides |
+
+---
+
+## store/usage.js
+
+### `getUsage(tenantId)`
+
+| Output | counters for current period |
+
+### `getUsageSummary(tenantId)`
+
+| Output | public-safe usage + limits snapshot |
+
+### `incrementUsage(tenantId, metric, amount?)`
+
+| For | Atomically increment a counter and persist |
+
+---
+
+## store/webhooks.js
+
+### `createWebhook({ tenantId, targetUrl, event })`
+
+| Output | `{ id, tenantId, targetUrl, event, createdAt }` |
+
+### `deleteWebhook({ tenantId, hookId?, targetUrl? })`
+
+| Output | `boolean` removed |
+
+### `getActiveWebhooks(tenantId, event)`
+
+| Output | `Webhook[]` |
+
+### `listWebhooksForTenant(tenantId)`
+
+| Output | `Webhook[]` |
+
+---
+
+## lib/tokens.js
+
+### `issueSessionToken({ tenantId, veraeToken, expiresAt })`
+
+| Output | `string` (`zmt_…`) |
+
+### `parseSessionToken(token)`
+
+| Output | `{ tenantId, veraeToken, expiresAt, nonce }\|null` |
+
+### `generateApiKey()`
+
+| Output | `string` (`zmw_…`) |
+
+### `isApiKey(value)`
+
+| Output | `boolean` |
+
+### `extractBearerToken(header)`
+
+| Input | `Authorization` header string |
+| Output | token string or `null` |
+
+---
+
+## clients/veraeClient.js
+
+All methods that hit the network log under `DEBUG_VERAE=http`.
+
+### `veraeClient.login(credentials)`
+
+| Input | `{ username, password }` |
+| Output | `{ token, expiresAt, user }` |
+
+### `veraeClient.validate(token)`
+
+| Input | Verae JWT |
+| Output | validation object |
+
+### `veraeClient.createTimestamp(token, body)`
+
+| Input | JWT, `{ data, hashAlg? }` |
+| Output | `{ jobId }` |
+
+### `veraeClient.createBatchTimestamp(token, body)`
+
+| Input | JWT, `{ items: [{ data, hashAlg? }] }` |
+| Output | `{ jobIds: string[] }` |
+
+### `veraeClient.getStatus(token, jobId)`
+
+| Output | StatusResponse |
+
+### `veraeClient.getBatchStatus(token, body)`
+
+| Input | `{ jobIds: string[] }` |
+| Output | `{ results }` |
+
+### `veraeClient.verify(token, body)`
+
+| Input | `{ certificate }` |
+| Output | `{ valid, timestamp?, blockIndex? }` |
+
+### `veraeClient.verifyBatch(token, body)`
+
+| Input | `{ certificates: string[] }` |
+| Output | `{ results }` |
+
+### `veraeClient.waitForJob(token, jobId, { maxAttempts, intervalMs })`
+
+| Output | terminal StatusResponse or throws `GATEWAY_TIMEOUT` |
+
+---
+
+## services/authService.js
+
+Debug namespace: `auth`.
+
+### `loginWithCredentials({ username, password, tenant? })`
+
+| Output | `{ accessToken, expiresAt, tenant, user }` |
+
+### `loginWithApiKey(apiKey)`
+
+| Output | same as loginWithCredentials after tenant lookup |
+
+### `resolveAuthContext(rawToken)`
+
+| Input | API key or session token |
+| Output | `{ tenantId, tenant, veraeToken, authMethod }` |
+
+### `validateSession(rawToken)`
+
+| Output | `{ valid, tenantId, authMethod, user }` |
+
+---
+
+## services/entitlementService.js
+
+Debug namespace: `billing`.
+
+### `checkEntitlement(tenantId, action, { amount? })`
+
+| Actions | `timestamp`, `verify`, `batch_timestamp` |
+| Throws | `402 QUOTA_EXCEEDED`, `403 PLAN_UPGRADE_REQUIRED` |
+| Output | `{ tenant, limits, usage }` |
+
+### `recordUsage(tenantId, action, { amount? })`
+
+| Output | `void` |
+
+---
+
+## middleware/authenticate.js
+
+### `authenticate(req, res, next)`
+
+| For | Populate `req.auth` via `resolveAuthContext` |
+| Reads | `Authorization: Bearer` or `x-api-key` |
+
+## middleware/rateLimit.js
+
+### `rateLimit(req, res, next)`
+
+| For | Enforce plan `requestsPerMinute` |
+| Throws | `429 RATE_LIMITED` |
+
+---
+
+## services/timestampService.js
+
+Debug: `jobs`, `billing`, `http`.
+
+### `createTimestamp(ctx, body)`
+
+| Input | auth context, `{ data, hashAlg? }` |
+| Output | `{ jobId }` |
+| Side effects | usage++, enqueue watch (NATS or in-process) |
+
+### `createTimestampAndWait(ctx, body)`
+
+| Output | StatusResponse (or pending on timeout when NATS wait enabled) |
+
+### `createBatchTimestamp(ctx, body)`
+
+| Output | `{ jobIds }` |
+
+### `getJobStatus(ctx, jobId)` / `getBatchJobStatus` / `getJobVerification`
+
+| Output | status payloads from Verae |
+
+---
+
+## services/verifyService.js
+
+### `verifyTimestamp(ctx, body)` / `verifyBatch(ctx, body)`
+
+| Output | verify results; records usage |
+
+---
+
+## services/webhookService.js
+
+Debug: `webhooks`.
+
+### `subscribe(ctx, { targetUrl, event })`
+
+| Output | webhook record |
+
+### `unsubscribe(ctx, { hookId?, targetUrl? })`
+
+| Output | `{ removed: true }` |
+
+### `deliverWebhook(targetUrl, payload)`
+
+| Output | `{ ok: boolean, status: number }` |
+
+---
+
+## services/tenantService.js
+
+### `selfServeSignup({ email, name, veraeUsername, veraePassword })`
+
+| Output | `{ tenant, apiKey, zapierSetup }` |
+
+### `provisionTenant({ id?, name, plan, veraeUsername, veraePassword, contract?, metadata?, audience? })`
+
+| Output | `{ tenant, apiKey }` |
+
+### `listProvisionedTenants()`
+
+| Output | public tenant summaries (no passwords) |
+
+---
+
+## nats/subjects.js
+
+### Constants
+
+`SUBJECTS.JOBS_WATCH`, `JOBS_EVENTS`, `WEBHOOKS_DELIVER`, `USAGE`  
+`STREAMS.ZAPIER_JOBS`, `ZAPIER_EVENTS`, `ZAPIER_WEBHOOKS`
+
+---
+
+## nats/connection.js
+
+### `connectNats(url?)`
+
+| Output | `{ nc, js, jsm }` NATS connection handles |
+
+### `ensureStreams(jsm)`
+
+| For | Idempotent stream create |
+| Output | `Promise<void>` |
+
+### `closeNats()`
+
+| Output | `Promise<void>` |
+
+---
+
+## nats/publishers.js
+
+### `enqueueWatch(jobWatchPayload)`
+
+| Input | see architecture/nats-subjects.md |
+| Output | `Promise<{ seq }>` |
+
+### `publishJobEvent(eventPayload)`
+
+| Output | `Promise<void>` |
+
+### `enqueueWebhook(deliverPayload)`
+
+| Output | `Promise<void>` |
+
+---
+
+## workers/jobPollerWorker.js
+
+### `startJobPollerWorker()`
+
+| For | Pull-consume watch queue; poll Verae; emit events |
+| Output | stop handle `{ stop() }` |
+
+## workers/webhookWorker.js
+
+### `startWebhookWorker()`
+
+| For | Deliver POSTs to Zapier with retry via JetStream |
+| Output | `{ stop() }` |
+
+## workers/inProcessJobPoller.js
+
+### `startInProcessJobPoller()` / `stopInProcessJobPoller()`
+
+| For | Phase 6 fallback when `NATS_ENABLED=false` |
+
+---
+
+## Routes (all under `/zapier`)
+
+| Method | Path | Handler purpose |
+|--------|------|-----------------|
+| POST | `/v1/auth/login` | Credentials or api_key → session |
+| GET | `/v1/auth/me` | Validate connection for Zapier |
+| POST | `/v1/timestamp` | Async create |
+| POST | `/v1/timestamp/wait` | Create and wait |
+| POST | `/v1/timestamp/batch` | Batch create |
+| POST | `/v1/verify` | Verify certificate |
+| GET | `/v1/status/:jobId` | Job status |
+| POST | `/v1/webhooks/subscribe` | REST Hook subscribe |
+| DELETE | `/v1/webhooks/unsubscribe` | REST Hook unsubscribe |
+| POST | `/v1/signup` | Self-serve tenant |
+| POST | `/v1/admin/tenants` | Admin provision |
+
+---
+
+## Zapier app modules
+
+### `authentication.test` / fields
+
+| Input from user | `api_key` |
+| Test URL | `GET {MIDDLEWARE_BASE_URL}/zapier/v1/auth/me` |
+
+### `beforeRequest: addApiKey`
+
+| Sets | `Authorization: Bearer ${bundle.authData.api_key}` |
+
+### Creates
+
+| Key | Middleware path |
+|-----|-----------------|
+| `timestamp_and_wait` | POST `/zapier/v1/timestamp/wait` |
+| `create_timestamp` | POST `/zapier/v1/timestamp` |
+| `verify_timestamp` | POST `/zapier/v1/verify` |
+| `batch_timestamp` | POST `/zapier/v1/timestamp/batch` |
+
+### Search `job_status`
+
+| Path | GET `/zapier/v1/status/{jobId}` |
+
+### Trigger `timestamp_completed`
+
+| Subscribe | POST `/zapier/v1/webhooks/subscribe` |
+| Unsubscribe | DELETE `/zapier/v1/webhooks/unsubscribe` |
+| Perform | return `[bundle.cleanedRequest]` |
--- a/docs/developer/modules/nats.md
+++ b/docs/developer/modules/nats.md
@ -0,0 +1,28 @@
+# Module: `nats`
+
+**Path:** `verae-zapier-middleware/src/nats/`  
+**Phase:** 7  
+**Debug namespace:** `nats`
+
+## Files
+
+| File | Purpose |
+|------|---------|
+| `subjects.js` | Subject, stream, consumer name constants |
+| `connection.js` | Connect, ensure streams, close |
+| `publishers.js` | enqueueWatch, publishJobEvent, enqueueWebhook |
+
+## Function contracts
+
+Full I/O tables: [function-reference.md](function-reference.md#natssubjectsjs)  
+Payload schemas: [../../architecture/nats-subjects.md](../../architecture/nats-subjects.md)
+
+## Implementation status
+
+Stubs throw until **GATE 7**. Call sites must check `config.natsEnabled` and fall back to in-process poller (Phase 6).
+
+## Security
+
+- Prefer `tokenRef` over raw Verae JWT in messages.
+- All publish paths log via `createDebugger('nats')` with redaction.
+- NATS must remain private (not exposed to Zapier).
--- a/docs/developer/modules/workers.md
+++ b/docs/developer/modules/workers.md
@ -0,0 +1,35 @@
+# Module: `workers`
+
+**Path:** `verae-zapier-middleware/src/workers/`  
+**Phases:** 6 (in-process), 8–9 (NATS)
+
+## Files
+
+| File | Phase | Purpose |
+|------|-------|---------|
+| `inProcessJobPoller.js` | 6 | Fallback poller when `NATS_ENABLED=false` |
+| `jobPollerWorker.js` | 8 | JetStream job watch consumer |
+| `webhookWorker.js` | 8 | JetStream webhook deliver consumer |
+
+## `startJobPollerWorker()`
+
+| | |
+|--|--|
+| **For** | Poll Verae until job terminal; publish events |
+| **Input** | none (uses NATS + config) |
+| **Output** | `Promise<{ stop: () => Promise<void> }>` |
+| **Debug** | `jobs`, `http`, `nats` |
+
+## `startWebhookWorker()`
+
+| | |
+|--|--|
+| **For** | POST completion payloads to Zapier hook URLs |
+| **Output** | `Promise<{ stop: () => Promise<void> }>` |
+| **Debug** | `webhooks`, `nats` |
+
+## Rules
+
+- Only one of in-process poller **or** NATS job worker should run.
+- Never log raw JWTs or full webhook secrets.
+- Queue groups allow horizontal scale without double delivery.