Milestone 0: import zappier billing, Verae middleware, and Zapier research

Compose-ready workspace: packages/zappier (rate card, portal, Stripe),
packages/verae-zapier-middleware (timestamp + NATS), packages/verae-zapier
(CLI app), vendor/zapier-platform, and research/zapier vendor corpus.

Gate 0 structure checks pass. Product code and research are not yet wired.
This commit is contained in:
George Lambert 2026-09-09 02:37:36 -04:00
commit b4150c8250
1364 changed files with 6814366 additions and 0 deletions

View file

@ -0,0 +1,80 @@
openapi: 3.0.0
info:
title: Verae Zapier Middleware API
description: |
Zapier-facing HTTP surface. Zapier CLI app calls these routes only.
Async job watching and webhook delivery may use NATS behind this API.
version: 0.1.0
servers:
- url: http://localhost:3100
description: Local middleware
paths:
/health:
get:
summary: Liveness probe
operationId: health
responses:
'200':
description: OK
/zapier/v1/auth/me:
get:
summary: Validate API key connection (Zapier auth test)
operationId: authMe
security:
- BearerAuth: []
responses:
'200':
description: Tenant + plan + usage
'401':
description: Unauthorized
/zapier/v1/timestamp:
post:
summary: Create timestamp (async jobId)
operationId: createTimestamp
security:
- BearerAuth: []
responses:
'202':
description: Accepted
/zapier/v1/timestamp/wait:
post:
summary: Create timestamp and wait for completion
operationId: createTimestampAndWait
security:
- BearerAuth: []
responses:
'200':
description: Completed or failed status
/zapier/v1/verify:
post:
summary: Verify certificate
operationId: verifyTimestamp
security:
- BearerAuth: []
responses:
'200':
description: Verification result
/zapier/v1/webhooks/subscribe:
post:
summary: REST Hook subscribe
operationId: webhookSubscribe
security:
- BearerAuth: []
responses:
'201':
description: Subscribed
/zapier/v1/webhooks/unsubscribe:
delete:
summary: REST Hook unsubscribe
operationId: webhookUnsubscribe
security:
- BearerAuth: []
responses:
'200':
description: Unsubscribed
components:
securitySchemes:
BearerAuth:
type: http
scheme: bearer
description: Middleware API key (zmw_…) or session token (zmt_…)

View file

@ -0,0 +1,101 @@
# NATS Subject Topology
All subjects are prefixed with `verae.zapier.` to isolate this platform from other Verae messaging.
## Streams
| Stream | Subjects | Retention | Purpose |
|--------|----------|-----------|---------|
| `ZAPIER_JOBS` | `verae.zapier.jobs.watch` | Work queue | Poll Verae for job status |
| `ZAPIER_EVENTS` | `verae.zapier.jobs.events` | Limits (time) | Terminal job outcomes |
| `ZAPIER_WEBHOOKS` | `verae.zapier.webhooks.deliver` | Work queue | POST to Zapier hook URLs |
| `ZAPIER_USAGE` (optional) | `verae.zapier.usage` | Limits | Billing export |
## Subjects
### `verae.zapier.jobs.watch`
**Published by:** HTTP edge after successful `POST /api/timestamp` (or batch item).
**Consumed by:** `job-poller` durable consumer (queue group).
**Payload**
| Field | Type | Required | Description |
|-------|------|----------|-------------|
| `tenantId` | string | yes | Owning tenant |
| `jobId` | string | yes | Verae job id |
| `tokenRef` | string | preferred | Opaque ref to resolve Verae credentials (avoid raw JWT) |
| `veraeToken` | string | discouraged | Only if tokenRef unavailable; redacted in logs |
| `enqueuedAt` | ISO-8601 | yes | Enqueue time |
| `attempt` | number | yes | Delivery attempt (0-based) |
| `maxAttempts` | number | yes | Stop after this many polls |
| `intervalMs` | number | yes | Suggested delay between polls |
| `traceId` | string | yes | Correlation id for debug |
### `verae.zapier.jobs.events`
**Published by:** Job poller when status is `completed`, `failed`, or `timeout`.
**Consumed by:** Event router → webhook enqueue; optional waiters on HTTP edge.
**Payload**
| Field | Type | Required | Description |
|-------|------|----------|-------------|
| `event` | string | yes | `timestamp.completed` \| `timestamp.failed` \| `timestamp.timeout` |
| `tenantId` | string | yes | Tenant id |
| `jobId` | string | yes | Job id |
| `status` | object | yes | Verae `StatusResponse` shape (or synthetic timeout) |
| `traceId` | string | yes | Correlation id |
| `emittedAt` | ISO-8601 | yes | Event time |
### `verae.zapier.webhooks.deliver`
**Published by:** Event router for each matching subscription.
**Consumed by:** `webhook-deliver` queue group.
**Payload**
| Field | Type | Required | Description |
|-------|------|----------|-------------|
| `hookId` | string | yes | Stored subscription id |
| `tenantId` | string | yes | Tenant |
| `targetUrl` | string | yes | Zapier REST Hook URL |
| `event` | string | yes | Event name |
| `payload` | object | yes | Body POSTed to Zapier |
| `attempt` | number | yes | Attempt count |
| `traceId` | string | yes | Correlation id |
### `verae.zapier.usage` (optional)
| Field | Type | Description |
|-------|------|-------------|
| `tenantId` | string | Tenant |
| `action` | string | `timestamp` \| `verify` \| `status` \| … |
| `amount` | number | Increment |
| `at` | ISO-8601 | Timestamp |
## Consumers
| Name | Stream | Mode | Notes |
|------|--------|------|-------|
| `job-poller` | `ZAPIER_JOBS` | Pull, queue | Nak with delay when still pending |
| `event-webhook-router` | `ZAPIER_EVENTS` | Push/pull | Fan-out to deliver subjects |
| `webhook-deliver` | `ZAPIER_WEBHOOKS` | Pull, queue | HTTP POST with backoff |
| `usage-writer` | `ZAPIER_USAGE` | Optional | Persist metering |
## Ack semantics
| Situation | Action |
|-----------|--------|
| Job still `pending` | `Nak` with delay ≈ `intervalMs` or republish with `attempt+1` |
| Job terminal | Publish event, `Ack` watch message |
| Webhook HTTP 2xx | `Ack` |
| Webhook HTTP 5xx / network | `Nak` / redelivery until `max_deliver` |
| Poison message | Term after max_deliver; write DLQ log with `DEBUG_VERAE=webhooks` |
## Security rules
1. Prefer `tokenRef` over embedding Verae JWTs in messages.
2. NATS must use private network + auth (Phase 15: mTLS).
3. Debug logs must redact tokens and `targetUrl` query secrets if any.
4. Treat `targetUrl` as untrusted egress (timeouts, size limits, SSRF allowlist later).

View file

@ -0,0 +1,132 @@
# Architecture Overview
## Purpose
Connect Zapier automations to Verae blockchain timestamping without:
- exposing raw Verae JWTs to end users,
- requiring Zapier to poll async jobs,
- coupling billing/plans to the core timestamping API,
- running multi-instance middleware with in-memory only job queues.
## Components
### 1. `verae-zapier` (Zapier Platform CLI app)
**Runs on:** Zapiers cloud when a Zap step executes.
**Responsibilities:**
- Custom auth field `api_key` (middleware-issued `zmw_…` keys).
- Map Zapier actions/searches/triggers to middleware HTTPS routes.
- Attach `Authorization: Bearer <api_key>` on every request.
- Translate middleware `402` / `403` into user-visible Zapier errors.
**Does not:**
- Call `api.veraetime.net` directly.
- Speak NATS.
- Enforce plan quotas (middleware does).
### 2. `verae-zapier-middleware` HTTP edge
**Runs on:** Your infrastructure (public HTTPS).
**Responsibilities:**
- Tenant identity and API keys.
- Auth bridge: API key → Verae login → JWT (server-side).
- Entitlement checks and usage metering.
- Synchronous API surface under `/zapier/v1/*`.
- REST Hook subscribe/unsubscribe storage.
- Publish async work to NATS when `NATS_ENABLED=true`.
### 3. NATS + JetStream
**Runs on:** Private network with middleware.
**Responsibilities:**
- Durable work queue for job status polling.
- Event stream for terminal job states.
- Work queue for Zapier webhook HTTP delivery with retries.
### 4. Workers
**Runs on:** Same deploy as middleware or separate worker processes.
| Worker | Consumes | Calls |
|--------|----------|-------|
| Job poller | `verae.zapier.jobs.watch` | `GET /api/status/{jobId}` on Verae |
| Event router | `verae.zapier.jobs.events` | Enqueues webhook deliveries |
| Webhook deliver | `verae.zapier.webhooks.deliver` | `POST` Zapier `targetUrl` |
### 5. `api.veraetime.net` (Verae Timestamping Service)
**Source of truth** for login, timestamp jobs, status, and verification.
OpenAPI: production Swagger / `Verae-Swagger.yaml`.
## Request flows
### A. Create Timestamp (async)
```text
Zapier → POST /zapier/v1/timestamp
Middleware: authenticate, checkEntitlement, POST /api/timestamp
Middleware: publish jobs.watch → return 202 { jobId }
Worker: poll status until terminal → publish jobs.events
Event router: match webhooks → publish webhooks.deliver
Webhook worker: POST hooks.zapier.com/...
```
### B. Create Timestamp and Wait
```text
Zapier → POST /zapier/v1/timestamp/wait
Middleware: create + wait for jobs.events (or in-process wait if NATS off)
→ return StatusResponse (completed/failed) or pending+jobId on timeout
```
### C. Auth connection test
```text
Zapier → GET /zapier/v1/auth/me Authorization: Bearer zmw_…
Middleware: resolve API key → tenant → optional validate Verae token
→ { tenantId, plan, usage, ... }
```
## Feature flags
| Flag | Effect |
|------|--------|
| `NATS_ENABLED=false` | In-process job poller; still full HTTP API (Phase 6 path) |
| `NATS_ENABLED=true` | JetStream workers; no in-process poller |
| `MOCK_VERAE=true` | No live Verae; deterministic mock jobs for tests |
| `DEBUG_VERAE=…` | Runtime failure tracing (see debugging.md) |
## Security boundaries
```text
Public Internet
├─ Zapier → Middleware HTTPS only
└─ Middleware → Zapier webhook HTTPS only
Private
├─ Middleware ↔ NATS
└─ Middleware/Workers → api.veraetime.net HTTPS
```
Never expose NATS ports to the public internet.
## Scaling model
- **HTTP edge:** scale replicas behind a load balancer (stateless except shared store).
- **NATS consumers:** queue groups — adding workers increases poll/deliver throughput.
- **Store:** file JSON is MVP single-node; Postgres/Redis required for multi-node (Phase 15).
## Related documents
- [nats-subjects.md](nats-subjects.md) — subjects, streams, payload schemas
- [../plans/phase-gates.md](../plans/phase-gates.md) — test gates
- [../../TODO.md](../../TODO.md) — full implementation order

View file

@ -0,0 +1,147 @@
# Runtime Debugging and Failure Tracing
## Goals
- Trace failures across **HTTP → NATS → Verae → Zapier webhook** without redeploying.
- Keep secrets out of logs even when debug is enabled.
- Allow selective namespaces so production noise stays low.
## Enabling debug (runtime)
Debug is **off** when `DEBUG_VERAE` is unset or empty.
```bash
# Everything
export DEBUG_VERAE=1
# or
export DEBUG_VERAE=*
# Selected namespaces (comma-separated)
export DEBUG_VERAE=auth,nats,jobs,webhooks,http,billing,trace
# Minimum level: debug | info | warn | error
export DEBUG_VERAE_LEVEL=debug
# Optional: write to file as well as stderr
export DEBUG_VERAE_FILE=/var/log/verae-zapier-debug.log
```
Restart is **not** required if the process reads env only at boot — current implementation reads env at process start. To change flags:
```bash
# systemd / docker: update env and restart one replica
# or send future SIGHUP support (Phase 15)
```
### Docker example
```yaml
environment:
DEBUG_VERAE: "auth,jobs,nats,webhooks"
DEBUG_VERAE_LEVEL: "debug"
```
## Namespaces
| Namespace | What it traces |
|-----------|----------------|
| `auth` | API key resolve, session parse, login outcomes (no passwords) |
| `billing` | Entitlement checks, quota decisions, plan limits |
| `http` | Outbound Verae requests: method, path, status, duration |
| `nats` | Connect, publish, consume, ack/nak, stream ensure |
| `jobs` | Watch enqueue, poll attempts, terminal transitions |
| `webhooks` | Subscribe, deliver attempts, HTTP status to Zapier |
| `trace` | Correlation id enter/exit spans |
| `app` | Boot, config summary (redacted), shutdown |
`DEBUG_VERAE=1` or `*` enables **all** namespaces.
## Correlation IDs
Every inbound HTTP request should get a `traceId` (generated or from `X-Trace-Id` header).
That id is:
- returned optionally as `X-Trace-Id` on responses,
- attached to NATS payloads as `traceId`,
- included in every debug line for that flow.
Example log line:
```text
2026-08-11T16:00:00.000Z DEBUG jobs [trace=a1b2c3d4] poll attempt=3 jobId=550e… status=pending
```
## Redaction rules
Always redacted (replaced with `[REDACTED]`):
- Headers: `authorization`, `x-api-key`, `cookie`
- Fields named: `password`, `veraePassword`, `token`, `veraeToken`, `accessToken`, `apiKey`, `api_key`, `secret`
- String values matching: `Bearer …`, `zmw_…`, `zmt_…`, long JWTs (`eyJ…`)
`targetUrl` host is kept; query string may be stripped if it contains tokens.
## Using debug for common failures
### 401 from middleware
```bash
DEBUG_VERAE=auth,trace npm start
# reproduce Zapier connection test
# look for resolveAuthContext failures
```
### Job never completes / trigger never fires
```bash
DEBUG_VERAE=jobs,nats,webhooks,http
# confirm: watch published → poll status → event emitted → deliver POST status
```
### 402 quota
```bash
DEBUG_VERAE=billing
# confirm plan limits vs usage counters
```
### Verae upstream errors
```bash
DEBUG_VERAE=http,jobs
# status codes and paths only; body may be summarized
```
## Programmatic API
See [modules/debug.md](modules/debug.md) and source under `verae-zapier-middleware/src/debug/`.
```js
import { createDebugger } from '../debug/logger.js';
import { withTrace, getTraceId } from '../debug/trace.js';
const log = createDebugger('jobs');
log.debug('poll start', { jobId });
log.error('poll failed', { jobId, err: err.message });
```
## Tests
Phase 1 gate verifies:
- silence when disabled,
- namespace filtering,
- redaction,
- trace id propagation.
```bash
npm run gate:1
```
## Production caution
- Prefer staging with narrowed namespaces.
- Never commit files containing live debug output with customer data.
- CI should run with debug off except dedicated debug unit tests that assert redaction.

View file

@ -0,0 +1,45 @@
# Developer Module Index
Each module document lists **every exported function**, its purpose, inputs, outputs, and side effects.
Implementation status follows [TODO.md](../../../TODO.md) phases. Docs describe the **target contract** so implementers and reviewers share one API surface.
## Middleware (`verae-zapier-middleware/src`)
| Module doc | Source path | Phase |
|------------|-------------|-------|
| [debug.md](debug.md) | `debug/` | 1 |
| [config.md](config.md) | `config.js` | 2 |
| [errors.md](errors.md) | `errors.js` | 2 |
| [app.md](app.md) | `app.js`, `index.js` | 2 |
| [store-db.md](store-db.md) | `store/db.js` | 3 |
| [store-tenants.md](store-tenants.md) | `store/tenants.js` | 3 |
| [store-usage.md](store-usage.md) | `store/usage.js` | 3 |
| [store-webhooks.md](store-webhooks.md) | `store/webhooks.js` | 3 |
| [tokens.md](tokens.md) | `lib/tokens.js` | 4 |
| [veraeClient.md](veraeClient.md) | `clients/veraeClient.js` | 4 |
| [authService.md](authService.md) | `services/authService.js` | 5 |
| [entitlementService.md](entitlementService.md) | `services/entitlementService.js` | 5 |
| [middleware-http.md](middleware-http.md) | `middleware/*.js` | 5 |
| [timestampService.md](timestampService.md) | `services/timestampService.js` | 6 |
| [verifyService.md](verifyService.md) | `services/verifyService.js` | 6 |
| [webhookService.md](webhookService.md) | `services/webhookService.js` | 6 |
| [routes.md](routes.md) | `routes/*.js` | 6 |
| [nats.md](nats.md) | `nats/*.js` | 7 |
| [workers.md](workers.md) | `workers/*.js` | 89 |
| [tenantService.md](tenantService.md) | `services/tenantService.js` | 10 |
## Zapier app (`verae-zapier`)
| Module doc | Source path | Phase |
|------------|-------------|-------|
| [zapier-app.md](zapier-app.md) | `index.js`, `authentication.js`, creates/searches/triggers | 11 |
## Documentation rules
When you add or change an export:
1. Update JSDoc in the source file (`@param`, `@returns`, purpose sentence).
2. Update the matching module markdown table.
3. Add/adjust unit tests in the phase that owns the module.
4. Instrument boundaries with `createDebugger('<namespace>')`.

View file

@ -0,0 +1,27 @@
# Module: `config`
**Path:** `verae-zapier-middleware/src/config.js`
**Phase:** 2
**Debug namespace:** `app`
## Purpose
Central runtime settings for HTTP, Verae upstream, NATS flags, and plan limits.
## Exports
### `loadEnvFile()`
Loads `.env` without overriding existing env vars. Called at module load.
### `config`
See [function-reference.md](function-reference.md#configjs) for field table.
### `PLAN_LIMITS`
Default quotas per plan name (`free` | `starter` | `pro` | `enterprise`).
## Environment variables
Documented in `verae-zapier-middleware/.env.example` and root `README.md`.

View file

@ -0,0 +1,103 @@
# Module: `debug`
**Path:** `verae-zapier-middleware/src/debug/`
**Phase:** 1
**Purpose:** Runtime-toggleable failure tracing with secret redaction and correlation IDs.
## Files
| File | Role |
|------|------|
| `config.js` | Parse env into enable/namespaces/level |
| `redact.js` | Strip secrets from log metadata |
| `logger.js` | Namespaced `createDebugger` |
| `trace-context.js` | ALS store (no logger import) |
| `trace.js` | `withTrace`, Express middleware |
| `index.js` | Public barrel export |
---
## `parseDebugVeraeEnv(raw)`
| | |
|--|--|
| **For** | Interpret `DEBUG_VERAE` env string |
| **Input** | `raw: string \| undefined` |
| **Output** | `{ enabled: boolean, namespaces: Set<string>\|null }` |
| **Notes** | `1`/`*` → all namespaces; `"auth,nats"` → allow-list |
## `loadDebugConfig(env?)`
| | |
|--|--|
| **For** | Build full debug config from environment |
| **Input** | `env?: NodeJS.ProcessEnv` (default `process.env`) |
| **Output** | `DebugConfig` `{ enabled, namespaces, level, filePath }` |
## `shouldLog(config, namespace, messageLevel)`
| | |
|--|--|
| **For** | Decide if a line should emit |
| **Input** | config, namespace string, level |
| **Output** | `boolean` |
## `redact(input)`
| | |
|--|--|
| **For** | Deep-redact secrets for safe logging |
| **Input** | any JSON-like value |
| **Output** | redacted clone |
## `createDebugger(namespace)`
| | |
|--|--|
| **For** | Get `{ debug, info, warn, error }` logger for a namespace |
| **Input** | `namespace: string` |
| **Output** | `Debugger` |
| **Side effects** | Writes to stderr (or test sink) when enabled |
### `Debugger.debug|info|warn|error(message, meta?)`
| | |
|--|--|
| **Input** | `message: string`, `meta?: Record<string, unknown>` |
| **Output** | `void` |
## `withTrace(options, fn)`
| | |
|--|--|
| **For** | Run work under a correlation id |
| **Input** | `{ traceId?, span? }`, `fn: () => T\|Promise<T>` |
| **Output** | `T\|Promise<T>` |
## `getTraceId()`
| | |
|--|--|
| **For** | Read active correlation id |
| **Output** | `string\|null` |
## `traceMiddleware(req, res, next)`
| | |
|--|--|
| **For** | Per-request Express trace context |
| **Input** | Express `(req, res, next)` |
| **Output** | `void` |
| **Sets** | `req.traceId`, response header `X-Trace-Id` |
## `setDebugTestSink(sink)`
| | |
|--|--|
| **For** | Capture log lines in unit tests |
| **Input** | `((line: string) => void) \| null` |
| **Output** | `void` |
## Env reference
See [../debugging.md](../debugging.md).

View file

@ -0,0 +1,486 @@
# Function Reference — All Modules
Canonical I/O contracts for the Verae Zapier middleware and Zapier app.
Source of truth for reviewers; keep in sync with JSDoc in code.
Namespaces for debug: `auth`, `billing`, `http`, `nats`, `jobs`, `webhooks`, `trace`, `app`.
---
## config.js
### `loadEnvFile()`
| | |
|--|--|
| **For** | Load `.env` into `process.env` if keys are unset |
| **Input** | none (reads file next to package root) |
| **Output** | `void` |
### `config` (exported object)
| Field | Type | Description |
|-------|------|-------------|
| `port` | number | HTTP port |
| `host` | string | Bind address |
| `veraeApiBaseUrl` | string | Upstream Verae base URL (no trailing slash) |
| `mockVerae` | boolean | Use mock client |
| `natsEnabled` | boolean | Use JetStream workers |
| `natsUrl` | string | NATS connection URL |
| `tokenSecret` | string | HMAC secret for `zmt_` tokens |
| `jobPollIntervalMs` | number | Poll delay |
| `jobPollMaxAttempts` | number | Max poll attempts |
| `storePath` | string | Absolute path to store file |
| `upgradeUrl` | string | Billing upgrade link for 402 bodies |
| `adminSecret` | string | Admin route secret |
### `PLAN_LIMITS`
Map of plan name → `{ timestamps, verifications, batch, batchMaxItems, requestsPerMinute }`.
`null` means unlimited.
---
## errors.js
### `class AppError extends Error`
| | |
|--|--|
| **For** | Structured operational errors returned as JSON |
| **Constructor** | `(message: string, { status?, code?, details? })` |
| **Properties** | `status: number`, `code: string`, `details: unknown` |
### `asyncHandler(fn)`
| | |
|--|--|
| **For** | Wrap async Express handlers so rejections hit error middleware |
| **Input** | `(req, res, next) => Promise<any>` |
| **Output** | Express middleware function |
### `sendError(res, err)`
| | |
|--|--|
| **For** | Write JSON error response; log 5xx with debug |
| **Input** | Express `res`, `Error\|AppError` |
| **Output** | `void` |
---
## app.js / index.js
### `createApp()`
| | |
|--|--|
| **For** | Build Express application (no listen) |
| **Input** | none |
| **Output** | `express.Application` |
| **Mounts** | `GET /health`, `/zapier/*`, error handler, `traceMiddleware` |
### `main()` (index.js)
| | |
|--|--|
| **For** | Boot store, optional NATS, workers, listen |
| **Input** | none |
| **Output** | `Promise<void>` |
---
## store/db.js
### `loadStore()`
| | |
|--|--|
| **For** | Load JSON store into memory |
| **Output** | store object |
### `getStore()`
| | |
|--|--|
| **For** | Access in-memory store |
| **Output** | `{ tenants, apiKeys, usage, webhooks, jobWatchers }` |
### `persist()`
| | |
|--|--|
| **For** | Flush store to disk |
| **Output** | `void` |
---
## store/tenants.js
### `getTenant(tenantId)`
| Input | `tenantId: string` |
| Output | `Tenant\|null` |
### `getTenantByApiKey(apiKey)`
| Input | `apiKey: string` |
| Output | `Tenant\|null` |
### `listTenants()`
| Output | `Tenant[]` |
### `createTenant({ id, name, plan, veraeUsername, veraePassword, contract?, apiKey?, metadata? })`
| Output | `{ tenant: Tenant, apiKey: string }` |
### `resolveLimits(tenant)`
| Output | Effective limits including enterprise contract overrides |
---
## store/usage.js
### `getUsage(tenantId)`
| Output | counters for current period |
### `getUsageSummary(tenantId)`
| Output | public-safe usage + limits snapshot |
### `incrementUsage(tenantId, metric, amount?)`
| For | Atomically increment a counter and persist |
---
## store/webhooks.js
### `createWebhook({ tenantId, targetUrl, event })`
| Output | `{ id, tenantId, targetUrl, event, createdAt }` |
### `deleteWebhook({ tenantId, hookId?, targetUrl? })`
| Output | `boolean` removed |
### `getActiveWebhooks(tenantId, event)`
| Output | `Webhook[]` |
### `listWebhooksForTenant(tenantId)`
| Output | `Webhook[]` |
---
## lib/tokens.js
### `issueSessionToken({ tenantId, veraeToken, expiresAt })`
| Output | `string` (`zmt_…`) |
### `parseSessionToken(token)`
| Output | `{ tenantId, veraeToken, expiresAt, nonce }\|null` |
### `generateApiKey()`
| Output | `string` (`zmw_…`) |
### `isApiKey(value)`
| Output | `boolean` |
### `extractBearerToken(header)`
| Input | `Authorization` header string |
| Output | token string or `null` |
---
## clients/veraeClient.js
All methods that hit the network log under `DEBUG_VERAE=http`.
### `veraeClient.login(credentials)`
| Input | `{ username, password }` |
| Output | `{ token, expiresAt, user }` |
### `veraeClient.validate(token)`
| Input | Verae JWT |
| Output | validation object |
### `veraeClient.createTimestamp(token, body)`
| Input | JWT, `{ data, hashAlg? }` |
| Output | `{ jobId }` |
### `veraeClient.createBatchTimestamp(token, body)`
| Input | JWT, `{ items: [{ data, hashAlg? }] }` |
| Output | `{ jobIds: string[] }` |
### `veraeClient.getStatus(token, jobId)`
| Output | StatusResponse |
### `veraeClient.getBatchStatus(token, body)`
| Input | `{ jobIds: string[] }` |
| Output | `{ results }` |
### `veraeClient.verify(token, body)`
| Input | `{ certificate }` |
| Output | `{ valid, timestamp?, blockIndex? }` |
### `veraeClient.verifyBatch(token, body)`
| Input | `{ certificates: string[] }` |
| Output | `{ results }` |
### `veraeClient.waitForJob(token, jobId, { maxAttempts, intervalMs })`
| Output | terminal StatusResponse or throws `GATEWAY_TIMEOUT` |
---
## services/authService.js
Debug namespace: `auth`.
### `loginWithCredentials({ username, password, tenant? })`
| Output | `{ accessToken, expiresAt, tenant, user }` |
### `loginWithApiKey(apiKey)`
| Output | same as loginWithCredentials after tenant lookup |
### `resolveAuthContext(rawToken)`
| Input | API key or session token |
| Output | `{ tenantId, tenant, veraeToken, authMethod }` |
### `validateSession(rawToken)`
| Output | `{ valid, tenantId, authMethod, user }` |
---
## services/entitlementService.js
Debug namespace: `billing`.
### `checkEntitlement(tenantId, action, { amount? })`
| Actions | `timestamp`, `verify`, `batch_timestamp` |
| Throws | `402 QUOTA_EXCEEDED`, `403 PLAN_UPGRADE_REQUIRED` |
| Output | `{ tenant, limits, usage }` |
### `recordUsage(tenantId, action, { amount? })`
| Output | `void` |
---
## middleware/authenticate.js
### `authenticate(req, res, next)`
| For | Populate `req.auth` via `resolveAuthContext` |
| Reads | `Authorization: Bearer` or `x-api-key` |
## middleware/rateLimit.js
### `rateLimit(req, res, next)`
| For | Enforce plan `requestsPerMinute` |
| Throws | `429 RATE_LIMITED` |
---
## services/timestampService.js
Debug: `jobs`, `billing`, `http`.
### `createTimestamp(ctx, body)`
| Input | auth context, `{ data, hashAlg? }` |
| Output | `{ jobId }` |
| Side effects | usage++, enqueue watch (NATS or in-process) |
### `createTimestampAndWait(ctx, body)`
| Output | StatusResponse (or pending on timeout when NATS wait enabled) |
### `createBatchTimestamp(ctx, body)`
| Output | `{ jobIds }` |
### `getJobStatus(ctx, jobId)` / `getBatchJobStatus` / `getJobVerification`
| Output | status payloads from Verae |
---
## services/verifyService.js
### `verifyTimestamp(ctx, body)` / `verifyBatch(ctx, body)`
| Output | verify results; records usage |
---
## services/webhookService.js
Debug: `webhooks`.
### `subscribe(ctx, { targetUrl, event })`
| Output | webhook record |
### `unsubscribe(ctx, { hookId?, targetUrl? })`
| Output | `{ removed: true }` |
### `deliverWebhook(targetUrl, payload)`
| Output | `{ ok: boolean, status: number }` |
---
## services/tenantService.js
### `selfServeSignup({ email, name, veraeUsername, veraePassword })`
| Output | `{ tenant, apiKey, zapierSetup }` |
### `provisionTenant({ id?, name, plan, veraeUsername, veraePassword, contract?, metadata?, audience? })`
| Output | `{ tenant, apiKey }` |
### `listProvisionedTenants()`
| Output | public tenant summaries (no passwords) |
---
## nats/subjects.js
### Constants
`SUBJECTS.JOBS_WATCH`, `JOBS_EVENTS`, `WEBHOOKS_DELIVER`, `USAGE`
`STREAMS.ZAPIER_JOBS`, `ZAPIER_EVENTS`, `ZAPIER_WEBHOOKS`
---
## nats/connection.js
### `connectNats(url?)`
| Output | `{ nc, js, jsm }` NATS connection handles |
### `ensureStreams(jsm)`
| For | Idempotent stream create |
| Output | `Promise<void>` |
### `closeNats()`
| Output | `Promise<void>` |
---
## nats/publishers.js
### `enqueueWatch(jobWatchPayload)`
| Input | see architecture/nats-subjects.md |
| Output | `Promise<{ seq }>` |
### `publishJobEvent(eventPayload)`
| Output | `Promise<void>` |
### `enqueueWebhook(deliverPayload)`
| Output | `Promise<void>` |
---
## workers/jobPollerWorker.js
### `startJobPollerWorker()`
| For | Pull-consume watch queue; poll Verae; emit events |
| Output | stop handle `{ stop() }` |
## workers/webhookWorker.js
### `startWebhookWorker()`
| For | Deliver POSTs to Zapier with retry via JetStream |
| Output | `{ stop() }` |
## workers/inProcessJobPoller.js
### `startInProcessJobPoller()` / `stopInProcessJobPoller()`
| For | Phase 6 fallback when `NATS_ENABLED=false` |
---
## Routes (all under `/zapier`)
| Method | Path | Handler purpose |
|--------|------|-----------------|
| POST | `/v1/auth/login` | Credentials or api_key → session |
| GET | `/v1/auth/me` | Validate connection for Zapier |
| POST | `/v1/timestamp` | Async create |
| POST | `/v1/timestamp/wait` | Create and wait |
| POST | `/v1/timestamp/batch` | Batch create |
| POST | `/v1/verify` | Verify certificate |
| GET | `/v1/status/:jobId` | Job status |
| POST | `/v1/webhooks/subscribe` | REST Hook subscribe |
| DELETE | `/v1/webhooks/unsubscribe` | REST Hook unsubscribe |
| POST | `/v1/signup` | Self-serve tenant |
| POST | `/v1/admin/tenants` | Admin provision |
---
## Zapier app modules
### `authentication.test` / fields
| Input from user | `api_key` |
| Test URL | `GET {MIDDLEWARE_BASE_URL}/zapier/v1/auth/me` |
### `beforeRequest: addApiKey`
| Sets | `Authorization: Bearer ${bundle.authData.api_key}` |
### Creates
| Key | Middleware path |
|-----|-----------------|
| `timestamp_and_wait` | POST `/zapier/v1/timestamp/wait` |
| `create_timestamp` | POST `/zapier/v1/timestamp` |
| `verify_timestamp` | POST `/zapier/v1/verify` |
| `batch_timestamp` | POST `/zapier/v1/timestamp/batch` |
### Search `job_status`
| Path | GET `/zapier/v1/status/{jobId}` |
### Trigger `timestamp_completed`
| Subscribe | POST `/zapier/v1/webhooks/subscribe` |
| Unsubscribe | DELETE `/zapier/v1/webhooks/unsubscribe` |
| Perform | return `[bundle.cleanedRequest]` |

View file

@ -0,0 +1,28 @@
# Module: `nats`
**Path:** `verae-zapier-middleware/src/nats/`
**Phase:** 7
**Debug namespace:** `nats`
## Files
| File | Purpose |
|------|---------|
| `subjects.js` | Subject, stream, consumer name constants |
| `connection.js` | Connect, ensure streams, close |
| `publishers.js` | enqueueWatch, publishJobEvent, enqueueWebhook |
## Function contracts
Full I/O tables: [function-reference.md](function-reference.md#natssubjectsjs)
Payload schemas: [../../architecture/nats-subjects.md](../../architecture/nats-subjects.md)
## Implementation status
Stubs throw until **GATE 7**. Call sites must check `config.natsEnabled` and fall back to in-process poller (Phase 6).
## Security
- Prefer `tokenRef` over raw Verae JWT in messages.
- All publish paths log via `createDebugger('nats')` with redaction.
- NATS must remain private (not exposed to Zapier).

View file

@ -0,0 +1,35 @@
# Module: `workers`
**Path:** `verae-zapier-middleware/src/workers/`
**Phases:** 6 (in-process), 89 (NATS)
## Files
| File | Phase | Purpose |
|------|-------|---------|
| `inProcessJobPoller.js` | 6 | Fallback poller when `NATS_ENABLED=false` |
| `jobPollerWorker.js` | 8 | JetStream job watch consumer |
| `webhookWorker.js` | 8 | JetStream webhook deliver consumer |
## `startJobPollerWorker()`
| | |
|--|--|
| **For** | Poll Verae until job terminal; publish events |
| **Input** | none (uses NATS + config) |
| **Output** | `Promise<{ stop: () => Promise<void> }>` |
| **Debug** | `jobs`, `http`, `nats` |
## `startWebhookWorker()`
| | |
|--|--|
| **For** | POST completion payloads to Zapier hook URLs |
| **Output** | `Promise<{ stop: () => Promise<void> }>` |
| **Debug** | `webhooks`, `nats` |
## Rules
- Only one of in-process poller **or** NATS job worker should run.
- Never log raw JWTs or full webhook secrets.
- Queue groups allow horizontal scale without double delivery.

View file

@ -0,0 +1,189 @@
# Phase Gates — Commands and Acceptance Criteria
Gates enforce **test-before-next-step**. Run from repository root:
```bash
cd /Users/marchon/datacubes/verae-zapier-api
npm run gate:N
```
`npm run gate:all` runs gates `0``12` sequentially and **exits non-zero** on first failure.
## Gate 0 — Structure and documentation
```bash
npm run gate:0
```
| Check | Requirement |
|-------|-------------|
| Layout | `TODO.md`, `README.md`, middleware + zapier dirs, docs tree |
| Architecture | `docs/architecture/overview.md`, `nats-subjects.md` |
| Developer | `docs/developer/debugging.md`, `docs/developer/modules/*.md` |
| Plan | `docs/plans/phase-gates.md` |
## Gate 1 — Debug facility
```bash
npm run gate:1
```
Tests: `verae-zapier-middleware/test/unit/debug*.test.js`
| Check | Requirement |
|-------|-------------|
| Off by default | No output when `DEBUG_VERAE` unset |
| Namespace filter | `auth` does not emit `nats` lines |
| Redaction | Tokens/passwords not present in formatted output |
| Trace | Child spans inherit `traceId` |
## Gate 2 — HTTP shell
```bash
npm run gate:2
```
| Check | Requirement |
|-------|-------------|
| Health | `GET /health` → 200 `{ status: "ok" }` |
| Errors | Thrown `AppError` → JSON `{ error, code }` |
| Config | Reads NATS and debug env keys |
## Gate 3 — Stores
```bash
npm run gate:3
```
| Check | Requirement |
|-------|-------------|
| Tenants | API key maps to tenant |
| Limits | Plan + enterprise contract |
| Webhooks | Tenant isolation |
| Persist | Reload preserves state |
## Gate 4 — Tokens + Verae client
```bash
npm run gate:4
```
| Check | Requirement |
|-------|-------------|
| HMAC | Bad signature rejected |
| Mock job | create → wait → completed |
| HTTP debug | No Authorization values logged |
## Gate 5 — Auth + entitlements
```bash
npm run gate:5
```
| Check | Requirement |
|-------|-------------|
| Me | Valid API key |
| 401 | Invalid key |
| 402 | Over quota |
## Gate 6 — Sync HTTP API (NATS off)
```bash
NATS_ENABLED=false MOCK_VERAE=true npm run gate:6
```
| Check | Requirement |
|-------|-------------|
| Timestamp | 202 + jobId |
| Wait | completed status |
| Verify | boolean valid |
| Webhooks | subscribe stored |
## Gate 7 — NATS infrastructure
```bash
npm run gate:7
```
| Check | Requirement |
|-------|-------------|
| Streams | Created/idempotent ensure |
| Pub/sub | One message round-trip |
| Flag off | App boots without NATS |
## Gate 8 — Workers
```bash
NATS_ENABLED=true MOCK_VERAE=true npm run gate:8
```
| Check | Requirement |
|-------|-------------|
| Watch→event | Terminal event published |
| Webhook | Mock receiver got POST |
| Queue group | No double complete |
## Gate 9 — Wait via NATS
```bash
npm run gate:9
```
| Check | Requirement |
|-------|-------------|
| Success | Wait returns completed |
| Timeout | Returns pending + jobId |
## Gate 10 — Tenancy
```bash
npm run gate:10
```
| Check | Requirement |
|-------|-------------|
| Signup | free + apiKey |
| Enterprise | requires contract |
## Gate 11 — Zapier package
```bash
npm run gate:11
```
| Check | Requirement |
|-------|-------------|
| Auth module | fields + test URL |
| Creates | correct middleware paths |
| Trigger | subscribe shapes |
## Gate 12 — E2E local compose
```bash
npm run gate:12
```
| Check | Requirement |
|-------|-------------|
| Stack | health green |
| Paths | async + wait + hook |
## Gate 13 — Production smoke (opt-in)
```bash
PRODUCTION_SMOKE=1 npm run gate:13
```
Requires real secrets; skipped in default `gate:all`.
## Gate 14 — Human acceptance
Manual checklist in `TODO.md` (not automated).
## Gate 15 — Hardening
Checklist + load notes in `TODO.md`.
## Recording results
After each green gate, update the **Gate log** table in `TODO.md` with date and `pass`.