marchon/master-zapier-plan-draft
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1

Milestone 0: import zappier billing, Verae middleware, and Zapier research

Browse source 
Compose-ready workspace: packages/zappier (rate card, portal, Stripe),
packages/verae-zapier-middleware (timestamp + NATS), packages/verae-zapier
(CLI app), vendor/zapier-platform, and research/zapier vendor corpus.

Gate 0 structure checks pass. Product code and research are not yet wired.
This commit is contained in:
George Lambert 2026-09-09 02:37:36 -04:00
commit b4150c8250
1364 changed files with 6814366 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

33
docs/02-architecture/composition.md Normal file
View file

@ -0,0 +1,33 @@
# Composition: zappier commercial edge + Verae adapter
```text
Users → Zapier UI
Zapier cloud runs packages/verae-zapier
--HTTPS, x-api-key--> packages/zappier /v1/*
(meter, quote, 401/403, usage)
--internal HTTPS--> packages/verae-zapier-middleware /zapier/v1/*
--sync--> api.veraetime.net or MOCK_VERAE
--NATS--> workers --> Verae + Zapier REST Hooks
Humans → zappier /portal signup, API key, usage, reloads, invoices
Ops → zappier /admin rate card, tiers, customers, PO invoices
```
## Ownership
| Concern | Package |
|---------|---------|
| API keys, custom pricing, Stripe, invoices, portal | `packages/zappier` |
| Timestamp/verify/status, NATS, REST Hooks, mock Verae | `packages/verae-zapier-middleware` |
| Zapier Platform nouns (creates/searches/triggers) | `packages/verae-zapier` |
| Official SDK reference | `vendor/zapier-platform` |
Zapier never talks to NATS or `api.veraetime.net`.
Do not reimplement Stripe, invoices, or the rate-card UI inside the Verae middleware.
Public keys are zappier `x-api-key` (issued at portal signup). Middleware `zmw_` / `PLAN_LIMITS` stay internal until composition PR 4 removes them from the public path.
Demo zappier routes `/v1/transform` and `/v1/storage` are a metering sandbox, not Verae timestamping or encrypted LTS.
Payment invoices (zappier) are not certified timestamp receipts (Verae feature i).

101
docs/02-architecture/nats-subjects.md Normal file
View file

@ -0,0 +1,101 @@
# NATS Subject Topology
All subjects are prefixed with `verae.zapier.` to isolate this platform from other Verae messaging.
## Streams
| Stream | Subjects | Retention | Purpose |
|--------|----------|-----------|---------|
| `ZAPIER_JOBS` | `verae.zapier.jobs.watch` | Work queue | Poll Verae for job status |
| `ZAPIER_EVENTS` | `verae.zapier.jobs.events` | Limits (time) | Terminal job outcomes |
| `ZAPIER_WEBHOOKS` | `verae.zapier.webhooks.deliver` | Work queue | POST to Zapier hook URLs |
| `ZAPIER_USAGE` (optional) | `verae.zapier.usage` | Limits | Billing export |
## Subjects
### `verae.zapier.jobs.watch`
**Published by:** HTTP edge after successful `POST /api/timestamp` (or batch item).
**Consumed by:** `job-poller` durable consumer (queue group).
**Payload**
| Field | Type | Required | Description |
|-------|------|----------|-------------|
| `tenantId` | string | yes | Owning tenant |
| `jobId` | string | yes | Verae job id |
| `tokenRef` | string | preferred | Opaque ref to resolve Verae credentials (avoid raw JWT) |
| `veraeToken` | string | discouraged | Only if tokenRef unavailable; redacted in logs |
| `enqueuedAt` | ISO-8601 | yes | Enqueue time |
| `attempt` | number | yes | Delivery attempt (0-based) |
| `maxAttempts` | number | yes | Stop after this many polls |
| `intervalMs` | number | yes | Suggested delay between polls |
| `traceId` | string | yes | Correlation id for debug |
### `verae.zapier.jobs.events`
**Published by:** Job poller when status is `completed`, `failed`, or `timeout`.
**Consumed by:** Event router → webhook enqueue; optional waiters on HTTP edge.
**Payload**
| Field | Type | Required | Description |
|-------|------|----------|-------------|
| `event` | string | yes | `timestamp.completed` \| `timestamp.failed` \| `timestamp.timeout` |
| `tenantId` | string | yes | Tenant id |
| `jobId` | string | yes | Job id |
| `status` | object | yes | Verae `StatusResponse` shape (or synthetic timeout) |
| `traceId` | string | yes | Correlation id |
| `emittedAt` | ISO-8601 | yes | Event time |
### `verae.zapier.webhooks.deliver`
**Published by:** Event router for each matching subscription.
**Consumed by:** `webhook-deliver` queue group.
**Payload**
| Field | Type | Required | Description |
|-------|------|----------|-------------|
| `hookId` | string | yes | Stored subscription id |
| `tenantId` | string | yes | Tenant |
| `targetUrl` | string | yes | Zapier REST Hook URL |
| `event` | string | yes | Event name |
| `payload` | object | yes | Body POSTed to Zapier |
| `attempt` | number | yes | Attempt count |
| `traceId` | string | yes | Correlation id |
### `verae.zapier.usage` (optional)
| Field | Type | Description |
|-------|------|-------------|
| `tenantId` | string | Tenant |
| `action` | string | `timestamp` \| `verify` \| `status` \| … |
| `amount` | number | Increment |
| `at` | ISO-8601 | Timestamp |
## Consumers
| Name | Stream | Mode | Notes |
|------|--------|------|-------|
| `job-poller` | `ZAPIER_JOBS` | Pull, queue | Nak with delay when still pending |
| `event-webhook-router` | `ZAPIER_EVENTS` | Push/pull | Fan-out to deliver subjects |
| `webhook-deliver` | `ZAPIER_WEBHOOKS` | Pull, queue | HTTP POST with backoff |
| `usage-writer` | `ZAPIER_USAGE` | Optional | Persist metering |
## Ack semantics
| Situation | Action |
|-----------|--------|
| Job still `pending` | `Nak` with delay ≈ `intervalMs` or republish with `attempt+1` |
| Job terminal | Publish event, `Ack` watch message |
| Webhook HTTP 2xx | `Ack` |
| Webhook HTTP 5xx / network | `Nak` / redelivery until `max_deliver` |
| Poison message | Term after max_deliver; write DLQ log with `DEBUG_VERAE=webhooks` |
## Security rules
1. Prefer `tokenRef` over embedding Verae JWTs in messages.
2. NATS must use private network + auth (Phase 15: mTLS).
3. Debug logs must redact tokens and `targetUrl` query secrets if any.
4. Treat `targetUrl` as untrusted egress (timeouts, size limits, SSRF allowlist later).

132
docs/02-architecture/overview.md Normal file
View file

@ -0,0 +1,132 @@
# Architecture Overview
## Purpose
Connect Zapier automations to Verae blockchain timestamping without:
- exposing raw Verae JWTs to end users,
- requiring Zapier to poll async jobs,
- coupling billing/plans to the core timestamping API,
- running multi-instance middleware with in-memory only job queues.
## Components
### 1. `verae-zapier` (Zapier Platform CLI app)
**Runs on:** Zapier’s cloud when a Zap step executes.
**Responsibilities:**
- Custom auth field `api_key` (middleware-issued `zmw_…` keys).
- Map Zapier actions/searches/triggers to middleware HTTPS routes.
- Attach `Authorization: Bearer <api_key>` on every request.
- Translate middleware `402` / `403` into user-visible Zapier errors.
**Does not:**
- Call `api.veraetime.net` directly.
- Speak NATS.
- Enforce plan quotas (middleware does).
### 2. `verae-zapier-middleware` HTTP edge
**Runs on:** Your infrastructure (public HTTPS).
**Responsibilities:**
- Tenant identity and API keys.
- Auth bridge: API key → Verae login → JWT (server-side).
- Entitlement checks and usage metering.
- Synchronous API surface under `/zapier/v1/*`.
- REST Hook subscribe/unsubscribe storage.
- Publish async work to NATS when `NATS_ENABLED=true`.
### 3. NATS + JetStream
**Runs on:** Private network with middleware.
**Responsibilities:**
- Durable work queue for job status polling.
- Event stream for terminal job states.
- Work queue for Zapier webhook HTTP delivery with retries.
### 4. Workers
**Runs on:** Same deploy as middleware or separate worker processes.
| Worker | Consumes | Calls |
|--------|----------|-------|
| Job poller | `verae.zapier.jobs.watch` | `GET /api/status/{jobId}` on Verae |
| Event router | `verae.zapier.jobs.events` | Enqueues webhook deliveries |
| Webhook deliver | `verae.zapier.webhooks.deliver` | `POST` Zapier `targetUrl` |
### 5. `api.veraetime.net` (Verae Timestamping Service)
**Source of truth** for login, timestamp jobs, status, and verification.
OpenAPI: production Swagger / `Verae-Swagger.yaml`.
## Request flows
### A. Create Timestamp (async)
```text
Zapier → POST /zapier/v1/timestamp
Middleware: authenticate, checkEntitlement, POST /api/timestamp
Middleware: publish jobs.watch → return 202 { jobId }
Worker: poll status until terminal → publish jobs.events
Event router: match webhooks → publish webhooks.deliver
Webhook worker: POST hooks.zapier.com/...
```
### B. Create Timestamp and Wait
```text
Zapier → POST /zapier/v1/timestamp/wait
Middleware: create + wait for jobs.events (or in-process wait if NATS off)
→ return StatusResponse (completed/failed) or pending+jobId on timeout
```
### C. Auth connection test
```text
Zapier → GET /zapier/v1/auth/me Authorization: Bearer zmw_…
Middleware: resolve API key → tenant → optional validate Verae token
→ { tenantId, plan, usage, ... }
```
## Feature flags
| Flag | Effect |
|------|--------|
| `NATS_ENABLED=false` | In-process job poller; still full HTTP API (Phase 6 path) |
| `NATS_ENABLED=true` | JetStream workers; no in-process poller |
| `MOCK_VERAE=true` | No live Verae; deterministic mock jobs for tests |
| `DEBUG_VERAE=…` | Runtime failure tracing (see debugging.md) |
## Security boundaries
```text
Public Internet
├─ Zapier → Middleware HTTPS only
└─ Middleware → Zapier webhook HTTPS only
Private
├─ Middleware ↔ NATS
└─ Middleware/Workers → api.veraetime.net HTTPS
```
Never expose NATS ports to the public internet.
## Scaling model
- **HTTP edge:** scale replicas behind a load balancer (stateless except shared store).
- **NATS consumers:** queue groups — adding workers increases poll/deliver throughput.
- **Store:** file JSON is MVP single-node; Postgres/Redis required for multi-node (Phase 15).
## Related documents
- [nats-subjects.md](nats-subjects.md) — subjects, streams, payload schemas
- [../plans/phase-gates.md](../plans/phase-gates.md) — test gates
- [../../TODO.md](../../TODO.md) — full implementation order