Milestone 0: import zappier billing, Verae middleware, and Zapier research

Compose-ready workspace: packages/zappier (rate card, portal, Stripe), packages/verae-zapier-middleware (timestamp + NATS), packages/verae-zapier (CLI app), vendor/zapier-platform, and research/zapier vendor corpus. Gate 0 structure checks pass. Product code and research are not yet wired.
2026-09-09 02:37:36 -04:00 · 2026-09-09 02:37:36 -04:00 · b4150c8250
commit b4150c8250
1364 changed files with 6814366 additions and 0 deletions
--- a/research/zapier/PLATFORM-REFERENCE.md
+++ b/research/zapier/PLATFORM-REFERENCE.md
@ -0,0 +1,374 @@
+# START HERE — Grok playbook for any Zapier work
+
+**Grok must begin every Zapier task by loading this document, or the same playbook in Mongo:**
+
+```js
+db.platform_reference.findOne({ kind: "guide", key: "build-new-connector" })
+```
+
+Then route:
+
+| User wants… | Next |
+|-------------|------|
+| AI client / no-code actions across 9k apps | [MCP-REFERENCE.md](MCP-REFERENCE.md) and `kind: "mcp_function"` |
+| **Any Zapier function** (CLI, `z.*`, SDK) | [FUNCTIONS-REFERENCE.md](FUNCTIONS-REFERENCE.md) and `kind: "cli_function" \| "core_function" \| "sdk_function"` (playbook: `guide/zapier-functions`) |
+| Code that calls existing Zapier apps | `kind: "sdk_function"` (78 commands + `zapier.runAction` twins) |
+| A new directory integration to publish | `cli_function` + `core_function` + `schema_json` + `template` · golden app `scratch/oauth2-typescript` |
+| Public REST (Workflow / Actions / Inbox) | `kind: "api_function"` |
+| Embed Zapier in a product | docs sections `embed`, `white-label`, `openapi` |
+| Skill | `/zapier-build` (`.grok/skills/zapier-build`) |
+
+---
+
+# Zapier platform reference (for building connectors and applications)
+
+Separate from the vendor catalog (`apps`, `templates`, `help_articles`). This is the **developer toolkit** Grok should read before writing a new Zapier integration, SDK client, MCP install, or embed.
+
+**Mongo collection:** `zapier.platform_reference` on NS1 (~1,507 documents, text index `ref_text`).  
+**Local clones:** `repos/` (16 official GitHub repos, shallow).  
+**Local dump:** `raw/platform-reference.jsonl` (~10 MB).  
+**Official docs dump:** NS1 `raw/docs/` (409 markdown pages) plus `raw/site-extras/` and `raw/openapi/`.
+
+Query after `./scripts/mongo-tunnel.sh` and `source ~/.mcp-env`:
+
+```js
+// mongosh "$ZAPIER_MONGO_URI"
+use zapier
+db.platform_reference.find({ kind: "cli_command", key: "init" })
+db.platform_reference.find({ $text: { $search: "oauth2 refreshAccessToken" } }, { score: { $meta: "textScore" } }).sort({ score: { $meta: "textScore" } })
+db.platform_reference.find({ kind: "core_function" })
+db.platform_reference.findOne({ kind: "guide", key: "build-new-connector" })
+```
+
+Reload:
+
+```bash
+ssh ns1 'cd /home/marchon/zapier-research && \
+  export ZAPIER_RESEARCH_ROOT=/home/marchon/zapier-research && \
+  export ZAPIER_MONGO_URI="mongodb://zapier:$(cat ~/.config/zapier-mongo/root.pass)@127.0.0.1:27017/zapier?authSource=admin" && \
+  .venv/bin/python scripts/ingest-platform-reference.py'
+```
+
+---
+
+## Which surface to use
+
+| Goal | Surface | Package / repo | Collection `kind` |
+|------|---------|----------------|-------------------|
+| Publish an app to the Zapier directory | **Platform CLI** (or Platform UI) | `zapier-platform-cli` + `zapier-platform-core` — `zapier/zapier-platform` | `cli_command`, `core_function`, `schema_type`, `example_app` |
+| Call existing Zapier apps from code | **Zapier SDK** | `@zapier/zapier-sdk` / `@zapier/zapier-sdk-cli` — `zapier/sdk` | `sdk_command`, `official_doc` section `sdk` |
+| Connect an AI client with no code | **Zapier MCP** | hosted `mcp.zapier.com` + `zapier/zapier-mcp` plugins | `mcp_function` (17 tools) + [MCP-REFERENCE.md](MCP-REFERENCE.md) |
+| Route “install Zapier” for an agent | **install-zapier** | `npx @zapier/install-zapier` — `zapier/install-zapier` | `skill`, `repo` |
+| Prototype a local connector artifact | **Connectors** (not production) | `zapier/connectors` | `connector` |
+| Embed Zapier in your product | Powered by Zapier / White Label | docs + OpenAPI | `official_doc` `embed` / `white-label`, `openapi` |
+
+Do **not** mix `zapier-platform …` (build integrations) with `zapier-sdk …` (consume integrations).  
+Do **not** recommend retired AI Actions / NLA.  
+Do **not** quote “9,000+ apps” for Connectors (prototype, ~29 apps).
+
+Start from `kind: "guide", key: "build-new-connector"` in Mongo.
+
+---
+
+## Collection map
+
+| `kind` | n | What it is |
+|--------|--:|------------|
+| `official_doc` | 409 | Every `docs.zapier.com` page (markdown), tagged by `section` |
+| `repo_doc` | 759 | Markdown from the cloned GitHub repos |
+| `skill` | 63 | `SKILL.md` files (SDK, install-zapier, connectors, MCP plugins) |
+| `schema_type` | 63 | `AppSchema`, `AuthenticationOAuth2ConfigSchema`, `TriggerSchema`, … |
+| `cli_command` | 40 | `zapier-platform` commands (full usage, flags, examples) |
+| `example_app` | 31 | Official templates under `zapier-platform/example-apps/*` |
+| `connector` | 29 | Prototype connectors (Sheets, Notion, Linear, …) |
+| `core_function` | 22 | `z.request`, `z.errors.*`, cache, cursor, dehydrate, … |
+| `sdk_command` | 20 | `zapier-sdk` / SDK CLI: `list-apps`, `run-action`, … |
+| `site_extra` | 22 | Leftover public zapier.com / developer surfaces + llms.txt |
+| `repo` | 16 | Cloned repo index + README |
+| `core_type` | 9 | `Bundle`, `ZObject`, perform signatures, middleware |
+| `openapi` | 5 | Actions, Connections, Trigger Inbox, Promotions, Workflow API schema |
+| `mcp_function` | 17 | Hosted Zapier MCP meta-tools (typed I/O, internals, examples) |
+| `cli_function` | 40 | `zapier-platform` commands (MCP-style: typed flags, internals, examples) |
+| `sdk_function` | 78 | `zapier-sdk` CLI + `zapier.camelCase` TypeScript twins |
+| `api_function` | 55 | Workflow / Actions / Connections / Trigger Inbox / Promotions endpoints |
+| `platform_news` | 49 | Platform CLI/core changelog 2025–2026 |
+| `template` | 32 | Full example-app sources + golden `scratch/oauth2-typescript` |
+| `schema_json` | 2 | `exported-schema.json` + official `definition.json` |
+| `guide` | 4 | **Start:** `build-new-connector`. Then `zapier-mcp`, `zapier-functions`, `coding-set` |
+
+`official_doc.section` breakdown: integration-builder 200, embed 78, api-reference 53, mcp 44, white-label 21, sdk 10, plus install/connectors.
+
+---
+
+## Platform CLI (`zapier-platform`)
+
+Install: `npm install -g zapier-platform-cli`  
+Deprecated alias: `zapier` (same commands). Prefer `zapier-platform`.
+
+### Scaffold → ship
+
+```bash
+zapier-platform login
+zapier-platform init my-app --template oauth2 --language typescript
+cd my-app && npm install
+zapier-platform scaffold trigger contact
+zapier-platform scaffold create contact
+zapier-platform validate
+zapier-platform invoke auth start
+zapier-platform invoke auth test
+zapier-platform invoke trigger new_contact
+zapier-platform test
+zapier-platform register "My App"
+zapier-platform push
+zapier-platform promote 1.0.0
+```
+
+`init --template` values: `basic-auth`, `callback`, `custom-auth`, `digest-auth`, `dynamic-dropdown`, `files`, `line-items`, `minimal`, `oauth1-trello`, `oauth2`, `openai`, `search-or-create`, `session-auth`. Add `--language typescript` and/or `--module esm`.
+
+### Full command list
+
+| Command | Usage | When |
+|---------|-------|------|
+| `login` / `logout` | `zapier-platform login` | Auth the CLI to developer.zapier.com |
+| `init` | `init PATH` | New project from a template |
+| `convert` | `convert PATH` | Visual Builder → CLI (`-i` integration id, `-v` version) |
+| `scaffold` | `scaffold ACTIONTYPE NOUN` | Add `trigger\|search\|create\|resource` |
+| `link` | `link` | Attach cwd to an existing integration |
+| `register` | `register [TITLE]` | Create the integration on Zapier |
+| `validate` | `validate` | Schema + style checks (gates push) |
+| `test` | `test` | Jest via platform test runner |
+| `invoke` | `invoke [TYPE] [KEY]` | Local / relay (`-a` auth id) / remote (`-r`) |
+| `describe` | `describe` | Print triggers/searches/creates as Zapier sees them |
+| `build` | `build` | `build/build.zip` + `source.zip` |
+| `upload` | `upload` | Upload a zip without full push |
+| `push` | `push` | Build + upload the `package.json` version (versions must be sequential) |
+| `pull` | `pull` | Pull remote definition |
+| `versions` | `versions` | List deployed versions |
+| `promote` | `promote VERSION` | Make this the public default |
+| `migrate` | `migrate FROM TO [PERCENT]` | Move users (same major) |
+| `deprecate` | `deprecate VERSION DATE` | DATE ≥ 3 weeks out; users emailed at T-14d |
+| `canary:create` | `canary:create FROM TO -p PCT -d SECS` | Temporary traffic split |
+| `canary:list` / `canary:delete` | | Inspect / cancel canary |
+| `env:get` / `env:set` / `env:unset` | `env:set 1.0.0 KEY=val` | Per-version secrets (`CLIENT_ID`, …) |
+| `logs` | `logs` | HTTP / console logs |
+| `history` | `history` | Audit trail |
+| `integrations` (`apps`) | | Integrations you admin |
+| `team:add` / `get` / `remove` | roles: `admin`, `collaborator`, `subscriber` | |
+| `users:add` / `get` / `links` / `remove` | Invite testers to a private version | |
+| `delete:version` / `delete:integration` | Only if no users/Zaps | |
+| `jobs` | Background job status | |
+| `legacy` | Legacy Web Builder helpers | |
+| `analytics` | CLI telemetry opt-in/out | |
+
+Full flags and examples live in `kind: "cli_command"`. Source: `repos/zapier-platform/packages/cli/docs/cli.md`.
+
+`invoke` modes: local (default, `.env` / `authData_*`), relay (`-a <authentication-id>`), remote (`-r`). Local limitations: no hook subscribe, hydration, file upload, buffered creates, search-or-create.
+
+---
+
+## Runtime functions (`z`, first argument to every `perform`)
+
+Every trigger/action/search is `(z: ZObject, bundle: Bundle) => …`.
+
+| Function | Signature | Notes |
+|----------|-----------|--------|
+| `z.request` | `(url, options?) → Promise<HttpResponse>` | **Always use this**, not axios/fetch. Auth middleware + logs. `json`, `form`, `params`, `raw`, `skipThrowForStatus`. |
+| `z.console.*` | `log\|info\|warn\|error` | Shows up in `zapier-platform logs` |
+| `z.dehydrate` | `(func, inputData?, ttl?) → string` | Lazy object pointer |
+| `z.dehydrateFile` | same | Lazy file |
+| `z.stashFile` | `(buf\|stream\|url, …) → string` | Public short-lived file URL |
+| `z.cursor.get` / `set` | polling cursor | Persist last-seen id/ts |
+| `z.generateCallbackUrl` | `() → string` | Resume long creates (`performResume`) |
+| `z.hash` | `(alg, data, enc?, inEnc?)` | Usually `sha256` |
+| `z.JSON.parse` / `stringify` | | Parse throws a clean user error |
+| `z.cache.get/set/delete` | per-auth JSON cache | `set(key, val, ttl?, scope?, nx?)` |
+| `z.errors.Error` | `(message, code?, status?)` | User-visible failure |
+| `z.errors.HaltedError` | | Stop run **without** failing the Zap |
+| `z.errors.RefreshAuthError` | | 401 → refresh OAuth/session and retry |
+| `z.errors.ExpiredAuthError` | | Dead connection; user must reconnect |
+| `z.errors.ThrottledError` | `(message, delaySeconds?)` | Retry-After |
+| `z.errors.ResponseError` | `(response)` | Usually auto-thrown by `z.request` |
+| `createAppTester` | from `zapier-platform-core` | Unit-test performs |
+| `zapier.tools.env.inject` | load `.env` | Tests |
+
+Return rules:
+
+- **Trigger / search** → array of objects (search may envelope `{results, paging_token}`).
+- **Create** → **one object** (not an array).
+- Polling items need `id` (or `primary: true` output fields).
+- REST Hook: `operation.type = 'hook'` plus `performSubscribe`, `performUnsubscribe`, `performList`.
+
+`bundle` fields: `authData`, `inputData`, `inputDataRaw`, `meta` (`isLoadingSample`, `isFillingDynamicDropdown`, `isPopulatingDedupe`, `isBulkRead`, `limit`, `page`, `timezone`, `paging_token`, `withSearch`), plus hook extras `cleanedRequest`, `rawRequest`, `targetUrl`, `subscribeData`.
+
+See `kind: "core_function"` and `kind: "core_type"`.
+
+---
+
+## App definition and schema
+
+Minimum `index.js` / `src/index.ts`:
+
+```js
+module.exports = {
+  version: require('./package.json').version,
+  platformVersion: require('zapier-platform-core').version,
+  authentication: { /* type + config + test + connectionLabel */ },
+  beforeRequest: [],  // (request, z, bundle) => request
+  afterResponse: [],  // (response, z, bundle) => response
+  hydrators: {},
+  triggers: {},
+  searches: {},
+  creates: {},
+  resources: {},
+};
+```
+
+Canonical types in `kind: "schema_type"` (generated from `zapier-platform-schema` 19.x):
+
+- App: `AppSchema`, `AppFlagsSchema`, `VersionSchema`
+- Auth: `AuthenticationSchema` + `AuthenticationOAuth2ConfigSchema`, `…SessionConfigSchema`, `…BasicConfigSchema`, `…DigestConfigSchema`, `…CustomConfigSchema`, `…OAuth1ConfigSchema`
+- Operations: `TriggerSchema`, `CreateSchema`, `SearchSchema`, `SearchOrCreateSchema`, `ResourceSchema`
+- HTTP: `RequestSchema`, `RedirectRequestSchema`
+- Fields: `PlainInputFieldSchema`, `PlainOutputFieldSchema`, `FieldChoicesSchema`, `InputFieldGroupsSchema`
+- Runtime extras: `HydratorsSchema`, `MiddlewaresSchema`, `ThrottleObjectSchema`, `LockObjectSchema`, `BufferConfigSchema`
+
+Full generated doc: `repos/zapier-platform/packages/schema/docs/build/schema.md`.
+
+### Auth recipes
+
+| `authentication.type` | Example app | Mechanism |
+|-----------------------|-------------|-----------|
+| `oauth2` | `example-apps/oauth2` | `authorizeUrl`, `getAccessToken`, `refreshAccessToken`, `autoRefresh` |
+| `oauth1` | `oauth1-trello`, `oauth1-twitter`, `oauth1-tumblr` | HMAC signed requests |
+| `session` | `session-auth` | `sessionConfig.perform` exchanges creds for a token in `authData` |
+| `basic` | `basic-auth` | Username/password; platform sets header |
+| `digest` | `digest-auth` | Challenge-response |
+| `custom` | `custom-auth` | API key / header / query; attach in `beforeRequest` |
+
+On HTTP 401: `throw new z.errors.RefreshAuthError()` if refreshable, else `ExpiredAuthError()`.
+
+### Example apps (clone + `init --template`)
+
+`babel`, `basic-auth` (+ TS), `callback`, `create`, `custom-auth` (+ TS), `digest-auth` (+ TS), `dynamic-dropdown`, `files`, `github`, `line-items`, `middleware`, `minimal`, `minimal-esm`, `oauth1-trello` (+ TS), `oauth1-tumblr`, `oauth1-twitter`, `oauth2` (+ TS), `onedrive`, `openai`, `resource`, `rest-hooks`, `search`, `search-or-create`, `session-auth` (+ TS), `trigger`.
+
+Each is `kind: "example_app"` with README + file list.
+
+---
+
+## Zapier SDK CLI (consume apps — different package)
+
+`npx @zapier/zapier-sdk-cli` / `zapier-sdk`. Docs: `official_doc` keys `sdk/cli-reference`, `sdk/reference`, `sdk/quickstart`.
+
+| Command | Purpose |
+|---------|---------|
+| `signup` / `login` / `logout` | Account + SDK credentials |
+| `list-apps` | Discover apps at runtime (do not hardcode keys) |
+| `get-app` | One app |
+| `list-actions` / `get-action` | Actions for an app |
+| `list-action-input-fields` / `…-schema` / `…-choices` | Input form |
+| `run-action` | Execute and wait |
+| `create-action-run` / `get-action-run` | Async run |
+| `create-connection` / `find-first-connection` / `find-unique-connection` | User connections |
+| `get-profile` | Who am I |
+| `create-client-credentials` / `list-` / `delete-` | Deploy without browser login |
+
+Source repo: `repos/sdk` (includes `skills/zapier-sdk/SKILL.md`).
+
+---
+
+## MCP, install, connectors
+
+Full typed reference: [MCP-REFERENCE.md](MCP-REFERENCE.md) and `kind: "mcp_function"`.
+
+```js
+db.platform_reference.findOne({ kind: "guide", key: "zapier-mcp" })
+db.platform_reference.find({ kind: "mcp_function" }).sort({ key: 1 })
+```
+
+- **MCP docs:** `section: "mcp"` (quickstart, auth, 20+ client guides, troubleshooting).
+- **Install router:** `docs.zapier.com/install` + `npx @zapier/install-zapier` (`repos/install-zapier`).
+- **Connectors (prototype):** `repos/connectors/apps/{algolia,alpaca,clay,dataforseo,discord,dropbox,elevenlabs,firecrawl,gitlab,google-*,harvest,heygen,linear,microsoft-*,notion,resend,runway,telegram,trello,youtube}`.
+
+---
+
+## Public APIs (OpenAPI)
+
+`kind: "openapi"`:
+
+| Key | URL |
+|-----|-----|
+| `actions.yaml` | Stored Actions API |
+| `connections.yaml` | Connection webhooks (White Label) |
+| `trigger-inbox.yaml` | Trigger Inbox |
+| `promotions-openapi.yaml` | Promotions / sponsored automation |
+| `workflow-api-schema.json` | Live `https://api.zapier.com/schema` (Powered by Zapier Workflow API) |
+
+Rendered endpoint docs: `section: "api-reference"` and `section: "embed"`.
+
+---
+
+## Cloned GitHub repos (`repos/`)
+
+| Repo | Role |
+|------|------|
+| `zapier-platform` | Canonical monorepo: CLI, core, schema, 31 example apps |
+| `zapier-platform-cli` / `-core` / `-schema` | Archived standalone copies; use the monorepo |
+| `sdk` | Agent-readable SDK docs + skill |
+| `zapier-mcp` | Hosted MCP plugin manifests |
+| `connectors` | Prototype local connectors |
+| `install-zapier` | Agent installer |
+| `agent-skills` / `marketplace` / `gtm-cheat-codes` | Agent skills / marketplace listing |
+| `visual-builder` | Archived Platform UI tutorial |
+| `resthooks` | REST Hooks spec (instant triggers) |
+| `zapier-platform-example-app-{github,oauth2,minimal}` | Standalone example mirrors |
+
+Re-clone: `ROOT=/path scripts/clone-zapier-repos.sh`.
+
+---
+
+## What was left on zapier.com (and what is not public)
+
+Pulled in this pass:
+
+- Full `docs.zapier.com` sitemap (409 pages) + `llms.txt` / `llms-full.txt` (2.5 MB)
+- Root `zapier.com` sitemap developer surfaces: developer-platform, embed-tools, integrations, partner-program, MCP, Functions (Code by Zapier), custom connections, opensource, agents
+- `developer.zapier.com` (shell → docs), legacy `platform.zapier.com` / v2 docs HTML
+- OpenAPI specs above
+
+Already in other collections: 9,986 apps, 332,441 templates, 1,272 help articles, commercial/API-doc/auth scrapes.
+
+**Not available publicly (do not invent):**
+
+- Partnership / sales emails
+- Private REST schemas of the 9,986 vendor APIs
+- Auth type for ~8k apps that never published help/API-doc signals
+- Internal Zapier implementation IDs mapping to OAuth vs API key
+
+---
+
+## Grok query recipes
+
+```js
+// Build a new OAuth2 integration
+db.platform_reference.find({
+  $or: [
+    { kind: "guide" },
+    { kind: "cli_command", key: { $in: ["init", "scaffold", "invoke", "push", "validate"] } },
+    { kind: "example_app", key: /oauth2/ },
+    { kind: "schema_type", key: /AuthenticationOAuth2|AppSchema|TriggerSchema|CreateSchema/ },
+    { kind: "core_function", key: /^z\.(request|errors)/ },
+    { kind: "official_doc", key: /oauth|cli-tutorial|core/ },
+  ]
+})
+
+// Instant (REST Hook) trigger
+db.platform_reference.find({
+  $or: [
+    { key: /hook|rest-hooks/i },
+    { kind: "schema_type", key: /Hook|Trigger/ },
+  ]
+})
+
+// Embed Workflow API
+db.platform_reference.find({ section: { $in: ["embed", "api-reference", "white-label"] } })
+```