master-zapier-plan-draft/research/zapier/PLATFORM-REFERENCE.md
George Lambert b4150c8250 Milestone 0: import zappier billing, Verae middleware, and Zapier research
Compose-ready workspace: packages/zappier (rate card, portal, Stripe),
packages/verae-zapier-middleware (timestamp + NATS), packages/verae-zapier
(CLI app), vendor/zapier-platform, and research/zapier vendor corpus.

Gate 0 structure checks pass. Product code and research are not yet wired.
2026-09-09 02:37:36 -04:00

374 lines
18 KiB
Markdown
Raw Permalink Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# START HERE — Grok playbook for any Zapier work
**Grok must begin every Zapier task by loading this document, or the same playbook in Mongo:**
```js
db.platform_reference.findOne({ kind: "guide", key: "build-new-connector" })
```
Then route:
| User wants… | Next |
|-------------|------|
| AI client / no-code actions across 9k apps | [MCP-REFERENCE.md](MCP-REFERENCE.md) and `kind: "mcp_function"` |
| **Any Zapier function** (CLI, `z.*`, SDK) | [FUNCTIONS-REFERENCE.md](FUNCTIONS-REFERENCE.md) and `kind: "cli_function" \| "core_function" \| "sdk_function"` (playbook: `guide/zapier-functions`) |
| Code that calls existing Zapier apps | `kind: "sdk_function"` (78 commands + `zapier.runAction` twins) |
| A new directory integration to publish | `cli_function` + `core_function` + `schema_json` + `template` · golden app `scratch/oauth2-typescript` |
| Public REST (Workflow / Actions / Inbox) | `kind: "api_function"` |
| Embed Zapier in a product | docs sections `embed`, `white-label`, `openapi` |
| Skill | `/zapier-build` (`.grok/skills/zapier-build`) |
---
# Zapier platform reference (for building connectors and applications)
Separate from the vendor catalog (`apps`, `templates`, `help_articles`). This is the **developer toolkit** Grok should read before writing a new Zapier integration, SDK client, MCP install, or embed.
**Mongo collection:** `zapier.platform_reference` on NS1 (~1,507 documents, text index `ref_text`).
**Local clones:** `repos/` (16 official GitHub repos, shallow).
**Local dump:** `raw/platform-reference.jsonl` (~10 MB).
**Official docs dump:** NS1 `raw/docs/` (409 markdown pages) plus `raw/site-extras/` and `raw/openapi/`.
Query after `./scripts/mongo-tunnel.sh` and `source ~/.mcp-env`:
```js
// mongosh "$ZAPIER_MONGO_URI"
use zapier
db.platform_reference.find({ kind: "cli_command", key: "init" })
db.platform_reference.find({ $text: { $search: "oauth2 refreshAccessToken" } }, { score: { $meta: "textScore" } }).sort({ score: { $meta: "textScore" } })
db.platform_reference.find({ kind: "core_function" })
db.platform_reference.findOne({ kind: "guide", key: "build-new-connector" })
```
Reload:
```bash
ssh ns1 'cd /home/marchon/zapier-research && \
export ZAPIER_RESEARCH_ROOT=/home/marchon/zapier-research && \
export ZAPIER_MONGO_URI="mongodb://zapier:$(cat ~/.config/zapier-mongo/root.pass)@127.0.0.1:27017/zapier?authSource=admin" && \
.venv/bin/python scripts/ingest-platform-reference.py'
```
---
## Which surface to use
| Goal | Surface | Package / repo | Collection `kind` |
|------|---------|----------------|-------------------|
| Publish an app to the Zapier directory | **Platform CLI** (or Platform UI) | `zapier-platform-cli` + `zapier-platform-core``zapier/zapier-platform` | `cli_command`, `core_function`, `schema_type`, `example_app` |
| Call existing Zapier apps from code | **Zapier SDK** | `@zapier/zapier-sdk` / `@zapier/zapier-sdk-cli``zapier/sdk` | `sdk_command`, `official_doc` section `sdk` |
| Connect an AI client with no code | **Zapier MCP** | hosted `mcp.zapier.com` + `zapier/zapier-mcp` plugins | `mcp_function` (17 tools) + [MCP-REFERENCE.md](MCP-REFERENCE.md) |
| Route “install Zapier” for an agent | **install-zapier** | `npx @zapier/install-zapier``zapier/install-zapier` | `skill`, `repo` |
| Prototype a local connector artifact | **Connectors** (not production) | `zapier/connectors` | `connector` |
| Embed Zapier in your product | Powered by Zapier / White Label | docs + OpenAPI | `official_doc` `embed` / `white-label`, `openapi` |
Do **not** mix `zapier-platform …` (build integrations) with `zapier-sdk …` (consume integrations).
Do **not** recommend retired AI Actions / NLA.
Do **not** quote “9,000+ apps” for Connectors (prototype, ~29 apps).
Start from `kind: "guide", key: "build-new-connector"` in Mongo.
---
## Collection map
| `kind` | n | What it is |
|--------|--:|------------|
| `official_doc` | 409 | Every `docs.zapier.com` page (markdown), tagged by `section` |
| `repo_doc` | 759 | Markdown from the cloned GitHub repos |
| `skill` | 63 | `SKILL.md` files (SDK, install-zapier, connectors, MCP plugins) |
| `schema_type` | 63 | `AppSchema`, `AuthenticationOAuth2ConfigSchema`, `TriggerSchema`, … |
| `cli_command` | 40 | `zapier-platform` commands (full usage, flags, examples) |
| `example_app` | 31 | Official templates under `zapier-platform/example-apps/*` |
| `connector` | 29 | Prototype connectors (Sheets, Notion, Linear, …) |
| `core_function` | 22 | `z.request`, `z.errors.*`, cache, cursor, dehydrate, … |
| `sdk_command` | 20 | `zapier-sdk` / SDK CLI: `list-apps`, `run-action`, … |
| `site_extra` | 22 | Leftover public zapier.com / developer surfaces + llms.txt |
| `repo` | 16 | Cloned repo index + README |
| `core_type` | 9 | `Bundle`, `ZObject`, perform signatures, middleware |
| `openapi` | 5 | Actions, Connections, Trigger Inbox, Promotions, Workflow API schema |
| `mcp_function` | 17 | Hosted Zapier MCP meta-tools (typed I/O, internals, examples) |
| `cli_function` | 40 | `zapier-platform` commands (MCP-style: typed flags, internals, examples) |
| `sdk_function` | 78 | `zapier-sdk` CLI + `zapier.camelCase` TypeScript twins |
| `api_function` | 55 | Workflow / Actions / Connections / Trigger Inbox / Promotions endpoints |
| `platform_news` | 49 | Platform CLI/core changelog 20252026 |
| `template` | 32 | Full example-app sources + golden `scratch/oauth2-typescript` |
| `schema_json` | 2 | `exported-schema.json` + official `definition.json` |
| `guide` | 4 | **Start:** `build-new-connector`. Then `zapier-mcp`, `zapier-functions`, `coding-set` |
`official_doc.section` breakdown: integration-builder 200, embed 78, api-reference 53, mcp 44, white-label 21, sdk 10, plus install/connectors.
---
## Platform CLI (`zapier-platform`)
Install: `npm install -g zapier-platform-cli`
Deprecated alias: `zapier` (same commands). Prefer `zapier-platform`.
### Scaffold → ship
```bash
zapier-platform login
zapier-platform init my-app --template oauth2 --language typescript
cd my-app && npm install
zapier-platform scaffold trigger contact
zapier-platform scaffold create contact
zapier-platform validate
zapier-platform invoke auth start
zapier-platform invoke auth test
zapier-platform invoke trigger new_contact
zapier-platform test
zapier-platform register "My App"
zapier-platform push
zapier-platform promote 1.0.0
```
`init --template` values: `basic-auth`, `callback`, `custom-auth`, `digest-auth`, `dynamic-dropdown`, `files`, `line-items`, `minimal`, `oauth1-trello`, `oauth2`, `openai`, `search-or-create`, `session-auth`. Add `--language typescript` and/or `--module esm`.
### Full command list
| Command | Usage | When |
|---------|-------|------|
| `login` / `logout` | `zapier-platform login` | Auth the CLI to developer.zapier.com |
| `init` | `init PATH` | New project from a template |
| `convert` | `convert PATH` | Visual Builder → CLI (`-i` integration id, `-v` version) |
| `scaffold` | `scaffold ACTIONTYPE NOUN` | Add `trigger\|search\|create\|resource` |
| `link` | `link` | Attach cwd to an existing integration |
| `register` | `register [TITLE]` | Create the integration on Zapier |
| `validate` | `validate` | Schema + style checks (gates push) |
| `test` | `test` | Jest via platform test runner |
| `invoke` | `invoke [TYPE] [KEY]` | Local / relay (`-a` auth id) / remote (`-r`) |
| `describe` | `describe` | Print triggers/searches/creates as Zapier sees them |
| `build` | `build` | `build/build.zip` + `source.zip` |
| `upload` | `upload` | Upload a zip without full push |
| `push` | `push` | Build + upload the `package.json` version (versions must be sequential) |
| `pull` | `pull` | Pull remote definition |
| `versions` | `versions` | List deployed versions |
| `promote` | `promote VERSION` | Make this the public default |
| `migrate` | `migrate FROM TO [PERCENT]` | Move users (same major) |
| `deprecate` | `deprecate VERSION DATE` | DATE ≥ 3 weeks out; users emailed at T-14d |
| `canary:create` | `canary:create FROM TO -p PCT -d SECS` | Temporary traffic split |
| `canary:list` / `canary:delete` | | Inspect / cancel canary |
| `env:get` / `env:set` / `env:unset` | `env:set 1.0.0 KEY=val` | Per-version secrets (`CLIENT_ID`, …) |
| `logs` | `logs` | HTTP / console logs |
| `history` | `history` | Audit trail |
| `integrations` (`apps`) | | Integrations you admin |
| `team:add` / `get` / `remove` | roles: `admin`, `collaborator`, `subscriber` | |
| `users:add` / `get` / `links` / `remove` | Invite testers to a private version | |
| `delete:version` / `delete:integration` | Only if no users/Zaps | |
| `jobs` | Background job status | |
| `legacy` | Legacy Web Builder helpers | |
| `analytics` | CLI telemetry opt-in/out | |
Full flags and examples live in `kind: "cli_command"`. Source: `repos/zapier-platform/packages/cli/docs/cli.md`.
`invoke` modes: local (default, `.env` / `authData_*`), relay (`-a <authentication-id>`), remote (`-r`). Local limitations: no hook subscribe, hydration, file upload, buffered creates, search-or-create.
---
## Runtime functions (`z`, first argument to every `perform`)
Every trigger/action/search is `(z: ZObject, bundle: Bundle) => …`.
| Function | Signature | Notes |
|----------|-----------|--------|
| `z.request` | `(url, options?) → Promise<HttpResponse>` | **Always use this**, not axios/fetch. Auth middleware + logs. `json`, `form`, `params`, `raw`, `skipThrowForStatus`. |
| `z.console.*` | `log\|info\|warn\|error` | Shows up in `zapier-platform logs` |
| `z.dehydrate` | `(func, inputData?, ttl?) → string` | Lazy object pointer |
| `z.dehydrateFile` | same | Lazy file |
| `z.stashFile` | `(buf\|stream\|url, …) → string` | Public short-lived file URL |
| `z.cursor.get` / `set` | polling cursor | Persist last-seen id/ts |
| `z.generateCallbackUrl` | `() → string` | Resume long creates (`performResume`) |
| `z.hash` | `(alg, data, enc?, inEnc?)` | Usually `sha256` |
| `z.JSON.parse` / `stringify` | | Parse throws a clean user error |
| `z.cache.get/set/delete` | per-auth JSON cache | `set(key, val, ttl?, scope?, nx?)` |
| `z.errors.Error` | `(message, code?, status?)` | User-visible failure |
| `z.errors.HaltedError` | | Stop run **without** failing the Zap |
| `z.errors.RefreshAuthError` | | 401 → refresh OAuth/session and retry |
| `z.errors.ExpiredAuthError` | | Dead connection; user must reconnect |
| `z.errors.ThrottledError` | `(message, delaySeconds?)` | Retry-After |
| `z.errors.ResponseError` | `(response)` | Usually auto-thrown by `z.request` |
| `createAppTester` | from `zapier-platform-core` | Unit-test performs |
| `zapier.tools.env.inject` | load `.env` | Tests |
Return rules:
- **Trigger / search** → array of objects (search may envelope `{results, paging_token}`).
- **Create** → **one object** (not an array).
- Polling items need `id` (or `primary: true` output fields).
- REST Hook: `operation.type = 'hook'` plus `performSubscribe`, `performUnsubscribe`, `performList`.
`bundle` fields: `authData`, `inputData`, `inputDataRaw`, `meta` (`isLoadingSample`, `isFillingDynamicDropdown`, `isPopulatingDedupe`, `isBulkRead`, `limit`, `page`, `timezone`, `paging_token`, `withSearch`), plus hook extras `cleanedRequest`, `rawRequest`, `targetUrl`, `subscribeData`.
See `kind: "core_function"` and `kind: "core_type"`.
---
## App definition and schema
Minimum `index.js` / `src/index.ts`:
```js
module.exports = {
version: require('./package.json').version,
platformVersion: require('zapier-platform-core').version,
authentication: { /* type + config + test + connectionLabel */ },
beforeRequest: [], // (request, z, bundle) => request
afterResponse: [], // (response, z, bundle) => response
hydrators: {},
triggers: {},
searches: {},
creates: {},
resources: {},
};
```
Canonical types in `kind: "schema_type"` (generated from `zapier-platform-schema` 19.x):
- App: `AppSchema`, `AppFlagsSchema`, `VersionSchema`
- Auth: `AuthenticationSchema` + `AuthenticationOAuth2ConfigSchema`, `…SessionConfigSchema`, `…BasicConfigSchema`, `…DigestConfigSchema`, `…CustomConfigSchema`, `…OAuth1ConfigSchema`
- Operations: `TriggerSchema`, `CreateSchema`, `SearchSchema`, `SearchOrCreateSchema`, `ResourceSchema`
- HTTP: `RequestSchema`, `RedirectRequestSchema`
- Fields: `PlainInputFieldSchema`, `PlainOutputFieldSchema`, `FieldChoicesSchema`, `InputFieldGroupsSchema`
- Runtime extras: `HydratorsSchema`, `MiddlewaresSchema`, `ThrottleObjectSchema`, `LockObjectSchema`, `BufferConfigSchema`
Full generated doc: `repos/zapier-platform/packages/schema/docs/build/schema.md`.
### Auth recipes
| `authentication.type` | Example app | Mechanism |
|-----------------------|-------------|-----------|
| `oauth2` | `example-apps/oauth2` | `authorizeUrl`, `getAccessToken`, `refreshAccessToken`, `autoRefresh` |
| `oauth1` | `oauth1-trello`, `oauth1-twitter`, `oauth1-tumblr` | HMAC signed requests |
| `session` | `session-auth` | `sessionConfig.perform` exchanges creds for a token in `authData` |
| `basic` | `basic-auth` | Username/password; platform sets header |
| `digest` | `digest-auth` | Challenge-response |
| `custom` | `custom-auth` | API key / header / query; attach in `beforeRequest` |
On HTTP 401: `throw new z.errors.RefreshAuthError()` if refreshable, else `ExpiredAuthError()`.
### Example apps (clone + `init --template`)
`babel`, `basic-auth` (+ TS), `callback`, `create`, `custom-auth` (+ TS), `digest-auth` (+ TS), `dynamic-dropdown`, `files`, `github`, `line-items`, `middleware`, `minimal`, `minimal-esm`, `oauth1-trello` (+ TS), `oauth1-tumblr`, `oauth1-twitter`, `oauth2` (+ TS), `onedrive`, `openai`, `resource`, `rest-hooks`, `search`, `search-or-create`, `session-auth` (+ TS), `trigger`.
Each is `kind: "example_app"` with README + file list.
---
## Zapier SDK CLI (consume apps — different package)
`npx @zapier/zapier-sdk-cli` / `zapier-sdk`. Docs: `official_doc` keys `sdk/cli-reference`, `sdk/reference`, `sdk/quickstart`.
| Command | Purpose |
|---------|---------|
| `signup` / `login` / `logout` | Account + SDK credentials |
| `list-apps` | Discover apps at runtime (do not hardcode keys) |
| `get-app` | One app |
| `list-actions` / `get-action` | Actions for an app |
| `list-action-input-fields` / `…-schema` / `…-choices` | Input form |
| `run-action` | Execute and wait |
| `create-action-run` / `get-action-run` | Async run |
| `create-connection` / `find-first-connection` / `find-unique-connection` | User connections |
| `get-profile` | Who am I |
| `create-client-credentials` / `list-` / `delete-` | Deploy without browser login |
Source repo: `repos/sdk` (includes `skills/zapier-sdk/SKILL.md`).
---
## MCP, install, connectors
Full typed reference: [MCP-REFERENCE.md](MCP-REFERENCE.md) and `kind: "mcp_function"`.
```js
db.platform_reference.findOne({ kind: "guide", key: "zapier-mcp" })
db.platform_reference.find({ kind: "mcp_function" }).sort({ key: 1 })
```
- **MCP docs:** `section: "mcp"` (quickstart, auth, 20+ client guides, troubleshooting).
- **Install router:** `docs.zapier.com/install` + `npx @zapier/install-zapier` (`repos/install-zapier`).
- **Connectors (prototype):** `repos/connectors/apps/{algolia,alpaca,clay,dataforseo,discord,dropbox,elevenlabs,firecrawl,gitlab,google-*,harvest,heygen,linear,microsoft-*,notion,resend,runway,telegram,trello,youtube}`.
---
## Public APIs (OpenAPI)
`kind: "openapi"`:
| Key | URL |
|-----|-----|
| `actions.yaml` | Stored Actions API |
| `connections.yaml` | Connection webhooks (White Label) |
| `trigger-inbox.yaml` | Trigger Inbox |
| `promotions-openapi.yaml` | Promotions / sponsored automation |
| `workflow-api-schema.json` | Live `https://api.zapier.com/schema` (Powered by Zapier Workflow API) |
Rendered endpoint docs: `section: "api-reference"` and `section: "embed"`.
---
## Cloned GitHub repos (`repos/`)
| Repo | Role |
|------|------|
| `zapier-platform` | Canonical monorepo: CLI, core, schema, 31 example apps |
| `zapier-platform-cli` / `-core` / `-schema` | Archived standalone copies; use the monorepo |
| `sdk` | Agent-readable SDK docs + skill |
| `zapier-mcp` | Hosted MCP plugin manifests |
| `connectors` | Prototype local connectors |
| `install-zapier` | Agent installer |
| `agent-skills` / `marketplace` / `gtm-cheat-codes` | Agent skills / marketplace listing |
| `visual-builder` | Archived Platform UI tutorial |
| `resthooks` | REST Hooks spec (instant triggers) |
| `zapier-platform-example-app-{github,oauth2,minimal}` | Standalone example mirrors |
Re-clone: `ROOT=/path scripts/clone-zapier-repos.sh`.
---
## What was left on zapier.com (and what is not public)
Pulled in this pass:
- Full `docs.zapier.com` sitemap (409 pages) + `llms.txt` / `llms-full.txt` (2.5 MB)
- Root `zapier.com` sitemap developer surfaces: developer-platform, embed-tools, integrations, partner-program, MCP, Functions (Code by Zapier), custom connections, opensource, agents
- `developer.zapier.com` (shell → docs), legacy `platform.zapier.com` / v2 docs HTML
- OpenAPI specs above
Already in other collections: 9,986 apps, 332,441 templates, 1,272 help articles, commercial/API-doc/auth scrapes.
**Not available publicly (do not invent):**
- Partnership / sales emails
- Private REST schemas of the 9,986 vendor APIs
- Auth type for ~8k apps that never published help/API-doc signals
- Internal Zapier implementation IDs mapping to OAuth vs API key
---
## Grok query recipes
```js
// Build a new OAuth2 integration
db.platform_reference.find({
$or: [
{ kind: "guide" },
{ kind: "cli_command", key: { $in: ["init", "scaffold", "invoke", "push", "validate"] } },
{ kind: "example_app", key: /oauth2/ },
{ kind: "schema_type", key: /AuthenticationOAuth2|AppSchema|TriggerSchema|CreateSchema/ },
{ kind: "core_function", key: /^z\.(request|errors)/ },
{ kind: "official_doc", key: /oauth|cli-tutorial|core/ },
]
})
// Instant (REST Hook) trigger
db.platform_reference.find({
$or: [
{ key: /hook|rest-hooks/i },
{ kind: "schema_type", key: /Hook|Trigger/ },
]
})
// Embed Workflow API
db.platform_reference.find({ section: { $in: ["embed", "api-reference", "white-label"] } })
```