Visual pipelines

A visual pipeline is a workflow you build on a canvas: a trigger, then steps that call connections, run Python, map data, branch, loop, wait for approval or call other pipelines. It is a workspace resource, like a QuickFlow. It needs no pipeline project, environment or deployment.

Use a visual pipeline for event-driven work across apps and data. Use a pipeline project when the work lives in a Git repository, and a QuickFlow for a plain source-to-destination sync.

Pipelines and folders

Visual pipelines are listed under VegaFlow › Visual pipelines. Folders organise the list:

• Every workspace has a Default folder. It cannot be renamed, moved or deleted.
• Folders nest up to three levels. Deleting a folder moves its pipelines and subfolders to Default.
• Folders carry no permissions. Members who can create visual pipelines can manage folders. Moving a pipeline needs update on that pipeline.

Pipeline names are unique within a workspace. Deleting a pipeline archives it: it leaves the list, its triggers stop, and its name can be reused. Its runs stay readable to people who hold read on it until retention removes them.

A pipeline can be exported as JSON and imported into a workspace. The export holds the graph and the portable parts of its run settings.

Draft, compile and publish

The editor works on the pipeline's draft. Saving keeps the draft; Compile checks it and lists any issues above the canvas. Publish turns the compiled draft into a new numbered version. Versions are immutable: runs started earlier keep the version they started with.

Runs, schedules, webhooks and app events always use the active published version. Editing the draft changes nothing until you publish.

Run settings

Each pipeline has its own run settings, edited on its Settings page:

Saving checks that you may use each cluster, connection and secret and act as the service principal. Every save creates a new immutable snapshot. A run keeps the snapshot it started with, including when it is retried, so changing settings never changes a run that already started.

A pipeline cannot run until its run settings are saved.

Triggers

The trigger is the first step on the canvas and is published with the version. Automatic triggers fire only while the pipeline is Active: published, with its Active switch on. Manual runs and calls from other pipelines are allowed whenever a version is published.

Pipelines can also be started by a call step in another pipeline. The caller's run-as identity needs execute on the called pipeline, and the called pipeline runs with its own run settings.

Schedules

A schedule sets what happens when a run is still active at the next time (overlap: skip, queue or allow) and when times were missed while the pipeline was off (missed: skip, run the latest, or catch up). Turning the pipeline off pauses the schedule.

Webhooks

Creating a webhook gives the pipeline a URL and a signing secret. The secret is shown once; rotate it to get a new one. Each delivery starts a run with this event:

{
  "body": { "...": "the request body" },
  "headers": { "content-type": "application/json" },
  "received_at": "2026-10-27T09:00:00Z",
  "delivery_id": "8f9c..."
}

App events

Pick a connection, the app's event and its settings. Polling events take an interval from 5 seconds to one day. The connection, and the signing secret when the app needs one, must be allowed in the run settings. Events arrive as:

{ "id": "evt_123", "type": "issue.opened", "time": "2026-10-27T09:00:00Z", "data": { "...": "..." } }

Events received while the pipeline is off are dropped, so turning it on never replays a backlog.

Sample events

Listen for a test event on a webhook or app-event trigger keeps the first event received in the next two minutes as the trigger's sample. The event still starts a run if the pipeline is Active. Tests from the draft use the sample when the trigger has no pinned event.

Test from the draft

Tests run the saved draft without publishing it. A test is real: it uses the pipeline's run settings and calls apps for real. Testing needs update on the pipeline.

Only top-level steps can be chosen. A step inside a loop or branch runs with its container, which can be chosen.

When a test skips earlier steps, each step it skips that a tested step reads from is given an output instead of running:

1. its pinned output, if it has one;
2. otherwise its output from the most recent test that ran it successfully.

If a needed step has neither, the test is refused and names that step. Pin it, or test from an earlier step.

Pins are kept per step of the draft, by step ID, and survive moving the step. Pin a step's output in the step panel's Test tab. Pinning the trigger sets the test event. Without a pinned or sample event, the editor asks for one.

Each test compiles the draft into a test build. Test builds never become versions: they cannot be published, activated or used by triggers.

While a test runs, the canvas shows each step's state, and the panel below the canvas lists the steps with their output and logs. Steps that were given an output show Reused output. Test runs also appear on the Runs page as test.

Debug with a past run's data

Debug with a past run's data loads a run's step data into the editor. Change the draft, then test from the failed step: the event and earlier steps' outputs come from that run instead of pins or earlier tests.

Runs, retry and retry from a step

The Runs page lists executions with their trigger, state and timing. Open one to see its runs, the canvas with each step's state, every step's occurrences, output and logs, and the run's output.

A failed run can be retried. The retry is a new run in the same execution, with the failed run's run settings.

• Start from the beginning, or from a top-level step. Starting from a step reuses the failed run's recorded output for the steps before it; the chosen step and everything after it run again. The failed step is selected by default.
• Retry with the same version, or pick another published version, for example one that fixes the failed step.

If an earlier step's output is no longer available, for example a step inside a loop or an output past retention, the retry is refused and names the step.

Permissions

The creator gets every permission on the pipeline. See access control.

API

All routes are under {api-base}/orgs/{org_id}/workspaces/{workspace_id}/vegaflow/visual-pipelines.

Test a draft from a step:

POST /orgs/{org_id}/workspaces/{workspace_id}/vegaflow/visual-pipelines/{id}/test-runs
Content-Type: application/json

{ "mode": "from_step", "node_id": "build_summary" }

mode is whole, from_step or step. input sets the test event; without it the trigger's pin, then its sample, is used. seed_run ({"execution_id", "run_id"}) uses a past run's event and outputs instead.

Retry a failed run from a step with a fixed version:

POST /orgs/{org_id}/workspaces/{workspace_id}/vegaflow/visual-pipelines/{id}/executions/{execution_id}/runs/{run_id}/retry
Content-Type: application/json

{ "from_node": "build_summary", "version_id": "{version_id}" }

Pin a step's output:

PUT /orgs/{org_id}/workspaces/{workspace_id}/vegaflow/visual-pipelines/{id}/draft/pins/{node_id}
Content-Type: application/json

{ "value": { "total": 3 } }

Pinned values are limited to 1 MiB.