Pipelines: Automation That Ships With the App

Every application accumulates automation. A lead comes in and needs to be scored, enriched, and routed. A nightly job reconciles two systems. A webhook fires and three things should happen in order, unless the second one fails, in which case something else should happen.

This logic usually ends up in one of two bad places: buried in a controller method that has quietly become 400 lines, or exiled to an external automation service where it is invisible to your version control, untestable locally, and one billing lapse away from silently stopping.

Tiknix pipelines are the third option — declarative JSON steps stored in the repository, executed by the app itself.

A pipeline is a file

{
  "slug": "demo-hello",
  "name": "Demo — hello",
  "description": "shell -> transform -> branch -> notify",
  "context_schema": { "who": { "type": "string", "required": true } },
  "steps": [
    { "name": "greet", "type": "shell",
      "config": { "command": "echo '{\"greeting\":\"hello {context.who}\"}'" },
      "on_success": "next", "on_fail": "exit" },

    { "name": "extract", "type": "transform",
      "config": { "mode": "jsonpath", "input": "{greet.output}", "path": "greeting" },
      "on_success": "next" },

    { "name": "checkn", "type": "branch",
      "config": { "left": "{greet.output.n}", "op": "gte", "right": "40" },
      "on_success": "goto:big", "on_fail": "goto:small" }
  ]
}

It lives in pipelines/, next to the code it automates. It is in Git. It diffs. It code-reviews. It ships with a deploy and rolls back with one. That last property is the whole argument: automation that lives outside your repository is automation you cannot roll back.

Running one is a single call from any controller:

$result = \app\Pipeline\Runner::run('lead-triage', ['email' => $email]);

The step types

Two of those are worth dwelling on. agent means an AI step is a step — a bounded unit with defined inputs and outputs sitting between deterministic steps, not a free-floating assistant with access to everything. That is the right shape for putting a model in production: it does one thing, its output feeds a known consumer, and the steps around it are ordinary code.

connection means credentials never appear in the pipeline file. The step references a connection by name; the encrypted token lives in the database (see Third-Party Credentials Without Sprawl). A pipeline is safe to commit because there is nothing secret in it.

Data flows through references

Steps read each other's output with a simple reference syntax:

• {context.who} — a value the caller passed in
• {greet.output} — a previous step's whole output
• {greet.output.n} — a field within it
• {time.date} — a built-in

Flow control is per-step: on_success and on_fail each take next, exit, or goto:<step>. That is enough for sequences, conditionals, retries, and early exits, and it stops well short of being a programming language — which is the correct place to stop. A pipeline you can read top to bottom is worth more than one that can express anything.

context_schema declares what the pipeline requires from its caller, so a missing input fails immediately and legibly rather than surfacing as an empty string four steps later.

Adding a step type is dropping a file

// lib/Pipeline/StepRegistry.php — auto-discovery
foreach (glob(__DIR__ . '/Steps/*Step.php') as $file) {
    $cls = 'app\\Pipeline\\Steps\\' . basename($file, '.php');
    // ...must implement StepInterface; indexed by its type() token
}

No registration list, no switch statement, no three places to update. Write Steps/SlackStep.php, implement the interface, and slack is a valid step type everywhere — in the runner, in the editor, and in the component list agents read.

The registry's docblock is candid about why it works this way: an earlier system had three registries and they drifted. One source of truth, derived rather than duplicated, is the fix. Each step declares rich fields once; the agent-facing config surface is derived from those fields rather than maintained alongside them.

Pipelines are a first-class agent surface

There is a full set of MCP tools for pipelines — pipeline_list, pipeline_get, pipeline_set, pipeline_run, pipeline_run_get, pipeline_continue, pipeline_delete, and pipeline_components.

pipeline_components is the important one. It returns the schema of every available step type, so an agent asked to "build me a lead triage flow" can enumerate its actual building blocks and their real configuration fields instead of guessing at a format. The agent then writes a JSON file — a reviewable artifact in a constrained format, not opaque code in a controller.

That is a considerably better division of labor than "let the agent write the automation wherever." The format is small enough to validate, small enough for a human to read in thirty seconds, and versioned like everything else.

Scheduling

lib/Pipeline/Cron.php is a compact five-field cron matcher — *, lists, ranges, and steps — driven by a tick that asks, once a minute, "does this expression fire now?" Familiar syntax, no new scheduling concept to learn, and the schedule lives with the pipeline.