Lab 2 — A Multi-Pack Pipeline

labs pipelines cross-pack composition

Lab 2 — A Multi-Pack Pipeline

Time: 30 minutes · Level: intermediate

Target audience

Developers who finished Lab 1 and want to compose commands instead of writing glue code.

Prerequisites

  • Lab 1 completed: developer mode enabled, a publishing namespace, and a published pack. Bump version from 0.0.0 before each publish.
  • A Huitzo API token for calling the REST API (see REST API).
  • Basic familiarity with Pydantic models.

Learning objectives

By the end of this lab you can:

  1. Declare a pipeline in huitzo.yaml and explain how stages are chained.
  2. Reference a command published by a different pack, and grant the permission that requires.
  3. Run the pipeline through the REST API and read the per-stage result.
  4. Trigger a stage failure at run time and see which stage failed.

1. Plan the shape

You will build two packs:

  • Provider pack — publishes one command, clean, that normalises a string. (Reuse your Lab 1 pack if you prefer: add a clean command to it and publish a new version.)
  • Pipeline pack — declares a pipeline with three stages: clean (from the provider pack), shout (a command in the pipeline pack), and frame (a command in the pipeline pack).

Stage n's output keys must structurally match stage n+1's input keys; the executor checks that on every transition. Each stage is a regular @command with typed Pydantic models; see Your First Pack, Step 2 for the command shape.

2. Publish the provider

Scaffold and publish the provider exactly as in Lab 1. Note its full identity, @yourscope/provider-pack, and the command's namespace:command name (in the block below, provider is the provider pack's namespace).

3. Declare the pipeline and the grant

In the pipeline pack's huitzo.yaml, add a pipelines: block whose first stage uses the cross-pack form @scope/pack:namespace:command, and the two local stages use namespace:command. Then grant the permission the cross-pack stage needs:

# pseudocode — the parts of huitzo.yaml that are new in this lab
permissions:
  - "storage:read"
  - "commands:execute:@yourscope/provider-pack"
policy:
  allowed_actions:
    - "storage:read"
    - "commands:execute:@yourscope/provider-pack"

pipelines:
  shout-and-frame:
    timeout: 60
    stages:
      - name: clean
        command: "@yourscope/provider-pack:provider:clean"
      - name: shout
        command: "labpipe:shout"
      - name: frame
        command: "labpipe:frame"

Every permission must also appear in policy.allowed_actions, and the @ and full scope are required: there is no de-scoped spelling. A grant only says your pack may call the target; at execution the platform also requires that your tenant may run the target command, so naming another tenant's private pack authorizes nothing.

4. Implement the two local stages

Write shout (uppercase the text; raise ValueError if the text is empty, so you can trigger a failure in step 7) and frame (wrap it) as ordinary @command functions with Pydantic argument and result models whose field names line up with the previous stage. Keep bodies non-blocking.

5. Publish and run

# pseudocode
huitzo pack validate
huitzo pack publish

validate catches a missing grant for a cross-pack stage as an error, so fix any it reports before publishing. Then run the pipeline:

# pseudocode — REST request (see the REST API reference for auth)
POST /api/v1/pipelines/{scope}/{pack}/{name}/execute
Authorization: Bearer <your token>

{ "initial_args": { "text": "  hello   pipeline  " } }

6. Read the result

A successful inline run returns HTTP 200 with the result under data.pipeline_run: status is SUCCESS, stage_results has one entry per stage with its duration, and final_output is a list of chunk envelopes holding the last stage's data. (If you get HTTP 202 with a task_id, the run was dispatched asynchronously; this lab assumes the inline 200 — see the REST API for the async path.)

7. Fail a stage at run time

Run the pipeline again with { "initial_args": { "text": "" } }. The shout stage raises, the pipeline stops (fail-fast), and the response is an error rather than a SUCCESS result. Find which stage failed and confirm the frame stage did not run.

8. Break the grant on purpose

Remove the commands:execute: line from permissions only (keep at least one other permission listed, such as storage:read, so the list still exists — a manifest with no permissions: list at all produces only a warning), and run huitzo pack validate again. You should see an error naming the stage and the missing permission. Restore the line. This is the install-time half of the cross-pack gate; the runtime half re-checks at execution and also requires that your tenant may run the target command (a private pack belonging to another tenant is denied even when the grant is present).

Completion check

You are done when the pipeline run returns status: SUCCESS, has exactly three entries in stage_results, and the first entry's stage name is the cross-pack clean stage. You should also have seen the run-time failure in step 7 and the validation error in step 8.

Follow-on resources