67. GitLab cron-polling event dispatch

Date: 2026-06-13

Status

Accepted

Update (2026-07, #5556): The child-pipeline output driver described in this ADR was replaced by direct API-triggered pipelines (POST /projects/:id/pipeline). The poller now creates standalone pipelines with dispatch variables instead of generating child pipeline YAML. This eliminates the bridge job, YAML generation, and 2-level pipeline nesting. The cron-polling input driver and dispatch core are unchanged. Superseded sections: "Relationship to the dispatch driver architecture" (child-pipeline output driver), "Pipeline nesting" under GitLab tier considerations, and the architecture diagram showing parent-child pipeline flow.

Trust boundary change: With child pipelines, dispatch variables were computed server-side by the trusted poller and injected via the trigger YAML artifact — only the poller could produce them. With API-triggered pipelines, any user with pipeline-create access on the protected branch can POST arbitrary variables (STAGE, EVENT_TYPE, EVENT_PAYLOAD_B64, RESOURCE_KEY, IS_FORK, MR_AUTHOR_ID, ACTOR_ID, STATUS_IID, FULLSEND_POLL_JOB_URL, ORIGINATING_URL, REPO_FULL_NAME). The in-job authorization gate and fork protection read these attacker-supplied variables. Mitigation #1 is implemented: the agent job uses the Pipelines API (GET /projects/:id/pipelines/$CI_PIPELINE_ID) to fetch the server-side .source field and .user.id, then branches with a deny-by-default case statement. For API-triggered pipelines (.source == "api"), it verifies the pipeline creator matches the bot PAT identity. MR child pipelines (.source == "parent_pipeline") skip this check since their creator is the MR author. A missing or unrecognized .source aborts the job (fail-closed). Both dispatch paths now depend on a successful pipeline-record read. The .source field is server-computed and cannot be overridden by pipeline variables, unlike the CI_PIPELINE_SOURCE env var. Residual risk: CI_API_V4_URL and CI_PIPELINE_ID are still overridable, so a sophisticated attacker can redirect the API calls. Mitigation #2 (HMAC signing, #5572) reduces this residual risk: the poller signs dispatch variables with FULLSEND_DISPATCH_SECRET using HMAC-SHA256 and the agent job verifies the signature before trusting any dispatch variable. The HMAC computation itself uses no CI-provided URLs, but the verification is gated on PIPELINE_SOURCE (derived from CI_API_V4_URL), so the risk is reduced rather than fully closed. FULLSEND_DISPATCH_SECRET MUST be configured as a protected, masked CI/CD variable — pipeline variables can be overridden by API-triggered pipelines, so protection is required to prevent a Developer+ user from supplying their own secret and computing a valid HMAC over forged variables.

New permission requirement: The bot PAT must have merge or push access to the protected branch to create pipelines via the API endpoint. The child-pipeline path had no such requirement (the trigger ran inside an existing pipeline context).

Pipeline visibility change: Dispatched pipelines are first-class pipelines on the default branch, not nested children. Agent failures mark the latest pipeline on main as failed. The scaffold sets workflow:auto_cancel:on_new_commit:none to prevent new commits from canceling queued agent pipelines. This setting applies globally (including MR pipelines) because GitLab does not scope auto_cancel per pipeline source. MR dispatch jobs are fast (<30s) so the impact on MR pipeline redundancy is negligible.

Observability trade-off: The old trigger: strategy: depend mirrored child pipeline pass/fail into the poll job's own status. API-triggered pipelines are fire-and-forget — the poll job reports success after creating the pipeline, regardless of downstream agent outcome. Dispatched pipeline URLs are logged for manual inspection.

Context

Fullsend needs to detect and react to GitLab events — new issues, merge requests, comments, and label changes — so that agent stages (triage, code, review, fix, retro) can be dispatched automatically. On GitHub, native event triggers (pull_request_target, issues, issue_comment) handle this within GitHub Actions. GitLab has no equivalent for most event types.

GitLab's CI/CD pipeline trigger sources are: push, merge_request_event, schedule, trigger, web, api, and parent_pipeline. Of these, only merge_request_event maps to an agent-relevant event. Issue creation, comment posting, and label changes have no native CI pipeline trigger. GitLab supports per-repo installation mode only (no per-org); the pipeline runs inside the enrolled project on the protected default branch.

See ADR 0028 for the original GitLab support architecture discussion. ADR 0028 documented a webhook bridge approach; this ADR supersedes that direction based on the operational complexity analysis in Options 1–3 below. ADR 0045 defines the forge-portable harness schema that GitLab stage templates must conform to.

Options

Option 1: Webhook bridge Cloud Function

Deploy a GCP Cloud Function that receives GitLab webhook POST requests, validates the X-GitLab-Token header, and calls the Pipeline Trigger API to dispatch agent stages.

Rejected. Requires external infrastructure (Cloud Function) that must be deployed, monitored, and secured. Exposes a public HTTPS endpoint — an inbound attack surface. Requires three credential types per project (bot PAT, webhook secret, trigger token). Creates a complex deployment story for self-hosted GitLab behind corporate firewalls (VPN peering, on-premise containers, or Cloud Run + VPC Connector). The bridge cannot be eliminated even in a hybrid model — if any event type uses webhooks, the full bridge must be deployed.

Option 2: Webhook-only (all events via bridge)

Use the webhook bridge for all events, eliminating native CI triggers.

Rejected. Still requires the bridge with all its operational complexity. The correct response to "if we need webhooks for some events, why not all?" is to eliminate the bridge entirely, not to double down on it.

Option 3: Native merge request (MR) events + webhook bridge for issues/comments

Use GitLab's native merge_request_event for MR events, keep the webhook bridge only for issues and comments.

Rejected. Still requires the bridge Cloud Function. The bridge's operational cost is dominated by deployment, monitoring, and credential management — not by event type count.

Option 4: Pure cron-polling (no native CI triggers)

Poll for all events including MR creation and updates.

Rejected. MR events have a viable native CI path (merge_request_event + include: local:) with sub-minute latency and zero additional infrastructure. Polling for MRs adds unnecessary latency to the most frequent, most latency-sensitive operation (code review).

Decision

GitLab event dispatch uses a two-path model:

1. Native CI triggers for MR events. MR creation, update, and reopen trigger pipelines via GitLab's merge_request_event pipeline source. MR merge does not fire merge_request_event — GitLab creates a push event on the target branch instead, so MR merge detection uses the cron-poller (see event routing table below). The dispatch template is loaded via include: local: from the protected default branch, ensuring untrusted MR branches cannot modify dispatch logic.

2. Cron-polled events for everything else. A scheduled pipeline runs every N minutes (5 minutes on Premium/Ultimate, 60 minutes on Free tier), queries the GitLab API for new issues, comments, and label changes since the last poll, and dispatches agent stages via parent-child pipelines.

No external infrastructure is required for event dispatch — no webhook bridge, no webhook secrets, no trigger tokens.

Relationship to the dispatch driver architecture

ADR 0061 defines the dispatch pipeline: input driver → authorize → enumerate harnesses → CEL triggers → output driver. This ADR implements a gitlab-poll input driver for fullsend poll and a child-pipeline output driver for GitLab CI, following the same composition as other poll input drivers (e.g. jira-poll):

gitlab-poll input driver → per-event coordination → dispatch core → child-pipeline output driver

The gitlab-poll input driver discovers events and emits NormalizedEvent values. The dispatch core — authorization (ADR 0054) and harness CEL trigger evaluation (ADR 0061) — is shared with fullsend dispatch and other poll input drivers. The poll input driver does not duplicate trigger routing or authorization logic.

ENROLLED PROJECT                           GCP (optional, for inference)
────────────────                           ────
.gitlab-ci.yml (root pipeline)             WIF pool/provider (inference OIDC validation)
.gitlab/ci/fullsend-dispatch.yml (MR routing)
.gitlab/ci/fullsend-poll.yml (cron-poller)
.gitlab/ci/fullsend-agent.yml (generic stage)
  (replaces per-stage templates — see PR #3193)
.fullsend/ (config workspace)

MR events (native CI):
  MR opened/updated/reopened → merge_request_event → fullsend-dispatch.yml → review stage

MR merge (cron):
  Pipeline schedule → fullsend-poll.yml → MR merged_at > watermark → retro stage

Issues, comments, labels (cron):
  Pipeline schedule (5 min) → fullsend-poll.yml → GitLab API → dispatch agent stage

Credentials:
  Pipeline job → protected CI/CD variable FULLSEND_FORGE_TOKEN → bot PAT

Credential model

A Maintainer-role project access token with api scope, created during fullsend admin install. Maintainer role is required because the poller updates CI/CD variables (watermark and label state persistence) via the API, which requires Maintainer-level access. The bot PAT is stored as a protected, masked CI/CD variable (FULLSEND_FORGE_TOKEN).

Key properties:

• Single credential type. One bot PAT per project handles all REST and GraphQL operations. No webhook secrets, trigger tokens, or mint service.
• Bot identity. The project access token creates a dedicated bot user, providing attributable identity equivalent to GitHub Apps.
• GraphQL support. Unlike CI_JOB_TOKEN, the bot PAT authenticates GraphQL — required for GitLab's Work Items API.
• CI_DEBUG_TRACE guard. The scaffold script aborts when debug trace is enabled, preventing PAT exposure in job logs.
• Inference credential support. Vertex AI inference credentials (FULLSEND_GCP_PROJECT_ID, FULLSEND_GCP_WIF_PROVIDER, FULLSEND_GCP_REGION) are configured via OIDC/WIF when inference is enabled, so that agent jobs can authenticate to Vertex AI.
• OIDC issuer reachability requirement. Inference WIF requires the GitLab instance's OIDC discovery endpoints to be publicly reachable by GCP's Security Token Service (STS). During the WIF token exchange, GCP's STS resolves the GitLab instance hostname to validate the JWT issuer. Internal or private GitLab instances (e.g., those accessible only via VPN or corporate DNS) will fail with Error code invalid_grant: Error connecting to the given credential's issuer. Inference is not available for GitLab instances that are not resolvable in public DNS.

Cron poller (gitlab-poll input driver)

The gitlab-poll input driver runs as fullsend poll inside the fullsend container image, invoked by a scheduled pipeline on the protected default branch. It reads a timestamp watermark, queries the GitLab API for events since the last poll, emits a NormalizedEvent per detected change, passes events to the dispatch core for authorization and harness CEL evaluation, and advances the watermark.

Change detection for labels uses client-side state diffing — the input driver tracks previously-seen labels per issue and emits events only for newly-added labels. This compensates for the lack of a changes object that webhook payloads provide.

Multi-frequency polling (Premium/Ultimate): Two pipeline schedules — a fast poll (every 5 minutes, slash commands only) and a slow poll (every 15 minutes, full event scan). On Free tier, a single hourly poll is the only option. Each mode uses a separate watermark (FULLSEND_LAST_POLL_AT_FAST / FULLSEND_LAST_POLL_AT_FULL) so that fast polls do not advance past label/note events that only the full poll handles. A consequence is that slash commands discovered by a fast poll may be re-discovered by the next full poll. resource_group serialization prevents concurrent execution, so a duplicate dispatch queues behind the first. The second run executes the same stage against already-processed state (the agent sees no new work) and exits as a no-op, wasting one pipeline invocation's CI minutes. This is an accepted tradeoff — the alternative (sharing a processed-note-IDs set or cross-reading watermarks between modes) adds state coupling that complicates the independent-schedule design.

Update (2026-08, #5959): The dual-schedule architecture above was replaced by a single */5 * * * * schedule with automatic full-poll promotion. The poller now decides at runtime whether to run a fast poll or full poll based on elapsed time since the last full poll (FULLSEND_LAST_POLL_AT_FULL). This eliminates the tier distinction (Premium vs Free), the separate fast/full schedules, and the FULLSEND_POLL_MODE variable. The fast-poll watermark (FULLSEND_LAST_POLL_AT_FAST) is still used for slash-command-only cycles. Free tier in-CI polling is no longer supported by this schedule (Free tier's minimum interval is 60 minutes); Free tier users should use off-system polling (fullsend poll on a VM or Kubernetes CronJob) as documented in "GitLab tier considerations" below. Superseded sections: "Multi-frequency polling" above, the "5 minutes on Premium/Ultimate, 60 minutes on Free tier" reference in the cron-poller introduction, "Multi-frequency polling" and fast-poll MR note limitation under "Slash command latency", the Free tier 60-minute interval references in "GitLab tier considerations", and the "5 minutes on Premium, 60 minutes on Free" latency in "Consequences". Superseded by #6077 below.

Update (2026-08, #6077): The single auto-promoting schedule from #5959 was reverted to two independent schedules with explicit mode selection. The auto-promote logic coupled slash-command latency to full-poll duration and used a single resource_group, causing GitLab to cancel the in-progress poll when the next schedule fired. The new architecture:

• Slash poll: */5 * * * * with FULLSEND_POLL_MODE=slash — processes only /fs-* slash commands, fast and lightweight.
• Event poll: 2,17,32,47 * * * * with FULLSEND_POLL_MODE=events — full event discovery (labels, MR merges, non-command notes).
• Each schedule uses a per-mode resource group (fullsend-poll-slash / fullsend-poll-events) so they never cancel each other. Resource group process modes differ by purpose: newest_first for slash (latest command wins, stale polls are preempted) and oldest_first for events (long-running discovery completes before the next cycle starts).
• The --mode CLI flag (also FULLSEND_POLL_MODE env var) selects the mode explicitly; empty uses the events discovery path but does not filter /fs-* notes (backward compatibility with pre-dual-schedule installations where a single schedule handled all event types).
• The shouldFullPoll auto-promote logic and FullPollInterval are removed.
• Superseded sections: "MR note limitation (fast-poll)" (slash commands on MRs are now handled by the dedicated slash poll schedule, not gated behind full-poll cycles), and the "Multi-frequency polling" reference under "Slash command latency" (replaced by the independent schedule architecture above).

Event routing

The design goal is functional event-type parity with GitHub — users see the same labels, slash commands, and stage dispatches regardless of forge (latency differs: cron-polled events have 5–60 minute delay vs sub-second on GitHub). Routing is performed by harness CEL trigger expressions (ADR 0061) evaluated in the dispatch core, not by the gitlab-poll input driver. The table below documents how each detected change maps to a NormalizedEvent and transport path, not trigger configuration.

Detected Change	Transport	Stage
Issue label ready-to-code added	Cron poll (label state diff)	code
Issue label ready-for-review added	Cron poll (label state diff)	review
Issue note starting with /fs-{triage,code,review,fix,retro,prioritize}	Cron poll (note body prefix)	corresponding stage
Issue note (non-command) on issue with needs-info label	Cron poll (label check); Reporter+ or issue author	triage
MR opened/updated/reopened	Native CI (merge_request_event)	review
MR merged	Cron poll (MR merged_at > watermark)	retro
MR note with <!-- fullsend:changes-requested -->	Cron poll (note body marker)	fix (same-project MRs only)

Bot-authored comments are skipped to prevent re-triggering loops (exception: the changes-requested marker from the review agent).

Slash command latency

Slash commands (/fs-*) are the only latency-sensitive operation. Mitigations:

• Labels as primary triggers. Applying ready-for-review or ready-to-code labels is discoverable and visible. Labels on issues are detected via cron-poll (5–60 minute latency); labels on MRs can also be detected via native CI merge_request_event when applied alongside an MR update.
• Multi-frequency polling keeps slash command latency to 5 minutes on Premium/Ultimate.
• Manual pipeline trigger via the GitLab UI as a power-user escape hatch.
• Off-system polling via fullsend poll on a standalone VM or Kubernetes CronJob, at any desired interval. This reintroduces external infrastructure but is architecturally simpler than a webhook bridge — see GitLab tier considerations below for details.

MR note limitation (fast-poll): GitLab's merge_request_event pipeline source fires on MR creation, update, and reopen — not on merge, close, or individual MR comments. Comment-based triggers on MRs (/fs-fix, /fs-code) must therefore use the cron-poller. Within the cron-poller, these commands on MR notes are only acted upon during the full-poll cycle (every 15 minutes on Premium/Ultimate), not the fast poll. The fast-poll path does not fetch MR source/target project IDs, so the fork MR protection check (deny-by-default when unknown) blocks these stages. This adds up to 10 minutes of latency beyond the fast-poll interval. Fetching MR details per note in fast-poll would add API calls that defeat its lightweight purpose. In practice, fix stages are typically triggered by the review bot's changes-requested marker (which uses the full-poll path), not human slash commands.

Quick Action risk: GitLab may silently strip unrecognized /-prefixed lines. If confirmed empirically, GitLab should use an alternative prefix (fs:triage or @fullsend triage). ADR 0042 permits forge-specific syntax.

GitLab tier considerations

Feature	Free	Premium	Ultimate
Schedule minimum interval	60 min	5 min	5 min
Project access tokens (SaaS)	Not available	Available	Available
CODEOWNERS enforcement	Not available	Available	Available
CI minutes (shared runners)	400/month	10,000/month	50,000/month
Parent-child pipeline nesting	2 levels	2 levels	2 levels

Pipeline nesting: The cron-poller uses exactly 2 levels of trigger: include: child pipeline nesting — the GitLab maximum. The poll runs inline in the root scheduled pipeline (no child pipeline). Level 1: the root pipeline triggers a dynamically generated dispatch child pipeline (via trigger: include: artifact:). Level 2: the dispatch child pipeline triggers per-stage child pipelines (via trigger: include: .gitlab/ci/fullsend-agent.yml). This is at the nesting ceiling — no additional trigger: include: levels can be added without restructuring. See GitLab CI/CD pipeline nesting.

Free tier is functional but degraded: 60-minute poll interval, no project access tokens on gitlab.com (must use personal access token), no CODEOWNERS guardrails, and CI minute quota is insufficient for polling on shared runners. Self-hosted runners are required. As an alternative, Free tier users can run fullsend poll on an external scheduler (cron on a VM, Kubernetes CronJob, etc.) at any desired interval. This reintroduces external infrastructure but is architecturally simpler than a webhook bridge — the poller is entirely outbound (no public endpoint, no inbound payload parsing) and uses the same code path as the in-CI poller.

Premium (recommended minimum): 5-minute polling, project access tokens, CODEOWNERS enforcement, adequate CI minutes for a single project.

fullsend admin install adapts poll frequency and interaction model to the detected tier.

Security model

The security model follows the project's threat priority order (external injection > insider > drift > supply chain):

• No inbound attack surface. Polling is entirely outbound — no public endpoint, no webhook parser, no shared-secret authentication.
• Protected branch enforcement. workflow:rules require $CI_COMMIT_REF_PROTECTED == "true" for scheduled pipelines.
• Protected CI/CD variables. All fullsend CI/CD variables are marked protected — accessible only to pipelines on protected branches.
• CI_DEBUG_TRACE guard. Install-time validation and runtime abort if debug tracing is detected. This guard is the sole defense against PAT exposure via debug tracing — GitLab logs CI/CD variables at job init, before any script runs. Known limitation: install-time validation checks project-level and group-level variables but cannot query instance-level CI/CD variables (requires admin API access).
• Event data sanitization. Attacker-controlled content is base64-encoded before passing to child pipelines.
• Fork MR protection. Fix/code stages are skipped when source_project_id != target_project_id.
• Slash command authorization. Only users with Developer-level (30+) project access can trigger agent stages via /fs-* commands. Exception: non-command comments on issues with the needs-info label trigger triage with a reduced authorization gate — the commenter must have at least Reporter-level (20+) project access or be the issue author. This mirrors the GitHub path where ADR 0054 requires author_association != NONE or issue authorship for non-command triage triggers, preventing unauthenticated cost exposure on public projects.

Security properties of the credential model:

Threat vector	Mitigation
CI_DEBUG_TRACE by Maintainer	PAT exposed at job init before script guard runs; guard limits further damage but cannot prevent initial exposure
Maintainer marks branch as protected	Token exposed (protected variable accessible)
GitLab database compromise	PAT stored in GitLab as protected CI/CD variable
Audit trail	GitLab audit logs (Premium+)

Forge abstraction

ADR 0005 requires new forges to implement forge.Client. This ADR extends the forge interface with new methods (some GitLab-specific, some forge-neutral):

• IsProtectedBranch — maps to GitHub branch protection API and GitLab protected branches API
• CreatePipelineSchedule / DeletePipelineSchedule — GitLab-native; GitHub returns ErrNotSupported
• UpdateCIVariable — for poll watermark management

A new ErrNotSupported sentinel (complementing the existing forge sentinel errors) allows forge implementations to reject inapplicable operations. GitHub-only methods (ListOrgInstallations, GetAppClientID) move to a GitHubExtensions extension interface. This requires interface evolution beyond pure implementation — adding methods to forge.Client and refactoring GitHub-specific methods into an extension interface. This is anticipated growth of the abstraction boundary, not a violation of ADR 0005's design; the changes to appsetup.go and admin.go are limited to calling new forge-neutral methods rather than adding forge-conditional logic.

Consequences

What becomes easier:

• No external infrastructure for event dispatch. No Cloud Function, no webhook bridge. Self-hosted GitLab requires only outbound HTTPS.
• Single credential per project. One bot PAT, stored as a protected CI/CD variable. No webhook secrets, trigger tokens, or mint service changes.
• Stronger event authenticity. Events read directly from the GitLab API, not from potentially spoofed webhook payloads.
• No event loss. Polling reads from the source of truth. Webhooks can fail silently or auto-disable after 4 consecutive failures.
• Simpler emergency shutdown. Disable the pipeline schedule or revoke the bot PAT. No bridge to tear down.
• MR review latency is unaffected. Native merge_request_event provides sub-second triggering for the highest-frequency operation.
• Tier-adaptive. Works on all GitLab tiers with graceful degradation.
• GCP required only for inference. The agent runtime works without GCP, but repos install requires inference credentials (WIF provider and GCP project) for all new installations.

What becomes harder or changes:

• Issue/comment event latency. Up to 5 minutes on Premium, 60 minutes on Free. Acceptable for asynchronous agent operations, poor for interactive use on Free tier.
• CI minute consumption. Polling runs continuously. At 5-minute intervals: ~8,640 min/month on shared runners. Self-hosted runners are not billed.
• State management. The poller must track watermarks, deduplicate events across overlapping windows, and diff label state. This state is internal to the GitLab forge implementation and does not leak into the forge.Client interface, preserving the forge-neutral contract from ADR 0005.
• Slash command latency. Up to 5 minutes vs sub-second with webhooks. Labels mitigate this for common operations.
• Quick Action stripping. GitLab may strip /fs-* commands from comments. Requires testing and potentially alternative syntax.
• Per-repo only. No centralized config or credential management across projects.
• api scope is broad. Narrower scopes are not available in GitLab today.

Risks (ordered by threat priority):

1. YAML injection in child pipeline generation. Attacker-controlled issue/MR content could break child pipeline YAML syntax. Mitigated by base64 encoding of event payloads passed to child pipelines.
2. Prompt injection via polled events. Attacker-controlled issue/MR content reaches the agent at inference time. This risk is identical across all forges and is handled by the existing agent harness security layer, not by the transport mechanism.
3. Watermark tampering. A Maintainer could skip or replay events by modifying the watermark variables. Mitigated by protected variable status and event deduplication.
4. Schedule modification. A Maintainer could retarget the schedule to a non-protected branch. Mitigated by protected variable status (bot PAT not exposed on non-protected branches).
5. Missed events from API quirks. The Notes API lacks created_after; the Events API after parameter is date-only. Mitigated by 30-second watermark overlap and dual-frequency polling as reconciliation.

Comparison with GitHub:

Concern	GitHub	GitLab (this ADR)
Primary credential	App installation token via mint	Bot PAT (WIF or CI/CD variable)
MR/PR event dispatch	pull_request_target	merge_request_event
Issue/comment dispatch	Native events (sub-second)	Cron polling (5 min)
External infrastructure	Mint Cloud Function	None for event dispatch
Credential types	App key + installation token	Single bot PAT

Implementation covers poller pseudocode, forge interface changes, CI/CD template scaffolding, and install flow.

References

• ADR 0002 — initial fullsend design (webhook + dispatch service, label state machine)
• ADR 0033 — per-repo installation model (the only supported mode for GitLab)
• ADR 0054 — authorization on all dispatch paths (slash command ACL)
• ADR 0061 — harness CEL triggers, dispatch drivers, and NormalizedEvent schema
• ADR 0063 — polling-based work discovery via dispatch drivers (fullsend poll, input/output driver architecture)
• NormalizedEvent v1