CI/CD and Pipeline Design for Tier 0¶
Part of Privileged Access, PIM, and What Belongs in CI — see that page for the EAM/NIST/CAF/security-levels model this design applies.
Related implementations: - GRINNTEC: Service Principal Vending — Central machine identity vending under Lane A - GRINNTEC: Tenant Root Bootstrap — Day-0 root of trust bootstrap sequence
The Dual-Lane Delivery Model: Lane A vs. Lane B¶
To reconcile CI/CD automation with the strict requirements of NIST AC-6 (Least Privilege), NIST AC-5 (Separation of Duties), and Microsoft EAM, the platform architecture establishes two delivery pathways:
| Property | Lane A (Tier 0 / Control Plane) | Lane B (Tier 1 & 2 / Workloads) |
|---|---|---|
| Repositories | azure-tenant-root, azure-service-principals, azure-priv-tier-0 |
subscription-vending, azure-management-groups, workload repos |
| CI Stage Scope | fmt, tflint, validate, checkov, plan |
fmt, tflint, validate, checkov, plan, apply |
| CI Machine Identity | Scoped Read-Only (sp-pla-tf-*-ro) |
Scoped Read-Write (sp-*-rw bounded to resource/sub) |
| CI Apply Capability | Permanently Disabled in pipeline YAML | Automated on merge to main |
| Deployment Gate | Human review + PIM activation + preflight script | MR review and approval |
| Execution Context | Local human CLI under authenticated az login |
GitLab CI shared / protected runner |
| Primary Controls | EAM Plane Isolation, NIST AC-6(5), AC-2(6), AC-5 | Branch protection, scoped OIDC, Checkov posture |
Why Service Principal Creation is strictly Lane A¶
Vending a machine identity requires creating Entra ID App Registrations, managing directory role assignments, granting Microsoft Graph application permissions, and assigning RBAC at management group or subscription scopes.
An automated pipeline capable of vending service principals would require standing Application Administrator and Privileged Role Administrator permissions. This creates an unacceptable vulnerability: anyone who merges code or compromises the CI runner could vend arbitrary identities with tenant-wide administrative rights. For this reason, azure-service-principals operates exclusively under Lane A.
What if this were applied differently — a pipeline holding Owner/UAA at root¶
It's worth walking through the counterfactual explicitly, because the complexity this tenant has built (two repos, a human CLI gate, PIM approval, senior-engineer-only eligibility) is a direct, traceable response to specific standards violations — not complexity for its own sake.
To be precise about which standard is doing the work here: CAF doesn't flatly prohibit a pipeline holding privileged access (see the model). What follows describes a pipeline that fails CAF's own stated conditions for when that's acceptable — no equivalent governance, no built-in escalation limit — not merely a hypothetical that falls short of this tenant's stricter-than-required choice.
Suppose azure-platform-identity's CI-applied service principal held Owner/UAA at the Tenant Root Group instead of azure-priv-tier-0's human-applied identity.
flowchart TD
subgraph "Counterfactual: CI holds Owner at root"
MR["Merge request to azure-platform-identity"] --> CIM["CI auto-applies on merge<br/>(no second gate)"]
CIM --> LIVE1["Owner at tenant root, live<br/>— no human judged this specific action"]
end
subgraph "What this tenant actually does"
MR2["Merge request to azure-priv-tier-0"] --> PLAN["CI plans only"]
PLAN --> REVIEW["Senior-engineer review"]
REVIEW --> HUMANAPPLY["Human, local az login,<br/>PIM-activated Global Administrator"]
HUMANAPPLY --> ACT["PIM activation request"]
ACT --> APPROVE["Named approver reviews + justification"]
APPROVE --> LIVE2["Owner at tenant root, live,<br/>time-bound, alerted"]
end
Here's specifically what breaks, standard by standard:
- CAF's own conditions for a privileged pipeline. No PIM-equivalent governance exists for this SP (it's a standing grant, not time-bound or approved per-use), and nothing structurally stops it from applying a role assignment wider than intended — both of CAF's stated conditions fail, not just this tenant's stricter internal rule.
- EAM's plane-isolation principle. EAM's whole model rests on lower-trust systems never being able to manage higher-trust ones. A CI runner triggered by anyone with merge rights — or by anyone who compromises that runner or its OIDC token — is a lower-trust surface than a human on a managed device with an activated privileged role. Giving it Control-plane authority collapses the isolation the model depends on.
- AC-6(5) Privileged Accounts. This control requires privileged accounts be restricted to explicitly authorized individuals for explicitly authorized purposes. A CI identity used for routine merges is neither — it's a shared, purpose-generic identity, and "someone merged a PR" is not an authorized privileged purpose.
- AC-5 Separation of Duties. The reviewer approving a Terraform diff is judging "is this code correct," not "should Global-Admin-equivalent access go live right now." Merge-to-apply collapses those into one action with one judgment call, removing the second, purpose-specific check separation of duties requires.
- AC-2(6) Dynamic Privilege Management. A CI identity holding standing
Ownerat root — even if only to execute a Terraform apply — is a standing privileged grant, the opposite of the just-in-time model NIST and EAM both call for.
The concrete failure path: a merge to main — malicious, or just a mistake — auto-applies immediately. No approval step exists to catch it, because the control that would normally require one (pim_require_approval, a named approver) is defined by the same automated pipeline that just applied the change. One bad merge is full tenant compromise, with no human ever having made a judgment call about that specific action.
That failure path is exactly why the added complexity is necessary, not incidental:
- The two-repo split (
solution-priv-tier-0/azure-priv-tier-0) exists so a compromised pipeline identity can't widen its own allow-list — this is CAF's own named condition (prevent self-escalation) satisfied directly, and it also addresses AC-6(5) (bounding what the "purpose" can ever be) and the EAM isolation principle (the allow-list is unreachable from the lower-trust repo). - The human-CLI-only apply exists so no merge, by itself, can ever make Tier 0 access live — this is where the design goes beyond CAF's minimum (equivalent governance) rather than attempting to simulate it for a service principal; it also addresses AC-5 (separates "code review" from "privilege activation") and AC-2(6) (nothing is standing; every use requires reactivation).
- Mandatory PIM approval on every grant, hardcoded so it can't be toggled off by editing a variable, exists so activation itself has a second, purpose-specific check — addresses AC-6(5) directly.
- Restricting Tier 0 PIM eligibility to senior engineers exists because AC-5's separation only works if the approver pool is distinct from, and at least as trusted as, the requester pool.
Every one of those is a mitigation for a named failure mode above, not a default level of caution applied everywhere. That's also why mg-platform and mg-landingzones don't get the same treatment — the failure mode that justifies it (self-referential Control-plane compromise) doesn't apply there.
What's safe to automate vs. what needs a human gate¶
| Safe to fully automate (CI, merge = apply) | Requires a manual, out-of-band gate |
|---|---|
| Creating groups and membership for non-privileged access | Any change to a Tier 0 role's PIM eligibility list |
Standing Reader / cost-management-style roles at Tier 1/2 scopes |
Any change to pim_require_approval on a Tier 0 grant — the pipeline that enforces the control cannot be trusted to remove it unsupervised |
| PIM eligibility for Tier 1/2 roles that already require approval | Any role assignment (Owner/UAA/Reader) at tenant root or mg-grinntec |
| Read-only or reporting pipelines | Anything that lets an identity grant itself more privilege (Privileged Role Administrator, or an app with admin-consent rights) |
Automation may define policy (AC-6 least-privilege scoping). Only a supervised, logged, time-bound human action may exercise Tier 0 privilege (AC-2(6), AC-5).
Who is even eligible to hold Tier 0 PIM — engineer seniority¶
The tables above answer which PIM role a human needs to activate. They don't answer who should be allowed to be eligible for it in the first place — that's what AC-5 (Separation of Duties) actually requires: the approver pool must be distinct from, and no less trusted than, the requester pool.
| Engineer level | Tier 0 (CLI, human apply) | Tier 1/2 (CI, auto-apply) |
|---|---|---|
| Standard engineer | Not PIM-eligible for anything at Tier 0. All of it routes to a senior engineer. | Full participation — merge rights to azure-platform-identity are enough; no personal privileged role needed |
| Senior engineer | PIM-eligible for the roles the change requires, per Entra directory roles and Azure RBAC by management group. Every activation still requires approval + justification + time-bound + alerting. | Same as standard — seniority isn't a factor at Tier 1/2 |
Why the line sits there
Tier 0 tasks aren't harder to write Terraform for — adding one UPN to a PIM eligibility list is trivial at any level. What seniority gates is judgment about blast radius: recognising that a seemingly small change is really about who can eventually reach GA/PRA-adjacent territory. azure-priv-tier-0's required-approvers should name senior engineers specifically, or AC-5's separation is nominal only.