Accelerating Optimizely CMS and Commerce upgrades with agentic AI (Part 1 of 2)

Mon, 11 May 2026 07:50:54 GMT

How Niteco's Upgrade Machine uses orchestrated AI coding agents to deliver a buildable baseline and a running CMS, then hands over for stabilization and deployment.

Niteco Engineering | Engineering Insights | 10 min read

Blog series

This is Part 1 of a 2-part series on accelerating Optimizely upgrades with agentic AI.

Part 1 (this post): CMS 11 and Commerce 13 → CMS 12 and Commerce 14

Part 2 (publishing soon): CMS 12 and Commerce 14 → CMS 13 and Commerce 15, and how we scope platform uplift vs capability adoption

Figure: Niteco Agentic Upgrade Machine (CMS 11 and Commerce 13 → CMS 12 and Commerce 14)

TL;DR
We built an Upgrade Machine: an agentic AI system composed of a main orchestrator, specialized subagents, and a growing skills library. It compresses the most repetitive phase of an Optimizely upgrade by producing a PR-ready codebase that builds successfully in Release mode and boots the CMS backend cleanly on the target platform.
From that point, a Niteco delivery team takes over to stabilize integrations, validate behavior, and ship to production, typically within two to six weeks depending on footprint.

1. Why Optimizely upgrades stall (and why timing matters now)

An Optimizely upgrade is rarely just a package update. It touches runtime assumptions, custom code, integrations, routing, dependency injection wiring, configuration, content models, and cutover readiness. The CMS 11 and Commerce 13 move to CMS 12 and Commerce 14 is a major inflection point because it sits on top of a broader platform modernization. Patterns that were stable for years can suddenly require structural refactoring.

There is also a lifecycle reality. Optimizely has communicated end-of-support timing for CMS 11 and Commerce 13 in 2026. Many teams reference an end-of-support window starting in April 2026, with an extension to October 2026. The exact dates and entitlements can vary, so you should verify against official Optimizely lifecycle communications and your agreement. The practical implication is consistent: delaying this upgrade increases platform risk because vendor support and patch cadence wind down.

In our experience, the true schedule killers are discovery gaps: undocumented integrations, legacy helpers, and configuration wiring no one wants to touch. These gaps force teams into slow compile-fix cycles and late-stage runtime surprises. Our objective with the Upgrade Machine is straightforward: remove the repetitive refactor grind from the critical path and give engineers a clean, buildable baseline to stabilize from.

2. What traditional automation solves, and where it stops

Rule-based upgrade tooling is genuinely useful for deterministic transformations. Retargeting frameworks, aligning package references, updating project files, and applying known API substitutions should be automated. Most upgrades should start there, and ours do.

The gap appears when upgrades meet context-heavy work: cross-file refactors, ambiguous intent, or situations where multiple fixes could compile but only one aligns with the correct platform pattern. A rule engine cannot reason about intent. It either applies a transformation safely or escalates. In practice, those escalations accumulate into a multi-week manual backlog. That is the territory where agentic AI changes the economics.

3. The agentic approach: what we built (Part 1 focus)

The Upgrade Machine is an orchestrated setup of AI coding agents operating inside a controlled workflow.

A main agent orchestrates the run: it sequences phases, enforces quality gates, and tracks progress in a persistent log.

Specialist subagents handle narrow domains: code fixes, reviewer checks, unit-test scaffolding, and skill enrichment.

A shared skills library encodes typical upgrade patterns derived from real delivery work. Each skill has clear triggers, a defined transformation approach, and validation rules.

Two design choices matter most.

First, autonomy with guardrails. Agents can run autonomously, but only within a workflow that enforces checkpoints, reviewer gates, and escalation thresholds. When uncertainty is high, the system pauses and asks for a developer’s decision rather than guessing.

Second, compounding knowledge. Every engagement adds patterns to the skills library. The system improves because it is built to capture what works, validate it across multiple cases, then promote it into safe automation.

For this post (Part 1), our definition of done is narrow and measurable: the solution builds successfully in Release mode on the CMS 12 and Commerce 14 target, and the CMS backend starts cleanly.

4. Upgrade Machine – an agent harness architecture

Agent harness (execution layer)

Under the visible “agents and tools” sits an agent harness. This is not a model, it is the execution layer that makes agentic work safe and repeatable in real engineering environments. It provides:

Run isolation and reproducibility: creates an isolated workspace and enforces a checkpointed, branch-based flow so changes are easy to audit, pause, and resume.

State and context management: packages the minimum required context for each subagent, tracks progress state, and prevents uncontrolled context drift across long runs.

Tool governance: controls which tools an agent can use, in what sequence, and under what policy constraints (build, scan, test, report) so automation stays deterministic.

Quality gates and escalation: enforces reviewer gates and uncertainty thresholds, escalating to a human decision when intent is ambiguous or risk of behavior change is high.

Observability and artifacts: captures structured logs, diffs, and run outputs (reports, checklists, remaining-decisions list) so delivery teams can stabilize and deploy with confidence.

Skills Library

This is an institutional memory. It contains validated patterns in categories such as migration patterns, CMS and Commerce patterns, DI and configuration patterns, and integration client patterns. Patterns move through a lifecycle: observed, validated, then automation safe.

Specialist Subagents

These are narrow, purpose-built agents invoked by the orchestrator:

Code-Fix Agent: applies migration fixes guided by the skills library

Reviewer Agent: validates output against the skills library and rejects fixes that compile but violate migration intent

Unit-Test Agent: scaffolds and runs tests where feasible

Skill-Enrichment Agent: captures new patterns discovered during the run and feeds them back into the skills library

Toolchain Integrations

Agents do not work in abstraction. They execute real tooling such as dotnet build, dependency scanning, test runners, and report generation. Outputs are logged and attached to the run report.

Main Agent

The orchestrator plans and drives the run, assigns work, tracks state and progress logs, enforces guardrails, and compiles the final report for developers and stakeholders.

Build-Fix Loop

This is the heartbeat. Build output is classified against a migration debt index, fixes are dispatched, reviewer gating is applied, checkpoint commits are created, and the loop repeats until the debt index is empty. Only then does the workflow promote to Release build verification.

5. How a run works end-to-end (CMS 11 and Commerce 13 → CMS 12 and Commerce 14)

A run moves through phases with explicit exit conditions and artifacts.

Phase 1: Pre-flight and isolation

The orchestrator assesses solution structure, dependencies, and likely breaking zones. Work runs in an isolated branch. Baselines remain untouched. Every phase produces checkpoints, so a run is auditable and resumable.

Phase 2: Guided migration

The machine applies upgrade transformations that convert known legacy patterns into modern equivalents and aligns dependencies for the target platform. This phase is migration-first: correctness of the target platform shape is prioritized over superficial compilation success.

Phase 3: Continuous validation loop (the heartbeat)

This is where agentic AI earns its value.

The orchestrator builds and classifies issues against migration debt patterns.

Code-fix is dispatched in controlled batches.

A reviewer gate validates that fixes are aligned to accepted patterns, not quick shims.

Checkpoints are committed only after reviewer acceptance.

If progress stalls or intent becomes ambiguous, the system escalates to a developer with options and rationale.

Phase 4: Release build verification and CMS boot validation

A Debug build is not enough. Release-mode build and CMS startup validation surface wiring issues that otherwise appear late. The pipeline repeats until Release build is clean and CMS backend starts cleanly.

Phase 5: Output artifacts

At completion, the machine produces:

PR-ready branch with checkpoint commits

Structured change log grouped by pattern families

Build logs and run summary

Remaining decisions list (items intentionally escalated to humans)

Stabilization checklist tailored to the solution footprint

Skill enrichment notes for future runs

6. Skills library: deep enough to trust, not detailed enough to copy

The skills library is the single most important asset in the system and the reason results improve across engagements. A skill is a documented pattern with:

Trigger: how the system detects the pattern (error signature, code shape, dependency signal)

Transformation: how code is changed safely

Validation: how correctness is verified beyond “it compiles”

Skills are not born autonomous. Patterns are captured from real engagements, validated across multiple independent codebases, and only then promoted to automation-safe. This prevents guesswork and reduces hidden regressions.

Representative categories include:

Project and dependency modernization patterns

DI and configuration patterns aligned to modern platform expectations

Routing and runtime initialization patterns

Integration client modernization patterns (safe client setup, resilience, async correctness)

Commerce-related upgrade patterns where the upgrade path diverges

Safe cleanup patterns that require explicit human sign-off

7. Human-in-the-loop and guardrails

Agentic systems must be able to pause. Ours is designed to escalate.

When the orchestrator hits an uncertainty threshold, it stops and asks for input. It proposes options and tradeoffs, then proceeds only after a decision is confirmed.

Guardrails ensure changes remain safe and auditable:

PR-style diff flow

Checkpoint commits at every phase

Full action logs

Reproducible runs

Mandatory human review before merge

No deployment actions by the machine

For engineering leaders, the risk profile is familiar: a disciplined internal branch-based workflow, just faster.

8. Definition of done and the handover model (Part 1)

We are deliberate about boundaries.

Done for Part 1 (fully autonomous agentic workflow) means:

The solution builds successfully in Release mode on the CMS 12 and Commerce 14 target

The CMS backend starts cleanly

Artifacts are produced: change log, build logs, remaining decisions list, stabilization checklist

What the delivery team does next (typically 2 to 6 weeks)

Stabilize regressions surfaced by functional and exploratory testing

Validate integrations, scheduled jobs, and data flows

Address performance and security hardening as required

Prepare deployment, rehearse cutover, and ship to production

This split keeps the model trustworthy. The machine accelerates engineering foundation work. The delivery team owns production readiness.

Figure: From pipeline start to clean backend build in 4h 38m

9. Mapping to the 2-step journey (and what comes next)

Most teams plan a two-step journey, and we structure machine runs around that shape.

Step 1 (this post): CMS 11 and Commerce 13 → CMS 12 and Commerce 14

This is the largest lift. The machine concentrates effort here because this is where repetitive refactoring and modernization posture work dominate.

Step 2 (coming soon): CMS 12 and Commerce 14 → CMS 13 and Commerce 15

Once the platform is modernized, the next uplift is typically narrower and faster. Part 2 will cover how we apply the same workflow with a tighter skill set, and how we separate platform uplift from adoption of new capabilities.

10. What you receive from a run

Every engagement produces a consistent artifact set:

PR-ready upgraded branch

Structured change report grouped by patterns

Build logs and run summaries

Remaining decisions list

Stabilization checklist tailored to the solution footprint

Optional internal playbook updates (pattern learning, skill enrichment)

11. FAQ

Is this a product or a delivery accelerator?
A delivery accelerator used by Niteco engineers inside client engagements.

Does it replace developer work?
No. It reduces to minimal all repetitive refactors and compile-fix cycles. Engineers retain ownership of correctness and production readiness.

Does it handle integrations and testing automatically?
It accelerates code modernization and structured validation. Full integration certification and business acceptance remain delivery responsibilities.

How do you keep changes safe and auditable?
PR-style workflow, action logs, checkpoint commits, controlled escalation, and mandatory human review.

12. Closing and contact

Optimizely upgrades do not have to be drawn-out projects dominated by repetitive refactoring. Agentic AI, applied with discipline, guardrails, and a skills library that compounds across engagements, is a practical way to accelerate the foundation work and shorten time-to-value. If you are planning a CMS or Commerce upgrade and want a walkthrough of the approach, you can contact https://niteco.com/contact-us/ and reference “Upgrade Machine”.

Part 2 of this series will be published very soon.

Blog posts by Hung Le Hoang