Chapter 27: The Three Phases of a Customer Project

Adapted public course. Concepts, procedures and examples are adapted from the internal textbook. Case sizes, timings, improvement figures and target thresholds are illustrative, not site delivery results or universal standards. Verify tools, platforms and skills in your environment. Prompts do not grant permissions and retry counts do not authorize recovery. Preserve work and verify targets, sharing and external-state effects first.

Lesson explanation

27.1 The Industry Benchmark: Scoping, Validation, Delivery

Consider a typical scene. Your team signs with a manufacturing company to build intelligent equipment-failure ticket classification and dispatch. At kickoff, the CIO is direct: “Three months, in production, thirty percent less handling time.” After the call, you face ticket data scattered across three systems, exceptions nobody can fully explain, and actual users in a factory a thousand miles away. The first question is neither how to code nor whether AI should draft a solution. It is what rhythm the project should follow.

The preceding parts supplied tools from session to project level. Customer projects have a lifecycle that determines when to use those tools and when to put them away. The industry benchmark offers three phases. According to The Pragmatic Engineer’s August 2025 interview with OpenAI’s FDE leader: early scoping involves several days at the customer site; validation builds evals, adds features, and improves against evals; delivery involves several days on site each week.

Phase one: scoping — trade several days on site for three judgments. Scoping asks whether this is a real problem, whether data can be obtained, and whether the proposed solution is broadly feasible. None asks how to write the code. Why on site rather than remote meetings? Real problems live in frontline complaints rather than executive slides, and actual data lives in customer systems rather than interface documents. Two days on site can expose an answer such as “export approval takes at least three days” that a month of remote questioning misses. Scoping produces no running deliverable. It produces an honest decision: enter validation or stop here. The willingness to say no is this phase’s most underrated professional ability. Politely moving a doomed project into validation can consume three months of trust on both sides.

Phase two: validation — improve against evals. Its core is building evals, adding features, and improving against them, rather than feature writing alone. Improvement needs a measurable slope: rerun after prompt changes, model switches, and solution adjustments. Without an evaluation set, iteration becomes free fall. Chapter 21 establishes that evaluation approach, including the minimum viable eval, five quality dimensions, golden sets, and regression. Validation has a specific object: the solution’s quality on real customer data, not its appearance in a demo environment. Its exit criterion is equally clear: do results on the evaluation set support the production commitment?

Phase three: delivery — several days on site each week. Around production rollout, the rhythm changes to weekly on-site work. The last mile lives in customer networks, permissions, change procedures, and people: who holds up approval, which workshop supervisor distrusts machine dispatch, whom to call for an early-morning alert. These answers are in the field. Embedded work is shortening the feedback loop, not standing by as a caretaker. If adoption stalls, sit beside users that day and see the obstacle. When a failure pattern appears, return it as an eval case that day. Chapter 28 explains stakeholder collaboration and when presence is essential versus asynchronous work sufficient.

These phases provide a project-level timetable. A legitimate question follows: with the six-step workflow, the three-step process, Job’s eight stages, and now three customer phases, which should an engineer follow? The next section reconciles them.

27.2 Four Process Models: Four Zoom Levels, Not Four Alternatives

The four models do not conflict because they operate at different granularities. The six-step workflow is session-level; the three-step process is for high-risk sessions; eight-stage construction automates a project; and the three phases govern customer projects. Like map zoom levels, each answers a different question. A city map and a street map are useful at different scales.

Process model	Granularity	Source	Applies to	User	When
Six-step workflow: decompose, issue instructions, code, accept, branch, update blueprint	Session: one AI coding task for one feature	Chapter 7	Routine features with clear requirements and controllable risk	Any developer using AI coding	Default mode for an individual feature
Three-step process: research, strategize, implement	High-risk session: withhold execution authority to require slow thinking	Chapter 19	Above-moderate complexity, unfamiliar domains, or costly solution mistakes	Developers facing high uncertainty	Escalate from default when work exceeds half a day or decisions have broad impact
Eight-stage fully automated construction: Job	Project: complete chain from zero to deployment	Chapter 17	Low-risk new projects with mature blueprints, complete testing grids, clear stacks	Users of the skill system	Starting a new project whose path is proven automatable
Three phases: scoping, validation, delivery	Customer project: weeks to months of delivery	This chapter	Embedded projects accountable for customer outcomes	FDE	Entire customer-project lifecycle

The models nest. The three phases form the outer shell. During validation, if the project meets the Section 19.3 authority boundary integrated in Chapter 17 — a mature blueprint, verified technical path, low risk, and complete testing grid — eight stages can build it from zero. Failure downgrades execution mode only. Beyond an already approved noncritical fallback, changes to acceptance standards, customer commitments, or scope need an authorized human decision; failed gates cannot become PASS by lowering the bar. Individual features then use six steps, with high-risk work escalated to the three-step process. Conversely, choose a process at one granularity for a particular question. You do not execute the six-step workflow in a customer kickoff or enter the delivery phase while writing a function.

Choose from coarse to fine with three questions. Which phase is the customer project in? Scoping, validation, or delivery. How should this project begin? Eight-stage automation if its boundaries hold; otherwise Chapter 16’s manual blueprint route. How should this feature be built? Six steps for routine work, three for high risk. Each level constrains the next: validation goals determine which evals to build, and their protected floor determines whether full automation is acceptable.

This also resolves an audit issue. Earlier chapters presented the four models without explaining granularity, inviting a four-way choice or apparent contradiction. The clearest example pits Chapter 19’s ban on substantive code before the problem is understood against Chapter 17’s fully automated zero-to-deployment workflow. One withholds execution; the other grants it. The mapping resolves the tension: the prohibition defends high-risk sessions; full automation authorizes low-risk projects. Their respective chapters define the applicable boundaries. Different granularities do not compete. Chapter 15’s skill overview now also cross-references this chapter.

27.3 Practical Scoping: What Must Several Days on Site Produce?

Scoping is easiest to underweight. It consumes the project’s most expensive time — the FDE personally on site — yet returns intangible findings. Here is an operational approach using Chapter 11’s Event Storming, extended to customers.

On-site interviews: bring Event Storming into the customer’s meeting room. Chapter 11’s question still holds: ask what events occur in the business, not what features the system needs. For equipment tickets, what happens between an alarm and an engineer arriving? A frontline operator supplies past-tense events: “The alarm sounded, somebody shouted in the on-call group, Engineer Wang thought it was false but checked anyway, then filled out a ticket.” This is more trustworthy than a requirements document because it describes the actual process. Extend it in three ways.

First, interview by stakeholder level. Operators provide events; process owners provide rules and exceptions; executives provide the value narrative and budget boundary. Accounts will almost certainly differ. An executive wants intelligent dispatch, while operators know three unwritten dispatch rules held only in experienced technicians’ heads. When accounts diverge, follow frontline evidence for event flow and executive direction for priorities. Chapter 28 covers language and artifacts for each group.

Second, use the advantage of being present. At every “what happened next,” get up and inspect the actual screen, ticket, or spreadsheet. Remote interviews reveal the intended process; presence reveals the actual process. Their difference is a rich source of requirements: undocumented rules, improvisations, and workarounds often determine whether an AI solution succeeds.

Third, bring disagreement records to the customer site. Chapter 11’s record format — disagreement, decision, reason, affected scope, date — gains value here. Customer organizations have frequent internal disagreements, likely to reopen when projects restart or contacts change. One record can spare your future self an entire repeated argument.

Data inventory: half of an AI project’s success is in the data, which the customer controls. Your own data may be close at hand; customer data has sovereignty, formats, and gatekeepers. Produce an inventory during scoping (the following is a fictional teaching example; supply provenance, approval basis, and verification dates in practice):

System/source	Data available	Format and quality	Access path	Timeliness	Compliance constraints
Ticket system	About 40,000 historical tickets over 18 months	Structured fields plus long free text of uneven quality	API export, with IT approval taking about 3 working days	T+1	Project environment only; no transmission to external models
On-call group records	Two years of instant messages	Unstructured, including images	No export channel; platform administrator must retrieve manually	Uncertain	Contains personal information; usable after de-identification
Equipment register	Complete equipment records	Structured and well maintained	DBA access to a read-only database	Real time	No special restrictions
Knowledge base: wiki	Failure-handling manuals	Semi-structured, not updated for three years	Self-service export	Static	No special restrictions

The table answers whether validation has usable raw material. Without accessible historical data, Chapter 21’s evaluation set cannot be built; surface that during scoping, not after validation begins. It also exposes the classic trap: data always exists in meetings but access rights disappear at export time. The access-path and timeliness columns force that issue into view.

Rapid PoC: boundaries matter more than speed. Scoping often includes a one- or two-day minimal demo to test a key assumption about a data/model combination. Its purpose is aligning understanding, not delivering early. Keep three boundaries. First, no commitments: explicitly say that anything demonstrated will be revalidated against the evaluation set during validation. Second, no production: do not connect the PoC to live business flows or retain production data. Third, stay connected to evals: record every case that persuades the customer as it happens. These become validation’s first seed cases, corresponding to real inputs in Chapter 21’s 6 + 2 + 2 mix. A clear sign of collapsing boundaries is the customer offering acceptance objections to the PoC. Reframe immediately through the three phases: “That is exactly what we will address during validation.”

Scoping exit criteria. Leave the site with four things: an event flow and initial REQUIREMENTS.md in Chapter 11’s format, a populated data inventory, eval seed cases, and an explicit proceed-or-stop decision. Without the last two, scoping is unfinished. Avoiding judgment by scheduling next week’s meeting turns expensive presence into cheap remote waiting.

[Exercise] Design Three Days of Scoping for a Fictional Customer

Task: A chain of dental clinics wants assistance with medical-record summaries and follow-up reminders. You have three days on site. Produce: (1) interviewees grouped into frontline, middle management, and executives, with three event-based questions per person; (2) a data inventory with at least four rows and concrete access-path, timeliness, and compliance entries; (3) a rapid-PoC scope statement expressing the three boundaries in words you would actually say on site; (4) an exit-criteria self-check identifying which of the four scoping outputs you are most likely to miss and why.

Guidance: Notice two gaps. First, event questions: by your second interviewee, executives may be unable to answer operational event questions and operators unable to answer value questions. That is the point of layered interviewing. Second, compliance: medical records can be physically present while not a single field may leave. The inventory’s final column becomes a governing solution constraint rather than boilerplate.

Reference direction for instructors: Check that event flow follows frontline evidence rather than executives’ words translated into events; that data inventory includes who can approve access and how long approval takes, rather than simply claiming data exists; and that the three PoC boundaries become usable spoken statements. A common counterexample copies a feature wishlist for three days while addressing none of the four exit criteria.

---

Sources for the industry facts in this chapter: OpenAI’s three customer-project phases — several days on site for scoping, building and improving against evals during validation, and several days on site weekly during delivery — come from The Pragmatic Engineer’s August 2025 interview with its FDE leader. Event Storming and the disagreement-record format come from Chapter 11. The unified mapping of four processes, three scoping questions, data inventory, and three PoC boundaries are this book’s methodological development, not a particular company’s internal practices. No additional industry statistics are cited. Unsourced cases, data volumes, target percentages, approval times, and exercise cadences are teaching assumptions, not customer measurements or industry benchmarks.

---

Independent exercise

Use fictional or authorized deidentified material. Answer independently before revealing the reference. Save chapter-27.md with versions, decisions, evidence and gaps.

For scoping, validation and delivery, specify inputs, outputs, exit and stop conditions. Respond to a request to put the demo directly into production.

Required chapter artifact: client-project stage-exit table

The submission for this independent exercise must contain the evidence below. The existing prompts supply content but do not replace these acceptance items.

• O1: The client-project stage-exit table connects scope, validate and handover in input-output order
• O2: Every step in the client-project stage-exit table states its owner, input, output and completion rule
• O3: The artifact names owners, evidence and exit conditions for all three stages of one client project

Reveal reference feedback (answer first)

Reference feedback

Scoping confirms the problem, access and business ownership; validation uses authorized data and independent evaluation; delivery requires handover, recovery and adoption observation. Demos cannot bypass gates. List missing evidence and owners and agree on controlled validation without an unconfirmed release promise.

Self-review and next steps

Check whether your decision is explicit, evidence reproducible and unknowns honestly recorded. The reference illustrates one defensible approach, not a unique answer. Seek peer review for alternatives with equivalent evidence. Mark unsupported parts unfinished and revisit the corresponding step.