1
button on the client's CRM row starts the whole kickoff
7.7 min
from the press to Ready, measured on a live re-run
4,569
assessment objectives checked count-for-count on the clone
0
lines of per-client code: every script targets whichever base it is told to
The whole pipeline
Two buttons on one CRM row. Each goes through the same gate to a different runner.
Call
Write-back to the Clients row
Dashed box: outside service
The two jobs run on different containers on purpose. Enrich needs the internet and a language model, so it runs on the runner that has egress. Kickoff only talks to the NocoDB API, so it runs on the runner that has none. Both report through alerts only when something fails.
1
Identify: enrich the CRM row
Find the legal entity and its identifiers, and never guess which one it is.
The lookup, in order
- Website first, only when there are no identifiers yet. If the row has a domain but no UEI or CAGE, the job fetches the homepage (20-second timeout, first 400 KB, scripts and markup stripped, 8,000 characters kept) and a language model returns a one-sentence description, an industry, and a legal name only if the exact legal entity name is stated. If the fetch fails, the hop is skipped.
- SAM.gov second, most exact key first. The Entity API is searched by SAM UEI, then HLO CAGE, then by name. The name from the website is only ever a search term. It is never written as the legal name unless SAM.gov confirms it.
- Write what was confirmed. Legal name, UEI, HLO CAGE, registration status with its expiry, physical address and up to five NAICS codes.
What each result does to the row
| SAM.gov returns | Written to the row | Enrichment |
|---|
| An exact UEI or CAGE hit | All identifiers | Enriched |
| One match on a name search | All identifiers, plus a note to verify it is the right entity | Enriched |
| Several matches on a name search | Nothing. The top six candidates (name, UEI, CAGE, city, status) go in Enrichment Notes | Needs confirm |
| No match | A note saying so | Not found |
"Needs confirm" is the design working. The consultant picks the right entity, pastes its UEI, and presses Enrich again, which now takes the exact-match path.
Where the identity lands in the client base
| Table | Holds | Why it is shaped this way |
|---|
Client_Profile | One row: Legal Name, DBA, SAM UEI, HLO CAGE, Target Level, Engagement Stage, Primary Contact, and later the SSP's system identity fields | Legal Name is the single source for the {{company_name}} variable used in every generated statement and document |
CAGE_Codes | One row per facility or division: CAGE Code, registered entity, address, Is Primary | The HLO CAGE names the one parent organization, so it is a single field. A client can have many facility CAGEs, each with its own entity name and address, so those get a table |
2
Clone: kick off the engagement
A copy of the template, not a shared database with a client column.
Every client gets its own NocoDB base. There are no links between bases, so one client's rows cannot appear in another client's views, exports or jobs. The template is the working crosswalk: frameworks, requirements, assessment objectives, policy statements, triage questions and technologies, with every client table staged empty.
What travels with the clone
| Part of the template | In the clone | Why |
|---|
| Reference data (37 tables in all, including 1,735 requirements and 4,569 assessment objectives) | Copied in full | Every assessment needs it, and a copy lets a client base be worked without touching the template |
| Views, including the 11 "what to do next" views and the phase marker on every table | Copied | Built once in the template, so every client inherits the same guided surface |
| Webhooks behind the action buttons | Copied | The webhook URLs name no base. n8n reads the base from each call, so a button in a client base runs the job against that base |
| Client tables (people, inventory, flows, evidence, assessments) | Copied empty | They are staged empty in the template, so nothing needs pruning after the copy |
The kickoff, in run order
- Check for an existing base. If the row already carries an
Engagement Base Id, the job skips the clone and resumes the seeding against that base.
- Duplicate the template with data, views and hooks. The API answers at once with the new base id; the copy itself runs in the background.
- Wait for the tables with an uncached read, up to 4 minutes. Then clear the table lookup cache for the new base (see the decision log for why).
- Wait for the data until the clone's Frameworks, Requirements and Assessment Objectives counts equal the template's, checked every 6 seconds for up to 20 minutes. Counting against the template means the wait calibrates itself as the template grows.
- Rename the base to the company name, and write the base id, link and time back to the Clients row.
- Seed the identity:
Client_Profile (Engagement Stage "Onboarding"), the facility CAGE codes from the CRM, and the client variables, so {{company_name}} already resolves in step 7.
- Open one assessment per selected level. Each is titled "Company: CMMC Level N readiness assessment", linked to its framework, then populated with its objectives and requirement rows, the review console, and draft implementation statements seeded from the policy library. A CMMC Level 2 assessment comes out at 110 requirements and 320 objectives.
- Open the client's intake surfaces: a read-only list of evidence requests plus "Submit evidence" and "Submit a document" forms, each with its public share link turned on.
- Mark the row Stage "Active" and Kickoff Status "Ready". Any failure on the way sets "Failed" and exits non-zero, which is what raises the alert.
Re-pressing is safe after step 5. Every later step finds or patches what exists instead of adding a second copy: the profile, each CAGE code, each assessment by title, each framework link, the variables and the objectives. Before step 5, a re-press clones again; that gap is listed under known gaps below.
3
Target: base-aware automation
One set of scripts, pointed at whichever client base pressed the button.
What stays the same across clones, and what doesn't
| Stable in every clone | New in every clone |
|---|
| Table titles and column titles | The base id |
Foreign-key field names such as frameworks_id | Every table id and column id |
| Business keys: a requirement's number within its framework, an objective within its requirement | Link column ids and view ids |
So scripts look tables up by title inside the target base and never carry a hard-coded id. Sixteen engagement scripts and 86 hard-coded ids were converted when cloning was built.
How a job finds its base
- The NocoDB webhook sends the calling base's id with the row. n8n passes it to the runner as
base.
- The runner checks the id against a strict pattern and hands it to the job only as the
NOCODB_BASE_ID environment variable, never on a command line or through a shell.
- The job resolves every table by title within that base. With no base given, it targets the template, so shared-base behavior never changed.
- Jobs run one at a time per job and base. Different clients' jobs run side by side; a second press of the same job on the same base gets "busy". For Enrich and Kick off engagement, n8n then retries every 30 seconds, up to eight times, before alerting.
- For these two buttons the runner clears the trigger field itself, confirms the clear by reading it back, and only then runs the job. If the clear can't be confirmed, the job doesn't run.
Reference corrections after the clone
A clone is a snapshot, so a fix to a control or a policy statement in the template does not reach existing clients by itself. push_reference_data.py carries it out, touching only an allowlist of 15 reference tables. Client tables are never read or written.
| Template row vs client row, matched by business key | Action |
|---|
| Matched, with a field that differs | Update those fields |
| Missing in the client base | Insert it |
| Only in the client base | Flag it, never delete |
| Two client rows share the key | Refuse, and report the ambiguity |
| A column is missing or renamed in the client base | Skip that table and flag it; other tables and bases still run |
Matching is by business key, not row id, so a re-run finds nothing to do. It dry-runs by default and writes only with --apply.
Decision log
The calls that shaped onboarding, and when they were made.
| Decision | Why | When |
|---|
| One base per client, cloned from the full crosswalk | Hard isolation; the paid sync feature replaced by a push script | 2026-07-17 |
| Client identity tables built into the template | Adding them after the first clone would mean retrofitting every client | 2026-07-20 |
| Resolve tables by title; pass the base as an environment variable | The same scripts drive any base with no per-client code | 2026-07-23 |
| Clone with the webhooks | Buttons work in the new base immediately; routing follows the caller's base id | 2026-07-23 |
| The guided views live in the template, with no filters | Clones inherit them, and view filters were what broke duplication | 2026-07-27 |
| One-button kickoff from the CRM, rename included | Rename turned out to work with the API identity, so no step is left for a person | 2026-07-29 |
| Several levels can be kicked off at once; CAGE and UEI are optional | One assessment per level; a base can be stood up before the identifiers are known | 2026-07-29 |
| Enrich flags, never guesses | A wrong legal entity poisons every document generated after it | 2026-07-29 |
| Clear the trigger inside the runner, confirmed by read-back | A write-back loop once fired 64 enrich runs in 25 seconds | 2026-09-17 |
| Busy means retry, then alert | A second press during a run should wait, not vanish silently | 2026-09-17 |
Two traps the clone taught us
- The copy is asynchronous, and a cache made it look hung. The new base id comes back before its tables exist. Looking a table up in that moment cached an empty table list for the new base, so every later lookup failed and the data wait could never finish. The job now waits with an uncached read and drops the cache entry before using it.
- One malformed view filter stops the whole copy. Filters saved without a logical operator made NocoDB's duplicate job fail mid-clone. The data and the scripts that wrote those filters were fixed; a re-run reached Ready in 7.7 minutes with all 84 filters copied cleanly.
Known gaps
- The resume point is written after the clone. The base id reaches the Clients row only once the copy and rename finish, so a failure during the longest step leaves an orphan base, and a re-press clones again. The fix is to write the id the moment the duplicate call returns.
- A killed job stays "Provisioning". Nothing alerts if the runner is stopped mid-kickoff, and the run's progress isn't visible while it happens.
- Kickoffs for different clients share one lock. The lock is per job and calling base, and every kickoff is called from the CRM base. A second kickoff pressed during a run retries for about four minutes, then alerts instead of running.
- Reference pushes are manual and untested on a real client. Targets are named by hand, link associations aren't synced, and whether policy-statement fixes should reach existing clients automatically is still an open decision.
- Deleting a base leaves its database schema behind. Teardown is a guarded script that drops the schema too, and is deliberately not a button.
Engagement Stage is a label only. Nothing reads it yet.