SkillPixel LMS — Tenant Domain Provisioning

Two kinds of tenant hostname, two different pipelines.

Managed subdomains (<slug>.skillpixel.vn) are ours end to end. Custom domains (tenantc.com / learn.tenantc.com) belong to the customer. They need different tooling, so we ship them in different phases.

Backend host routing is already done and beta-verified. What was missing when this page was written (2026-08-21) was the edge work: DNS record, DigitalOcean domain, certificate, Firebase authorized domain — all manual back then. This page proposed how to automate it, and the proposal has since been built and released — see the status card below.

Legend FACTtrue today, verified DECIDEDalready agreed SHIPPEDbuilt and in production (v0.0.40) PROPOSALnot yet agreed DECISION NEEDEDneeds a call from Khang REJECTEDconsidered, not doing

Status update — 2026-08-31

This proposal was accepted, built, and released SHIPPED v0.0.40

Custom domains (Phase 4, SP-477): implemented 2026-08-26 as stacked PRs #316 (SP-617) → #317 (SP-618) → #318 (SP-619) → #319 (SP-477), merged 2026-08-27, released to production 2026-08-30 in v0.0.40. All four §3 decisions are settled — see the decision list below.
Managed subdomains: Option A shipped as provision-tenant-domain.yml (SP-633, PR #329). The "move it behind the platform API later" step arrived early: the backend dispatches the workflow as a GitHub App and platform-admin gained an Infra panel (SP-634, PR #332); the managed zone is enforced per environment (SP-636, PR #342).
One departure from §3: ownership proof is our own TXT challenge — _skillpixel-challenge.<host> = skillpixel-domain-verification=<token>, verified_at permanent, no expiry in v1 (SP-617) — not Cloudflare's pre-validation. Certificates still come from Cloudflare for SaaS.
Beta-verified 2026-08-27: tenant rsa and rsa.beta.skillpixel.vn provisioned end to end (platform-admin dispatch → Cloudflare CNAME → DO ALIAS domain and certificate → Firebase authorized domain) in about 2 minutes. Custom domains stay intentionally unsupported on beta: the Cloudflare environment is left unset there and activation answers a clear 503.
Still open on prod: enable Cloudflare for SaaS on skillpixel.vn with fallback origin customers.skillpixel.vn; add CLOUDFLARE_API_TOKEN / CLOUDFLARE_ZONE_ID and the three GITHUB_APP_* variables to Infisical prod; grant the backend Firebase SA firebaseauth.configs.update. Until then, custom-domain activation and provisioning dispatch answer 503 on prod.

0TL;DR

Managed subdomains

`<slug>.skillpixel.vn`

Keep per-hostname registration on DigitalOcean + Cloudflare. DECIDED 2026-08-02 Proven on beta and prod. Cost $0.
Automate it now as a GitHub Actions workflow_dispatch workflow. SHIPPED v0.0.40 SP-633, PR #329.
Move it behind the platform API later, not now. SHIPPED v0.0.40 Arrived early: backend dispatches the workflow as a GitHub App from platform-admin (SP-634, PR #332).
Do not pursue the DO wildcard. REJECTED

Custom domains

`tenantc.com` / `learn.tenantc.com`

Use Cloudflare for SaaS custom hostnames with lms-prod as fallback origin. SHIPPED v0.0.40 SP-619, PR #318.
Gate activation on verification (verified_at) and on the Firebase authorized-domain cap — warn at 75, block at 90. SHIPPED v0.0.40 Verification is our own TXT challenge (SP-617); guardrails in SP-618.
Apex (tenantc.com) and subdomains (learn.tenantc.com) both supported. Apex needs the customer's DNS to support ALIAS/CNAME flattening, or a www fallback — see §3.
Start only after managed subdomains have run on prod for a few weeks.

1Where we are today FACT

Everything in this section is true right now. No proposals here.

Edge and DNS

Prodskillpixel.vn is one DO App Platform app, lms-prod. Ingress: /api + /health → backend, / → static frontend. The frontend calls the API same-origin.

BetaSplit-host: DO static site lms-beta-frontend at beta.skillpixel.vn plus Cloud Run backend lms-backend-beta. Because of the split, beta uses the X-Tenant-Host hint header.

DNSZone skillpixel.vn on Cloudflare (NS cartman.ns.cloudflare.com / noor.ns.cloudflare.com).

PolicyALL DNS stays on Cloudflare. No NS delegation to DigitalOcean.

Backend routing DONE · BETA-VERIFIED 2026-08-20

Host resolves to a tenant only when it matches an ACTIVE tenant_domain of an ACTIVE tenant.
Platform hostnames (skillpixel.vn, www.skillpixel.vn, beta.skillpixel.vn) short-circuit to the default tenant.
Unknown hosts return 400/404. They never fall through to the default tenant.
Write path canonicalizes (IDNA/punycode, lowercase) and rejects reserved labels — admin api app assets beta dev mail staging static www — plus the platform apex skillpixel.vn and localhost. Customer apex domains such as tenantc.com are accepted (verified 2026-08-21 against normalize_host_header / is_reserved_hostname). It also refuses PLATFORM type (SP-606).
Activate/deactivate invalidates routing caches across all instances within about 1 second, via a Redis generation key (SP-607).

Gotcha: tenants created via POST /api/v1/platform/tenants start as PROVISIONED. Domains only route once the tenant is ACTIVE.

Domain rows and DO app domains right now

Where	Hostnames
Beta `tenant_domains`	`beta.skillpixel.vn` (PLATFORM); `historyclass.beta.skillpixel.vn`, `tenanta.beta.skillpixel.vn` (MANAGED_SUBDOMAIN)
Prod DO app	`skillpixel.vn` (PRIMARY), `canary.skillpixel.vn` (ALIAS)
Beta DO app	`tenanta`, `nosuch`, `historyclass` — all `.beta.skillpixel.vn`, all ALIAS

Fields that already exist on tenant_domains: status (PENDING/ACTIVE/DISABLED), domain_type (PLATFORM/MANAGED_SUBDOMAIN/CUSTOM), is_primary, verified_at.

What is manual today

From docs/specs/001-multi-tenant/edge-routing.md. Five steps per hostname, all by hand.

DigitalOcean: add <host> as an ALIAS domain on the app via doctl apps update. Exact domains auto-validate over CNAME; the Let's Encrypt cert lands in about 2 minutes.
Cloudflare: CNAME <host> → the app's default ingress hostname, DNS only (grey cloud).
Prod only, optional: flip to Proxied once DO shows ACTIVE. Universal SSL covers single-level *.skillpixel.vn. Beta must stay DNS-only — Universal SSL does not cover the nested wildcard *.beta.skillpixel.vn.
Firebase: add <host> to authorized domains. No wildcard support.
Our API: PATCH the tenant domain status to ACTIVE.

2Managed subdomains — recommended path SHIPPED v0.0.40

Same five manual steps as today, run by a workflow instead of a person. Shipped as designed: SP-633 (workflow), SP-634 (dispatch from platform-admin), SP-636 (zone enforcement).

platform-adminadd domain (PENDING)

→

GitHub Actionsprovision-tenant-domain

→

CloudflareCNAME, DNS only

→

DigitalOceanALIAS domain + cert

→

workflowpoll until ACTIVE

→

prod onlyproxied flip

→

Firebaseauthorized domain

→

our APIdomain ACTIVE

Three options

A — GitHub Actions workflow SHIPPED v0.0.40

provision-tenant-domain.yml, workflow_dispatch, inputs hostname, environment (beta|prod), action (add|remove).

Secrets are already in Actions — DO token and Firebase service account. The Cloudflare token is the only new one.
Every run is audited in the Actions log.
No backend change. Can ship in a day.
Later it can be triggered from platform-admin through the GitHub API, with no rewrite.

As built: shipped as SP-633 (PR #329) with exactly these inputs. The platform-admin trigger followed immediately — SP-634 (PR #332) dispatches the run as GitHub App managed-domain-provisioner and finds it by its deterministic run name; SP-636 (PR #342) pins the managed zone per environment and rejects out-of-zone hostnames with 422. First real use: rsa.beta.skillpixel.vn, provisioned in about 2 minutes on 2026-08-27.

B — Backend side effect END STATE · PHASE 2+/5

Provisioning runs as a side effect of update_domain(status=ACTIVE).

Needs a real state machine: PENDING → DNS_CREATED → DO_REGISTERED → ACTIVE.
Needs retries and async polling — the DO cert takes minutes, so it cannot live in a request.
Tie it to verified_at when we get there.
Not a prerequisite for SP-476.

Partially shipped: the trigger now lives behind the platform API — the backend dispatches the workflow as a GitHub App (SP-634) — but provisioning itself still runs in GitHub Actions, not as an in-backend side effect. Backend side effects exist only for CUSTOM domains (Cloudflare hostname + Firebase on activation, SP-619/SP-477).

C — Local script SKIP

Fast to write, but it puts prod secrets on laptops and leaves no audit trail. Only worth it as a stepping stone to A, and even then A is about the same effort.

Workflow steps

Validate. Hostname must be <label>.skillpixel.vn (prod) or <label>.beta.skillpixel.vn (beta). The label must pass the same rules as SP-606: lowercase, IDNA-valid, not reserved. Refuse anything else.
Cloudflare. POST /client/v4/zones/{zone_id}/dns_records with type CNAME, name <host>, content = the app default ingress (lms-prod-xxxx.ondigitalocean.app or the beta equivalent — read it with doctl apps get --format DefaultIngress), proxied:false. Idempotent: look up an existing record first.
DigitalOcean. doctl apps spec get <app-id> > spec.yaml, append {domain: <host>, type: ALIAS}, then doctl apps update <app-id> --spec spec.yaml. App IDs: prod e4f6e538-4564-414d-8e24-3fc73d0fb6ac (lms-prod), beta 8fd569c5-f115-4ffe-ad10-fac66a3f78e1 (lms-beta-frontend).
Poll. doctl apps get <app-id> until the domain status is ACTIVE. Timeout 10 minutes. On timeout, leave state as-is and fail loudly — do not roll back DNS.
Prod only. PATCH the Cloudflare record to proxied:true. On beta, never.
Firebase. Add <host> to authorized domains via the Identity Toolkit Admin API: GET/PATCH https://identitytoolkit.googleapis.com/admin/v2/projects/{project}/config, field authorizedDomains, read-modify-write. Projects: beta skill-pixel-beta, prod is the prod Firebase project.
Optional last step. PATCH /api/v1/platform/tenants/{tenant_id}/domains/{domain_id} with {"status":"ACTIVE"} using a platform-admin token — or leave it to the operator in platform-admin so a human confirms.

Concurrency: a spec update triggers a redeploy of that app. The workflow must share the release workflow's concurrency group so it can never race a production release.

remove runs the same steps in reverse: set the domain DISABLED first, then Firebase, then DO, then Cloudflare.

Secrets needed

Secret	Status	Notes
`CLOUDFLARE_DNS_API_TOKEN`	NEW	API token scoped Zone → DNS → Edit on `skillpixel.vn`. The local wrangler OAuth login has no DNS scope, so it cannot be reused.
`DIGITALOCEAN_ACCESS_TOKEN`	EXISTS	Already used by the release workflows.
Firebase service account JSON	EXISTS	Already in Actions / Infisical.

Two things we are not doing REJECTED

DO wildcard domain. wildcard: true needs DNS-01 TXT validation, and DO never surfaces the TXT flow for CNAME-managed (externally hosted DNS) domains. The only fixes are DO-managed DNS — against policy — or a DO support ticket.
Cloudflare Worker router. Rejected 2026-08-02: too complex, it conflicts with the same-origin /api rules, and Origin Rules host override is Enterprise-only.

Failure modes

Failure	What happens	What to do
DNS created, DO step fails	CNAME points at the app but the app does not serve that host.	Re-run the workflow. Every step is idempotent.
DO ACTIVE, Firebase fails	Site loads, but login breaks on that host.	Surface the error clearly in the run summary and re-run step 6.
Beta record set to proxied by mistake	TLS handshake fails at the Cloudflare edge — no cert for the nested wildcard.	The workflow refuses to proxy on beta. Fix by hand if it ever happens.
Tenant still PROVISIONED	Routing returns 400 even though DNS and cert are fine.	Check tenant status before provisioning; move the tenant to ACTIVE.

First use: historyclass.skillpixel.vn on prod, as part of SP-476 (Phase 2).

3Custom domains — recommended approach SHIPPED v0.0.40 · PHASE 4 · SP-477

Different problem: we do not own the DNS, so we cannot just add a record.

tenant adminenters tenantc.com / learn.tenantc.com

→

our APIcreate CF custom hostname, store PENDING, show TXT/CNAME instructions

→

customersets DNS

→

pollerCF status active → set verified_at

→

operatorset ACTIVE

→

platformFirebase add, canonical host flip, 301 from managed subdomain

Why Cloudflare for SaaS instead of building it

Verification is theirs. Cloudflare runs the TXT/HTTP ownership check and we read the result over the API.
Certificates are theirs. Cloudflare issues and renews the cert for the customer's hostname. No DO domain registration at all.
No backend change. The customer CNAMEs to a fallback origin (for example customers.skillpixel.vn) which points at lms-prod, so /api stays same-origin and Host resolution works as-is — SP-606/SP-607 already handle the hostname.
Cost. 100 custom hostnames included, then roughly $0.10 per hostname per month. Verify current pricing before committing.

As built (2026-08-26): one departure — verification is ours, not Cloudflare's. Ownership is proven by our own TXT challenge (_skillpixel-challenge.<host>, SP-617) and the Cloudflare custom hostname is only created at activation (SP-619). Everything else shipped as written: Cloudflare issues the certificates, the fallback origin is customers.skillpixel.vn, and backend host routing needed no change.

State machine

State	Means	Routes?	Side effects on entry
PENDING	CF custom hostname created, waiting on customer DNS.	No	Show TXT/CNAME instructions to the tenant admin.
VERIFIED	CF reports the hostname active. `verified_at` is set.	No	Set `verified_at`. Nothing else.
ACTIVE	Live for real users.	Yes	Add to Firebase authorized domains; canonical host becomes the custom domain; the managed subdomain 301s to it.
DISABLED	Turned off.	No	Remove from Firebase; canonical host falls back to the managed subdomain.

ACTIVE is refused when verified_at is null, or when the Firebase cap is hit — warn at 75, block at 90 active custom domains (spec §2.x). This closes the known debt item: "Custom domains cannot reach ACTIVE without verification" (debt.md).

Apex (root) domains FACT

Cloudflare for SaaS verification (TXT/HTTP) and certificate issuance work the same for an apex domain as for a subdomain. The only apex problem is DNS: a root domain cannot carry a CNAME under standard DNS, and our fallback origin is a hostname, not a fixed IP — DO App Platform has no static IP.

Situation	What the customer does	Result
DNS supports ALIAS / ANAME / CNAME flattening at the apex	Point `tenantc.com` at the fallback origin the same way as a subdomain. Cloudflare, AWS Route 53, DNSimple, Namecheap, NS1, DNS Made Easy and Google Cloud DNS (ALIAS) all support this.	Apex works directly.
DNS provider does not support it	Move DNS to Cloudflare — free plan, CNAME flattening is built in, nameserver change only, the registrar stays where it is. This is the default onboarding instruction.	Apex works directly.
Customer cannot change DNS provider	`www.tenantc.com` becomes the served primary host; the customer uses registrar/DNS "domain forwarding" for `tenantc.com` → `https://www.tenantc.com` (GoDaddy, Namecheap and the Google Domains successors all offer it).	Apex reaches the site via a redirect.

Product rule PROPOSAL — onboarding always registers both tenantc.com and www.tenantc.com for a custom-domain tenant. The customer picks which one is primary (is_primary already exists); the other one 301s to the primary. Verification is per hostname, so both must be verified before ACTIVE.

Rejected: Cloudflare "Apex Proxying" / BYOIP — Enterprise-only. REJECTED
Deferred (v2 option): our own redirect host on a fixed IP — for example a small DO droplet with a reserved IP — so customers on non-ALIAS DNS can use an A record for apex instead of registrar forwarding. DECISION NEEDED only for whether we build this v2 redirect host.

Backend / API changes needed

Create the CF custom hostname when a domain of type CUSTOM is created.
Status poller. Options: a cron fanned out per tenant, or one global job. Recommend one global job — this is platform-level work, not tenant-level.
Cap counter query plus the check inside update_domain.
Firebase add on ACTIVE, remove on DISABLED.
TenantUrlResolver needs no change — its canonical host rule already says custom domain first, then managed subdomain.

Decisions — all settled before Phase 4 shipped (locked 2026-08-26)

Apex domainsDECIDED
Apex domains are supported from v1 via ALIAS/CNAME-flattening, Cloudflare DNS migration, or www-primary + registrar forwarding (see Apex card). Still open, deferred: whether to run our own fixed-IP redirect host (v2). Not built in v1.
Shared auth domainDECIDED 2026-08-26
Keep per-domain authorized domains in v1, with guardrails instead of a redesign: warn at 75 and block at 90 active domains, exposed via GET /platform/domains/capacity and a capacity line in platform-admin (SP-618, PR #317). A shared auth.skillpixel.vn authDomain with redirect-based sign-in stays the fallback if the cap ever becomes a real constraint.
Scope boundaryDECIDED
Email sender domain, OG/SEO tags and canonical URLs on custom domains are tenant settings, not routing. Kept out of SP-477 scope as proposed. Per-tenant sender identity shipped separately in Phase 3 (SP-614).
Who flips ACTIVEDECIDED 2026-08-26
Operator click in platform-admin, behind a readiness-checklist dialog — tenant active, domain verified, Firebase capacity, Firebase configured, Cloudflare configured — with the backend enforcing every check independently of the UI (SP-477, PR #319). An unconfigured Cloudflare or Firebase environment returns 503 rather than silently skipping the side effects. Shipped in v0.0.40.

4Side by side

	DNS owner	TLS issuer	Verification	DO registration?	Firebase domain	Automation home	Cost	Phase / ticket
Managed subdomain	Us — Cloudflare zone `skillpixel.vn`	DO (Let's Encrypt); Cloudflare Universal SSL once proxied on prod	None — we own the zone	Yes, ALIAS per host	One per host	GitHub Actions `provision-tenant-domain.yml`	$0	Phase 2 — SP-476
Custom domain `tenantc.com` / `learn.tenantc.com`	Customer — apex + subdomains (see §3)	Cloudflare for SaaS	Cloudflare TXT/HTTP check → `verified_at`	No — fallback origin only	One per host, capped (warn 75 / block 90)	Backend API + global status poller	100 included, then ~$0.10/host/month	Phase 4 — SP-477

5Rollout order — where it stands (2026-08-31)

Create the Cloudflare DNS-edit API token and add it to GitHub secrets as CLOUDFLARE_DNS_API_TOKEN. DONE — exists as an Actions repo secret, alongside FIREBASE_SA_JSON_BETA / FIREBASE_SA_JSON_PROD.
Build provision-tenant-domain.yml. DONE — tracked as SP-633 instead (PR #329), shipped in v0.0.40.
Dry-run on beta, then run remove and confirm it cleans up. DONE (add path) — verified end to end 2026-08-27 with rsa.beta.skillpixel.vn, dispatched from platform-admin. The remove path has not been exercised on a live run yet.
Provision historyclass.skillpixel.vn on prod, then activate the domain in platform-admin. OPEN — prod still serves only the default tenant; this is the first real prod use when a tenant launches there.
SP-477 design review using section 3, and settle the four decisions. DONE — decisions locked 2026-08-26, Phase 4 shipped in v0.0.40 (2026-08-30). Custom-domain activation on prod additionally waits on the ops enablement listed in the status card above.
Move provisioning behind the platform API — option B. DONE (trigger) — backend dispatches the workflow as a GitHub App (SP-634); the work itself stays in GitHub Actions by design.