| Priority | Issue | Details |
|---|---|---|
| High | AWX project does not auto-sync | The lab templates run from AWX project 10 (crc-awx-labops), which has update revision on launch disabled. Merging a fix to main has no effect until the project is synced manually. Currently synced to f4293d5 (PR #42). |
| High | Windows licensing gaps | POD07-SRV and POD11-SRV are still unactivated (License Status: Notification); replacement Datacenter keys have been requested and both pods are held back from auto-advance (crc_held_back_pods: 7,11). Neither server may be issued to a student until activated. Both domain controllers are now permanently licensed retail Datacenter — see Domain Controllers. |
| Medium | Guacamole sees every student as the proxy, not their own address — fix after the current cohort | The edge proxy (crc-proxy-gateway-01, 192.168.1.55) forwards requests to Guacamole without Tomcat trusting X-Forwarded-For, so every login is attributed to 192.168.1.55. Guacamole's InMemoryAuthenticationFailureTracker bans an address after 5 failures, so five mistyped passwords from any one student temporarily lock out the whole cohort, and the per-user login history records the proxy instead of the real client. Fix: add REMOTE_IP_VALVE_ENABLED=true (plus the internal proxy/trusted-header settings) to the guacamole service in /opt/guacamole/docker-compose.yml and recreate the container — that drops live sessions, hence the deferral. |
| Medium | Student sessions end with Connection to guacd timed out — fix after the current cohort |
Long student desktop sessions are being torn down abnormally rather than logged out (GuacamoleWebSocketTunnelEndpoint - Connection to guacd terminated abnormally). Neither nginx layer sets proxy_read_timeout / proxy_send_timeout, so nginx's 60s default is the leading suspect for an idle tunnel being closed under the session — confirm against guacd's own log before changing anything. Candidate fix: raise the websocket timeouts on the guac.01 / guac.02 vhosts on the edge proxy and on the Guacamole VM's own vhost, and set api-session-timeout in guacamole.properties. |
| Medium | Lab credential rotation has no AWX bridge | The encrypted store, the Start Here reveal and the staff rotate/audit path are live on production, and Pod01–Pod03's working passwords are seeded. A rotation stages the new password as pending_push and students keep seeing the password the lab accepts, but nothing applies a staged password to AD and Guacamole yet — so rotations are unusable until that playbook is built. |
| Medium | LabOps AI open items | PBS protection for VMID 100 (drcc-labops-01) is not configured; the read-only AWX service token svc-drcc-labops-ai-ro has not been issued; non-owner/student denial and Refuse-in-Direct-Chat are unproven on production. All five write switches are false. See LabOps AI. |
| Medium | Legacy Guacamole tiles still granted | student07 and student11 still carry the legacy POD07-DC / POD11-DC connections from the shared-DC era. Every other student has exactly one tile, PODXX-SRV. |
| Medium | Guacamole passwords never rotated between cohorts | Cohort-1 learners can still sign in to Guacamole (they fail at the RDP step). Rotate student01–student20 as part of the credential work. |
| Medium | Guacamole PODXX-GW tiles hidden |
The 20 gateway tiles were created with protocol http, which guacd does not implement, so they failed and looped. Students' READ permission was removed; the firewall is reached from a browser inside PODXX-SRV. |
| Medium | IA M3-L2 domain-wide policy | In member-server mode the lab is graded against the session host's local account policy, which is the student's own to change. The shared acs-p01.local Default Domain Password Policy stays hardened centrally; a per-pod fine-grained policy is still the durable fix for any shared-DC fallback. |
| Medium | Seed templates overwrite student work | AC, IA, SI and SC seed playbooks have no "already seeded" guard (only MP/PE check .families/<F>.seeded). Never launch a seed template while a class is in progress — use the reset templates. |
| Medium | Live Ops mapping coverage | Only a small fraction of the ~228 Moodle learners map to a portal account, so most Live Operations rows carry no lab assignment. This resolves as cohorts register through the portal. |
| Low | ide.digitalrcc.com is down |
The IDE vhost serves but its backend (192.168.1.61:3000) is not running, so the name times out / returns 504 — the same as the old crc.ide.tcecure.com. Unrelated to the rename. Decide whether to remove the vhost entirely. |
| Low | MP seeding drive letters | MP attaches VHDX media and drive-letter assignment is machine-global. MP now seeds per member server so pods no longer contend, but two MP jobs must not target the same host at once. |
| Task | Status | Notes |
|---|---|---|
| Build the AWX rotation bridge for lab credentials | Pending | Apply a pending_push password from the portal store to the AD account and the Guacamole user, then mark it active |
| Student learning report (transcript) | Pending | Sectioned per-control-family report card issued alongside the completion certificate, from Moodle completions plus lab verification. Moodle currently stores no completion dates or scores, so no date or grade column |
Retire the legacy *.tcecure.com names |
Pending | Safe once a cohort has run entirely on the new names; also delete labops.drcc.digitalrcc.com and set the eight DNS TTLs back to Automatic |
| Replacement Datacenter keys for POD07 / POD11 | Pending | Supplier contacted; the other 18 member servers are activated |
| Per-pod fine-grained password policy | Pending | Only needed for a shared-DC fallback now that IA M3-L2 grades the local policy |
| Push the final guide-update branch | Blocked | Revised AC/IA/SI/SC/MP/PE DOCX/PDF committed on devin/1787417238-guide-updates; every push to the tcecure remote returns HTTP 403 through the git proxy (fetch works). Reported to Cognition |
| Confirm snapshot rollback reset logic | Pending | Verify that rolling back student workstation snapshots correctly resets lab state |
| Re-disable password auth on PVE1 | Pending | Currently enabled for convenience; should be key-only |
| Disable rpcbind on PVE1 and PVE2 | Pending | Security hardening |
| Dreamwall firewall rules cleanup | Pending | Remove stale rules, document active rules |
| Git-based deploy on VM 105 | Pending | Automated deployment for the tracker host |
| Remove workstation hosts from AWX inventory | Pending | ws01-p01 … ws01-p10 entries in CRC-LabPods are placeholders superseded by crc_pod_servers |
| Task | Date | Notes |
|---|---|---|
| Repositories caught up with production | Sep 2026 | drcc-lab-companion #12/#14/#16, crc-awx-labops #43–#46 and tracker #15 merged — snapshot capture, Live Ops collectors, the unseeded-AC/SC reset fix, the read-only local_drccops Moodle plugin and Guacamole role, and the hostname cutover. AWX project 10 synced to bf85db9 |
| Staff star ratings on finished cohorts | Sep 2026 | drcc-lab-companion PR #19 — manual 1–5 stars per student per cohort on /admin/progress, manager-only, hidden from students |
| Finished cohorts archived out of the active list | Sep 2026 | PR #18 — cohort 1's 11 rows set to completed and moved to a Finished cohorts section linking to the frozen record at /admin/progress?cohort=1 |
| Current cohort imported through the portal | Sep 2026 | Three learners seated on Pod01–Pod03 with portal invites, queue and seat emails delivered over the relay |
| Student mail delivers for real | Sep 2026 | PR #17 — EMAIL_DELIVERY_MODE=live, Google Workspace relay, branded queue/seat templates, late imports seated into a running cohort |
| DigitalRCC mail authenticated end to end | Sep 2026 | no-reply@digitalrcc.com, SPF + DMARC added at Namecheap, DKIM signing, "via" suffix removed, Moodle renamed to Digital Resilience Community Clinic / DigitalRCC. See Email Delivery |
| Branded Supabase auth templates installed | Sep 2026 | Invite, recovery and confirmation templates with the real logo and the safe /auth/invite?token_hash= route; verified with a live invite |
Public names moved to digitalrcc.com |
Sep 2026 | lms, awx, guac.01, guac.02, wiki, ide, training, labops — certificates issued and dual-serving with the old names. See Reverse Proxy Configuration |
| Host-rights waivers retired | Sep 2026 | PR #42 — IA M2-L1, IA M3-L2 and MP M1-L1/M1-L2 are graded for real in member-server mode; AWX project 10 synced to f4293d5 |
| Student logon retired on DC01/DC02 | Sep 2026 | Students sign in only to their own PODXX-SRV; the DCs serve DNS, LDAP/LDAPS, Kerberos, GC and SYSVOL/NETLOGON only, which also removes the RDS grace-period exposure |
| Pod01/Pod03 member-server pilots, then fleet rollout | Sep 2026 | Real Guacamole + RDP sign-in proved local-admin boundary, ADUC against the DCs, stored-credential task, evidence write and cross-pod denial |
| All six families reset and reseeded in member-server mode | Sep 2026 | Tracker reads 0% / not_started on all 20 pods; AD identities rotated and re-scoped |
| Live Operations and Impact split | Sep 2026 | PR #13 — admin Live Ops over real Moodle activity plus a PII-free donor Impact view with Presentation Mode |
| Per-cohort progress snapshots | Sep 2026 | PR #11 — interim rows while a cohort runs, frozen final rows afterwards; cohort 1's receipt reconstructed from pre-reset verify jobs |
| Moodle-backed Community Impact | Sep 2026 | PR #8 — read-only Moodle projection, roster, filters and Presentation Mode |
| All 20 pod member servers built, hardened and mapped | Sep 2026 | POD01-SRV–POD20-SRV (VMs 401–420) domain-joined in OU=PodServers, matching studentXX as the only local admin / RDP user |
Seed / verify / reset retargeted to crc_pod_servers |
Sep 2026 | All six families verified fleet-wide; crc_shared_dcs kept as the rollback path |
Guacamole PODXX-SRV connections issued |
Sep 2026 | 20 connections at 10.50.XX.20, granted only to the matching student, no stored passwords |
| Portal issues the member server as the session host | Sep 2026 | PR #7 — PODXX-SRV / 10.50.XX.20, DC01-P01 / DC02-P01 as directory controllers only |
| Cohort 1 evidence archived on DC01 | Sep 2026 | C:\CohortArchive\COHORT1-20260902 — 3,675 files, SHA-256 manifest, per-pod counts, restore notes |
| LabOps AI Direct Chat + Phase 2 security controls | Aug 2026 | /admin/labops/chat, gateway/agent secret split, per-investigation workspace, default-deny egress, cross-run denial, restart recovery |
| Read-only Proxmox health monitoring | Aug 2026 | drcc-monitor@pve + lab-status token (audit-only), poller publishing outbound every 45s — port 8006 stays unexposed |
| Authenticated per-pod progress API | Aug 2026 | GET /api/v1/pods/<NN>/progress on the tracker host, consumed server-side by my.digitalrcc.com |
| Pod firewall reachable from the lab desktop | Aug 2026 | 11 wedged gateway VMs restarted and a boot-persistent pod-routing.service added, isolation preserved |
| Hardened domain password policy applied | Aug 2026 | 12 chars, all four character classes, lockout 10, history 24, max age 90 |
| Guides and all 20 quick starts re-rendered | Aug 2026 | Print-stylesheet PDFs, AC CMMC mapping corrected to 3.1.1/3.1.2 |
| All six families seeded + verified on pods 1–20 | Aug 2026 | AC, IA, SI, SC, MP, PE (57 labs) |
| Auto-verify + auto-advance schedules live | Aug 2026 | Schedules 7–13, every 30 min, advance_enabled and certificates_enabled on |
| Training Tracker extended to all six families | Aug 2026 | MCP v2.5 on 192.168.1.61 queries verify templates 13, 16, 19, 22, 28, 31 |
| Remove WS01 connections from Guacamole | June 2026 | Deleted 20 PODXX-WS01 connections |
| CDW zone fully operational | May 2026 | VMs 300-306 on PVE2 vmbr50 (192.168.50.0/24) |
| Wazuh SIEM deployed (CDW) | May 2026 | Docker-based Wazuh 4.9.2 on crc-cdw-monitor-01 |
| Expand from 10 to 20 pods | April 2026 | OU structure, users, groups, delegation for Pod11-Pod20 |
| Stand up Wiki.js | April 2026 | VM 106, Docker, PostgreSQL |