<ai>
Last updated Mon 12 Oct 2026. Nothing here is urgent; skip or push back on any of it. Since Saturday I went looking for everything that should be in the archive and isn't: data that never left the colo, links that point nowhere, copies that stopped part-way. Below is my reconstruction of how the export happened, what's missing and where it might still be, then the asks, grouped as before.
The short version
- At least 548 TB of Esperanto storage existed in July 2024 (the last full
df, IT-2454; not counting the VM datastore, the Veeam pool and the IO FreeNAS). 74.9 TB of it reached mordast, and 65.9 TB of that is the CAD home, mostly foundry kits. Of Esperanto's own engineering storage, about 1.8% arrived (8.9 of ~482 TB; the 482 TB includes duplicate copies such as the 22 TB A0 tarball, so the share of distinct data is a little higher). - The full-chip layout is not here. The A0 tape-out release (30.6 TB by
du; soc_top alone 8.4 TiB) moved in 2023 to sc-truenas03, a 22 TB tarball, AWS S3 and a LaCie drive, and none of those reached mordast except a 10 GB fragment of the AWS copy (small files from 10 of the 26 blocks). Alchip received the full-chip OASIS files in 2020. - The June 2025 Wasabi upload of
/engstopped at the user directorykpn
and didn't follow symlinks. 123 of the 261 named/engowners were never uploaded, and 58 of the 92 top directories that were uploaded were still transferring when it stopped (inferred from upload times)./project/pd,/scratch, the Jenkins workspaces, syseng and the three TrueNAS servers were never uploaded at all. - So the best surviving copies are whatever stayed at the colo and in the cloud accounts: sc-truenas01/02/03 (including Veeam backups of
/engup to 10 Jan 2025), the old IO FreeNAS, the AWS account(s), and Google shared drives holding at least 16 TiB of ex-staff tarballs. Meanwhile stash holds the only copy of what did arrive, protected only by two snapshots on the same pool A12. - Security items found while looking:
/etc/auto.home-selecton mordast is still world-writable, and/archive/gdrive,/indexand/home/githubare mounted read-write and writable by any account A12. Credentials in the archive (AWS keys, the esperanto.ai wildcard TLS key, lab server SSH keys) are readable by every account on mordast, even the mode-600 ones. The lab flash PC, whose key is also in the archive, has ssh commands to esperanto-soc2 (whose host key it holds) and to roman@10.20.10.1, the old lab gateway address A06.
If you answer only one thing: what happened to the colo hardware after June 2025 A01? Almost every large gap has its best surviving copy on one of those boxes or in the AWS account.
Every number here has a path, a ticket or a log line behind it, and I say inferred
where it's my guess. The full evidence is in my home on mordast, /home.local/yaroslav/claude/vault-2026-10-11/synthesis/missing/ (missing.md, missing.json; readable by root only; I'll send whatever you want).
All 20 asks at a glance
In the order I'd take them: priority 1 first. The details, with commands, are further down, grouped by who can do them.
- A01What happened to the colo hardware after June 2025?question · priority 1
- A12Two permission fixes on mordast and stash, plus a fresh snapshotroot · priority 1
- A03AWS: list the buckets and restore the A0 copies from Glacierbrowser · priority 1
- A04Google Workspace: copy the shared drives that weren't copiedbrowser · priority 1
- A16Alchip: the full-chip OASIS and ICC2 databasesthrough you · priority 1
- A10Lab hosts: what are esperanto-soc2…10 and esperanto-flash?root · priority 1
- A06Rotate or revoke the credentials in the archivebrowser · priority 1
- A02If an old box survives: what to copy first, and how to do it safelyroot · priority 1
- A05Export the linked power Sheets and decks by id, and leavers' Drivesbrowser
- A07Any Atlassian backup from before the site was deactivated?question
- A08Who ran the June 2025 upload, and why did it stop at
kpn
?question - A09What was stash's empty
archive/engmeant to receive?question - A14Was the Perforce depot saved anywhere?question
- A17Amkor, Radian/Boyd and Meta: package and cooling modelsthrough you
- A18Ex-Esperanto engineers: personal copies of the power datapeople
- A11Make the three unreadable mirror repos readableroot
- A13Tailscale SSH check period for mordastbrowser
- A15Is gitlab.com/esperantotech alive, and what is the device
gitlab
?question - A19Teradyne: a dump of the ATE test-program repositorythrough you
- A20Do any Synopsys or Siemens licences still work?question
Saturday's 16 items: 1 done, 8 updated or answered, 5 still open, 2 dropped
| 10 Oct item | Now | What changed |
|---|---|---|
1. Mount /archive/eng and /archive/esperanto read-only; pin all mounts in /etc/fstab | ✓Done | Checked today: /etc/fstab lines 25–29 (written 11 Oct 15:52) pin all five; both new mounts are up, nfs 4.2, read-only (atlassian, gdrive and /index are read-write: A12). Thank you. /archive/eng is empty (0 entries, a few hundred KiB used; the 26 TiB that df shows is the pool's free space): question A09. |
2. chmod the unreadable objects of gepardo-soc and customers/denso | ○Still open (extended) | Unchanged today (objects still mode 0600, owner shelley); esperanto/qqq has the same problem (360,726 objects). Now A11. |
3. /etc/auto.home-select is world-writable | ○Still open | stat today: -rwxrwxrwx root, modified 11 Oct 15:52 together with fstab, mode unchanged. Now A12. |
| 4. Tailscale SSH check period | ○Still open | I can't check it from mordast. Now A13. |
5. Share Original Esperanto Shared Driveand VLSI Engineering; ~300 vendor PDFs got 403 | ↻Updated | Correction: the 403s were 252 distinct objects (27.4 GB), mostly All Hands decks and Meet recordings, not vendor PDFs. The ask now names 10 shared drives (7 by id), the ex-staff tarball drive first. Now A04. |
| 6. Jira histories cut at 40; does an Atlassian XML backup exist? | ↻Updated | Confirmed: 650 issues, 24,834 of 433,529 history entries. The same backup would also bring back the private MFGTST project, 45 attachments on archived pages and the TEAM space. Now A07. |
7a. git lfs fetch on the mirror (maxion_top.def.gz, PLL GDS) | –Dropped | Not needed: the design files are in the mirror's .git/lfs, and maxion_top.def.gz (114 MB) is also in many /eng checkouts. Only 67 ML/tool LFS objects (2.04 GB) are missing. |
| 7b. Re-export the power/thermal Google Sheets behind dangling shortcuts | ↻Updated | Google-native files in the copied drive did export (3,585 of 3,586), and 8 of the 11 named shortcut targets are already here in the Aug 2025 Drive subset. What's missing are Sheets and decks that were never in the copied drive. Now A05 (a list of ids). |
| 7c. Is there a newer thermal/power characterization than the 2017–18 work? | ↻Answered | Yes, but not on mordast: shire-level PT-PX runs with FSDBs (Jenkins, /scratch PowerRuns2, alilic archive_ptpx), soc_top RedHawk-SC EM/IR (A0 release, rupeshr) and the 2024 card power campaigns (mv-gb09). No die-level thermal simulation survives anywhere. See A01 A02 A03 A10 A18. |
| Q: Perforce or SVN? GitHub repos left out of the mirror? | ↻Answered | Perforce: the journal is here but content for only 112 of 23,677 depot paths A14. SVN: hosted by Teradyne, only an r23 working copy here A19. The mirror has 513 repos (not 435), 511 of them cloned from gitlab.com, so none of the gitlab.com repos is missing; 44 on-prem-only projects (32.8 GB) are only in the Sep 2023 GitLab backup tars on mordast. What's missing is MRs, issues, CI and wikis A15. |
| Q: Does a full-chip GDS or OASIS exist anywhere? | ↻Answered | Not on mordast. It was /project/pd/release/A0; see the short version above and A01 A02 A03 A16. |
| Q: The bigger NFS archive: what has to happen to switch it on? | ↻Updated | /archive/eng is mounted and empty; its name matches sc-truenas01's archive/eng, but its export goes to a different subnet A09. The 53 home exports on stash (plus home/github) are the current team's homes, not landing spots. |
| Q: Where is the index your agent built over the Atlassian dump? | –Dropped | Not needed: the archive browser's own index covers all 44,649 issues and 5,595 pages. |
| Q: TEAM (Team Meetings) space: empty or no permission? | ○Still open | Folded into A07. |
| Q: Do any Synopsys or Siemens licences still work? | ○Still open | A20. |
| Q: The layout file (OASIS or GDS, clear of TSMC) from 6 Oct | ↻Updated | Now with names: the full-chip databases Alchip received, and where Esperanto's own copies went A16 A01 A02 A03. |
How the archive got here
This is what I could reconstruct from tickets, shell histories, logs and file dates. You know parts of this first-hand, the 2025–26 steps especially, so please correct anything that's wrong; where I say inferred
, it's my guess.
Three things shaped what arrived. First, storage was chronically short of space, so from 2020 IT moved leaver homes and finished projects from the Pure to sc-truenas01 and left symlinks behind. Second, when the colo closed in June 2025, the copy out was an rclone upload of three Pure volumes to Wasabi: it didn't follow symlinks, so nothing that had been moved came along, and the /eng walk ended part-way through the alphabet. Third, the TrueNAS boxes, the Veeam backups and the AWS buckets that held everything else were never part of that upload.
- arrived complete
- arrived in part
- stayed behind, never exported
- planned, nothing arrived
The diagram scrolls sideways.
df of 18 Jul 2024 unless noted). Not drawn: database dumps from the colo VMs that went into /eng/git, a 126 GB Drive subset uploaded to Wasabi in Aug 2025, and the lab flash PC's /home, rsynced straight to mordast on 12 Mar 2026 (34 GB, the only transfer that skipped stash).All 30 transfers as a table (method, date, who, how complete, what was lost)
| # | From → to | When | How | Who | Size | How complete | Lost |
|---|---|---|---|---|---|---|---|
| F01 | Pure /eng → Wasabi | 2025-06-20 15:42Z .. 2025-06-23 ~22:30Z | rclone sync, alphabetical walk incl. 23 .snapshot dirs, no -L (symlinks skipped; shown for fe, inferred for eng) | Erich Kolb (inferred); account dave.ditzel | 8.1 TB, 4,798,940 files | Truncated at user "kpn" (123 named owner accounts after it); 58 of 92 uploaded top dirs still receiving files after 20:30 UTC (probably incomplete, inferred); all snapshot walks cut before "dyau" | After-kpn homes, symlinked subtrees, /eng/jenkins workspaces, most of Artifactory |
| F02 | Pure /home_cad, /fe, b4c → Wasabi | 2025-06-13 21:20Z .. 2025-06-23 | rclone sync | as F01 (inferred) | 66.7 TB, 11,317,944 files | home_cad 100% (65.93 TB; also followed the /cad/tools/ARCHIVE symlink or ran where /archive was mounted), fe 100% of real files (993 "Can't follow symlink" notices), b4c logs_* only | fe release "latest" symlinks; rest of /project/b4c |
| F03 | Pure /pd → AWS S3 | 2023-04-19 .. 2023-06-07 | aws s3 sync of /project/pd/release/A0 to a0-archive-* (3 runs) | Michael Ray (asked of Erich Kolb by Ray Borg) | 37.3 TB | Last run (a0-archive-et-soc-1, 30 May 2023) "Completed 38.7 TiB/~38.7 TiB, ~903 files remaining"; ~85 TB buckets each (multipart leftovers); "on s3" claimed Nov 2024 | State after 2024 unknown |
| F04 | Pure /pd → AWS S3 | 2023-09-06 .. 2023-09-20 | s3cmd sync, then aws s3 cp --recursive to s3://pure-pd-archive/A0 | Erich Kolb | Stopped after memshire_east_side (inferred from upload-time order); multi-TB per the Jan 2025 bill (inferred); Glacier lifecycle 14 Mar 2025 | soc_top, shire_top, minion_top probably never uploaded here | |
| F05 | Pure /pd → LaCie drive | 2023-06-19 .. 2023-07-24 | rsync to ext4 LaCie | Michael Ray | 37.8 TB | Disk full at 37.8 TB; restarted; never confirmed | Whereabouts unknown |
| F06 | Pure /pd → sc-truenas03 | 2023-08-28 .. 2023-11-29 | tar czf a0_backup.tgz in /eng/mray (439 h) + rsync of the tree; diff; rename A0_final; rm -rf on Pure | Michael Ray (tar), Patrick Lugliani (copy, diff, delete) | 37.3 TB | 100% (diff clean); intact 21 Nov 2024 | Pure original deleted 29 Nov 2023 |
| F07 | Pure /pd → Vendors | 2020-07-09 .. 2020-12-24 | OASIS / ICC2 DB hand-offs to Alchip for TSMC (PCWF, base, metal ECO) | Mike Pelham, PD team | >= 8 full-chip OASIS DBs and 4 ICC2 tarballs named; TMNK96 NTO 23 Dec 2020 | Vendor-side copy only | |
| F08 | Pure /pd → sc-truenas01 | 2022-08 .. 2024-11 | rsync + rm + symlink to /archive/exstaff and /archive/project/pd | Biju Albert, Michael Ray, Ray Borg | Moves, not copies: 29 PD ex-staff (Aug 2022), klingner 887.6 GB + bmaric 15.0 TB (Dec 2023), anilm Work_A0 + adolfod 451 GB (Nov 2024) | Never exported from TrueNAS | |
| F09 | Pure /eng → sc-truenas01 | 2020-07 .. 2025-01 (bulk Sep 2024 - Jan 2025) | rsync + rm -rf + ln -s of leaver homes and "please archive" subtrees | Ray Borg, Michael Ray | Moves: IT-2549 (Sep 2024), HR leaver tickets, Nov 2024 alilic archive_ptpx and dft/ATPG | 74 pre-kpn owners and the symlinked subtrees absent from Wasabi | |
| F10 | Pure /eng → sc-truenas02 | 2023-10 .. 2025-01-10 | Veeam file-share jobs ("PureENG to Truens02 backup", daily) + archive02/exstaff moves (Oct 2023, inferred) | Michael Ray | Newest /eng point 10 Jan 2025 (a second job ~8 Sep 2024); fe and home_cad daily too | Never exported | |
| F11 | Pure scratch, jenkins, artifactory → sc-truenas02 | 2024 .. ~2024-12-05 | Veeam file-share jobs (Pure Scratch, PureArtifactory, PureJENKINS to Truenas02) | Michael Ray | Scratch and Artifactory points ~5 Dec 2024; Jenkins row cut off in the screenshot | Never exported | |
| F12 | Pure /pd → AWS S3 | 2024-04-14 .. 2024-06-04 | Veeam "to s3 backup" jobs of all 8 file systems (PurePD, PureENG, PureJENKINS ...) to veeam-to-s3-backups-4-5-2* | Michael Ray (mradmin), Erich Kolb (S3 connector) | All 8 jobs Failed on 29 May 2024, disabled 4 Jun; restore points of ~4 Jun 2024 still catalogued in Jan 2025 (may be incomplete) | Only the Jun 2024 /project/pd point known | |
| F13 | Pure /eng → Wasabi | 2024-06-26 .. ~2024-07-28 | Veeam "PureENG to Wasabi" job (Veeam format) | Erich Kolb (ESPERANTO\ekadmin) | 10.5 TB | 10.5 TB chain; failed run 3 Jul 2024; a restore point ~28 Jul 2024 | Bucket not among the 7 buckets of 2026; account unknown |
| F14 | sc-truenas01 → Google Workspace | 2024-03-29 .. 2024-10-10 | tar czf per ex-staff home, browser upload to shared drives ET-EX-PD/ENG-Tarballed-Home-Directories | Michael Ray, Ray Borg | 18 TB, 33 files | 24 tarballs listed (at least 16.34 TiB: bmaric.tgz was still being written at 9.8 TiB and failed its compare) + 9 more names; uploads stalled, lost 7 May 2024, restarted; completion unrecorded | Not in the Jul 2026 Drive copy |
| F15 | sc-truenas01 → deleted | 2022-10 and 2024-07 .. 2024-11 | deletions (outage clean-up; Jul-Nov 2024 archive/pd clean-up) | IT (Biju Albert 2022; Ray Borg 2024, inferred) | 84.7 TB | ~50T in Oct 2022; ~77 TiB of archive/pd between 18 Jul and 21 Nov 2024 | Older PD releases and ex-staff PD areas (unknown which) |
| F16 | GitLab → Pure /eng | 2020-12 .. 2023-09-06 | gitlab-backup tars + loose copies into /eng/git | IT / DevOps | 2.24 TB, 16 files | 16 full backup tars (4 of the Jul 2023 ones truncated after the repositories); uploaded with F01 (git sorts before kpn) | Nothing after Sep 2023 |
| F17 | GitLab → stash | 2025-01-25 .. 2026-03-08 | git clone on Roman's Mac (Jan and Sep 2025), push to github.com/et-* (17-19 Sep 2025), copy to stash through macOS | Roman Shaposhnik; files owned by shelley | 0.053 TB, 513 repos | 513 repos (511 cloned from gitlab.com: 171 in Jan, 340 in Sep 2025; 2 with other origins, e.g. the bare qqq), bare git; design repos not pushed to GitHub; 1,180,890 LFS-rewrite objects mode 600 | MRs, issues, CI, wikis, registries; 1,347 esperanto-soc branches (but in the Sep 2023 tar) |
| F18 | Colo VMs → Pure /eng | 2022-04-04; .. 2025-06-22 | Perforce server-root copy (Apr 2022); cron DB dumps (pg_dump, mysqldump, grafana.db) | IT cron / unknown | 0.185 TB | Perforce journal complete but archive content for 112 of 23,677 depot paths; codegen_metrics, pcie_card_tracking, slurm, grafana dumped; sysqa, rtl_metrics, InfluxDB, Prometheus not | 11,021 live Perforce heads; 9 of 12 Grafana data sources |
| F19 | Colo VMs → sc-truenas02 | 2023-01 .. at least 2024-04 | Veeam VM-image and agent jobs (26 VMs; agents incl. mv-lab09) | Michael Ray, Biju Albert | "Linux VMs to TrueNAS02" (11 objects) running Apr 2024 | Never exported | |
| F20 | Lab servers → stash | 2025-07-28 .. 2025-12-18 | root disks copied to backup.<host>.img on data disks (skip-on-error, no padding), then 6 disks dd'd to stash | unknown (Ainekko side, inferred) | 22.5 TB | 10 nested images of 9 hosts; swpcie21/22/23/25/eap04 damaged (mostly re-alignable) | Unimaged hosts; swpcie21 190 GB tail |
| F21 | Lab servers → mordast | 2026-03-12 | rsync -av from the lab flash PC (/home) to ainekko@mordast | lab flash PC user "et" | 0.034 TB, 51,345 files | All of /home on the PC (52 per-user dirs empty by design) | - |
| F22 | AWS S3 → Wasabi | 2025-06-30 16:06Z .. 16:15Z | S3-to-S3 copy of s3://pure-pd-archive | unknown (Erich Kolb, inferred) | 0.0102 TB, 3,470 files | Only 3,470 STANDARD objects (10 of 26 blocks, small files) | Bulky subdirs (possibly Glacier-tiered, inferred) |
| F23 | Google Workspace → Wasabi | 2025-08-15 .. 2025-08-21 | Drive zip download ("licensable" subset, unzipped on a Mac) + Drive-for-desktop folder | unknown (inferred same Mac workflow) | 0.126 TB, 11,938 files | Hand-picked subset (17 top folders); keeps shortcut targets and same-name duplicates | - |
| F24 | Wasabi → stash | 2026-01-11 .. 2026-03-08 (verified to 2026-04-29) | rclone v1.50.2 per bucket, --exclude .snapshot | unknown NFS client; verification on stash as root | 75 TB, 16,133,505 files | 100% of live trees; snapshot copies skipped by design (982,466 eng / 3.34 TB kept) | Only redundant snapshot versions (19 snapshot-only files) |
| F25 | Wasabi → deleted | 2026-04-30 | wasabi_parallel_purge.py from a docker container | not recorded (executor field blank; asked in A08) | 110 TB, 32,690,599 files | All versions deleted, 0 errors | Wasabi copy (stash is now the only one) |
| F26 | Google Workspace → stash | 2026-07-10 17:25 .. 2026-07-11 20:20 PDT | rclone copy of the consolidated "Esperanto" shared drive | Shelley (file owner) | 2.53 TB, 124,413 files | 3,585 of 3,586 native files exported; 59 distinct dangling shortcuts, 252 distinct 403s, ~305 duplicates dropped; Gepardo tree twice | Other shared drives, My Drives |
| F27 | Atlassian Cloud → stash | 2026-07-10 .. 2026-07-12 | REST exporter (Roman's nekkoai/it exporter): Jira search expand=changelog; Confluence content API (status=current) | Shelley / Roman | 0.025 TB, 20,752 files | 44,649 issues, 136,615 comments, 10,165/10,165 Jira attachments; 5,595 pages, 10,524/10,587 attachments | Changelog > 40, MFGTST, 63 attachments, TEAM + 1 personal space, page history, remote links |
| F28 | stash → mordast | 2025-12 .. 2026-10-11 | NFS v4.2 mounts (nconnect=4), pinned in /etc/fstab | Shelley / Roman (root) | All stash archive datasets visible; /archive/eng mounted but empty | - | |
| F29 | sc-truenas01 → stash | export created 2025-12-15 17:09 PST | planned import into /mnt/tank/archive/eng (inferred from the name, which matches sc-truenas01's tank/archive/eng; the dataset has no quota of its own: df shows the pool's 26 TiB free) | unknown (Shelley / Roman) | 0 TB | 0 entries; exported to 10.20.10.0 and 10.10.1.0 (10.10.1.0 is the Pure / sc-truenas03 subnet, not sc-truenas01's 10.10.7.10) | The whole TrueNAS archive |
| F30 | Lab servers → Lab servers | ~2025-07-30 .. 2025-08-05 | lab servers re-provisioned as Ainekko hosts esperanto-soc2..10 (inferred) | Ainekko (Gianluca flashed cards on esperanto-soc4, 5 Aug 2025) | Unknown which machines and disks; names only | Root disks overwritten after the Jul 2025 backups (inferred); data disks may survive |
Names in Who
come from tickets, shell histories and file owners; inferred
means I matched them by timing or role.
Timeline
- Jun 2020sc-truenas01 becomes
/archive. From now on, data leaves the Pure by moving (rsync, delete, symlink), not copying. - Jul–Dec 2020A0 tape-out: full-chip OASIS and ICC2 databases go to Alchip for TSMC (product TMNK96); masks ordered 23 Dec. The final merge is written to
/project/pd/mpelham/…/TAPEOUT_121920on 22 Dec. - Aug 202229 ex-staff PD directories moved from
/project/pdto/archive/exstaff. - Oct 2022TrueNAS outage: about 50 T deleted to free space.
- Apr–Jul 2023The A0 release (37.3 TB per the Pure) synced to AWS buckets
a0-archive-*and rsynced to a LaCie drive (full at 37.8 TB). gitlab.com migration starts; last on-prem GitLab backup 6 Sep. - Sep 2023Erich Kolb uploads the A0 release to
s3://pure-pd-archive; it seems to stop after memshire_east_side (inferred from upload order). - Oct–Nov 2023A0 tree and a 22 TB
a0_backup.tgzland on sc-truenas03; the Pure copy is deleted on 29 Nov. The IO FreeNAS is retired but kept. - Apr–May 2024Aging reports of every file older than 180 days (3.4 GB gz) and ex-staff home tarballs (24 listed, at least 16.3 TiB) are uploaded to Google Drive; the uploads stall and restart. Eight Veeam backup jobs to AWS all fail on 29 May.
- 18 Jul 2024Last full
dfof all Esperanto NFS volumes: at least 547.8 TB used. - Jul–Nov 2024sc-truenas01
archive/pdshrinks from 96 to 18.6 TiB (~77 TiB deleted). Leaver homes move from/engto TrueNAS (Sep 2024 – Jan 2025); alilic archive_ptpx, anilm Work_A0 and adolfod's floorplans follow in late Nov. - 10 Jan 2025Newest known Veeam restore points:
/eng, fe and home_cad on sc-truenas02. - 25 Jan 2025171 gitlab.com repos cloned (the first part of the git mirror).
- 14 Mar 2025Glacier lifecycle set on the AWS PD archive bucket.
- 13–23 Jun 2025rclone upload to Wasabi (account dave.ditzel). home_cad and fe arrive complete; the
/engwalk reacheskpn
at 20:52 UTC on 23 Jun and stops there (the parallel walks end by ~22:30). - 24–30 Jun 2025Last edit of the QTS colo machine list (24 Jun); last staff day (30 Jun), when 10 GB of
s3://pure-pd-archiveis copied to Wasabi. - Jul–Aug 2025Lab server root disks copied to image files (28–30 Jul); cards flashed on esperanto-soc4 (5 Aug); Drive subsets uploaded to Wasabi (15–21 Aug).
- 17–19 Sep 2025340 more gitlab.com repos cloned; 490 pushed to github.com/et-*.
- Dec 2025Six lab disks imaged to stash (15–18 Dec). The empty
archive/engexport is created on 15 Dec. - Jan–Mar 2026Wasabi restored to stash (11 Jan – 8 Mar); the git mirror copied to stash through macOS; the lab flash PC rsynced to mordast (12 Mar).
- 30 Apr 2026Wasabi purged: 32.7 M object versions deleted, 0 errors. stash now holds the only copy of the 2025 uploads (plus ZFS snapshots of 13 Apr and 2 May on the same pool).
- 10–12 Jul 2026Drive rclone copy of the consolidated
Esperanto
shared drive, and the Atlassian REST export. - 9–11 Oct 2026Handover under NDA; mordast's fstab pinned with
/archive/engand/archive/esperanto.
Filled red dots mark losses, filled blue dots steps on the way to mordast. Sources: Jira IT-395, IT-974, IT-1246, IT-1249, IT-1725, IT-1771, IT-2338, IT-2454, IT-2549, IT-2617, IT-2635–2646, IT-2668, IT-2702, EIP-124, EIP-125; the Wasabi listings and purge attestation in /archive/esperanto/wasabi/; reflogs of the git mirror; debugfs stat of the lab images.
What's here and what's missing
Five pies for the main proportions, a block strip for the tape-out release, one bar chart for the volumes, and a table by kind of data. Blue is here, amber partly here, red missing, grey low value; every slice also has a text label, and each chart has its numbers underneath.
Esperanto storage in July 2024: what reached mordast
- Reached mordast (65.9 TB of it the CAD home) ✓ Here74.9 TB14%
- Pure volumes not uploaded ✕ Missing148 TB27%
- sc-truenas01 archive, never exported ✕ Missing270 TB49%
- sc-truenas03 A0 archive, never exported ✕ Missing55.0 TB10%
df. sc-truenas01 is 245 TiB (270 TB) by the 18 Jul df but 221 TiB (243 TB) on the 3 Jun pool page; the gap is archive/sw (54 T vs 27.7 TiB), unresolved. The 11 Oct report's ~170 TB used older, partial figures.Numbers
| Slice (clockwise from the top) | Status | Value | Share |
|---|---|---|---|
| Reached mordast (65.9 TB of it the CAD home) | Here | 74.9 TB | 14% |
| Pure volumes not uploaded | Missing | 148 TB | 27% |
| sc-truenas01 archive, never exported | Missing | 270 TB | 49% |
| sc-truenas03 A0 archive, never exported | Missing | 55.0 TB | 10% |
| Total | 548 TB | 100% |
The parts sum to 547.85 TB, not the 547.8 of the df: mordast holds /fe as it was in June 2025, 0.08 TB larger.
Source: Jira IT-2454 (df of 18 Jul 2024, TiB × 1.0995); vault.db sources.
The 261 named /eng owners of April 2024
- Home here, with content (≥ 1 MB) ✓ Here48 owners18%
- Home here, near-empty (< 1 MB) Partly here16 owners6.1%
- Sorts before “kpn” but no home: moved or deleted before June 2025 ✕ Missing74 owners28%
- Sorts after “kpn”: never uploaded ✕ Missing123 owners47%
Numbers
| Slice (clockwise from the top) | Status | Value | Share |
|---|---|---|---|
| Home here, with content (≥ 1 MB) | Here | 48 owners | 18% |
| Home here, near-empty (< 1 MB) | Partly here | 16 owners | 6.1% |
| Sorts before “kpn” but no home: moved or deleted before June 2025 | Missing | 74 owners | 28% |
| Sorts after “kpn”: never uploaded | Missing | 123 owners | 47% |
| Total | 261 owners | 100% |
Source: EIP-124 owner list (2 Apr 2024); vault.db depth-1 directories of the eng source.
Ingredients of an ET-SoC-1 thermal model
- Have ✓ Here6 ingredients43%
- Partial Partly here5 ingredients36%
- Missing ✕ Missing3 ingredients21%
Numbers
| Slice (clockwise from the top) | Status | Value | Share |
|---|---|---|---|
| Have | Here | 6 ingredients | 43% |
| Partial | Partly here | 5 ingredients | 36% |
| Missing | Missing | 3 ingredients | 21% |
| Total | 14 ingredients | 100% |
Source: synthesis/report.md and the inventory agent's checked list.
Google Drive files linked from Jira and Confluence
- In the Drive copy ✓ Here387 ids16%
- Only a same-name candidate Partly here240 ids10%
- Not in the copy ✕ Missing1,749 ids74%
Numbers
| Slice (clockwise from the top) | Status | Value | Share |
|---|---|---|---|
| In the Drive copy | Here | 387 ids | 16% |
| Only a same-name candidate | Partly here | 240 ids | 10% |
| Not in the copy | Missing | 1,749 ids | 74% |
| Total | 2,376 ids | 100% |
Source: gdrive.db (ids.verdict, missing_reason).
Perforce depot paths in the archived journal
- Archive content here: 112 paths (102 head files on disk) ✓ Here112 paths0.5%
- Deleted before Apr 2022 (mostly one PMIC SAMD20 demo-firmware tree; low value) – Low value12,544 paths53%
- Live, content missing (2.80 GB) ✕ Missing11,021 paths47%
Numbers
| Slice (clockwise from the top) | Status | Value | Share |
|---|---|---|---|
| Archive content here: 112 paths (102 head files on disk) | Here | 112 paths | 0.5% |
| Deleted before Apr 2022 (mostly one PMIC SAMD20 demo-firmware tree; low value) | Low value | 12,544 paths | 53% |
| Live, content missing (2.80 GB) | Missing | 11,021 paths | 47% |
| Total | 23,677 paths | 100% |
Source: the Perforce journal in esperanto-eng/git/perforce/servers/master/journals/, parsed by two agents.
The ET-SoC-1 A0 tape-out release, block by block
0.03%
of the release is on mordast: 10.2 GB of 30.6 TB (27.8 TiB by du).
- 10 blocks, 6.8 TiB: small files only Partly here
- 16 blocks, 21.0 TiB ✕ Missing
du. The 10 amber blocks are represented only by small files (10.2 GB in all, the largest 689 MB); none of the 26 has a full layout or power/IR map here. soc_top alone is 8.4 TiB.Numbers
| Block (du of 9 Oct 2023) | Status | Size |
|---|---|---|
ioshire_wrapper | Partly here | 3.0 TiB |
memshire_east_side | Partly here | 1.5 TiB |
maxshire_top | Partly here | 1.1 TiB |
ioshire_meshstop_wrapper | Partly here | 371 GiB |
maxion_shire_channel | Partly here | 312 GiB |
dft_control_unit | Partly here | 310 GiB |
MaxionTile | Partly here | 183 GiB |
ioshire_misc | Partly here | 37 GiB |
etcust_64x32_3r2w_vpurf | Partly here | 3.6 GiB |
etcust_128x64_1r1w_lram | Partly here | 2.9 GiB |
soc_top | Missing | 8.4 TiB |
shire_top | Missing | 4.6 TiB |
pshire_et_block | Missing | 1.8 TiB |
memshire_west_side | Missing | 1.6 TiB |
pu_noc_wrapper | Missing | 771 GiB |
shire_channel_wrap | Missing | 617 GiB |
neigh_top | Missing | 538 GiB |
memshire_east_top | Missing | 537 GiB |
shire_top_logic_wrapper | Missing | 508 GiB |
spio_noc_wrapper | Missing | 489 GiB |
memshire_west_top | Missing | 457 GiB |
spio | Missing | 277 GiB |
pu | Missing | 214 GiB |
minion_top | Missing | 137 GiB |
shire_cache_bank | Missing | 132 GiB |
mr75138 | Missing | 34 GiB |
| Total | 27.8 TiB |
Source: IT-1246 (du on sc-vnc02 after the rsync to sc-truenas03, Oct 2023); vault.db source aws-pd (depth-2 directories).
Each volume: last known size, and how much of it is on mordast
- On mordast ✓ Here
- Not on mordast ✕ Missing
/eng arrived at 8%. The sc-truenas02 row is backup copies of the Pure rows, so don't add it.Numbers
| System | Volume | Last known, TB | On mordast, TB | Note |
|---|---|---|---|---|
| Pure sc-pure01 (df, 18 Jul 2024) | /eng | 106.65 | 8.100 | homes after kpn, moved subtrees, jenkins, 207 of 256 Artifactory shards missing |
| Pure sc-pure01 (df, 18 Jul 2024) | /home_cad | 65.97 | 65.930 | complete |
| Pure sc-pure01 (df, 18 Jul 2024) | /pd (/project/pd) | 32.99 | 0.000 | never uploaded |
| Pure sc-pure01 (df, 18 Jul 2024) | /artifactory | 6.05 | 0.000 | never uploaded |
| Pure sc-pure01 (df, 18 Jul 2024) | /scratch | 5.94 | 0.000 | never uploaded (24.2 TB in Nov 2023) |
| Pure sc-pure01 (df, 18 Jul 2024) | /jenkins (/project/jenkins) | 4.29 | 0.000 | never uploaded |
| Pure sc-pure01 (df, 18 Jul 2024) | /fe | 0.79 | 0.785 | complete except symlinks (0.71 TB in the df; it grew) |
| Pure sc-pure01 (df, 18 Jul 2024) | /project_b4c + /syseng | 0.42 | 0.023 | only logs_* of b4c |
| sc-truenas01 (df, 18 Jul 2024) | archive/pd | 105.55 | 0.000 | fell to 18.6 TiB by Nov 2024 |
| sc-truenas01 (df, 18 Jul 2024) | archive/exstaff | 75.87 | 0.000 | 16.3 TiB of it tarred to Google in 2024 |
| sc-truenas01 (df, 18 Jul 2024) | archive/sw | 59.37 | 0.000 | Artifactory/SW backups |
| sc-truenas01 (df, 18 Jul 2024) | archive/eng | 23.09 | 0.000 | older leaver homes, Nov 2024 moves |
| sc-truenas01 (df, 18 Jul 2024) | home/eng | 5.94 | 0.000 | |
| sc-truenas03 | A0_final (tree + 22 TB tarball) | 54.98 | 0.010 | 10 GB AWS fragment on mordast |
| Not in that df | ESXi VM datastore (Nov 2023) | 36.17 | 0.000 | only app-level dumps reached /eng/git |
| Not in that df | sc-truenas02 Veeam (inferred) | 44.00 | 0.000 | backup copies of the Pure volumes: overlaps the rows above |
| Not in that df | IO FreeNAS (Jan 2024) | 12.09 | 0.000 | never backed up or exported |
Source: IT-2454 (df of 18 Jul 2024); IT-1246; IT-1877; vault.db sources.
By kind of data
| Kind of data | Status | Detail |
|---|---|---|
| CAD home: foundry kits, EDA tools, IP | ✓Here | 65.9 TB, complete (Pure /home_cad). |
Front-end volume /fe | ✓Here | 0.78 TB, complete except 993 symlinks. |
| Jira and Confluence | ✓Here | 44,649 issues, 5,595 pages, all 10,165 Jira attachments. Gaps: histories cut at 40 entries on 650 issues, the private MFGTST project, 63 Confluence attachments A07. |
| Git | ✓Here | 513 repos, complete for git to 3 Jun 2025, plus on-prem GitLab backups to Sep 2023. Not: gitlab.com merge requests, issues, CI logs, wikis A15. |
| Google Drive | Partly here | One shared drive (124,477 files, 2.53 TB) plus a 126 GB subset; 387 of 2,376 Drive ids linked from Jira/Confluence A04 A05. |
Engineering NAS /eng | Partly here | 8.1 of 107 TB: homes a…kpn (many cut mid-transfer), git/, ate/, 49 of 256 Artifactory shards A01 A08. |
| Lab servers | Partly here | 12 hosts imaged; the mv-gb servers and several 8 TB data disks never; two images damaged A10. |
| Perforce (board and package CAD) | Partly here | Journal and db of Apr 2022; content for 112 of 23,677 depot paths A14. |
| Service databases and telemetry | Partly here | 3 of 12 Grafana data sources dumped; Sysqa, rtl_metrics, InfluxDB, Prometheus not. |
| Full-chip layout (A0 tape-out release) | ✕Missing | 30.6 TB; a 10 GB fragment here A01 A02 A03 A16. |
PD work areas /project/pd | ✕Missing | 33 TB in Jul 2024 A01 A02. |
/eng homes after kpn | ✕Missing | 123 named owners, ~38 TB by 2022 quotas A01 A02 A08. |
CI workspaces (/eng/jenkins, /project/jenkins) | ✕Missing | 5.44 TB + 3.9 TiB; only the file listing is here. |
/scratch, incl. the A0 per-shire power runs | ✕Missing | 5.9 TB in Jul 2024 (24 TB in 2023). |
| TrueNAS archive: ex-staff homes, archived PD, SW backups | ✕Missing | 243 TB used in Jun 2024 A01 A02 A09. |
| Veeam backups (sc-truenas02, AWS, Wasabi) | ✕Missing | ~40 TiB on sc-truenas02 (inferred from an 11% gauge). |
| Old IO FreeNAS | ✕Missing | 12 TB, retired in 2023 but kept. |
Systems-engineering share /mnt/syseng | ✕Missing | 33 GiB: SLT/ATE logs, LabVIEW data. |
| Slack, mailboxes, Smartsheet, Datadog | ✕Missing | Never exported A08. |
| Copies held by vendors | ✕Missing | Alchip (full-chip OASIS), Amkor (package models), Teradyne (ATE SVN) A16 A17 A19. |
Where the missing parts might be
The 28 gaps I'd rank priority 1 or 2 (by how much they matter for the thermal model and how much data is involved), each with the best ideas for where a copy might still exist and who would know. A red left edge marks priority 1. Confidence (high / medium / low) is mine. The links in the last column go to the asks below; the 16 smaller gaps are listed after the table.
| What's missing | How much | Where it could still be | Who would know | Ask |
|---|---|---|---|---|
| M01 · priority 1The ET-SoC-1 A0 tape-out release: the full-chip layout (soc_top, shire_top, minion_top …) and its EM/IR runs | 30.6 TB by du (37.3 TB per the Pure); soc_top 8.4 TiB. 10.2 GB here (0.03%). |
| Ray Borg (Head of IT; opened IT-1246); Michael Ray (ran the AWS syncs, LaCie and tarball); Patrick Lugliani (copied to sc-truenas03, deleted the Pure copy); Erich Kolb (AWS); Anil Muddamsetti (used A0_final in late 2024); Dave Ditzel (AWS owner of record) | A01 A02 A03 A16 |
M02 · priority 1s3://pure-pd-archive: the AWS copy of A0 blocks, of which only a fragment arrived | 3,470 files / 10.2 GB here (10 of 26 blocks, small files only). The bucket itself was multi-TB (inferred from the Jan 2025 bill). |
| Erich Kolb (made the 2023 upload); Georgios Koloventzos (Jan 2025 bill analysis); Ray Borg (set the lifecycle); Dave Ditzel | A03 |
| M03 · priority 1Full-chip OASIS and ICC2 databases delivered to Alchip for TSMC (TMNK96, Jul–Dec 2020) | ≥ 8 full-chip OASIS DBs and 4 ICC2 tarballs named in 34 documents; the Dec 2020 final merge was 9.1 GiB. |
| Mike Pelham (built the merges, sent the DBs); Christian Klingner (PD lead, most Alchip mail); Darren Jones (VP VLSI, owned the relationship); Stephen Lee | A16 |
M04 · priority 1The final merged chip in /project/pd/mpelham/work/CHIP/INT_A0/TAPEOUT_121920, outside the release tree | Two files, 9.1 GiB + 995 MiB, 22 Dec 2020 (seen in a screenshot, FV-157). |
| Mike Pelham; Anil Muddamsetti | A01 A02 A03 A16 |
M05 · priority 1/project/pd work areas (PD runs, RedHawk EM/IR, ET-SoC-1.5/2 floorplans): never uploaded | 30 TiB (33 TB) in Jul 2024; peak 143 TiB in 2021. |
| Anil Muddamsetti (Sr. Manager ASIC PD); Mike Pelham; Adolfo Delacalle; Michael Ray (ran the Veeam PD jobs) | A01 A02 A03 |
M06 · priority 1sc-truenas01 archive datasets: archive/pd, exstaff, archive/eng, sw, home/eng | 221 TiB used on 3 Jun 2024; ~145 TiB by end-2024 after a clean-up (inferred). |
| Ray Borg; Michael Ray; Biju Albert (wrote the archive how-to); Patrick Lugliani | A01 A02 A09 |
M07 · priority 1Nov 2024 archive moves: alilic archive_ptpx (ET-SoC-1 and Gepardo PT-PX runs), anilm Work_A0, adolfod ET-SoC-1.5/2 floorplans, dft/ATPG | ≥ 944 GB known: archive_ptpx 11 GB + 482 GB, adolfod ~451 GB. Part of M06. |
| Aleksandar Lilic (made archive_ptpx; stash home aleksandar, inferred); Anil Muddamsetti; Adolfo Delacalle; Zoran Stanojevic (dft); Ray Borg (did the moves) | A01 A02 A09 A18 |
| M08 · priority 1Ex-staff PD homes: klingner, bmaric, psanchez, rohiniv, rupeshr … (soc_top RedHawk-SC EM/IR, SOC_Top bumps, ET-SoC-1.5 mPower runs) | 24 tarballs, at least 16.3 TiB listed on 11 Apr 2024 (bmaric.tgz was still being written at 9.8 TiB), plus 9 more names; how many finished uploading is unknown. |
| Michael Ray (made and uploaded the tarballs); Ray Borg (created the drives); Erich Kolb (drive member); Pedro Sanchez (his own psanchez home) | A01 A02 A04 A18 |
M09 · priority 1/eng homes after kpn: never uploaded (tberg, tbeale, rknath, rajl, shankarj, rsteeves, yhxu, zorans, mray …) | 123 named owners (165 directories after kpn in Mar 2024; Softwaresorts first and was uploaded); 38.1 TB by the Dec 2022 quotas. |
| Erich Kolb (likely ran the upload, inferred); Ray Borg; Michael Ray (Veeam); owners Tom Berg, Terry Beale, Rajib Nath, Raj Lam, Shankar Jayaratnam | A01 A02 A08 A18 |
M12 · priority 1/eng/jenkins workspaces and Pure /project/jenkins: per-shire PT-PX runs with gate-level FSDB and shire layouts | 5.44 TB / 9.9 M files (listing of 19 Nov 2024; 5.9 GB here) + 3.9 TiB; 122 FSDB files = 1.11 TB. |
| Georgios Koloventzos (Jenkins/Artifactory admin); Slavko Guduric; Xavier Reves (owned the ET-SoC-1 jobs); Aleksandar Lilic | A01 A02 |
| M13 · priority 1The Veeam backups of the Pure file systems and VMs | sc-truenas02 ~40 TiB used (inferred); points of /eng, fe, home_cad to 10 Jan 2025, scratch and artifactory to ~5 Dec 2024; an AWS chain (~4 Jun 2024); a 10.5 TB Wasabi chain (Jul 2024). |
| Michael Ray (built and ran the jobs); Erich Kolb (Wasabi job, S3 connector); Patrick Lugliani (rebuilt the Veeam server); Ray Borg | A01 A02 A03 A04 A08 |
| M14 · priority 1A0 full-chip per-shire PrimeTime-PX runs: PowerRuns2 (11 shires × 3 workloads, Feb 2021) and siblings | PowerRuns2: 4,975 files / 279.5 GB (101 reports, 100 FSDB); PowerRuns 134 GB, BIGRUNS 186 GB. |
| Rajib Nath (owner of ARCHI-4 and /scratch/noc); Ray Borg; Michael Ray | A01 A02 A18 |
| M19 · priority 1Lab card servers never imaged: mv-gb01…19 (2024 power traces on mv-gb09 and gb19), the 8 TB data disks of mv-swpcie22/25, mv-demo01, mv-lab07/09, lab-desktop | ~99 lab hosts named in Jira/Confluence; 12 imaged. |
| You (tailnet owner); Gianluca Guida (flashed cards on esperanto-soc4, Aug 2025); Aleksandar Lilic (wrote the 2024 traces); Shankar Jayaratnam (owned most mv-swpcie hosts) | A10 |
| M21 · priority 1Perforce Systems depot content: package SiP series, ET-SOC1-3Dcad.zip, stack-ups, the T31-235 heatsink drawings, the Rev3 dev-card board | 11,021 live head files (2.80 GB) without content; unique ET-SOC loss 112 files / 2.0 GB; changes after Apr 2022 also missing. |
| Bruce Klein (main client; stash home bruce, inferred); Bill Orner (Perforce admin); Jeff Kane (substrate/PCB layout) | A01 A14 A17 A18 |
M15 · priority 2Old IO FreeNAS 10.10.1.3 (/data, /home/{gfx,mmi,project}) | 11 TiB used of 16 (~12 TB). |
| Ray Borg (we still have it); Patrick Lugliani (retired it); Michael Ray (remounted it Jan 2024) | A01 A02 |
| M22 · priority 2Package and cooling models held by vendors: Amkor FEA model and theta characterisation, the 2019 Meta C1 SoC thermal models, heatsink drawings | 4 SoC thermal models + 1 PDML (2019); one Amkor warpage FEA (2021). No theta-JA/JC/psi-JT anywhere in the archive. |
| Han Park, Victor Kessler (Amkor); Bill Orner (uploaded the 2019 models); Jim Petroski (contractor, CFD); James Crook (Radian) | A17 |
M25 · priority 2Leavers' My Drives and mailboxes; the June 2025 leavers (Torres, Ward, Jayaratnam) are recorded as email only archived | 275 leaver tickets; ~69 record Drive transfers. No PST, mbox or Slack export anywhere. |
| Ray Borg (ran every off-boarding); Aleksandar Lilic (got Ivan Ratkovic's Drive); you (Workspace now) | A05 |
M26 · priority 2123 weekly Power_TopicsGoogle Slides decks (Aug 2020 – Feb 2023), linked from the HE Weekly Power Meetingspage | 122 decks + 1 folder. |
| Francisco Torres (presenter; left 30 Jun 2025); Devin Patel | A05 A18 |
| M27 · priority 2Product/test-engineering characterization Sheets and package/qual files linked from Confluence (2022–24) | ~16 P/TE and ~10 package ids; raw data behind two of them is already here. |
| Devin Patel; Raj Lam; Carl Wang; Matt Ward | A04 A05 |
| M28 · priority 2Power-estimate and PT-PX Sheets (ptpx_data_table_JB, Power Projection Spreadsheet, ET-SOC-2 PTPX experiments …) | ~12 ids, plus 3 dangling shortcuts. |
| Jim Burr; Ivan Ratkovic; Aleksandar Lilic; Raymond Tang | A05 A18 |
M10 · priority 2/eng homes before kpnthat had already left /eng (ivan, devinp, faizanms, fgispert …) | 74 named owners (121 directory names); 16.0 TB by the Dec 2022 quotas. Overlaps M06/M08. |
| Ray Borg; Michael Ray | A01 A02 A08 A09 |
| M11 · priority 2Homes a…kpn that arrived only in part (cning, cye, jiyer, dyau, ftorres, alilic …) | 58 of 92 top directories (7.9 of 8.1 TB) still transferring at the stop (inferred from upload times); e.g. cning: 1.53 TB in the last listing (27 Nov 2024; 3.08 TB a week earlier) vs 0.32 TB here. These are upper bounds, since some data was deleted before June 2025. |
| Erich Kolb; Aleksandar Lilic; Chun Ning | A08 |
M16 · priority 2/mnt/syseng: SLT/ATE logs, LabVIEW data, firmware releases | 33 GiB (Jul 2024). |
| Bill Orner; Carl Wang; Dennis Huang; Kirk Festing | A01 A03 |
| M23 · priority 2Systems-engineering CAD on Windows desktops and cloud PLM: SiP/APD and Allegro projects, PowerDC, the 2024 Celsius trial, Arena, Onshape | Unknown; nothing on the NAS. |
| Bruce Klein; Bill Orner; Jeff Kane | A14 A18 |
M24 · priority 2Google shared drives that were not copied (7 linked drive roots, et_exstaff, ET-EX-*, the Original Esperanto Shared Drive) | Unknown; the Jul 2026 copy took one consolidated drive (2.53 TB). |
| Ray Borg; Michael Ray; Devin Patel and Zhenyi Cai (managers of 0ADP4zyteTbOdUk9PVA); Shelley (ran the July copy); you | A04 |
M31 · priority 2The April 2024 aging reports: a listing of every file older than 180 days on /eng, /project/pd, fe, jenkins, artifactory, cad, syseng | 3.4 GB gzipped (242+ /eng reports, 16+ PD). |
| Michael Ray (made them); Ray Borg | A04 A05 |
M32 · priority 263 Confluence attachments not exported, 45 of them on 8 archived pages (the Feb 2022 Fmax/Power decks, volt-freq-shmoo.xlsx) | 57 MB. |
| Stephen Lee; Devin Patel; you (ran the export) | A07 A18 |
M33 · priority 2Jira project MFGTST (MFG Test, private): exported with 0 issues | ≥ 86 silicon bring-up / ATE triage issues. |
| Devin Patel (MFGTST admin); Carl Wang; you | A07 |
The 16 smaller gaps (priority 3)
- M20 Damaged lab images: mv-swpcie21 lacks a 190 GB tail, mv-swpcie23 12.4 GB, mv-eap04 ~74 MB; swpcie21/22/23/25 have shifted blocks (mostly recoverable). A10
- M36 esperanto-soc: 1,347 branches and 14 tags dropped at the gitlab.com move. Already here in the Sep 2023 GitLab bundle; nothing to ask.
- M18 Service databases not dumped: Sysqa, rtl_metrics, dvwebdb, InfluxDB, Prometheus (9 of 12 Grafana sources). A01
- M29 1,749 of 2,376 Drive ids linked from Jira/Confluence are not in the copy (incl. the board-tracking and E-Fuse Sheets). A05
- M30 Drive copy defects: 39 dangling shortcuts, ~180 same-name duplicates dropped, 252 download-restricted objects (27.4 GB). A04
- M35 gitlab.com era: merge requests, issues, CI logs, wikis and personal namespaces were not exported. A11 A15
- M38 ATE test-program SVN at Teradyne: only an r23 working copy of Oct 2020 here. A19
- M40 Slack (esperanto-tech), mailboxes, Smartsheet, Datadog: never exported. A08
- M41 The leased NetApp (
must return
) and an AWS FSx test (Aug–Nov 2024): what moved there is not recorded. A01 A08 - M17 Artifactory: 207 of 256 filestore shards (3.86 TB) missing; Pure
/artifactoryand the VM datastore never uploaded. A01 - M34 Other Atlassian gaps: remote links, 357 unexplained key gaps, page history, the TEAM space. A07
- M37 Retired GitLab VMs (sc-gitlabdev, sc-gitlab, sc-gitlab03) and the Barcelona
barcino
backup PC;/eng/bcn_gitlab_backup(204 GB) is not in the upload. A01 A15 - M39 Minor git gaps: 67 LFS objects (2.04 GB, ML/tool data), ~19 referenced repos not located, unreadable mirror objects. A11
- M42 Other AWS buckets (SDK releases, CI inputs; 7.7 TB of SW data in Jan 2025). A03
- M43 ADTechnology (Samsung design partner) FTP exchange for Gepardo / ET-SoC-2, 2023–24. Only if ET-SoC-2 thermal work starts.
- M44 Wasabi housekeeping: 19 snapshot-only files (85 MB) not restored; no unique content lost. A08
Root: mordast, stash, and the old boxes if they survive
These need root on mordast or stash, so you or Shelley. A12 is a few commands on mordast and stash; A02 only matters once we know what survived (A01); A10 is about the lab hosts on your tailnet; A11 is low priority.
A12 Two permission fixes: the automount script on mordast (from Saturday) and stash's open exports and read-write mounts, plus a fresh snapshot (updated)
/etc/auto.home-select is still -rwxrwxrwx root (checked today). It was edited on 11 Oct at 15:52, the same minute as /etc/fstab, but the mode stayed. autofs runs it as root on every /home lookup, so any account on mordast can take root. Since anyone could have changed it, it's worth one look at its 27 lines after the chmod.
sudo chmod 755 /etc/auto.home-select
sudo cat /etc/auto.home-select # one look: anyone could have edited it
Mounts on mordast. Only /archive/esperanto and /archive/eng are read-only. /archive/atlassian, /archive/gdrive and /index are mounted read-write (/etc/fstab lines 27–29), and so is /home/github (the systemd unit home-github.mount). /archive/gdrive (the only copy of the Drive export), /index and /home/github (mode 777) pass a write-access check for an ordinary account (checked with [ -w ]; nothing written). Please remount them read-only, or tell me what needs to write there.
On stash (verified from mordast with showmount -e stash, stat and ls .zfs/snapshot): archive/esperanto, archive/esperanto/atlassian, archive/esperanto/gdrive, agentic/index and home/github are exported to everyone (so are the 53 /mnt/tank/home/* user homes), and the top directories archive/esperanto/wasabi, archive/esperanto/lab and home/github are mode 777. Since the Wasabi purge this pool is the only copy of the 2025 uploads, and home/github the only copy of the design repos that were never pushed to GitHub; the only other protection is two ZFS snapshots of archive/esperanto on the same pool (esperanto-archive-2026-04-13 and -05-02). Worth considering: read-only exports limited to the hosts that need them (mordast's fstab mounts atlassian and gdrive read-write, so check what writes there first), 755 on those three directories, a fresh snapshot of both datasets, and eventually a second copy off stash.
# on stash, as root (all read-only except the snapshots and the chmod)
zfs list -t snapshot -o name,creation -r tank
zfs snapshot -r tank/archive/esperanto@handover-2026-10-12
zfs snapshot -r tank/home/github@handover-2026-10-12
chmod 755 /mnt/tank/archive/esperanto/wasabi /mnt/tank/archive/esperanto/lab \
/mnt/tank/home/github
A02 If any of the old boxes survive: copy these subtrees to stash first, without letting the box delete anything (new)
Only if the answer to A01 is yes, it still exists
. stash has about 26 TiB (~28.5 TB) free in its pool (df today), shared with the only copy of the archive and the team's 53 homes, so it isn't enough for everything. Here is the order I'd take things in, thermal-critical first; please stop at about 85% full (zpool list tank), since a full pool would also stall everyone's homes:
- sc-truenas03
/mnt/tank/backup/A0(=A0_final):soc_top(9.2 TB),shire_top(5.1 TB),minion_top,pshire_et_block,memshire_west_side(about 18 TB in all), and everypdqa/em_irdirectory. - sc-truenas01:
archive/eng/alilic(archive_ptpx),archive/project/pd/anilm/Work_A0,archive/project/pd/adolfod,archive/exstaff/{klingner,rupeshr,rohiniv,psanchez,ivan,devinp}. - sc-truenas02 Veeam file-level restores: the newest
PureENG to Truens02 backup
point for/eng/{tberg,tbeale,rknath,rajl,shankarj,rsteeves,yhxu,zorans,jenkins}(/eng/mrayonly if sc-truenas03 is gone: it probably still holds the 22 TBa0_backup.tgz, inferred, which alone would take most of stash's free space);Pure Scratch to Truenas02
for/scratch/noc/A0/Data/{PowerRuns2,PowerRuns,BIGRUNS}; the newest PD point for/pd/mpelham/work/CHIP/INT_A0/TAPEOUT_121920,/pd/anilm/Work_Redhawk_EMIR_*and/pd/release/gpd. (Veeam console: Home → Backups → Disk → the job → Restore guest files / file share, to an NFS or SMB target on stash.) A restore needs Veeam B&R 12 and, if the jobs were encrypted, their password (Michael Ray set them up). Veeam configuration backups from Feb 2023 are in the ET IT Shared Company Folder0AD0UWR09ToBsUk9PVA(IT-1075; see A04). - IO FreeNAS:
/mnt/tank/home/data/rknath/ARCHI-4/Data, then/data/{power*,tberg,yhxu,shankarj}.
Before an old TrueNAS goes back on a network, keep it unplugged or, on first boot, disable every task under Data Protection (Periodic Snapshot, Replication, Cloud Sync, Rsync). After a year offline their retention rules would delete expired snapshots on the first run, and those snapshots may hold the archive/pd data deleted in Jul–Nov 2024 (inferred; standard TrueNAS behaviour, not tested here). On sc-veeam01, disable all jobs before it can reach the repository. Even for a box that boots, the disks-only path below is the safest. Import the pool read-only on any ZFS host other than stash (stash's own pool is also called tank), by its numeric id, under an altroot, and send from a snapshot that already exists:
# a box that boots (network unplugged, tasks disabled): read-only on the source
ssh stash zfs create tank/archive/a0_final
cd /mnt/tank/backup/A0
rsync -aH --numeric-ids soc_top shire_top minion_top stash:/mnt/tank/archive/a0_final/
# disks only, on a spare ZFS host: nothing is written to them
zpool import # lists the pools and their numeric ids
zpool import -o readonly=on -N -R /mnt/rescue <numeric-id> # add -f if it says the pool was last used by another system
zfs list -o name,used,refer -r tank # please send us this output
zfs list -t snapshot -o name,creation -r tank
zfs send -R tank/archive/eng@<newest-existing-snapshot> \
| ssh stash zfs recv -u -o readonly=on -x mountpoint tank/archive/eng-truenas01
# if send -R stops with "cannot hold ... read-only": send each dataset's snapshot without -R,
# or import without -N and rsync from /mnt/rescue/...
Why: these are the only places the full-chip layout, the shire-level power runs and the PD work areas are known to have existed after 2023.
Would close or narrow: M01 M04 M05 M06 M07 M08 M09 M10 M12 M13 M14 M15
A10 Lab hosts: what are esperanto-soc2…10 and esperanto-flash, and can what's left be imaged before reuse? (new)
Your tailnet lists esperanto-soc2 (offline) through esperanto-soc10 and esperanto-flash. I haven't touched them. The lab flash PC's history (/archive/esperanto/lab/flash/et/.bash_history, 12 Mar 2026) has ssh esperanto-soc2 and ssh roman@10.20.10.1, and a card image gianluca.esperanto-soc4.bin is dated 5 Aug 2025, so I guess they are re-used Esperanto card hosts on the old Mountain View lab subnet (inferred). esperanto-flash is probably the lab flash PC itself: its history runs tailscale status and rsyncs itself to ainekko@mordast (inferred). 10.20.10.x was Esperanto's Mountain View lab subnet (gateway 10.20.10.1 in the Machine Setup from Scratch
page, mv-swpcie32 at 10.20.10.228 in IT-1175), and mordast (10.20.10.72) is on it now. Gianluca (stash home gianluca
, inferred Gianluca Guida) flashed cards on esperanto-soc4 on 5 Aug 2025 and would know which machines these are. Which physical machines are they (old hostname, serial), and do they still hold their pre-2025 disks?
The root disks were copied on 28–30 Jul 2025 and cards were flashed on esperanto-soc4 a week later, so old root disks may be overwritten already; data disks may survive. Never imaged at all: mv-gb01…19 (the Aug–Oct 2024 power traces under mv-gb09:/mnt/esperanto/et/power/_s6_traces_2024-08-28 and later, and mv-gb19:/mnt/esperanto/et), the 8 TB data disks of mv-swpcie22 and mv-swpcie25, mv-demo01, mv-lab07/09 and the lab-desktop file server. If the mv-swpcie21/23 or mv-eap04 disks still exist, a re-image would fix the copies that dropped blocks (swpcie21 lacks a 190 GB tail).
# on stash, with the lab disk attached; mark it read-only first, the map file lets it resume
blockdev --setro /dev/sdX
cd /mnt/tank/archive/esperanto/lab
ddrescue -d -r3 /dev/sdX <host>.img <host>.map
# or just the data directories; -R keeps each full path (.../<host>-files/mnt/esperanto, .../home/et, .../root)
rsync -aHAXR --numeric-ids /mnt/esperanto /home/et /root stash:/mnt/tank/archive/esperanto/lab/<host>-files/
Why: the 2024 card power and temperature campaigns on mv-gb09/gb19 are the most recent measured data, and they exist nowhere else.
A11 Make the three unreadable mirror repos readable (updated)
Still open today: the loose objects of esperanto/gepardo-soc (820,132 of 820,169) and customers/denso (64 of 102) are mode 0600, owner shelley, and so are 360,726 objects of the bare esperanto/qqq, whose tape_out tag exists nowhere else. The copy through macOS also left 78,804 AppleDouble ._* files; 117 ._pack-*.idx make git print non-monotonic index
. Low priority; faster on stash than over NFS.
sudo chmod -R a+rX /mnt/tank/home/github/esperanto/esperantotech/esperanto/gepardo-soc/.git/objects \
/mnt/tank/home/github/esperanto/esperantotech/customers/denso/.git/objects \
/mnt/tank/home/github/esperanto/esperantotech/esperanto/qqq/objects
find /mnt/tank/home/github/esperanto -name '._*' -path '*/.git/*' | wc -l # review, then add -delete
Why: the Gepardo (ET-SoC-2) history and the qqq tape_out tag become browsable.
In a browser
AWS console, Google Workspace admin, Tailscale.
A03 AWS: list the Esperanto buckets, and restore the PD archive from Glacier (new)
In the AWS accounts 828856276785 and 554873560799 (owner of record Dave Ditzel): please list all buckets, and for s3://pure-pd-archive the object count, total size and storage classes. A Glacier lifecycle was set on 14 Mar 2025 after the Jan 2025 bill said this bucket was most of the S3 bill
; mordast has only the 10.2 GB of STANDARD objects copied to Wasabi on 30 Jun 2025. Also check a0-archive-et-soc-1, a0-archive-us-west and it-1249-a0-archive (the 2023 A0 syncs; the last run logged Completed 38.7 TiB
) and veeam-to-s3-backups-4-5-2* (Veeam chains of all 8 Pure file systems, ~4 Jun 2024; restoring these also needs Veeam B&R 12 and the job's encryption password, if one was set). If they exist, restore and copy soc_top, shire_top and minion_top first: soc_top is most likely only in the a0-archive-* buckets, since the pure-pd-archive upload seems to have stopped after memshire_east_side. Bulk Glacier restores take up to two days; moving ~30 TB out of AWS costs a few thousand dollars at list price (my estimate).
aws s3 ls
aws s3 ls s3://pure-pd-archive/A0/ --recursive --summarize | tail -2
aws s3api list-objects-v2 --bucket pure-pd-archive --query 'Contents[].StorageClass' \
--output text | tr '\t' '\n' | sort | uniq -c
for b in a0-archive-et-soc-1 a0-archive-us-west it-1249-a0-archive; do
aws s3 ls s3://$b/ --recursive --summarize | tail -2
done
# then: S3 Batch Operations > Restore (Bulk) on the prefix that holds A0, and after it completes
aws s3 sync s3://<bucket that holds it, e.g. a0-archive-et-soc-1>/<prefix>/soc_top <dest> --force-glacier-transfer
Why: the cheapest place the full-chip layout may still exist; no hardware needed.
A04 Google Workspace: share or copy the shared drives that weren't copied (updated)
Read-only to yaroslav@nekko.ai, or rclone them to stash by id. In order:
0ADz9vEuaYsTGUk9PVA: the ex-staff home tarballs (24 listed in EIP-125, at least 16.3 TiB, since bmaric.tgz was still being written at 9.8 TiB; incl. bmaric, psanchez, nshah, rohiniv, rupeshr; a 15 Apr 2024 screenshot shows klingner, bmaric, kiran, jsola and jsanchez uploading), andET-EX-ENG-Tarballed-Home-Directories
. From these, rupeshr, rohiniv, klingner and psanchez first.et_exstaff
(departed staff laptop copies, PST mail, Gdocs).0AIMo64EKj195Uk9PVA(ET VLSI Shared Drive),0ADP4zyteTbOdUk9PVA(P/TE, inferred),0AD0UWR09ToBsUk9PVA(ET IT Shared Company Folder; holds the Veeam/TrueNAS configuration backups),0ABPxZKAuxOIxUk9PVA(All Hands Documents and Recordings),0AFeuUUGKHxLYUk9PVA(OPM),0AHK5s3IRLKXiUk9PVA, and theOriginal Esperanto Shared Drive
(a dangling shortcut at the root of the copy).
Correction to Saturday's item 5: the 403s were 252 objects (27.4 GB), mostly All Hands decks and Meet recordings, not vendor PDFs.
# on stash; lists every shared drive with its id
rclone backend drives gdrive:
rclone copy gdrive: /mnt/tank/archive/esperanto/gdrive-extra/exstaff \
--drive-team-drive 0ADz9vEuaYsTGUk9PVA --drive-acknowledge-abuse -P \
--include '{klingner,rupeshr,rohiniv,psanchez}*.tgz'
Why: the ex-staff PD homes hold the 2020 soc_top RedHawk-SC EM/IR runs (rupeshr), the SOC_Top bumps (rohiniv) and the ET-SoC-1.5 mPower runs (klingner); none of it is anywhere on mordast.
A05 Google Workspace: export the linked power and test Sheets and decks by id, and the leavers' My Drives (updated)
I have a list of 227 Drive ids in priority order (in my home on mordast: /home.local/yaroslav/claude/vault-2026-10-11/synthesis/missing/drive_ids_wanted.tsv, readable by root only; I'll send it): the 123 weekly Power_Topics decks (Aug 2020 – Feb 2023, Francisco Torres), the power Sheets (ET-SOC-2 PTPX experiments, Power Projection Spreadsheet, ptpx_data_table_JB, power measurement and estimate
), the P/TE characterization and package Sheets (board_vmin_corr_round2, Vmin vs process and temperature, ETSOC1 Die Related Information, Package X/Y), the board-tracking and E-Fuse Sheets, and the Apr 2024 aging-report folder 1dyaLxlkDN6jChTGuaZNYcPTTjeIXsV_m.
With a Workspace super-admin: do the accounts of Francisco Torres, Matt Ward and Shankar Jayaratnam (left 30 Jun 2025, email is only archived
) and of Tom Berg, Eric Soriano, Aleksandar Lilic and Devin Patel still exist? If so, please export their Drives (Takeout or Vault, Sheets as .xlsx). Correction to Saturday's item 7: 8 of the 11 power shortcuts named there are already on mordast.
rclone backend copyid gdrive: <file-id> /mnt/tank/archive/esperanto/gdrive-extra/byid/ \
--drive-export-formats xlsx,pptx,docx
# Google Vault: Export > Drive > the accounts above > include shared-drive items
Why: the weekly power decks and Sheets record per-block and measured power through tape-out and bring-up; the aging reports are the only file-level map of the homes and of /project/pd.
A06 Rotate or revoke credentials that are in the archive (new)
Paths only; I haven't used or copied any of them. They are readable by every account on mordast (checked: ekolb/.aws/credentials and the wildcard key are mode 0644; the lab disk images are 0644 too; and even mode-600 files open for a non-owner account, see the flash PC below). Restricting read access on the archive would also block the archive browser, so rotating is the real fix.
- AWS keys:
/archive/esperanto/wasabi/esperanto-eng/ekolb/.aws/credentialsandekolb/.s3cfg(6 Sep 2023);home_eng_jenkins/.aws/credentials(2019) andjenkins/.aws. - Wildcard TLS key for esperanto.ai:
ekolb/wildcard-esperanto.ai.key(May 2021). - Jenkins SSH keys, incl. AWS
.pemfiles:home_eng_jenkins/.ssh(51 files). - A Datadog API key in
ekolb/.history(2021); the sc-truenas03 web login in a comment of IT-1467; a vendor FTP login in IT-1701. - GitHub tokens in 75 lab
.git/configfiles, e.g.lab:sdb1:/esperanto/avuong/et_sdk_tools_1.6.0/et_repos/repos/gp-sdk/.git/config. - The lab flash PC's SSH key,
/archive/esperanto/lab/flash/et/.ssh/id_rsa(Oct 2023). Mode 600, ownernobody, but every account on mordast can read it anyway: the stash export appears to map every client user to the files' owner (inferred; checked 12 Oct withtest -rfrom a non-owner account, an access check that doesn't open the file). Itsauthorized_keyschanged on 28 Aug 2025, and its history ssh'es to esperanto-soc2 (host key inknown_hosts) and to roman@10.20.10.1 (the lab gateway in the old setup pages; no host key recorded), so the key may still be trusted on current hosts (inferred). - Lab server keys: 123 files named
id_rsa/id_ed25519/id_ecdsain the lab images (some are test keys), e.g.lab:sda1:/backup.mv-swpcie21.img:/root/.ssh/id_rsa,lab:sdf:/backup.mv-swpcie23.img:/root/.ssh/id_rsaand the sharedetaccount's key on several hosts. If any esperanto-socN is a re-used lab host that wasn't reinstalled, itsauthorized_keysmay still trust these.
# for each key id in those files: shows the IAM user and when it was last used
aws iam get-access-key-last-used --access-key-id <id>
aws iam update-access-key --user-name <user> --access-key-id <id> --status Inactive
# on the esperanto-soc hosts and the flash PC
cat ~et/.ssh/authorized_keys /root/.ssh/authorized_keys # drop keys that aren't current
Why: the AWS account was alive in 2025, and the flash-PC key may still be trusted on current hosts. A chmod on stash would not help either: the export maps every client to the owner (inferred).
A13 Tailscale SSH check period for mordast (from Saturday)
Unchanged from Saturday, and I can't verify it from mordast. In the nekko.ai policy, the SSH rule for yaroslav@nekko.ai to tag:lab-maintenance (mordast) needs either "action": "accept" or the longest check period, so agents can reach mordast non-interactively. Same for aifoundry1–3 if you're in there anyway.
{ "action": "check", "src": ["yaroslav@nekko.ai"], "dst": ["tag:lab-maintenance"],
"users": ["autogroup:nonroot"], "checkPeriod": "168h" }
Elsewhere: vendors and people
Requests that go outside, so through you.
A16 Alchip (through you): the full-chip OASIS and ICC2 databases of the A0 tape-out (updated)
Alchip received at least eight full-chip OASIS databases and four ICC2 tarballs for TSMC product TMNK96 (customer 0G33) between Jul and Dec 2020, e.g. soc_top_A0_FULL_071720_noIPtext.oasis, soc_top_full_merge_102720-2030layerV2.oasis, soc_top.nlib_1124_reduced.tgz and soc_top_nlib_1111.tgz, and probably the final soc_top_full_merge_121920_remove_uncon_VDD_0P65.oasis (inferred: masks were ordered on 23 Dec 2020). Mike Pelham was the contact. Would they confirm they keep the tape-out archive and release to the asset owner a layer-reduced view (die outline, block and macro placement) cleared by TSMC? This is the 6 Oct layout request, now with names; I can send the exact list of what the solver needs.
Why: an independent copy of the final layout that doesn't depend on any Esperanto hardware surviving.
A17 Amkor, the heatsink vendor and Meta (through you): package and cooling models (new)
Amkor: the ET-SoC-1 45×45 mm package FEA model (materials, geometry), any JEDEC theta-JC / psi-JT characterisation or a 2-resistor / Delphi model, the substrate SiP databases (TSM_4720, 0859819SP) and the stack-up. Radian (or Boyd): heatsink drawing T31-235 / INF-760355-MIS. Meta: the four SoC thermal models uploaded to the C1 SharePoint on 18 Apr 2019.
Why: no package thermal model or theta values exist anywhere in the archive.
A18 Ex-Esperanto engineers: personal copies of the missing power data (new)
Some are reachable inside the company (new Atlassian accounts since Aug 2025, or stash homes named after them; the matches are inferred):
- Aleksandar Lilic (
aleksandar
; could also be Aleksandar Ugarak):alilic/dev/archive_ptpx, the ET-SOC-2 PTPX experiments Sheet, the mv-gb09 traces, Ivan Ratkovic's Drive. - Bruce Klein (
bruce
): the Perforce depot, SiP/Allegro projects, the 2024 CelsiusDC runs on his Windows desktop. - Howard Davidson (
howard
): cooling and packaging contractor 2022–23. - Pedro Sanchez (
pedro
): his psanchez home, the 4.7 TB tarball in EIP-125 (M08). - Ying Hao Xu (
yinghao
): yhxu, one of the homes after kpn (M09). - Jayesh Iyer (
jayesh
): where the 2021–24 architecture/PD areas went. - Barry Pangrle (new account Aug 2025): the 2019 power model and the Ansys/RedHawk evaluation.
Is it all right if I ask these people directly? For those below, would you forward a short note? I'll draft it.
Through you, if you're in touch: Rajib Nath (PowerRuns2 PT-PX reports, ARCHI-4); Tom Berg and Terry Beale (gate-level shire/NoC power FSDBs, virus70 full-chip PT-PX); Francisco Torres (Power_Topics decks, tape-out PT-PX in /archive/eng/ftorres/power); Stephen Lee (Feb 2022 Fmax decks, V-f shmoo); Devin Patel (P/TE Sheets, Silicon Bring-Up Presentation (Devin).pptx
, which Atlassian itself lost); Chun Ning (cning soc* run areas).
Why: small, high-value files (reports, Sheets, decks) often survive on laptops even when the NAS copies are gone.
A19 Teradyne (through you): a dump of the ATE test-program repository (new)
tron.teradyne.com/Esperanto/ET_SOC1 hosted the ATE test programs; mordast has only an r23 working copy of Oct 2020 (190 files, 215 entries). Could they send a dump, or at least the final test program with its PVT/POWER blocks?
svnadmin dump <repo> | gzip > ET_SOC1.svn.gz
Why: temperature-dependent test limits and sensor trim.
Would close or narrow: M38
One-line questions
Some have a short table attached, but each is answerable in a line.
A01 What happened to the colo hardware after June 2025? (new)
For each box: still powered, in storage, wiped, returned or sold, and where? The QTS colo machine list was last edited on 24 Jun 2025 and still listed the TrueNAS boxes, sc-pure01, the IO NAS and the NetApp. Ray Borg (Head of IT; last seen in Jira 1 Jul 2025), Michael Ray, Erich Kolb and Patrick Lugliani ran them; Dave Ditzel is owner of record of the cloud accounts (AWS: A03).
If you don't know, would you forward this table to Ray Borg or Michael Ray? Vendor records could also tell: Pure1 shows whether sc-pure01 still phones home, iXsystems support can look up serials A1-100191 and A1-72549, and QTS should have the move-out and shipping records for the cage.
| Box | Identity | What it held | Last seen |
|---|---|---|---|
| sc-truenas03 | TrueNAS Mini XL+, SN A1-100191, 10.10.1.250 | the A0 tape-out tree + a0_backup.tgz; 49.1 TiB used | 21 Nov 2024 (IT-1958) |
| sc-truenas01 | iXsystems X10-S, SN A1-72549, 10.10.7.10 | /archive: pd, exstaff, eng, sw, home/eng; 221–245 TiB | Jul 2024 df; on the colo list 24 Jun 2025 |
| sc-truenas02 + sc-veeam01 | 10.10.8.10 | Veeam: /eng, fe, home_cad to 10 Jan 2025; scratch, artifactory to ~5 Dec 2024; archive02/exstaff | Jan 2025 (IT-2668) |
| sc-pure01 | FlashBlade, 26 blades, 10.10.1.60 | the live volumes: /eng after kpn, /pd, /scratch, jenkins, artifactory, syseng; maybe /eng/mray/a0_backup.tgz | in use Apr 2025 (IT-2734) |
| IO FreeNAS | io.esperanto.ai, 10.10.1.3 | /data incl. rknath ARCHI-4 power runs; 11 TiB | Jan 2024, we still have it(IT-1877) |
| ESXi hosts | sc-perforce (10.10.1.181), sc-metrics01, GitLab VMs | Perforce depot; Sysqa / rtl_metrics DBs | 2023–25 |
| LaCie drive | ext4 | A0 rsync, full at 37.8 TB | Jul 2023 (IT-1246) |
| Leased NetApp | Netapp HaaS, must return | planned Pure replacement; unknown what moved | Aug–Nov 2024 |
| Jenkins / compute servers | sc-server02, sc-server14, sc-server59, sc-b4c-s1/s2 | local Jenkins workspaces with the per-shire PT-PX runs and FSDBs (M12) | listing of 19 Nov 2024 |
If a TrueNAS box powers up (network unplugged, scheduled tasks disabled: see A02), the output of zfs list -o name,used,refer -r tank and zfs list -t snapshot -o name,used,creation -r tank would already tell us most of what's on it.
Why: almost every large gap (full-chip layout, PD work areas, PT-PX runs, ex-staff homes, the homes after kpn
) has its best surviving copy on one of these.
Would close or narrow: M01 M04 M05 M06 M07 M08 M09 M10 M12 M13 M14 M15 M16 M18 M21 M37 M41 M17
A08 Who ran the June 2025 Wasabi upload, why did /eng stop at “kpn”, and was anything else copied at shutdown? (new)
The rclone walk of /eng (20–23 Jun 2025, Wasabi account dave.ditzel) reached kpn
at 20:52 UTC on 23 Jun and stopped there (the parallel walks ended by ~22:30), with 58 of 92 top directories probably still transferring (inferred from upload times). Only a 3-hour excerpt of the parallel fe log survives (eng/ekolb/rclone.log), so I guess Erich Kolb ran it (inferred). Is there a full log or exit status? Was it resumed anywhere? Did /eng or /pd also go to a disk, the leased NetApp or AWS FSx? For the provenance notes only: who imaged the lab disks (28–30 Jul and 15–18 Dec 2025), and who ran the 30 Apr 2026 Wasabi purge (the attestation leaves the executor blank)? And does a Slack export of esperanto-tech exist? It would answer most of this. Last, the bucket the Veeam job PureENG to Wasabi
wrote in Jul 2024 (10.5 TB, run by Erich Kolb): was it in the dave.ditzel account, or in another Wasabi account that may still exist?
A09 What was stash /mnt/tank/archive/eng meant to receive? (updated)
It was created on 15 Dec 2025 at 17:09 PST, 21 minutes before the first lab disk image started writing to stash (17:31), and is empty (0 entries; the 26 TiB that df shows is just the pool's free space, the same on every stash dataset). It is exported to 10.20.10.0 (the old Mountain View lab subnet mordast is on) and 10.10.1.0, the colo subnet of sc-truenas03, the IO FreeNAS and the Pure. Its name matches sc-truenas01's tank/archive/eng, but sc-truenas01 was at 10.10.7.10. Which box was it for, and was that box on the lab network in December? (This replaces Saturday's question about the bigger NFS archive.)
ls -la /mnt/tank/archive/eng; zfs get creation,used,refquota tank/archive/eng
A07 Is there any Atlassian backup from before the site was deactivated? (updated)
A Jira XML backup or a Confluence site export would close: the histories cut at 40 entries (650 issues, 24,834 of 433,529 entries; also needed to explain 357 key gaps), the private MFGTST MFG Test
project (exported with 0 issues; at least 86 bring-up/ATE triage issues), 45 attachments on 8 archived pages (incl. PROJECTS page 2616426497 with volt-freq-shmoo.xlsx and the three Feb 2022 Fmax meeting DD status
decks), page history, remote links, and the TEAM space (empty, or no permission?). If the site can be reactivated even briefly, two calls recover most of it:
GET /wiki/rest/api/content/2616426497/child/attachment?status=archived
jira_changelog.py with a read-only token (per-issue changelog endpoint)
The script: jira_changelog.py (published next to this page).
Why: the Fmax/Power material is measured V-f and power data; MFGTST holds the silicon bring-up triage.
A14 Perforce: was the sc-perforce depot saved anywhere? (updated)
mordast has the server root of Apr 2022 (journal and db) but file content for only 112 of 23,677 depot paths, and the server kept running to at least May 2023 (IT-1222, IT-1285). Bruce Klein was the main client. If any of it survives, these paths first: //depot/Systems/ET-SOC/Substrate/..., //depot/Systems/PCIe-Dev-Card/Rev3/{PCB,Mechanical}/..., //depot/Systems/ET-SOC1 Test PCIe Card/..., //depot/Systems/BUB-1.0/... (not the PMIC SAMD20 demo-firmware tree).
# on the server: a checkpoint without rotating the journal, plus the Systems archives
# (the /opt/perforce path is inferred; the archived copy is under eng/git/perforce/servers/master/{root,journals,archives})
p4d -r /opt/perforce/servers/master/root -jd /tmp/p4-checkpoint
tar czf p4-systems.tgz /opt/perforce/servers/master/archives/depot/Systems
Why: package stack-up, substrate and heatsink geometry for the package thermal model.
A15 Is gitlab.com/esperantotech still alive, and what is the tailnet device “gitlab”? (updated)
The 513-repo mirror is complete for git (newest commit 3 Jun 2025), but merge requests, issues, CI pipelines and job logs, wikis, registries and personal namespaces of the gitlab.com era (Jul 2023 – 2025) were never exported. If the group still exists, per-project exports would keep the MRs, issues, wikis and pipeline history; job logs, registries and personal namespaces need separate API pulls or your account. Are the github.com/et-* orgs (490 repos pushed 17–19 Sep 2025) still there? What is the tailnet device gitlab (owner roman@): an old VM, a restore or a mirror? And do the retired sc-gitlabdev / sc-gitlab / sc-gitlab03 VM disks or the Barcelona barcino
backup PC still exist?
glab api --paginate 'groups/esperantotech/projects?include_subgroups=true&per_page=100' \
| jq -r '.[].path_with_namespace' | wc -l
glab api -X POST projects/<id>/export # one per project: MRs, issues, wiki, pipeline history (no job logs)
glab api projects/<id>/export/download > <name>.tar.gz
glab api projects/<id>/jobs/<job-id>/trace > <job-id>.log # job logs are not in the export
Why: CI logs and MRs record which power/PT-PX jobs ran and where their outputs went.
A20 Do any Synopsys or Siemens licences still work, and may we run them? (from Saturday)
Unchanged from Saturday. Needed to open NDM, OASIS or FSDB files if any of the above is recovered.