- - not enough to have a record
- - auditors will ask you to prove the data is not laying around
- proving all changes shipped are reviewed and linked to tracked work - - auditors will ask you to show live that you can’t approve your own changes
- - some auditors might ask you for an audit log to prove no unexpected branch rule changes occurred —- depending on the observation period, you might have to build your own audit log capture to prove this
- proving you performed penetration testing - - running a DR test might be more than a few hours depending on your data size and level of infra automation
- - this is often something that startups are ready to execute, but don’t invest a lot of time automating
- proving you have and enforce full-disk-encryption on all your employee laptops - - this is automated with MDM but a startup might not be running an MDM yet
- proving you are rotating credentials on the frequency you ascribe to in your policies - - automated reports are available for some credentials, e.g. AWS keys, but takes more work for smaller vendors
- - even with AWS, you might discover you forgot to rotate something, and it might be because it’s non-trivial to execute
- perform quarterly access reviews - - some systems are more difficult/time consuming to inspect against your employee and permissions list
- - ideally this is automated, but often times at a startup, you might not have fully automated authorization and access control, such that when employees change teams or leave the company, that you get notified and don’t miss it
- proving that you act on performance or reliability alerts - - auditors will ask you to show live some examples of past alerts and that someone handled it
- - auditors will often ask you to show live that these alerts are consistently configured for all your production system —- startups might not have the alerting and PagerDuty-like setup be fully automated (e.g. with Terraform)
Ultimately this combo worked:
1. https://pi.dev/packages/pi-tool-guard —- corrects key name synonyms and common structure errors, so tool calls succeed automatically (e.g if the model hallucinates old_str instead of oldText). It also wraps top level oldText/newText in an edits array if the tool didn’t do it.
2. https://pi.dev/packages/@aboutlo/pi-smart-edit - white-space-tolerant edits, as Qwen would sometimes add a fifth space to a four space indent
Hashline edit tools didn’t work well for me at all, they confused the model and it still failed to edit correctly. Also line removals would invalidate the rest of the file requiring re-reads. I tried pi-hashline-edit-pro, though I see it now keeps a database of hashes to help keep them stable across edits. Regardless Qwen kept thinking that the hashline prefixes were part of the source.