Compliance & Legal

How to collect soc 2 audit evidence as Chief of Staff and Founder's Office

Compliance & LegalFor Chief of Staff and Founder's Office3 apps12 steps~24 min to set up

SOC 2 audit season hits the chief of staff like a second job. You're the one corralling access review screenshots from the engineering lead, chasing the HR team for terminated-employee offboarding records in ADP, pulling vendor contracts out of Notion, and cross-referencing who had admin access to AWS last quarter. None of this lives in one place. You're threading together Slack threads, shared Google Drives, email chains, and spreadsheets you built yourself to track what evidence has been collected versus what's still outstanding. The auditor asks for something and you spend 45 minutes figuring out whether it exists, who has it, and whether it's the right version. This repeats for six weeks.

Build this on Starch → See the apps used

Compliance & LegalFor Chief of Staff and Founder's Office3 apps12 steps~24 min to set up

Outcome

What you'll set up

A centralized SOC 2 evidence tracker that pulls from your real systems — Slack, Notion, Gmail, ADP, AWS — and shows you exactly what's been collected, what's missing, and who owns each open item

Automated collection runs that pull access logs, payroll records, and system change events on a schedule so evidence gathering isn't a manual scramble every time the auditor asks a follow-up

A status dashboard you can share with your auditor and CEO that shows evidence completeness by control category without you having to update it by hand

The Starch recipe

Apps, data, and prompts

The combination of Starch apps, the data sources they pull from, and the prompts you use to drive them.

Apps used

Knowledge Management Task Manager

Data sources & config

Starch syncs your ADP data on a schedule (employees, org units, termination records) and syncs your Notion pages and databases on a schedule for policy docs and vendor agreements. Gmail is synced on a schedule so Starch can scan for security-related threads and vendor contracts. AWS Cost Explorer and CloudWatch are queried on-demand — no scheduled snapshot — when the access-change automation runs. Slack is connected from Starch's integration catalog so the agent can post the weekly digest. Any auditor portal or compliance tool without a direct API connection is automatable through your browser — no API needed.

Prompts to copy

Build me a SOC 2 evidence tracker with control categories (Access Control, Change Management, Availability, Confidentiality, HR/Offboarding), evidence status (Not Started, In Progress, Collected, Auditor Reviewed), owner assignment, due date, and a link field for each artifact. Pull the list of current employees and recent terminations from ADP so I can cross-reference access reviews against actual headcount.

Every Monday, check AWS Cost Explorer and CloudWatch for any new IAM policy changes or privilege escalations in the past 7 days and add them as evidence line items in the Access Control category of my SOC 2 tracker.

Scan my Gmail and Notion for any documents tagged 'vendor agreement,' 'DPA,' or 'security review' created in the last 12 months and surface them in the Confidentiality section of the tracker as candidate evidence items.

Create a weekly digest that Slacks me every Friday at 4pm with a count of: total controls, how many have collected evidence, how many are overdue, and which owners have open items assigned to them.

Run these in Starch → or paste them into your favorite agent

Walkthrough

Step-by-step

1 Connect ADP to Starch (scheduled sync) — this gives you a live employee and termination roster that becomes the backbone of your access review. Every terminated employee in ADP becomes a line item to verify offboarding in your tracker.

2 Connect Notion (scheduled sync) — Starch pulls your existing policy docs, runbooks, and vendor agreement databases so you're not recreating an index from scratch. Existing Notion pages map to specific SOC 2 control categories.

3 Connect Gmail (scheduled sync) — Starch scans for vendor security questionnaires, DPA signatures, and audit-related threads so evidence that lives in your inbox gets surfaced rather than buried.

4 Connect AWS on-demand — tell Starch to query CloudWatch and Cost Explorer for IAM changes, new privilege assignments, and configuration drift. This runs on a schedule you set (weekly is typical) and appends findings to the Access Control category.

5 Connect Slack from Starch's integration catalog so the agent can post status digests to your #compliance or #chief-of-staff channel without you manually updating a spreadsheet.

6 Describe your SOC 2 tracker to Starch in natural language — list your five Trust Service Criteria, the evidence types you need per category, and the owner field. Starch builds the app. No spreadsheet setup, no column mapping.

7 Do a first-pass evidence import: run the Notion and Gmail scans to auto-populate what already exists. You'll typically find 40–60% of required artifacts are already somewhere in your systems — they just weren't indexed.

8 Assign owners to open line items directly in the Starch app. Each owner gets a Slack notification via the connected Slack integration listing exactly what they owe and by when.

9 Set the weekly Monday automation — Starch pulls new AWS access events, checks ADP for any terminations since last run, and flags new gaps in the tracker automatically. You review a diff, not a blank canvas.

10 Share a read-only dashboard view with your auditor. The view shows evidence status by control category, links to each artifact, and last-updated timestamps — formatted for what auditors actually ask for, not what's convenient for your internal tooling.

11 As the audit progresses, use the Email Triage app to manage auditor follow-up threads — Starch summarizes long back-and-forth threads, drafts replies with references to the evidence already collected, and flags any auditor questions that have been sitting unanswered for more than 48 hours.

12 At audit close, run a completeness report: Starch queries the tracker and produces a summary of every control, its evidence artifact, collection date, and auditor sign-off status — your post-audit record for the next cycle.

See this running on Starch

Connect your tools, describe what you want, and the agent builds it. Closed beta is free.

Try it on Starch →

Worked example

Q1 2026 SOC 2 Type I Evidence Collection — 14 Weeks to Report Date

Sample numbers from a real run

Total controls mapped (CC + A + C + PI + P)	47
Evidence items pre-populated from Notion + Gmail scan	28
Open items requiring active collection at week 1	19
ADP terminations in scope for access review (past 12 months)	11
AWS IAM change events surfaced in first weekly run	6
Vendor contracts located via Gmail scan (DPAs + MSAs)	14
Hours saved vs. manual spreadsheet tracking (estimated over 14 weeks)	38

The company's SOC 2 Type I scope covers 47 controls across Common Criteria, Availability, and Confidentiality. When the chief of staff ran the initial Notion and Gmail scan, Starch surfaced 28 existing artifacts — policy docs, signed vendor DPAs, incident response runbooks — that already satisfied controls without any manual collection. That left 19 open items. The ADP sync identified 11 employees terminated in the past 12 months; Starch flagged each one as an access review line item and notified the IT lead via Slack with the specific systems to verify. The first AWS on-demand query pulled 6 IAM privilege changes from CloudWatch that weren't previously documented — those became evidence items under Access Control, filled with the raw log exports Starch retrieved. The auditor got a shared dashboard on day 3 of the engagement instead of week 6. Weekly Monday automations caught 3 additional gaps mid-audit before the auditor asked about them. Total active coordination time for the chief of staff: roughly 2 hours per week instead of the usual 8–10.

Measurement

How you'll know it's working

Evidence completeness rate by control category (target: 100% by report date, tracked weekly)

Average time from auditor request to evidence delivery (target: under 24 hours)

Number of open items with no assigned owner (target: zero at any given week)

Terminated-employee access review coverage (% of ADP terminations verified across all systems)

Weeks to audit readiness (baseline vs. Starch-assisted cycle)

Comparison

What this replaces

The other ways teams handle this today, and how the Starch version compares.

Vanta or Drata

Purpose-built compliance platforms with deeper pre-built control frameworks, but they cost $15–25K/year, require engineering to set up integrations, and don't connect to your operational data (HubSpot deals, ADP payroll, Notion docs) the way Starch does — so you still end up manually bridging gaps the tool doesn't cover.

Google Sheets + manual collection

Free and flexible, but you are the automation — every AWS log, ADP termination, and Gmail thread gets pulled by hand, which is exactly the 8-hour-per-week problem Starch exists to eliminate.

Notion database as evidence tracker

Works well for organizing docs you already have, but Notion can't query ADP, run scheduled AWS checks, or scan Gmail — it stores what you manually put in it, not what Starch can pull automatically.

Your auditing firm's evidence portal

Auditors often provide a shared request list in a portal (like Fieldguide or Auditboard), but those tools are designed for the auditor's workflow, not yours — they don't connect to your systems, so you're still doing the evidence collection entirely manually before uploading.

On Starch RECOMMENDED

One platform — knowledge management, task manager, founder inbox all running on connected data. Setup in plain English; numbers stay current via scheduled syncs and live agent queries.

Try it on Starch →

FAQ

Frequently asked questions

Does Starch actually pull audit logs from AWS, or does it just organize links to them?

Starch queries AWS Cost Explorer, CloudWatch, and related services on-demand when you run the automation. It retrieves the actual event data — IAM changes, configuration events — and surfaces them as structured line items in your tracker. AWS is on-demand only today, meaning it runs when triggered rather than on a continuous sync schedule, but you can schedule that trigger to run weekly or on whatever cadence your audit requires.

Is Starch SOC 2 certified itself? Should I be putting audit evidence into it?

Starch is not SOC 2 Type II certified today — that's worth knowing. For most chiefs of staff using Starch to organize and track evidence collection (rather than storing the raw sensitive artifacts themselves), this is a reasonable tradeoff. The tracker lives in Starch; the actual artifacts (payroll records, access logs, signed contracts) can remain in their source systems and be linked rather than uploaded. Talk to your auditor about what counts as 'in scope' for your environment.

We use Rippling instead of ADP or Paylocity. Can Starch pull employee and termination data from Rippling?

Rippling is available through Starch's integration catalog, so the agent can query it live when your tracker needs to check current headcount or recent terminations. It won't be a scheduled background sync the way ADP is, but it will pull fresh data each time the automation runs, which is sufficient for weekly access reviews.

What if some of our vendors don't have APIs and their security questionnaire portals are clunky web forms?

Starch automates any website through your browser — no API needed. If a vendor's security portal requires you to log in, navigate to a form, and fill in answers, Starch can automate that workflow. You'd describe the steps and Starch runs them. This is the same mechanism it uses for other browser-only workflows.

Can the auditor see the tracker directly, or do I have to export everything?

You can share a read-only view of the Starch dashboard directly with your auditor. They see evidence status, artifact links, and last-updated timestamps in real time without you exporting spreadsheets. Whether your specific auditor is comfortable working in a shared Starch view versus their own portal is a conversation worth having early — some prefer to pull everything into their own system regardless.

QuickBooks is on our SOC 2 scope for financial data controls. Can Starch pull QuickBooks data for evidence?

Starch syncs your QuickBooks data on a schedule — invoices, bills, payments, vendors, and journal entries are all available. One heads-up: QuickBooks report views (P&L summaries, Transaction List, Vendor Expenses) are temporarily disabled pending a fix, but entity-level data syncs normally. For most financial control evidence, the entity data is what auditors actually want.