Compliance & Legal

How to collect soc 2 audit evidence as Small IT and ITOps Teams

Compliance & LegalFor Small IT and ITOps Teams2 apps12 steps~24 min to set up

When your auditors ask for SOC 2 evidence, you spend two weeks pulling screenshots manually — Access reviews from Okta, change logs from GitHub, incident records from PagerDuty, ticket history from Jira, and AWS CloudTrail exports that finance had to help you get. None of it lives in one place. You're a two-person team supporting 300 employees, and the audit prep work lands entirely on one of you while the other keeps the lights on. Last year it took 11 days. You lost a contractor to offboarding gaps that showed up in the evidence review. The tools exist — Okta, GitHub, Jira, AWS, PagerDuty — but pulling coherent evidence from all of them at once, on a schedule, formatted for your auditor, is entirely manual today.

Build this on Starch → See the apps used

Compliance & LegalFor Small IT and ITOps Teams2 apps12 steps~24 min to set up

Outcome

What you'll set up

A continuously updated SOC 2 evidence dashboard that pulls access logs, change records, and incident history from Okta, GitHub, Jira, and AWS into a single auditor-ready view

A weekly automated evidence collection run that gathers access review data, open vulnerability tickets, and AWS configuration snapshots — formatted and filed before anyone asks

A knowledge base that documents your control descriptions, evidence ownership, and runbook links so any team member (or your auditor) can find what they need without asking you

The Starch recipe

Apps, data, and prompts

The combination of Starch apps, the data sources they pull from, and the prompts you use to drive them.

Apps used

Knowledge Management Task Manager

Data sources & config

Jira, GitHub, PagerDuty, and Okta are connected from Starch's integration catalog and queried live when the evidence dashboard or automation runs. Starch syncs your AWS data directly on a schedule via the AWS connection, pulling Cost Explorer anomalies and configuration data. Notion connects from the integration catalog for any existing runbooks you want to pull in. The Knowledge Management app stores control descriptions and evidence summaries natively in Starch.

Prompts to copy

Build me a SOC 2 evidence tracker that pulls open and closed access review tickets from Jira, lists all active users and recently deprovisioned accounts from Okta, shows the last 90 days of merged pull requests and deployments from GitHub, and surfaces any critical or high incidents from PagerDuty — organized by SOC 2 trust service criteria

Every Monday morning, collect this week's access changes from Okta, merge activity from GitHub, incident closures from PagerDuty, and AWS cost anomaly alerts, then generate a structured evidence summary and save it to our Knowledge Management wiki under the current audit period

Create a task list of all open SOC 2 control gaps — any Jira ticket tagged 'soc2' that's been open more than 30 days, any AWS finding unacknowledged for more than 7 days — with due dates and assigned owner, and alert me if anything goes overdue

Run these in Starch → or paste them into your favorite agent

Walkthrough

Step-by-step

1 Connect Jira from Starch's integration catalog — the agent queries your ticket history live, filtering for SOC 2-tagged issues, change management tickets, and access review records.

2 Connect GitHub from the integration catalog so Starch can query merged PRs, deployment activity, and repository access changes across the audit window — this covers your change management control evidence.

3 Connect PagerDuty from the integration catalog; the agent pulls incident records, resolution timelines, and on-call history — what auditors call your availability and incident response evidence.

4 Connect Okta from the integration catalog so Starch can query user provisioning events, group membership changes, MFA enrollment status, and recent deprovisioning actions — your access control evidence.

5 Wire the AWS scheduled-sync connection so Starch pulls Cost Explorer data and configuration snapshots on a recurring schedule — covering your infrastructure security and monitoring controls without a manual export.

6 Tell Starch to build your evidence dashboard: describe the trust service criteria you're being audited against (CC6, CC7, CC9, etc.) and which control maps to which data source — Starch builds the view.

7 Set up a weekly automation: describe what you want collected, formatted, and filed each Monday — Starch runs it on schedule and saves the output to your Knowledge Management wiki with the audit period as the folder.

8 Use the Knowledge Management app to document your control descriptions, evidence owners, and the exact query logic Starch is using — so your auditor can see the methodology, not just the outputs.

9 Build a task list in Starch for open control gaps: any ticket or finding that's been unresolved past your SLA threshold gets surfaced automatically, with owner and due date, so nothing ages out quietly.

10 One week before your auditor fieldwork starts, run a full evidence pull across all connected sources and export the structured summary — Starch formats it so you're handing your auditor a packet, not a folder of screenshots.

11 For any evidence source that doesn't have a direct integration — a vendor security questionnaire portal, a niche compliance tool — Starch automates it through your browser with no API needed.

12 After the audit, update the Knowledge Management wiki with what auditors actually asked for, so next year's prep starts from a known baseline instead of from memory.

See this running on Starch

Connect your tools, describe what you want, and the agent builds it. Closed beta is free.

Try it on Starch →

Worked example

Q1 2026 SOC 2 Type II Fieldwork — March Evidence Pull

Sample numbers from a real run

Okta access review records (90 days)	847
GitHub merged PRs with change tickets linked	312
PagerDuty incidents opened and closed in period	44
AWS configuration findings reviewed	19
Jira SOC2-tagged tickets resolved in period	63
Deprovisioning events with Jira offboarding ticket	28

It's March 9, 2026. Your auditor's fieldwork starts March 16. Last year this took one of you 11 days of manual pulls. This year, Starch has been running a Monday evidence collection automation since January. By March 9, you already have 12 weekly snapshots saved in Knowledge Management, each structured by trust service criteria. You run a final pull: Starch queries Okta and returns 847 access review events for the 90-day window, flags 3 accounts that were deprovisioned but whose Jira offboarding ticket was closed late. GitHub returns 312 merged PRs — Starch cross-references each against your change management Jira project and surfaces 7 that have no linked ticket, which is a finding you can remediate before the auditor sees it. PagerDuty returns 44 incidents; 2 critical ones had resolution times over your 4-hour SLA. You document the exceptions in Knowledge Management before handing the packet over. AWS shows 19 configuration findings from the period; 17 are acknowledged and closed. Total prep time for the final packet: 3 hours, not 11 days. The auditor receives a structured folder organized by control, not a ZIP of screenshots.

Measurement

How you'll know it's working

Hours spent on audit evidence prep per audit cycle (target: under 8 hours active work)

Percentage of SOC 2 controls with evidence collected automatically vs. manually

Number of control gaps (open findings past SLA) at the time auditor fieldwork begins

Deprovisioning events with a corresponding closed offboarding ticket — your access termination control metric

Time from auditor evidence request to evidence delivery (target: same business day)

Comparison

What this replaces

The other ways teams handle this today, and how the Starch version compares.

Drata or Vanta

Purpose-built compliance automation with strong auditor integrations, but adds another per-seat subscription and doesn't replace the custom cross-system evidence surfaces your team needs for the controls Drata doesn't cover — Starch builds those in between.

Manual export + Google Sheets

Zero additional cost and your auditor already knows how to read it, but every audit cycle starts from scratch, exports go stale the moment you save them, and one person owns all of it with no automation.

Notion runbooks + Jira tickets only

Works for documenting what you plan to do but gives you no live evidence — your auditor wants records of what happened, not policies about what should happen.

Okta's built-in access review reports

Solid for access control evidence specifically, but it's one system — you still manually pull GitHub, PagerDuty, AWS, and Jira separately and reconcile them in a spreadsheet.

On Starch RECOMMENDED

One platform — knowledge management, task manager all running on connected data. Setup in plain English; numbers stay current via scheduled syncs and live agent queries.

Try it on Starch →

FAQ

Frequently asked questions

Does Starch store my Okta and GitHub data, or does it query it fresh each time?

For Okta, GitHub, Jira, and PagerDuty connected from the integration catalog, the agent queries them live when your evidence dashboard or automation runs — the data isn't persistently stored in Starch's database. For AWS, Starch syncs on a schedule and stores a snapshot. If you need a persistent historical archive of every access event going back three years, that's a data warehouse use case Starch isn't designed for. For audit evidence windows — typically 90 days to 12 months — the live-query approach covers most controls.

Is Starch SOC 2 certified itself? Our auditor will ask.

Starch is not SOC 2 Type II certified today. That's worth knowing before you wire sensitive access log data through it. If your auditor or security policy requires vendors to be SOC 2 certified, flag that before you set up connections to Okta or GitHub through Starch.

Can Starch pull from Jamf or Kandji for device compliance evidence?

Jamf and Kandji aren't in Starch's scheduled-sync providers, but if either has a web portal you log into, Starch can automate it through your browser — no API needed. For the integration catalog, check whether your specific Jamf or Kandji plan exposes an API that Starch can reach from its 3,000+ app catalog. Either way, device compliance evidence is buildable — it just may route through browser automation rather than a direct API query.

We use Linear instead of Jira. Does that work?

Linear is reachable from Starch's integration catalog, so the agent can query it live. You'd describe the same evidence workflow — 'pull all SOC 2-tagged Linear issues closed in the last 90 days' — and Starch builds against Linear instead of Jira. The recipe doesn't change, just the connection.

Our auditor wants evidence formatted a specific way — tables, specific column names, date formats. Can Starch do that?

Yes. When you describe the evidence dashboard or the automation output, include the format your auditor expects — Starch structures the output accordingly. If your auditor's format changes between audit cycles, you update the description and Starch rebuilds the surface. You don't modify a schema or rewrite an export script.

What happens if an evidence pull fails mid-run — say, GitHub is rate-limiting or PagerDuty is down?

Starch will surface the failure rather than silently returning partial data. You'll know which source didn't return results and can re-run that piece manually or wait and retry. For weekly evidence collection automations, this means one incomplete run doesn't silently corrupt your audit trail — it flags for review.