Agent Skill Security: Secrets, Permissions & PII

What's at stake when you share a skill file

Agent skills are spreading fast because they work — drop a plain text instruction file and the AI agent gets instantly better at a specific task. But the same simplicity that makes skills easy to create also makes them easy to leak. A single SKILL.md or README can contain:

• API tokens and private keys someone pasted while testing
• Internal hostnames and staging IPs that map your network
• Customer email addresses or PII buried in an example log
• Local file paths (/Users/laojin/secret-contract.pdf) that expose your machine layout

When that file gets pushed to a public GitHub repo or shared in a community chat, the damage is done before anyone notices. Unlike a full code repo, a skill file rarely goes through the same review or CI pipeline. That gap is where most incidents start.

Our team scans every skill file with 11 pattern rules before publication. Since we adopted deny‑by‑default, zero secrets have escaped into our open‑source releases.

The four‑step security baseline

You don't need a dedicated security team to avoid most disasters. These four practices work for teams of any size — and they're the same ones we enforce at 365Skill.

1. Sensitive info scan, always before publish

Create a lightweight pre‑publish checklist that catches tokens, keys, email addresses, internal IPs, and local paths. Automate it when possible; run it manually before every commit if you're still building the automation. The scan must block a publish if it hits.

2. Least privilege for tools and access

A skill that only needs to read a calendar should not have write permission to the CRM. Define the exact set of tools and data scopes a skill requires, and nothing more. If a skill grows later, expand the scope explicitly — don't just hand it a master key from day one.

3. Deny‑by‑default publishing gate

Start with every skill private. Only make it public after a human verifies it's clean, and only if public distribution brings real value. The default shouldn't be “share everything”; it should be “no public release unless approved.”

4. Environment variable injection

Secrets never go into the skill file. They live in environment variables that the runtime injects when the skill executes. The file references a placeholder like $STRIPE_KEY, not the actual value. This single habit eliminates the most common class of leaks.

These 11 rules came from real incidents in our own cross‑border operations: a token pasted into a README during testing, a co‑worker's local path left in a log example, an internal staging IP shared in a community demo. None of them looked dangerous at the time — they rarely do.

Where teams slip — and the scars to prove it

The mechanics aren't hard; the human habits are. Three failure patterns show up repeatedly:

• Screenshots and logs — you take a screenshot to show the output, but a terminal history or environment dump in the background includes a secret. Public blog post; leaked key.
• README as architecture doc — someone writes a great README explaining how the skill connects to the internal ranking service, with the actual hostname and port. Instant network map for an attacker.
• “Get it working, then secure it” — the famous last words. The working version gets shared in a team group for feedback. By the time someone circulates it publicly, the original, un‑hardened file is the one that travels. Nobody remembers to go back and clean it.

We learned the hard way with early internal tools. An AI sales draft utility once shipped with a test mailbox password in a comment. It only reached three devices, but that was enough to spark the deny‑by‑default rule we now bake into every 365 product.

How 365Skill ships skills securely (Apache‑2.0)

365Skill runs on two principles that enforce security by architecture, not by hope:

Deny‑by‑default publishing. Every skill starts in a private repository. Before a skill becomes public, it must pass a scan against all 11 sensitive‑info rules. No exceptions, no “I'll fix it later.” The public repository only contains skills that have cleared that gate. There's a separate, private repo for internal‑only capabilities, and the two never mix.

Separation of public and private. The open‑source repo at 365Skill is the verified, clean surface. Internally we use additional skills that reference production systems, and those stay private permanently. This split means contributors and users of the public library never accidentally touch our live infrastructure, and we never accidentally leak internal trade secrets through a skill definition.

For anyone adopting agent skills, the minimum step is to set up a similar gate — a checklist, a pre‑commit hook, a CI check — that catches the 11 patterns above. It costs an afternoon and saves months of cleanup.

FAQ

Can I just put all secrets in environment variables and stop worrying?

Environment variables handle the storage part, but not the accidental leak. You still need to scan for placeholders that got replaced with real values during debugging, and for logs or examples that contain secret names. Variables plus scanning is the real minimum.

What if my skill genuinely needs to reference a local file path?

Replace the literal path with a placeholder like $DATA_DIR and document the expected folder structure separately. Keep the actual path out of any file that might be shared. If the path appears in an example, mask it before publishing.

How does deny‑by‑default work in a community‑driven project?

Contributors submit skills to a private staging area first. Only after a maintainer runs the scan and confirms the skill is both useful and clean does it move to the public repo. This adds a few hours of delay but eliminates reversals.

Do the 11 scanning rules cover everything?

They cover the most common and high‑impact leaks based on our own incidents and those we've seen in open‑source projects. They're not a replacement for application‑level security, but they catch over 90% of the casual mistakes that lead to exposure.

---

If you're building or using agent skills, start with the 11‑rule scan and a deny‑by‑default gate — it's the simplest insurance against a leak that can cost you accounts, customers, or trust. Explore the open‑source skills library at 365Skill to see how we're keeping it clean, and check our free AI tools for production‑ready capabilities you can plug into your operation today.