AI tools · Avinash Gosavi

Doing the Right Thing, Fast

2026-04-25T00:00:00+00:00

We recently deprecated the initial version of Cisco Catalyst Cloud Monitoring. That means the code that drove it — onboarding flows, device-specific paths, integrations that only existed for those devices — needed to go.

In the old world, this kind of cleanup almost never happens on schedule. It’s not that people don’t want to do it. It’s that the cost is high: you’re touching interconnected parts, there’s real risk of breaking something unrelated, and the work has to fit around ongoing feature development. So it gets triaged below the line. Unused code paths don’t disappear — they just quietly accumulate. Every engineer knows the feeling of working around code that should have been deleted six months ago.

This time was different.

What We Actually Did

I split the cleanup into 20-25 focused work streams. A few people helped with specific areas, but most of it was mine. Over two weeks, every stream went through the same cycle: identify what to remove, make the change, review it carefully, test, and move on.

Diagram: The cleanup split into 20 to 25 work streams, each run through the same cycle (see it in the post)
The cleanup was split into 20 to 25 focused work streams. Every stream went through the same four steps. Identify what to remove, with Codex finding the call sites. Make the change, with Codex generating the deletions and the ripple effects across the codebase. Review it carefully, diff by diff in VS Code. Test it manually. Then move on to the next stream. Every stream ran through this cycle over two weeks, and all of it is now deployed to production.
Same four steps, 20–25 times over.

The Meraki developer tools team’s jira-to-PR skill made the pipeline from ticket to code review almost frictionless. I wasn’t fighting process — I was just doing the work.

That work is now fully deployed to production. No errors. The legacy load is gone.

Where Codex Fit In

Codex handled the mechanical side of each stream — finding the right call sites, generating the deletions, updating the ripple effects across the codebase. I’ve been here since inception, so I know this code well. That matters. A new person would have needed more time to build the mental model before they could trust what was being removed. My 4.5 years gave me the confidence to move fast without second-guessing.

The way I verify is consistent: I do all my visual inspection in VS Code, diff by diff. When I wasn’t sure what Codex had generated, I asked it to explain the change in plain terms. That became a reliable part of my review loop — not a crutch, just a check.

Takeaway: experience and AI compound. The codebase knowledge lets you set the right scope. Codex handles the execution. The combination is what made 2 weeks possible instead of 2 quarters.

The Shift I Keep Coming Back To

What I used to put off, I now just do.

Not because the work got trivial — the cleanup was real work, done the right way, with proper reviews and manual testing. But the overhead that used to make it not worth starting has collapsed. The cost of doing the right thing is now low enough that “we’ll do it properly later” is no longer the default answer.

That’s the shift. You can do the right thing and do it fast. Not by cutting corners, but because the slow parts — finding call sites, writing boilerplate, generating consistent changes across files — aren’t slow anymore.

I’m writing less code than I used to. But I’m seeing more of my intentions land exactly as I wanted them to. That’s a trade I’ll take every time.

I’m genuinely excited to see where this goes next.

Read about how I approach repeatable workflows in Codex: Codex Skills: Baking Your Workflow Into the Tool

Related: Are These AI-Generated Tests Good? — a repeatable way to review the tests an LLM writes.

Are These AI-Generated Tests Good?

2026-02-28T00:00:00+00:00

I use AI to generate a lot of tests now. It saves time, but fast output can create false confidence.

Even if AI-generated code is 99% accurate, we still need tests for the same reason as always: catching regressions before users do.

From experience, I can usually spot missing cases during review. But while writing this down, I wanted more than instinct — I wanted a framework that is easy to explain and repeat. ISTQB definitions align well with how I already think, so I use them as the foundation.

Test Level vs Test Type

ISTQB defines test level as:

“A group of test activities that are organized and managed together.”

ISTQB defines test type as:

“A group of test activities aimed at testing specific characteristics.”

That gives me a clear way to evaluate AI-generated tests: classify by level, classify by type, then check coverage and signal.

Running Example

Feature: password reset

Expected behavior:

User requests reset link
Email is sent with token
Token expires
Token is single-use
Password updates
Old password no longer works

Low Signal vs High Signal

Low-signal test:

it "returns success" do
  post :create, params: { email: user.email }
  expect(response).to have_http_status(:ok)
end

Higher-signal test:

it "creates a single-use token and sends reset email" do
  expect {
    post :create, params: { email: user.email }
  }.to change(PasswordResetToken, :count).by(1)

  token = PasswordResetToken.last

  expect(token.used_at).to be_nil
  expect(ActionMailer::Base.deliveries.last.to).to include(user.email)
end

The second test asserts what changed — token count, token state, email delivery — instead of only that the controller returned 200. That distinction matters when reviewing AI output: AI tends to generate the first kind because it is easier to write and harder to fail.

Actionable Review Algorithm

When I review a PR with AI-generated tests, I run through this sequence:

Collect all new and changed tests in the PR.
Classify each test by ISTQB test level.
Classify each test by ISTQB test type.
Map tests to Jira acceptance criteria.
Cross-check requirement docs for edge cases and failure paths.
Check external impact coverage:
- API contracts
- Downstream consumers
- DB side effects
- Jobs and events
- Third-party boundaries
Weight confidence:
- Highest: system + acceptance + functional
- Medium: integration + change-related regression
- Lower: component-only checks
- Lowest: smoke/startup/load-only checks
Publish a short gap summary in the PR.

Diagram: The eight-step review of AI-generated tests in a pull request (see it in the post)
Step 1: collect all new and changed tests in the PR. Steps 2 and 3: classify each test by ISTQB test level, a group of activities organized and managed together, and by ISTQB test type, a group of activities aimed at specific characteristics. Step 4: map the tests to Jira acceptance criteria. Step 5: cross-check requirement docs for edge cases and failure paths. Step 6: check external impact coverage: API contracts, downstream consumers, DB side effects, jobs and events, and third-party boundaries. Step 7: weight confidence. Highest for system, acceptance and functional tests; medium for integration and change-related regression; lower for component-only checks; lowest for smoke, startup or load-only checks. Step 8: publish a short gap summary in the PR.
The same eight steps as a flow: classify, compare against what was asked for, then weigh what the tests actually prove.

Scoring Example

PR contains:

2 component functional tests
1 component integration test
1 system functional test
1 acceptance regression test
2 smoke checks

Initial result: Decent baseline, but incomplete if no system integration test validates the email provider failure path.

Action: Add one integration contract test and re-score.

AI Test Quality Checklist

Use this when reviewing a PR that includes AI-generated tests:

Tests classified by level (Component, Component Integration, System, System Integration, Acceptance)
Tests classified by type (Functional, Non-functional, Black-box, White-box, Change-related)
Jira acceptance criteria mapped to explicit assertions
Requirement doc edge cases and failure paths covered
External impact covered (API contracts, downstream consumers, jobs/events, side effects)
Regression protection added for changed behavior
High-signal vs low-signal mix summarized
Known gaps documented with a follow-up plan

AI helps me generate tests faster. This framework helps me trust what I ship.

Sources:

Related: how I work with LLMs day to day is in I Fell in Love with Coding Again, and how I review AI-generated changes diff by diff is in Doing the Right Thing, Fast.

Codex Skills: Baking Your Workflow Into the Tool

2026-02-17T00:00:00+00:00

A follow-up to I Fell in Love with Coding Again

In that post I shared a simple workflow: know the system and ask precise questions, break tasks into steps, and use automation for the boring parts. What I didn’t mention is how often I was repeating those same instructions. Once that pattern was clear, I started encoding it—that’s when Skills in Codex became a big part of my day.

Before diving in, it helps to be clear on the difference between the agent and a skill, and when to use which.

Agent vs. Skill: When to Use What

Aspect	Agent (Codex)	Skill
What it is	The general-purpose AI. You give it intent and context; it plans, reads code, edits, runs commands.	A packaged playbook: instructions, templates, optional scripts. One focused workflow.
Use when	Open-ended work: “explain this codebase,” “implement this feature,” “fix this bug,” “refactor X.”	You do the same workflow over and over and want it consistent and one-tap.
Where it lives	Codex itself (CLI, IDE, app).	A folder with `SKILL.md` (and optionally scripts/references). You or your team create or install skills.

Rule of thumb: If you find yourself pasting the same block of instructions or saying “like last time, but for X,” that’s a candidate for a skill. The agent does the work; the skill tells it how to do that kind of work every time.

What I Actually Use Skills For

Most of my skills are local, day-to-day—things that used to be manual or a wall of prompts. Two big buckets:

1. Jira: From a few details to a real ticket

I give Codex a short description and maybe a doc link. The skill:

Uses a ticket description template I keep in the skill (in references/ or in the instructions).
Sets the fields we care about: type, priority, board, assignee, labels, etc.
Creates the ticket and attaches it to the right board, assigned to the right team.

Before: Copy template, fill fields, attach, assign. After: “Create a Jira ticket for <brief description>” and the skill does the rest. Huge reduction in busywork.

2. Git: Housekeeping as a single flow

Our housekeeping is a chain of small steps. I turned them into small skills and then one combined skill that runs the full flow:

Step	What the skill does
New branch	Create branch from the right base (e.g. `main`) with our naming convention.
Update from origin	Stash current changes → pull from `origin/main` → re-apply stash → handle conflicts if needed.
Commit	Generate commit message from my template (what changed, why, scope).
Push	Push branch and set upstream if needed.

I can still run the small skills individually (e.g. “just create a commit message”), but the combined skill is what I use most: one prompt, full flow. That’s the pattern—small skills you can mix or run as one.

Diagram: Small git skills chained into one combined housekeeping skill (see it in the post)
One prompt runs the combined skill, which chains four small skills in order. New branch: create a branch from the right base, such as main, with the team's naming convention. Update from origin: stash current changes, pull from origin/main, re-apply the stash and handle conflicts if needed. Commit: generate the commit message from a template covering what changed, why and the scope. Push: push the branch and set upstream if needed. Each step is also a small skill that can be run on its own.
Small skills you can mix, or run as one.

3. Next: Codebase-specific skills

I’m now turning recurring codebase prompts into skills: “how we add a new API route,” “how we update the schema and migrations,” “how we add a component to the design system.” Same idea—capture the steps and templates once, then invoke with a short prompt. That’s the direction I want to expand: more repo-specific skills so my usual prompts become reliable, repeatable workflows.

How Skills Work (Without Slowing You Down)

Skills use progressive disclosure:

Startup: Codex loads only skill names and descriptions.
On demand: When you use a skill, it loads the full SKILL.md.

So you can have many skills; they don’t bloat context until invoked. Keeps the CLI fast and the session clean.

Where Skills Live

Scope	Location	Good for
Repo	e.g. `.codex/skills/` or `.agents/skills/`	Team conventions: Jira template, code review, “how we add a feature.”
User	Your config (e.g. `~/.codex/skills/` or `~/.agents/skills/`)	Personal workflows: Git flow, commit template, prompt-optimization.
System	Bundled with Codex	Built-ins (e.g. skill-creator).

I keep Jira and Git in my user skills (same flow across projects). Codebase-specific ones live in the repo so the team shares them.

How to Arrange SKILL.md: A Proper Format

Codex only requires a SKILL.md with frontmatter (name, description) and a body. How you arrange the body is up to you—but a consistent structure makes skills easier to write, read, and refine. Here’s a format I use and that you can adapt.

Required: Frontmatter

---
name: skill-name
description: One clear sentence: what this skill does and when Codex should use it.
---

name: Lowercase, hyphens OK (e.g. jira-ticket, git-housekeeping). Codex uses this for $skill-name.
description: This is what Codex uses to match your prompt. Be specific so the right skill triggers (or so you can pick it from the list).

Body: Section layout that works

After the frontmatter, structure the rest of the file so the agent knows what to do and when. A good default arrangement:

Section	Purpose
Goal	One short paragraph: what this skill achieves.
Inputs	What you (or the user) provide: e.g. brief description, doc link, branch name.
Outputs	What must be true when done: e.g. ticket on board and assigned, branch pushed.
Workflow (or Steps)	Numbered, imperative steps the agent follows.
Template (optional)	If the skill uses a fixed format (Jira body, commit message), put the template here or in `references/` and reference it in the steps.
Example triggers (optional)	Example phrases that should invoke this skill (helps implicit matching).
Constraints (optional)	What not to do, or rules (e.g. “Never assume assignee; always ask if missing”).

You don’t need every section for every skill—use what fits. For a tiny skill, Goal + Steps might be enough. For Jira, you’d add Inputs, Outputs, Template, and maybe Constraints.

Full SKILL.md template

---
name: my-skill-name
description: [What it does and when to use it. Codex uses this to select the skill.]
---

# My Skill Name

## Goal
[One short paragraph: what this skill achieves.]

## Inputs
- [What the user or context must provide, e.g. brief description, link, branch name.]

## Outputs
- [What must be true when the skill is done.]

## Workflow
1. [First imperative step.]
2. [Second step.]
3. [Continue until done.]

## Template
[Paste the template here, or: "Use the template in references/ticket-template.md and fill …"]

## Example triggers
- "Create a Jira for …"
- "File a ticket for …"

## Constraints
- [Any rules or things to avoid.]

Defining your own format

You can change the section names and order to match how you think. For example:

Repo-specific: Add a Context section (“This repo uses …”) or Key fields (“Always set: type, priority, board, assignee”).
Multi-step skills: Add Prerequisites (“Run only when on a feature branch”) or Sub-skills (“This skill calls the commit-message skill then the push skill”).
Strict templates: Put long templates in references/ or assets/ and keep SKILL.md to Goal, Steps, and “Use template at …”.

The important part is: arrange SKILL.md so the agent has a clear path—goal, what it gets, what it must produce, and the steps in between. Once you pick a layout you like, reuse it so every skill feels familiar and easy to iterate on.

Quick Examples (Less Wall of Text)

Jira skill (conceptual)

Input: short description + optional doc link.
Steps: fill ticket template, set type/priority/board/assignee/labels, create and attach.
Template and key fields live in the skill (references or SKILL.md).

Git – small skills

“Create branch from main with naming convention.”
“Update my branch from origin/main (stash, pull, re-apply, resolve conflicts).”
“Generate commit message from template.”
“Push and set upstream.”

Git – combined skill

Runs the four steps in order; I trigger once and get the full housekeeping flow.

Codebase-specific (target state)

“Add a new API route” → skill knows our routing pattern, where to put files, how we register.
“Add a design-system component” → skill knows structure, exports, and story pattern.

Using Skills in Codex

Explicit: Type $ in the CLI (or use /skills where available), pick the skill, then give your prompt.
Implicit: If the skill’s description and example triggers match your prompt, Codex can invoke it automatically.
Install more: Use $skill-installer (or equivalent) to add skills from catalogs or GitHub so you’re not starting from zero.

Take the Long View (And Your Coffee)

Useful skills don’t have to be perfect on day one. I’ve had the best results by:

Starting with one painful, repetitive task (for me: Jira and Git).
Turning it into a first version of a skill.
Using it in real work, noting what’s wrong or missing.
Refining the steps and templates over a few sessions—often while having a coffee.

So: agent for open-ended work, skill for repeatable workflows. Start with day-to-day pain (tickets, git, one codebase task), use a simple framework to create and refine skills, and let them improve over multiple iterations. That’s how skills have become one of the most useful parts of my setup—and how you can make them yours too.

Further reading: Codex Agent Skills, Codex CLI features, Slash commands, OpenAI skills on GitHub, and community resources like the Codex Skills Catalog and the Habr deep dive on Codex Skills.

Next: the jira-to-PR skill shows up again in Doing the Right Thing, Fast, a two-week cleanup split into 20–25 work streams.