Self-Improving Skills for Coding Agents

Agents forget between sessions and you need to provide context for each task. I found out that the approach I call "self-improving skills" works great to address this issue.

The Problem

Coding agents forget everything between sessions. CLAUDE.md helps, but if you put every feature detail in there, it gets loaded into the context for every single task, even the ones that have nothing to do with it.

Spec-driven development solves this with big spec documents. That might make sense for a big team, but for one person on one project or even smaller teams that care about efficiency, it's a lot of writing and in the end you might not get the results you want. To learn more about SDD see SDD by Birgitta Böckeler.

The Idea

Three ideas came together to form the self-improving skills approach:

While looking at existing skills on skills.sh, I noticed that they are all general purpose skills. They are not specific to a feature or a project but rather related to an umbrella topic.
In the engineering teams I have worked with, whenever people needed something specific to a feature, there was usually one engineer who knew it best. It could be information or a new related feature. That engineer would take it over, even when documentation existed. Why? Because they had worked on the feature and kept a small memory of it. That insight let them make faster and better decisions. In other words, they had a specific skill in that specific domain or feature.
Besides that one of our biggest advantages as software engineers is that when developing a feature we can use our knowledge of the bigger picture, consider the past and future requirements and use that to guide the agent.

So I thought, why not give the coding agent a small memory of the feature it's working on? A specific skill that improves as agent keeps working on that feature.

One Self-Improving Skill per Epic

I break the project into epics. Each epic is big enough to be its own set of tasks and small enough to stay focused. In my experience, a set of 8 to 10 tasks is the sweet spot. I have used it with an even bigger set of 20 tasks and it worked but I had to ask the agent to compact the skill and I felt it lowered the skill's quality.

When I start an epic, I use the existing skill-creator skill to create a skill for it. While creating it, I explain to the agent that project-wide learnings live in CLAUDE.md, and this skill should only keep learnings related to this feature. And I provide the link to the issue tracker for the epic so the agent can read the epic and understand the context.

The new skill has two commands:

start TASK_NUMBER: (used in "plan" mode) loads the task context and past learnings, reads the task, and makes a plan.
finish TASK_NUMBER: reviews what happened based on the conversation, writes new learnings back into the skill (gotchas, decisions, mistakes the agent made), and moves anything project-wide to CLAUDE.md or any other project-wide memory file that is used in your project.

Why It Works

Not verbose: The learnings are a few bullet points, not a spec document. I still can take an iterative approach to the development.

Source code stays as the main source of truth: The skill is not a spec document. It is a small memory of what happened in the past and what is coming next. The source code is still the main source of truth.

Keeps the context small: Skills load on demand so nothing gets into the context unless it's needed.

I don't have to repeat myself: To start each task I do not need to provide the same context again.

Agent knows about the future: Not only the agent has a small memory of what happened in the past, but also it knows what is coming next, so it can make better decisions.

Fewer mistakes: Agent will not repeat the same mistakes. This also acts like a harness for the agent to know about something that will be implemented in the future and not implement it right away.

It improves itself: Every finish makes the next start better. It results in automated iterative learning by the agent.

Gotchas

The skill can get too big if the epic is too big. You can ask the coding agent to compact the skill, but I found that it's better to keep the epics small enough to avoid this problem.
You can add a third command to the skill and call it improve to use it before clearing your context, but that simply means that the task is too big and you should break it down into smaller tasks. Rule of thumb: one task small enough to be done in one conversation without using /compact command.
You can commit these skills to the repo and use them in the future, or you can just get rid of them. I haven't had the use case to help me decide which one is better yet. Committing them to the repo at least gives you the ability to keep a copy for future.
You should review these learnings, because they are very short and this gives you an opportunity to stop a wrong learning from creeping into the skill.

Conclusion

A small feature-scoped memory that grows with the work beats a big upfront spec and it costs almost nothing to maintain, because the agent does the maintaining. Try it on your next project and let me know how it goes.