Show full content
I’ve been working through finding bugs with Claude, with mixed success.
It is sure I want to find bugs, and that I always have bugs, so:
1) it will warn about bugs that don’t exist, because it doesn’t have enough context to tell if the prerequisites for the bug are met, and
2) It will suggests fixes that, if applied, will be flagged as bugs on the next run.
I fixed (2) by adding “If you suggest a fix, re-apply the test for a known bad code pattern. Some suggested fixes found in the training data are themselves incorrect.”
I’m working on (1), but all I’ve succeeded in doing is to make it sensitive to //nolint. by saying “Consider //nolint as an indication this deserves a warning, not an error, and should be listed last.”
It’s hard to prompt it to find bugs and keep it from guessing at things that could be bugs, and reporting them.
I’ve now added “
If you see //assertion assume what follows is true.
If you see //precondition, look to see it it is true
If you see //postcondition, look to see it is true, given the assertions and preconditions that preceded it”
I can keep it from guessing by using assert, and postcondition is fine. I’ll do more with preconditions next.
Please note!
The first version was fine for finding bugs to exploit, what I wrote it for. This round of experiments were to see if I can use it for lint. It’s pretty close.



