3. GitHub / Open-Source Signals
Reverify: Instead of "LLM Verifies LLM" — Deterministic Tools Verify Every Agent Claim
One of the most interesting new small projects in today's GitHub signal is 2akouwu/reverify.
The idea is almost maximally simple:
LLM forms a hypothesis → deterministic tool verifies it → only then does it become a fact.
The project started with reverse engineering binaries, where hallucination is especially dangerous, but the architecture generalizes to regular code as well. Reverify works as a CLI and MCP server, stores separately verified and refuted facts, and can transfer this state between fresh agent sessions.
What the data showed. The author tested the system on 71 real Windows binaries. The model's answer "from memory" was incorrect in 97% of cases; the verifier did not accept any of the 71 incorrect claims. In a separate classifier-style dataset, the project claims 0 false VERIFIED among 475 deliberately false claims and no missed known-true claims. The benchmark, results, and CI are reproducible from the repository. This is the project author's benchmark, not an independent academic evaluation.
As of September 5, a third-party daily snapshot recorded about 866 stars for the repository; more significantly, the project already has 71 commits, a benchmark suite, CI, and replication instructions.
Why it matters. The architectural pattern is much broader than reverse engineering:
model answers
↓
objective system verifies
↓
verified state is saved
↓
the next agent receives facts, not a summary from the previous agent.
This is directly related to two problems from recent issues: the fallibility of LLM-as-a-Judge and state degradation during compaction/context reset.
What you need to understand now: deterministic verification, MCP, ground truth, context compaction, persistent state.
What it might change. For production coding agents, it's useful to literally walk through the workflow and ask: which agent claims can be turned from text into a verifiable object? Compiler, tests, types, schemas, database constraints, API introspection, and security scanners should take priority over the model's self-confidence.
Signal date: September 5.
Primary source: GitHub — reverify
Skills Continue to Grow Faster Than Full-Fledged Agent Frameworks
Yesterday we already noted mattpocock/skills, so I won't repeat the project description. But the dynamics themselves became a new signal: a third-party snapshot for September 5 counted approximately +2.7k stars in a day, compared to about +1.6k in the previous observation. The official anthropics/skills also gained about +500, and model-agnostic OpenCode about +300.
The exact intraday star values here are based on a third-party snapshot and should be considered approximate.
Why it matters. It seems the open-source market is increasingly optimizing not the agent runtime itself, but engineering behavior on top of the runtime:
how to clarify requirements → how to plan → how to test → how to review → how to avoid writing unnecessary code.
That is, the competitive layer is becoming portable engineering practices, not just prompts and models.
What you need to understand now: Agent Skills, agent harness, TDD, workflow, model-agnostic tooling.
What it might change. If a team is testing coding agents, it makes sense to benchmark not only models, but also one model with different sets of engineering skills/process rules.
Date: September 5.
Sources: GitHub repositories + historical snapshot.