Production agents
LinkedIn closes the support-agent loop with offline evaluation and A/B testing
LinkedIn combined RAG, evolutionary auto-prompting, and versioned evaluation
without retraining the foundation model. Prompt and retrieval changes pass
operational guardrails, are tested offline, and then enter a randomized
production A/B test.
Over two weeks, the company reports gains of 9.0 percentage points in QA
self-service, 4.8 points in cancellation self-service, and 30.6 points in
routing accuracy. These are absolute changes in specific support flows, not
LinkedIn-wide deflection. The abstract discloses neither sample sizes nor
confidence intervals.
Here, "self-evolving" does not mean permission for an agent to rewrite itself
without control. Search proposes a candidate, a version is stored, a gate rejects
the bad one, and a user experiment tests the business outcome. The candidate
evolves; production changes only after external evidence.
Source: LinkedIn's self-evolving support agent.
A personal skill is extracted from developer history—along with the risk of going stale
The study builds a compact skill from a particular developer's past
interactions: conventions, commands, preferences, and recurring corrections.
It then tests that skill on new tasks and separately looks for regressions where
an old preference conflicts with current work.
The current arXiv page was updated later, so the issue uses only the question and
method from v1, without numbers added afterward. Personal history carries privacy
and leakage risk, while a recurring convention in one project need not transfer
to another.
The important boundary is personalization. A skill is useful not because it
"knows the user," but because it makes concrete work rules explicit and
testable. It needs a scope, date, and revocation condition; otherwise memory
turns yesterday's habit into today's requirement.
Source: personalized skills for coding agents.