Newsletter archive
Applied Signals Weekly: 2026-09-20
Sep 14, 2026 to Sep 20, 2026
In this week’s edition
This week’s edition surfaces where agentic systems face the sharpest pressure points: reliability under coordination, evaluation discipline, and the practical mechanics that keep multi-agent behavior stable over time.
- An Empirical Study of Harness Design for Coding Agents — Coding harnesses shape how autonomous coding agents translate model capabilities into long-horizon software-engineering performance, yet existing work typically evaluates harnesses as monolithic systems, leaving the…
- Don't Mask the Environment: Observation Supervision Changes How Agents Explore… — Agent trajectories record what an agent does and what happens next. Yet standard supervised fine-tuning (SFT) applies loss only to agent-authored action tokens, using environment observations as context but not as…
- How Fyxer built an AI executive assistant people trust — Fyxer uses OpenAI models, fine-tuning, memory, and real user feedback to organize inboxes and draft emails in each user’s voice.
- AI agent regression testing with Agent Experiments in Arize AX — A cancellation-policy fix raised action safety and dropped average task completion from 0.89 to 0.72. This walkthrough shows how to regression-test agent changes with Agent Experiments in Arize AX.
- Hex turns complex analysis into visual reports with GPT‑6 Astra — GPT-6 Astra helps Hex’s data agents turn answers into interactive visualizations that employees are proud to share.
- How to find and debug agent failures your evals are missing — Evals measure failures you know how to name. Arize Signal continuously reviews production traces to find recurring trajectory failures you do not, then turns them into evidence for tests and fixes.
- RetireOPD: Self-Retiring On-Policy Distillation for Agentic Reinforcement… — Multi-turn agents trained with reinforcement learning (RL) receive a single scalar reward per trajectory, which motivates self on-policy distillation (OPD) to supply dense token-level supervision from a self-teacher with…
Worth watching
- evaluation + tool-use: hype or repeatable pattern?
- planning + tool-use: hype or repeatable pattern?
- evaluation + planning: hype or repeatable pattern?
Read the full analysis
- Trace-driven eval loops are becoming the limiting factor (weekly trend)
- Sep 20, 2026 tau^2-bench Fixed plans and HIL-UMI VLA post-training
- Sep 19, 2026 ClashBench preemption rates, Arize AX regression tests, ActObs RL initialization
- Sep 18, 2026 FMOS virtualization, tool hallucinations, and FARSIGHT trading-agent stress tests
- Sep 17, 2026 ScienceIDE, agentic-eCAL, and peer filtering limits in LLM societies
- Sep 16, 2026 Emergence World memory failures and KV-cache placement policies
- Sep 15, 2026 BudgetBench, PhysMent, Continual Search and Gavel evaluations
- Sep 14, 2026 Look Before You Leap guards and GAUGE close-pair misranking
Subscribe
Get Applied Signals weekly newsletter in your inbox