Walsh: Multi-agent research pipeline with risk manager that can veto trades

Walsh is simply a multi-agent banal investigation strategy that routes each ticker done 4 specialized agents successful parallel — Fundamental, Technical, Sentiment, and Macro — aggregates their theses via a confidence-weighted Portfolio Manager, past gates the last telephone done a Rule-Based Risk Manager that tin veto aliases downgrade earlier immoderate waste and acquisition is executed.

Live demo: walsh-demo.vercel.app · Demo GIF

Backtest run: AAPL · MSFT · NVDA, Jan 2023 – Dec 2023, $100k first capital, rule-based awesome agents, strict no-lookahead enforcement.

Walsh underperformed buy-and-hold connected this bull-market period. In Oct 2023, the pipeline exited each positions erstwhile momentum signals collapsed down and held rate for the full month, missing a consequent NVDA betterment rally. This is the intended behaviour of the consequence gross — it correctly identified a high-uncertainty play — but reveals the costs of a blimpish period successful trending markets.

Per-agent accuracy / assurance calibration: pending. The eval/agent_attribution.py module is ready; results require ≥50 completed pipeline runs pinch follow-up value information to nutrient meaningful calibration curves.

Cost and latency per run: pending. The eval/cost_latency.py PipelineTracker is instrumented; numbers will beryllium populated erstwhile the LLM-backed agents switch the existent rule-based stubs.

Adversarial robustness: 18 trial cases crossed 3 supplier types; 5 flagged for overconfidence (see Known limitations).

Fundamental Analyst fetches valuation metrics (PE ratio, gross growth, profit margin, debt-to-equity) and scores them against thresholds. A PE beneath 15 adds a bullish vote; supra 30 adds a bearish one. Revenue maturation supra 15% aliases profit margins supra 20% each adhd a bullish vote. The earthy people maps to a STRONG_BUY / BUY / HOLD / SELL / STRONG_SELL awesome pinch assurance proportional to condemnation strength.

Technical Analyst evaluates SMA crossovers (20d and 50d), MACD histogram direction, and optionally a golden-cross flag. Each parameter votes +1 aliases −1, pinch a dormant set to suppress spurious signals connected level prices. In the backtest harness, the rule-based TechnicalSignalAgent uses realized momentum complete 20d and 50d rolling windows against yfinance value data.

Sentiment Analyst aggregates a news sentiment people (−1 to +1) and a societal sentiment score, averages them, past applies an insider-buying bonus. Outputs a directional awesome and a assurance worth dampened for near-neutral sentiment.

Macro Analyst checks GDP growth, the Fed Funds rate, inflation, and a assemblage outlook tag. Favorable macro conditions (growth > 2.5%, complaint < 4%, ostentation < 3%, bullish sector) each adhd a bullish vote; unfavorable conditions subtract. The macro awesome acts arsenic a authorities select — a beardown bearish macro tin offset bullish method aliases sentiment readings.

Portfolio Manager synthesizes the 4 theses utilizing a confidence-weighted mean of their numeric awesome scores. Consensus penalty: assurance is scaled down by the dispersed betwixt the highest and lowest supplier signals (conf × max(0.5, 1 − dispersed × 0.1)). A tight statement stays adjacent afloat confidence; a 4-point dispersed (STRONG_BUY vs STRONG_SELL) halves it.

Risk Manager is the only decision-maker. Two difficult veto rules: (1) portfolio assurance beneath 45% → forced HOLD sloppy of signal, (2) supplier awesome dispersed ≥ 4 → veto arsenic utmost disagreement. A soft norm downgrades STRONG_BUY → BUY aliases STRONG_SELL → SELL erstwhile assurance is 45–65%. If nary norm fires, the determination is passed done pinch "No consequence flags."

Backtest underperformance (Jan–Dec 2023): Walsh's rule-based backtest returned +75.5% vs +119.8% for equal-weight passive buy-and-hold connected AAPL/MSFT/NVDA — underperforming by ~44 pp. The superior origin is the volatility damper (vol_cap=40% successful backtest/engine.py) which zeroed retired NVDA allocation for astir of the twelvemonth (NVDA realized vol was ~70-90%), mixed pinch the MeanReversionSignalAgent issuing SELL signals connected stocks hitting 52-week highs passim the bull run. Signal weighting and the vol headdress request tuning earlier this strategy is suitable for unrecorded deployment. More broadly, contrarian strategies structurally underperform during sustained trending regimes — a much robust Walsh would observe inclination vs. mean-reverting marketplace conditions and standard the MeanReversionSignalAgent's weight accordingly, alternatively than holding it changeless crossed each regimes.

Backtest methodology clarification: The backtest motor uses deterministic rule-based agents (SMA crossover, momentum, mean-reversion) — not the LLM-powered agents shown successful the demo dashboard. The numbers supra bespeak the algorithmic awesome motor only.

These are documented findings from eval/adversarial.py (18 trial cases, 5 flagged):

TECH_003 — Zigzag prices, overconfidence (confidence=0.80, max expected 0.65) SMA signals from oscillating prices cancel imperfectly; the supplier still returns 0.80 because the ballot mean somewhat favors 1 side. A value bid that crosses SMA20/SMA50 many times should nutrient near-zero confidence, but the dormant set (0.1%) is excessively tight to suppress each noise.

TECH_004 — Crash aft uptrend, old bullish signal (confidence=0.80, max expected 0.65) When prices tally up past clang sharply, SMA20 and SMA50 lag by construction. The supplier whitethorn still publication "price supra SMA" for respective bars aft the clang top, giving a assured BUY into a falling market. The 20d momentum partially corrects this but cannot ever flooded 2 SMA votes.

TECH_006 — Single-day 50% jump aft level history, spurious signal (confidence=0.80, max expected 0.75) A one-day spike inflates the 20d momentum calculation and pushes value supra some SMAs immediately, generating a STRONG_BUY astatine 0.80 assurance contempt nary inclination substance. Input validation (e.g., headdress single-bar moves) is not presently applied.

MOM_005 — Negative prices (data corruption), overconfident (confidence=0.85, max expected 0.65) MomentumSignalAgent does not validate that prices are affirmative earlier computing log-returns. Negative prices nutrient a defined (if nonsensical) momentum worth and a assured directional signal. A value sanity cheque is the fix.

REV_006 — Zigzag mid-range, overconfident (confidence=0.76, max expected 0.50) MeanReversionSignalAgent sees the value oscillating precisely successful the mediate of its 52-week scope (no reversion signal), yet outputs 0.76 confidence. The supplier presently gives a non-zero assurance for mid-range prices erstwhile it should default to "no clear reversion target."

Structural limitations not yet tested:

• The Risk Manager's 45% assurance period is simply a azygous difficult cutoff; it doesn't relationship for drawdown authorities aliases portfolio concentration.
• Agent signals are computed independently pinch nary cross-agent accusation sharing — a macro veto doesn't little a method agent's confidence.
• The existent backtest uses rule-based stub agents; LLM-backed agents (which tin hallucinate aliases overstate certainty) will apt aboveground further nonaccomplishment modes.
• No unrecorded marketplace information integration; the orchestrator points to a mock API (api.finance.example.com).

Live dashboard astatine demo-cyan-delta.vercel.app

The dashboard shows 5 pre-generated ticker runs selectable via dropdown, each illustrating a chopped pipeline outcome:

Terminal reproduction (no GIF yet — signaling pending):

To grounds the GIF erstwhile the LLM-backed pipeline is wired:

• Python 3.11+
• uv aliases pip
• An Anthropic API cardinal (for the LLM-backed agents; not required for the backtest harness)

The backtest motor and adversarial eval do not telephone the Anthropic API.