One of my pet peeves alongside the web betterment community is that so many of the decisions we create are vibes-based. Frameworks are “blazing fast,” tools volition “supercharge” your development, the correct idea volition advantage the two you and your users (DX and UX all in one!). All of this breathless boasting is normally offered alongside zero evidence. Alex Russell has called this “frontend’s misplaced decade”, Baldur Bjarnason has called programming a pop culture, and I’ve tried to oppose this trend in my blog alongside facts and benchmarks (at my best, anyway).
For a lengthy period it seemed akin our community’s allergy to evidence was driven by the desire for greater DX. In another words, devs fair liked the way it felt to category JSX in React, and the arguments promoting React or connected tools (“the virtual DOM is fast!”) were mostly post-hoc rationalizations to validate those first gut feelings. (Jonathan Haidt would apt concur alongside this assessment.)
But in the agentic era, whenever developers are no longer typing code by hand and are thus small constrained by DX, I motionless see vibes ruling the day (and not fair in the awareness of “vibe-coding”). Frontends are rewritten since several example or another is “better for agents.” (Did anyone think to ask the agents?) The resulting code is “faster,” the agents create “fewer mistakes,” etc. Once again, these claims are frequently made without evidence. Whenever I peruse one of these accounts, I think to myself, “Where are the benchmarks?”
Well I got curious, so I wrote a benchmark. The construction of this benchmark is beautiful straightforward:
- Design a small but non-trivial Pokémon app using the PokéAPI (those who have followed my work volition be unsurprised)
- Instruct agents to build the app alongside a particular JavaScript framework/meta-framework (e.g. Next.js, Nuxt, SvelteKit, etc.)
- Test the output against a spec conformance suite (to evade delegate cheating)
- Evaluate the results in conditions of betterment time, tokens spent, lines of code, and achievement and accessibility scores in Lighthouse
The app is fairly simple: you’ve got a catalog of Pokémon alongside their types, moves, and data from assorted games. You can search, filter, etc.:
I should commencement off alongside several caveats: this is not a ideal experiment. This app represents lone one holotype – it’s a comparatively content-based location alongside small interactivity, nearer to a Wikipedia than a Figma, GMail, etc. That said, I had to choose some app to build, and this one is at smallest uncomplicated to comprehend at a elevated level, and comparatively straightforward to spec out and test.
To try to equal the playing field, I had all framework/meta-framework use the authoritative MCP or skills for that tool. I wanted to evade accusations of “this example is not well-represented in the training data, that’s why we built our MCP, we anticipate group to use it,” etc. I additionally didn’t over-specify the exact architecture to use (e.g. SSR vs CSR), nor did I define that the app have to be as performant as possible. I merely gave the spec to Opus 5.5 alongside elevated attempt and told it to use the latest type of the stated framework. Each delegate ran in a sandbox and without any cognition that it would be tested in Lighthouse on achievement and accessibility, although it was stated the conformance test suite.
Note I additionally didn’t command any CSS methodology, apart from to evade using Tailwind, Bootstrap, or any another component framework, and to scheme according to a set of spec tokens to enforce several ocular consistency. I didn’t desire to create this test all concerning CSS, but alternatively concerning JS frameworks, CSR vs SSR, SPAs vs MPAs, etc. The resulting app is tested against a conformance suite according to the accessibility tree fairly than the raw DOM to evade mandating any particular HTML construction – one example uses web components which depend on shade DOM, so this was required.
One item I learned from this test is that everything I’m measuring is extremely stochastic. LLMs are random-word generators, so sometimes you get a low dice-roll and an delegate goes off on several weird tangent wasting a bunch of tokens, through no error of the example or its tools. Lighthouse itself can provision inconsistent results as well: Largest Contentful Paint (LCP) may decide that one depiction is the “largest” element, afterward choice another one on the next run. It’s difficult to have a absolutely fair experiment, but operating multiple iterations did help, and I’ve tried to be honest concerning the variance.
With all that throat-clearing out of the way, let’s get to the results.
Results
I’m going to interrupt this downward by multiple dimensions: disbursal to build, quality, and maintainability. For all framework/meta-framework, I took the median of 5 runs: in another words, I built the identical app 5 times in the identical framework. (Yes, this is the benevolent of zany item you can do alongside coding agents these days.)
The elimination was Angular, which unsuccessful to create a operating app twice, so I took 3 iterations instead. (This is explained below.)
Cost
| Framework | Cost ($) |
|---|---|
| Hono+htmx | $2.67 (2.23–3.19) |
| Astro | $2.76 (2.16–3.63) |
| Nuxt | $2.82 (2.39–4.25) |
| Vanilla | $2.98 (2.24–3.37) |
| Next.js | $3.14 (2.58–4.13) |
| Qwik | $3.27 (2.35–3.80) |
| TanStack Start | $3.54 (3.11–3.96) |
| SolidStart | $3.79 (2.84–4.81) |
| React Router | $3.81 (2.82–3.97) |
| SvelteKit | $4.28 (3.59–4.65) |
| Gracile+Lit | $4.65 (3.92–5.07) |
| Angular | $4.97 (3.19–6.82) |
| 02468 |
Median of 3-5 runs.
Sort Show as table
Note that this quotes the API prices immediately from Claude Code and doesn’t really portray how much prosperity I spent. I’m on a Max plan, so this was fair part of my general $100/month.
Note additionally the variance: React Router’s median is approximately $1 worse than that of Hono+htmx, but the cheapest type of React Router ($2.82) is motionless cheaper than the most costly type of Hono+htmx ($3.19). However several dependable patterns do emerge.
Most clearly, SvelteKit, Gracile+Lit, and Angular are additional costly to build than the another frameworks. For SvelteKit, this may be because of the fact that the agents peruse the SvelteKit v3 immigration guide in all run, apt since it’s too new to be in Opus 5.5’s training data. Similarly for Gracile, the agents didn’t cognize the example extremely fine and either cloned the Gracile repo or peruse its compiled origin from node_modules. For Angular it seems to be a function of the total code size (see below).
I should additionally citation that not all delegate made use of an MCP/skill/Claude Code plugin, nor did all example have one:
Click to see AI tool usageFor example, TanStack Start’s delegate lone discovered its accomplishment records in 2/5 runs (they’re shipped in the node_modules of the project). For Nuxt, the MCP was never used by the agents. For Next.js, agents lone peruse the accomplishment document in 1/5 runs and the AGENTS.md in 2/5 runs. This is a broad range of results, which points rear to the stochastic nature of LLMs. I motionless regard this a valid experiment, though, since I doubt many vibe-coders are shouting at their agents in all-caps to use the MCP (and plentifulness are likely skipping it entirely).
Also, if you’re wondering how I came up alongside Hono+htmx: essentially, I wanted htmx represented somehow, and Hono seemed akin a fine pairing (i.e. as a JavaScript substitute to Rails/Django/etc.). As for Gracile+Lit, I wanted web components represented via their standard-bearer example (Lit), and was lone conscious of Gracile as a meta-framework. If there are improved candidates, afterward those would be awesome for a follow-up experiment.
For “Vanilla,” note that the delegate was told to use “vanilla JavaScript or TypeScript alongside Vite, without a UI framework.” Surprisingly, this underperforms multiple frameworks, which seems to display that having some example can be improved than nothing (or improved than raw Vite, anyway). Vite was chosen since it undergirds many frameworks and represents fairly norm and agnostic JavaScript infrastructure.
The total period and tokens track beautiful closely alongside the cost:
| Framework | Time (minutes) |
|---|---|
| Astro | 8.2 min (6.2–9.7) |
| Hono+htmx | 8.6 min (5.9–9.6) |
| Nuxt | 9.0 min (8.0–17.6) |
| Vanilla | 10.1 min (6.9–18.6) |
| TanStack Start | 10.3 min (8.3–13.9) |
| Qwik | 10.7 min (7.1–12.9) |
| SvelteKit | 10.8 min (8.6–13.8) |
| Next.js | 10.9 min (8.3–11.9) |
| SolidStart | 11.2 min (8.2–15.9) |
| Gracile+Lit | 12.1 min (10.8–12.9) |
| React Router | 12.5 min (8.4–13.6) |
| Angular | 17.2 min (9.8–23.7) |
| 0510152025 |
Wall-clock minutes from the agent’s perspective. Median of 3-5 runs.
Sort Show as table
| Framework | Tokens (M) |
|---|---|
| Nuxt | 3.85 M (3.16–8.18) |
| Astro | 4.13 M (3.03–7.57) |
| Hono+htmx | 4.26 M (3.40–5.61) |
| Next.js | 4.99 M (3.54–8.82) |
| Vanilla | 5.00 M (3.44–6.32) |
| Qwik | 5.64 M (3.41–7.03) |
| TanStack Start | 7.07 M (5.85–7.99) |
| SolidStart | 7.13 M (5.02–10.07) |
| React Router | 7.58 M (4.55–8.18) |
| SvelteKit | 8.53 M (6.79–9.29) |
| Gracile+Lit | 9.04 M (7.36–11.10) |
| Angular | 9.76 M (4.92–14.94) |
| 051015 |
Cache reads, uncached input and output. Median of 3-5 runs.
Sort Show as table
The tokens, by the way, were reported by Claude Code. The duration is fair the total wall-clock time.
Quality
Of course, since I think that outcomes matter, I additionally tested the agents alongside Lighthouse. As application developers, I think it’s crucial that we not lone build things cheaply but additionally performantly and accessibly.
I should archetypal note two caveats concerning my benchmark:
- The Pokémon app is small and doesn’t portray a large, complex real-world application.
- Lighthouse and the Core Web Vitals in broad are biased against SPAs, so the MPA frameworks are nearly guaranteed to win on metrics akin Total Blocking Time (TBT), JavaScript bytes, etc. I considered adding several in-app achievement metrics but backed off since it added complexity and variability to the benchmark, and these would be metrics I invented as opposed to industry standards.
For simplicity’s sake, I’m lone reporting the scores for the residence page. There are another pages, but they all mark fairly likewise or they have untamed swings in metrics akin LCP, so it fair complicates things to contain them.
In short: the Lighthouse scores for Performance, Accessibility, and CLS (Cumulative Layout Shift) are nearly soaked and don’t change much by framework:
| Framework | CLS | Performance | Accessibility |
|---|---|---|---|
| Angular | 0.000 | 98 | 100 |
| Astro | 0.000 | 100 | 100 |
| Gracile+Lit | 0.000 | 100 | 100 |
| Hono+htmx | 0.000 | 100 | 100 |
| Next.js | 0.000 | 99 | 100 |
| Nuxt | 0.000 | 100 | 100 |
| Qwik | 0.000 | 100 | 100 |
| React Router | 0.000 | 100 | 100 |
| SolidStart | 0.000 | 100 | 100 |
| SvelteKit | 0.000 | 100 | 100 |
| TanStack Start | 0.000 | 99 | 100 |
| Vanilla | 0.111 | 97 | 100 |
So to get several additional engaging numbers, we need to appearance at another metrics. The most dependable one is the JavaScript bytes sent, which mostly tracks the client-side example size:
| Framework | JS (KiB) |
|---|---|
| Astro | 0 KiB (0–1) |
| Vanilla | 7 KiB (6–7) |
| Gracile+Lit | 8 KiB (1–13) |
| Hono+htmx | 18 KiB (18–18) |
| SvelteKit | 36 KiB (35–36) |
| SolidStart | 40 KiB (38–40) |
| Qwik | 40 KiB (40–41) |
| Nuxt | 84 KiB (83–85) |
| Angular | 93 KiB (81–102) |
| React Router | 115 KiB (115–119) |
| TanStack Start | 118 KiB (112–119) |
| Next.js | 142 KiB (136–143) |
| 050100150 |
As reported by Lighthouse. Median of 3-5 apps.
Sort Show as table
The Total Blocking Time (TBT) is engaging in that it’s not purely a function of the JavaScript bytes; several of the apps expend additional period on the chief thread than others. That said, the numbers are all fairly small:
| Framework | TBT (ms) |
|---|---|
| Astro | 0 ms (0–0) |
| Gracile+Lit | 0 ms (0–0) |
| Hono+htmx | 0 ms (0–0) |
| SvelteKit | 0 ms (0–1) |
| Vanilla | 0 ms (0–0) |
| TanStack Start | 1 ms (0–36) |
| Qwik | 3 ms (0–30) |
| SolidStart | 11 ms (0–12) |
| React Router | 17 ms (0–52) |
| Nuxt | 40 ms (14–54) |
| Next.js | 71 ms (66–121) |
| Angular | 103 ms (93–123) |
| 0255075100125 |
As reported by Lighthouse, simulated mobile throttling. Median of 5 Lighthouse runs per app, afterward median of 3-5 apps following that.
Sort Show as table
Largest Contentful Paint (LCP) is extremely changeable because of the aforementioned issues alongside the metric itself. As expected, though, it does broadly punish SPAs and reward MPAs:
| Framework | LCP (ms) |
|---|---|
| Astro | 810 ms (792–1662) |
| Gracile+Lit | 956 ms (796–980) |
| Hono+htmx | 978 ms (977–1662) |
| Qwik | 979 ms (954–1278) |
| SvelteKit | 1130 ms (1129–1131) |
| SolidStart | 1204 ms (1203–1278) |
| Vanilla | 1667 ms (764–1676) |
| Nuxt | 1727 ms (1427–1844) |
| React Router | 1729 ms (1729–1770) |
| Angular | 1954 ms (1351–1955) |
| TanStack Start | 2028 ms (1805–2104) |
| Next.js | 2090 ms (1028–2187) |
| 05001000150020002500 |
As reported by Lighthouse, simulated mobile throttling. Median of 5 Lighthouse runs per app, afterward median of 3-5 apps following that.
Sort Show as table
Note that the untamed variance in the Next.js numbers can be explained by the fact that one app avoided lazy-loading images and the other four didn’t. Astro and Vanilla have akin swings that appear to be caused by prioritization or LCP quirks.
In broad though, it’s unsurprising that the metric tends to reward MPAs that coat quickly alongside HTML fairly than SPAs that coat following client-side JavaScript runs (along alongside perchance multiple throttled round-trips to catch the Pokémon data). Ignoring variance, LCP current is mostly a function of JavaScript bundle size. (Qwik is a notable elimination here: technically it’s an SPA that uses client-side routing, but it scores among the MPA frameworks.)
Note although that several of the frameworks varied in whether they focused additional on the client or the server, or equal whether they used a client-side framework. For example, the Gracile delegate didn’t equal use Lit in two runs. For “Vanilla,” the delegate had complete latitude (aside from being told to use Vite), but motionless chose to build an SPA all 5 times.
SSR vs CSR is additionally partially anywhere the difficulty alongside Angular came from: in 2 runs, the delegate built an SSR app that rejected non-localhost (even although the instructions explicitly said that the app would be rearward a reverse proxy). I considered this a valid nonaccomplishment to adhere to the spec and decided not to re-run the 2 unsuccessful iterations.
Maintainability
There are many of ways to measure maintainability (including fancy things akin cyclomatic complexity), but I decided to keep it uncomplicated and study item tried-and-true: total lines of code.
| Framework | Lines of code |
|---|---|
| Hono+htmx | 1370 (1264–1489) |
| Nuxt | 1415 (1382–1458) |
| Astro | 1461 (1379–1503) |
| Vanilla | 1492 (1437–1742) |
| Next.js | 1539 (1452–1627) |
| React Router | 1564 (1421–1621) |
| SvelteKit | 1611 (1503–1816) |
| SolidStart | 1630 (1528–1894) |
| TanStack Start | 1697 (1506–1771) |
| Gracile+Lit | 1717 (1412–1748) |
| Qwik | 1744 (1646–1827) |
| Angular | 2009 (2003–2051) |
| 05001000150020002500 |
Number of non-blank origin document lines. Median of 3-5 apps.
Sort Show as table
To me this is among the additional engaging charts. In a sense, the entire item of a example is to decrease the amount of code you need to maintain. In Rich Harris’s memorable words:
“Frameworks are not tools for organizing your code; they are tools for organizing your mind.”
– Rich Harris, “Rethinking reactivity”
In the agentic era, this hits equal harder since the code size ultimately impacts the figure of tokens, including in forthcoming edits, definition it affects the literal disbursal to keep a codebase complete time. This doesn’t average that succinctness is the lone value that matters, but it’s certainly engaging to see that Hono+htmx can do approximately the identical item as Angular alongside ~70% of the code (and faster, to boot).
Conclusion
There are many ways you could enhance this benchmark. For example, you could command that the delegate use Service Workers for offline caching, or optimize the Pokémon depiction files, or use lazy-loading, fetchpriority, the Speculation Rules API, etc. Even although I cognize a lot of these achievement hacks, I deliberately didn’t steer the agents toward any of them, since it’s not what a representative vibe-coder would do.
On the another hand, this complete benchmark could be cheated by fair pre-generating all the HTML records as fixed resources (there are lone so many Pokémon). I specifically forebade that in the instructions and made it an definitive inspect in the test harness, since I felt that it wasn’t in the soul of the benchmark. (My aim is to simulate a ample quantity of possibly-dynamic data, which is additional challenging than a purely fixed site.)
You could additionally enhance the way I ran the benchmark by operating additional iterations, using distinct coding agents, trying distinct attempt levels, using distinct frameworks or distinct versions of the frameworks, trying alongside and without MCPs/skills, evaluation in additional browsers than fair Chromium, construction distinct types of apps, etc. I didn’t do these things mostly in the involvement of time. However, if anybody wants to run the benchmark themselves or fork it, it’s on GitHub, so I advance you to go nuts. I’ll happily study your results current (along alongside any mistakes or flaws you may find).
My goal current is not to compose the end-all-be-all benchmark for “I’m a vibe-coder, which example should I choose?” The item of this article is to display that it’s imaginable in the era of AI coding to motionless create decisions according to evidence fairly than vibes. The benchmark itself is mostly vibe-coded, although I did assessment the code, so if there are mistakes afterward delight regard them mine. I’d additionally be absolutely blessed to see competitor frameworks try to enhance on what I’ve done here.
I’m additionally not trying to vilify any particular framework. I didn’t cognize which one would “win” before operating the benchmark, although I did doubtful that MPAs would outperform SPAs (which I’ve cheerfully admitted throughout this post). You could certainly compose a additional SPA-favored benchmark by making the app a complex dashboard, a video game, a efficiency app, etc. You could additionally choice distinct metrics: Lighthouse and the Core Web Vitals are beautiful durable on SPAs, so you could measure in-app interactivity instead.
I’m additionally certain that the example authors themselves could enhance their frameworks or AI tools and do improved on this benchmark. I have a powerful suspicion, for example, that the current crop of MCPs and skills may be doing additional damage than good. So if this article is the final term on these frameworks’ performance, I’ll be sorely disappointed. I’m certain there are additionally emerging frameworks that are champing at the bit to do fine on benchmarks specified as this one.
If nothing else, I anticipation this article inspires you to use additional scrutiny to your choice of JavaScript example in this weird new era of vibe coding. Are you really picking your tech stack according to evidence? Or are you fair going alongside what you already know? That’s certainly understandable, particularly if you’re motionless reading, reviewing, and debugging the code yourself. But if you’ve gone mostly hands-free, afterward what attachment do you really have to the choices you made rear whenever you were typing code line-by-line akin an animal? If you’re re-evaluating the complete way you build software, perchance it’s value re-evaluating what you category following npm install? Just saying.
Personally, I don’t have any attachment to the frameworks I’ve evaluated above, apart from that I accept we should strive to build the finest imaginable experiences for our users. I cognize and regard many of the authors of the complete frameworks, and I cognize that they are operating tirelessly for the identical values I champion. My anticipation is fair to shove the complete community in a improved direction toward better, faster, and additional accessible frameworks to power the next generation of web apps.
Addendum
Framework versions used in the test (note Qwik had a place publish partway through the test):
| Framework | Version |
|---|---|
| Angular | @angular/core 22.2.3 |
| Astro | astro 7.4.2 |
| Gracile+Lit | @gracile/gracile 0.11.0, lit 3.3.3 |
| Hono+htmx | hono 4.13.14, htmx.org 2.0.11 |
| Next.js | next 16.4.3, react 19.3.1 |
| Nuxt | nuxt 4.6.2, vue 3.5.44 |
| Qwik | @builder.io/qwik-city 1.20.2 (iteration 1), 1.20.3 (iterations 2, 3, 4, 5), @builder.io/qwik 1.20.2 (iteration 1), 1.20.3 (iterations 2, 3, 4, 5) |
| React Router | react-router 8.5.0, react 19.3.1 |
| SolidStart | @solidjs/start 2.1.0, solid-js 1.9.17 |
| SvelteKit | @sveltejs/kit 3.1.0, svelte 5.57.2 |
| TanStack Start | @tanstack/react-start 1.169.4, react 19.3.1 |
| Vanilla | vite 8.3.5 |
You can comment on the fediverse.
