Vercel Fluid Compute for AI: sizing Active CPU vs memory
Fluid Compute can suit streamed AI routes on Vercel, but the cost depends on Active CPU, memory busy time, runtime, region and request duration.
An AI route on Vercel can look cheap if the code mostly waits for an LLM, vector database or tool API. Fluid Compute makes that workload more attractive because Active CPU billing does not count I/O wait time, but it does not make long streams free. Provisioned Memory is still billed while requests are in flight, including while the response is streaming.
The practical decision is not simply whether Fluid Compute is cheaper. The decision is whether the route belongs in an HTTP request lifecycle, which runtime and region it should use, how tightly to cap maxDuration, and how to estimate compute separately from model-token spend. If this route is part of a wider AI app on Vercel, pair this with a CI and preview setup such as the Vercel Pro CI/CD architecture for AI apps so you can test changes without surprise bills and keep side projects deployed on Vercel safely as usage grows.
The decision: should an AI route use Fluid Compute?
Use Fluid Compute for AI-heavy HTTP routes that spend most of their wall-clock time waiting on external I/O: model calls, embedding requests, vector lookups, database queries or third-party tool APIs. Vercel’s Fluid Compute documentation specifically positions it for I/O-bound AI applications, where multiple invocations can share one function instance.
The warning is just as important: a streamed AI response can keep the request open long after the route has finished its own CPU-heavy work. Vercel’s Functions usage and pricing documentation says Active CPU billing pauses during I/O waits, but Provisioned Memory continues while requests are in flight and only stops when the last in-flight request completes.
For an AI route, the right outcome is a route-level worksheet with these decisions filled in:
- Runtime: Node.js, Python, Edge, Bun or Rust, with concurrency assumptions stated.
- Region: chosen for latency and data proximity, not only the lowest compute rate.
- Memory size: default or configured, because memory GB-hours are part of the bill.
maxDuration: capped to the product’s real waiting tolerance, not set to the plan maximum by default.- Cost split: Vercel compute separated from model or AI Gateway token spend.
If the route is a normal request/response or streamed chat endpoint and it fits inside duration limits, Fluid Compute is a strong fit. If the work needs durable state, pause/resume behaviour or unbounded execution, treat it as a workflow decision rather than a longer HTTP timeout. For end-to-end AI feature design decisions, including when work should move off HTTP into jobs or queues, see the broader playbook in how to add AI to an existing SaaS without blowing it up and the real cost of building an AI MVP.
How Fluid Compute bills an AI route
Fluid Compute pricing has three main meters for Vercel Functions: Active CPU, Provisioned Memory and Function Invocations. Enterprise is custom, so public Pro rates should not be used to estimate an Enterprise contract.
| Billing meter | What Vercel measures | Why it matters for AI routes |
|---|
| Active CPU | CPU time while the function code actively consumes CPU. | Waiting on an AI model, database query or third-party API does not count as Active CPU according to Vercel’s pricing documentation. |
| Provisioned Memory | Allocated function memory in GB-hours across the instance lifetime while requests are in flight. | Memory billing continues during I/O waits and streamed responses until the last in-flight request completes. |
| Function Invocations | Each incoming request to the function. | Requests count regardless of success or failure, so retries, timeouts and abandoned generations still matter. |
This distinction is the core of Vercel Fluid Compute AI routes pricing. A route can have low Active CPU because most of its time is spent waiting on the model, while still accumulating memory GB-hours because the request remains open.
When all requests finish, Vercel pauses the instance and applies no CPU or memory charges until the next invocation. That helps intermittent workloads, but it does not remove the need to measure streamed duration and in-flight memory time.
The worked example: a streamed chat route with retrieval and one tool
Use this as an illustrative route shape, not observed data: a Next.js App Router POST /api/chat route accepts a user message, performs one vector lookup, may call one external tool, and streams an LLM answer back to the browser.
Vercel’s streaming guide recommends the Vercel AI SDK for streaming responses from LLMs and AI APIs, and its example notes Node.js 20 or later. That does not make Node.js mandatory for every AI route, but it is a common fit for a Next.js App Router streaming endpoint. If this route is part of a broader AI agent stack, make sure your deployment workflow matches the guidance in hybrid edge + CI deployment for AI agents so preview branches and production limits stay aligned.
| Route phase | Cost exposure | What to measure |
|---|
| Request parsing and prompt assembly | Active CPU and memory time. | CPU milliseconds and configured memory size. |
| Vector lookup | Low CPU if mostly waiting on the vector database; memory remains open. | Wall-clock wait time and any retries. |
| Optional tool call | CPU for preparation and parsing; I/O wait for the external API; memory remains open. | Frequency of tool use, timeout settings and failure rate. |
| Model stream | CPU pauses during model I/O waits, but memory billing continues while the stream is in flight. | Stream duration, user-abandonment behaviour and p95 completion time. |
| Model or AI Gateway usage | Separate from Fluid Compute. | Provider token usage or AI Gateway spend. |
The common failure case is assuming model wait time is free. The CPU part may be lower than a traditional always-busy server model, but the memory meter still runs while the request is open. If a slow stream improves neither conversion nor user satisfaction, it can become a compute and product problem at the same time.
The cost worksheet to build before choosing a plan
Build the worksheet from measured route values, not from a generic traffic guess. For each AI route, collect monthly invocations, measured Active CPU hours, measured Provisioned Memory GB-hours while requests are in flight, configured memory, runtime, region, maxDuration and separate model-token cost.
Vercel’s official Fluid Compute regional pricing table, used to verify the Active CPU and Provisioned Memory rates in the worksheet.
Vercel pricing values below are verified from Vercel sources as of 2026-11-11. Prices are in USD. Vercel’s pricing page states Pro pricing excludes VAT, GST and other applicable taxes.
| Plan or meter | Public pricing input | Use in the worksheet |
|---|
| Hobby | $0/month, including 4 Active CPU hours, 360 GB-hours Provisioned Memory and 1M Function Invocations per month for Fluid Compute. | Check whether measured usage stays inside the included allowances. Hobby cannot purchase additional usage according to the supplied pricing FAQ. |
| Pro | $20/month, including $20 of usage credit; additional Fluid Compute usage is metered on demand. | Estimate marginal usage after applying the plan’s usage credit. |
| Function Invocations | $0.60 per 1M invocations. | Count all requests, including failed requests. |
| Enterprise | Custom. | Do not estimate from public Pro rates; use the commercial terms in the contract. |
The formulas are simple, but the inputs need to come from the route:
- Invocation cost = monthly function invocations / 1,000,000 × $0.60.
- Active CPU cost = measured Active CPU hours × selected region’s Active CPU rate.
- Provisioned Memory cost = measured GB-hours while requests are in flight × selected region’s Provisioned Memory rate.
- Fluid Compute subtotal = invocation cost + Active CPU cost + Provisioned Memory cost, before included allowances, Pro usage credit or Enterprise custom terms.
- AI feature cost = Fluid Compute subtotal + separate model-token cost or AI Gateway/provider cost.
For the default documented region, iad1, the Pro marginal formula is:
Measured Active CPU hours × $0.128 + measured Provisioned Memory GB-hours × $0.0106 + measured invocations / 1,000,000 × $0.60
One official billing-unit pack in iad1 is useful for understanding the meters, not for estimating a real workload: 1 Active CPU-hour × $0.128 + 1 Provisioned Memory GB-hour × $0.0106 + 1M invocations × $0.60 = $0.7386.
Region can move the compute line. The supplied Vercel regional rates list iad1, cle1 and pdx1 at $0.128 per Active CPU-hour and $0.0106 per Provisioned Memory GB-hour. London, lhr1, is listed at $0.177 per Active CPU-hour and $0.0146 per Provisioned Memory GB-hour; Frankfurt, fra1, at $0.184 and $0.0152; Singapore, sin1, at $0.160 and $0.0133. The supplied highest-rate example, São Paulo gru1, is $0.221 and $0.0183.
Do not choose a region by price alone. If the route waits on a database, vector store or model endpoint in another region, the cheaper compute region may create slower streams and longer in-flight memory time. Region selection is an architecture decision first and a price optimisation second.
Keep token cost separate. Vercel’s AI Gateway pricing documentation says AI Gateway uses provider list prices with no token markup, and BYOK has no AI Gateway markup or fee. That still means model usage is a separate cost line from Fluid Compute. If token spend dominates the feature, optimise model choice, prompts, caching and provider usage before over-optimising the Vercel compute line. For a deeper breakdown of model-side pricing, you can cross-check your assumptions with the ChatGPT pricing guide for 2026.
Runtime, duration and streaming choices that change the bill
Fluid Compute is available for Node.js, Python, Edge, Bun and Rust. However, Vercel documents optimised in-function concurrency for Node.js and Python, not as a general guarantee across every runtime. If the cost case depends on multiple invocations sharing a function instance efficiently, check the runtime before making the claim.
Memory and vCPU settings also change the worksheet. Vercel’s Functions limits documentation lists the default Fluid Compute memory size as 2 GB / 1 vCPU. Pro and Enterprise can configure up to 4 GB / 2 vCPU. More memory may be necessary for some workloads, but it raises Provisioned Memory exposure while requests remain open.
Duration needs a product cap, not only a platform cap. With Fluid Compute enabled, Node.js, Bun and Python functions have a 300 second default maximum duration. Hobby has a 300 second maximum. Pro and Enterprise have an 800 second generally available maximum. Vercel’s duration configuration documentation also describes an 1800 second beta maximum for Pro and Enterprise, limited to supported Node.js, Bun and Python runtime versions, configured per function, with BYOC not supported above 800 seconds during the beta.
For request handlers, max duration includes time spent sending streamed responses. If a function exceeds its configured duration, Vercel terminates it and returns FUNCTION_INVOCATION_TIMEOUT. That makes an overgenerous maxDuration risky: it can let poor product behaviour stay open longer while memory billing continues.
Edge runtime has a different streaming constraint. Vercel documents that Edge runtime functions must begin sending a response within 25 seconds to maintain streaming beyond that point and can continue streaming for up to 300 seconds. For a long AI stream, this is a reason to re-check whether Edge is the correct route runtime.
When to keep it as an HTTP route and when to move it to Workflows
Keep the route as a Fluid Compute HTTP endpoint when the user is actively waiting for the answer, the stream fits inside the route’s configured duration, and failure can be returned to the request cleanly. A chat response, retrieval-augmented answer or short tool-assisted generation usually belongs here if the measured worksheet works.
Move the work out of the request lifecycle when it needs to pause, resume, maintain state or run beyond request-duration limits. Vercel directs workloads requiring unlimited execution time to Workflows, which can pause, resume and maintain state for minutes to months without duration limits.
The boundary is not only technical. If a user will not wait several minutes for the stream, keeping the HTTP connection open is often the wrong product shape even if the platform can support the duration. Use a workflow, job status page, notification or stored result when the work can outlive the user session.
The route-by-route decision checklist
Before scaling an AI-heavy Vercel route, make the decision per route rather than applying one global setting to every endpoint.
Official enablement guidance for Fluid Compute; use this instead of a private dashboard screenshot unless the author can safely capture a non-sensitive demo project.
- Confirm Fluid Compute is enabled for the project or configured route. Vercel says new projects have Fluid Compute enabled by default as of 23 April 2025; existing projects can enable it in the dashboard or with the
fluid property in vercel.json. Redeploy after dashboard changes.
- Check the runtime. Use Node.js or Python if the cost case depends on documented optimised in-function concurrency.
- Measure wall-clock streamed duration separately from Active CPU time. The failure case is treating model-wait time as fully free.
- Measure monthly invocations, Active CPU hours and Provisioned Memory GB-hours from Vercel Observability or billing export before forecasting growth.
- Select the region against the model endpoint, database, vector store and users, then run the same measured workload through that region’s CPU and memory rates.
- Cap
maxDuration to the route’s expected p95 and product timeout, not the largest value the plan allows.
- Separate Fluid Compute from model-token or AI Gateway cost so the team knows which lever actually moves the bill.
- If the task can outlive the request, needs durable state or may run for minutes to months, evaluate Vercel Workflows instead of forcing it into a long HTTP route.
The next action is to build the worksheet for one real route: invocations, Active CPU hours, Provisioned Memory GB-hours, runtime, region, memory size, maxDuration and token spend. If the route is mostly I/O-bound and fits the user-wait window, Fluid Compute is likely the right HTTP compute model. If the memory-open time or duration risk is the problem, redesign the route before traffic grows.