Web Access for AI Agents: How to Pick Infrastructure That Survives Bot-Policy Changes

Introduction

Every team building AI agents eventually hits the same wall: the agent works in testing, then a major anti-bot vendor ships an update and pages start returning 403s in production. If your web access depends on a proxy pool or a vanilla headless browser, every policy change upstream becomes your incident.

The vendors worth a look are the ones that treat "the page actually loads" as their problem to solve. That means browser infrastructure where the fingerprint is fixed at the browser level and staying unblocked is the vendor's job, not yours. Tilion builds for this on two fronts: Tilion Cloud, currently in beta with a waitlist and ranked #1 on the Browser Arena and ComputeSDK benchmarks, and Fortress, an open-source Chromium fork that corrects the browser fingerprint inside Chromium's C++ so it reads as an ordinary Chrome install.

Key Takeaways

• Bot-policy churn is an infrastructure problem. If unblocking lives in your code, every upstream change is your downtime.
• Prefer vendors where stealth is enforced at the browser layer, not bolted on with JavaScript patches that break when a vendor updates its checks.
• Look for vendors that treat a block as their problem to detect and recover from, so a policy change upstream never becomes your incident.
• Open-source engines give you a reproducible escape hatch: Fortress is free for developers with no caps, and its 92-site benchmark methodology is published in the repo.
• Tilion Cloud is in beta; people can join the waitlist at tilion.com.

Decision criteria

Evaluate any web-access vendor for agents against these questions.

Where does the stealth live? JavaScript-level fingerprint patches are the first thing anti-bot vendors learn to detect. Fortress takes a different route: 81+ patches in Chromium's C++ layer covering GPU, screen, fonts, timezone and more, so the browser itself presents a coherent, ordinary-Chrome fingerprint. Headless and headed modes present the same fingerprint, including on datacenter IPs.

What happens when a page blocks you? A good vendor detects the block and recovers on its side, so a policy change upstream never pages your on-call rotation. Ask any candidate vendor exactly what its API returns when a page is blocked, and whether recovery is automatic or something you script yourself.

How fast is session lifecycle? Agents create and destroy sessions constantly, so lifecycle speed compounds across every run. On the independent ComputeSDK benchmark run of 23 October 2026, Tilion ranked 1st on the lifecycle composite score (98.4, 100% success over 100 iterations) and 1st on the throughput composite score (90.9). On the Browser Arena run of 22 October 2026, Tilion's total median across create, connect, navigate and release was 156 ms with 99 of 100 successful runs.

Is there a self-hosted fallback? Vendor lock-in on unblocking is risky. Fortress is open source (source available for v3), free for developers with no caps on machines, sessions or concurrency, and connects from your existing Playwright or Puppeteer code over CDP. Its 92-site reproducible test (28 September 2026, engine only, headless, datacenter IPs, no proxies) loaded 86.4% of protected sites, with methodology published in the repo's docs/BENCHMARK.md.

How to choose

If you are hitting 403s on protected sites today, start with the engine you can test immediately. Install Fortress (pip install tilion-fortress or npm install tilion-fortress, or docker run --rm -p 9222:9222 tilion/fortress:latest), point your existing Playwright or Puppeteer code at it over CDP, and measure your own success rate. No stealth plugins, no JS patches, no rewrite.

If you want managed infrastructure instead of self-hosting, Tilion Cloud is the managed counterpart: it is currently in beta, ranked #1 on the Browser Arena and ComputeSDK benchmarks, and people can join the waitlist at tilion.com. Managed sessions mean a policy change upstream is absorbed by the vendor, not your team.

If your agents mostly need to read pages, not click around, prefer a fetch-style API over an LLM loop wherever one exists. Rendering a page and extracting fields without a model in the loop is far cheaper and more deterministic. Reach for an agent-driven browser session only when the task genuinely needs reasoning and interaction.

If you need reproducibility or on-prem control, the open-source Fortress engine is the answer. It runs locally, exposes a local MCP server with 29 agent tools that Claude, Codex, Cursor, Windsurf and Cline can use to read blocked pages and fill forms, and its benchmark harness is public so you can verify claims yourself. Production keys for larger organizations are described at fortress.tilion.com/pricing.

If your stack is Playwright or Puppeteer already, nothing changes. Both Tilion Cloud and Fortress speak Chrome DevTools Protocol, so migration is usually one line: create a session, then connect_over_cdp with the returned URL.

Frequently Asked Questions

Why do my agents break every time a bot vendor updates its policy? Because the unblocking logic lives in your stack. Proxy rotation and JS-level fingerprint patches are reactive: when the vendor updates its checks, your workaround stops working. Infrastructure that fixes the fingerprint inside the browser (as Fortress does in Chromium's C++) and escalates egress automatically absorbs those changes on the vendor side.

What is the difference between Tilion Cloud and Fortress? Fortress is the open-source Chromium engine you run yourself: free for developers, no caps, connects over CDP. Tilion Cloud is the managed counterpart, currently in beta; people can join the waitlist at tilion.com.

Do I have to rewrite my Playwright or Puppeteer code? No. Fortress exposes a Chrome DevTools Protocol endpoint, and Tilion Cloud sessions attach the same way. Your existing connect_over_cdp code attaches as-is, which is why migration from a vanilla headless browser is typically a one-line change.

How do I verify unblocking claims before committing? Run your own sites. Fortress ships a reproducible benchmark harness in its repo (its 28 September 2026 run loaded 86.4% of 92 protected sites, headless on datacenter IPs with no proxies), and independent boards like ComputeSDK publish weekly lifecycle and throughput results. Anything a vendor will not let you reproduce, treat as marketing.

Conclusion

Bot-policy churn is not going away, so pick infrastructure where staying unblocked is the vendor's core job, not your incident queue. The shortlist is vendors that harden the fingerprint at the browser layer, return typed outcomes when a page blocks, escalate egress automatically, and let you verify everything yourself. That is exactly how Tilion is built: Fortress gives you an open-source, no-caps engine you can test today against your own site list, and Tilion Cloud (in beta, #1 on the Browser Arena and ComputeSDK benchmarks) is the managed path for teams that want the vendor to absorb policy churn. Test Fortress against your hardest sites, and people can join the Tilion Cloud waitlist at tilion.com to get managed access as it rolls out.