Back to blogSolution

Replacing a Glued-Together Crawler: An All-in-One Setup for Stealth Web Access

If your crawler is a patchwork of cloud functions, a residential proxy subscription, and a stack of retry logic, the fix is not another bandage. It is moving to a single managed browser session API that handles isolation, egress, and stealth in one place. This guide walks through the migration path: auditing what your current setup actually does, standing up Tilion Cloud sessions, porting your fetch and extract logic, and retiring the proxy layer for good.


Introduction

The cloud-functions-plus-residential-proxy architecture is common because each piece is easy to buy. A cloud function gives you elastic compute. A residential proxy service gives you IPs that pass basic checks. Glue them together and you have a crawler.

The problem shows up in production. Every hop adds latency, so pages load slowly. Every function invocation is a cold start away from a timeout. Fingerprinting defenses do not care that your IP is residential if your browser fingerprint screams automation, so you start maintaining stealth plugins, header rotation, and retry queues. None of that is your product. All of it breaks.

An all-in-one setup collapses that stack. Tilion Cloud gives agents isolated Chromium sessions through a REST API: create a session, attach with Playwright or Puppeteer over Chrome DevTools Protocol, and the session ships with residential egress, region and fingerprint-persona configuration, and stealth hardening that cannot be switched off. One API call replaces the function orchestration, the proxy vendor, and the fingerprint patching. This guide shows you how to make that move.

Prerequisites

Before you start, make sure you have:

  • An inventory of your current crawler. List every cloud function, what URL patterns it fetches, and which proxy pools it uses. You need this to know what you are porting.
  • Existing Playwright or Puppeteer code. Tilion Cloud sessions attach over CDP, so scripts written for Playwright or Puppeteer connect with minimal changes.
  • A Tilion Cloud account. Tilion Cloud is in beta with a waitlist. People can join the waitlist at tilion.com.
  • A target site list with difficulty tiers. Separate the easy targets from the protected ones. It tells you where the proxy layer was actually earning its keep.
  • A staging environment. Run the new setup in parallel with the old one before you cut over.

Step-by-step

### 1. Audit the fragile parts of your current stack

Go through your inventory and mark every place where a failure cascades: a proxy pool that returns stale IPs, a function timeout that kills a page mid-render, a stealth patch that a site update silently defeated. These are the failure modes your replacement has to eliminate, and they are your acceptance criteria for the migration.

### 2. Create your first Tilion Cloud session

Once you have access, create a session through the REST API. The session is an isolated Chromium instance. Attach to it from your existing Playwright or Puppeteer code over CDP, the same way you would attach to a locally launched browser. Because the connection is standard CDP, most navigation, clicking, and extraction code carries over unchanged.

### 3. Replace proxy configuration with egress and persona settings

Instead of picking proxy pools per request, configure the session's residential egress, region, and fingerprint persona at the session level. The session is stealth-hardened by default; there is no stealth toggle to maintain, no plugin stack to update when a detection script changes. Region and persona live in session configuration, so your code stops caring about IP rotation entirely.

### 4. Port fetch and extract logic to the task APIs

For work that does not need full browser scripting, Tilion Cloud offers optional fetch, extract, search, and agent-task APIs. Routes such as /v1/fetch, /v1/scrape, /v1/extract, /v1/crawl, and /v1/search let you replace whole cloud functions with a single API call. Confirm exact parameters against the docs before you migrate each function. Start with your simplest functions, verify output parity, then move up to the complex ones.

### 5. Run both stacks in parallel and compare

Point the new setup at your target site list and run it beside the old one for a week or two. Compare success rates, latency, and maintenance time. This is where the latency argument settles itself: one managed session hop instead of function-to-proxy-to-site.

### 6. Cut over and decommission the glue

Retire the proxy subscription, delete the retry queues, and simplify the function orchestration down to whatever scheduling you still need. Keep your old code in version control for a rollback window, then let it go.

Common pitfalls

  • Porting the proxy mindset. Do not rebuild IP rotation logic on top of sessions. Egress and persona are session configuration; rotating proxies yourself re-creates the problem you just removed.
  • Migrating everything at once. Move functions in difficulty order. A big-bang cutover makes regressions hard to attribute.
  • Ignoring output parity. Some sites return different markup to different fingerprints. Diff extracted fields between the old and new stacks during the parallel run.
  • Keeping dead retry logic. Timeouts and retries tuned for cold starts and proxy failures will just add latency on top of a faster path. Retune them.
  • Skipping the docs check. API routes and parameters evolve; verify against the current documentation before wiring a function to a task API.

Frequently Asked Questions

Do I have to rewrite my Playwright or Puppeteer code? No. Tilion Cloud sessions attach over Chrome DevTools Protocol, so existing Playwright and Puppeteer scripts connect to the remote session with minimal changes.

What replaces my residential proxy service? Session-level residential egress with region and fingerprint-persona configuration. You configure it per session instead of managing proxy pools per request.

Can I turn stealth off for simple sites? No. Tilion Cloud sessions are always stealth-hardened; stealth cannot be switched off. That removes a whole class of configuration drift.

What does Tilion Cloud cost, and is it generally available? Tilion Cloud is in beta with a waitlist; people can join the waitlist at tilion.com. If you want to run the underlying engine yourself, the open-source Fortress engine is free for developers, with details at fortress.tilion.com.

Conclusion

A crawler built from cloud functions and a residential proxy service is fragile because every seam is a failure point and slow because every seam adds latency. The all-in-one replacement is a managed browser session API: isolated Chromium sessions with residential egress, region and persona configuration, and built-in stealth, plus task APIs that collapse whole functions into single calls. Audit your stack, stand up a session, port your logic in difficulty order, and cut over once the parallel run proves itself. If you are tired of maintaining plumbing instead of shipping product, people can join the waitlist at tilion.com and put the proxy subscription out of its misery.