Keep Playwright proxy sessions consistent

A sticky proxy address and a browser session solve different problems. Keeping the same exit IP does not preserve cookies. Keeping cookies does not hold the network route steady. A reliable browser collector gives both an explicit lifecycle.

Consider a public catalog that asks for a delivery country before showing availability. Your job selects a market, follows product links and records stock. If a later task inherits the wrong browser state, the output can look valid while describing a different market.

Separate browser state, routing and measurement context

Browser state includes cookies and origin storage. Proxy routing determines the connection path. Measurement context describes what the collected value means: for example, country, selected product variant and retrieval time. Diagnose these separately instead of calling all of them “the session.”

Playwright browser contexts provide isolated environments within a browser. Creating separate contexts keeps their cookies and storage apart. Opening another page inside an existing context does not give it that same clean-state boundary.

That isolation is useful for collection as well as testing. One context can hold the market selection for one job while another begins without those cookies. Decide which requests belong together before assigning work to pages.

Define the session before opening the page

Write down a small contract for each job. Use a logical session identifier, a browser-state policy and a proxy-configuration reference. Store the identifier in your collector's records; do not confuse it with a routing token embedded in a provider-specific username.

{
  "job_id": "catalog-check-001",
  "browser_state": "fresh_context",
  "proxy_config_ref": "approved-market-route",
  "requested_market": "GB",
  "observed_market": null,
  "retrieved_at": null
}

This is an illustrative application record, not a proxy credential or a Crawlhub API request. Populate the observed market from evidence on the page. Leave it unknown if the page gives you no dependable way to verify it. An IP's reported location alone does not prove which storefront or shipping market the page selected.

Choose the session's end condition as carefully as its start. A completed product batch, a market change or a failed continuity check can all justify closing the context. Keep a partial result separate from a complete batch so downstream reporting does not silently treat them as equivalent.

Assign proxy settings at the context boundary

Playwright documents proxy configuration at browser launch or for an individual browser context. Its network guide also distinguishes proxy credentials from the destination's HTTP authentication settings. Use the option intended for the connection you are configuring.

For jobs with different routing requirements, make the context configuration explicit in the job setup. Keep secrets in the worker's secret store and refer to a configuration revision in logs. Do not include complete authenticated endpoint URLs in failure reports.

A fresh context does not force a rotating proxy to choose a new address. A provider's session option also does not guarantee an address remains available indefinitely. Treat continuity as something to verify against the product's documented behavior and your job's actual requirements.

If a stable address is necessary, check it using an approved diagnostic endpoint before the job, and at an appropriate boundary afterward. If it changes, mark the affected observations for review. Avoid adding an address check to every page request unless that measurement is necessary.

Wait for the data your parser needs

Loading a document is only part of a browser collection job. A product panel can appear after navigation, after a region choice or after a separate data request. Define a readiness condition around the field or response your extractor actually needs.

Playwright discourages using networkidle as a general readiness signal in testing and provides locator assertions for specific conditions. For collection, apply that idea to the field contract: wait for the required element and check its value, rather than assuming network quiet means a usable record.

A visible price container can still contain a loading placeholder. Check the extracted amount and currency before saving. If stock is optional, preserve an unknown stock state without discarding an otherwise usable price. If price is required, route a missing price to a validation failure.

Use a bounded wait and preserve a failure category when it expires. “Required price did not become available” tells the operator what happened. “Proxy failed” mixes a rendering or extraction issue with a network diagnosis you have not established.

Reuse browser state only when the job needs it

Some authorized workflows deliberately reuse authenticated state. Playwright supports storing and loading browser state, and its authentication guide warns that saved state can contain sensitive material capable of impersonating the account. Keep it out of repositories and ordinary debug attachments.

For public collection jobs that do not need login, a clean context is often easier to explain. Start from the required market choice and rebuild only the state needed for that task. Reusing a large, old profile can import preferences that no one remembers setting.

If you intentionally reuse state, document the account or market boundary, the expiry policy and the condition that forces a fresh start. Loading a state file should be a visible choice in the job configuration, not an accidental consequence of reusing a worker directory.

Investigate drift before increasing rotation

When two observations disagree, compare the exact URL, variant, requested market, observed market and browser-state policy. Check whether the job changed routes or reused a context. Then inspect whether both pages reached the same readiness condition.

Run a small controlled replay: one product, one fresh context and one documented routing configuration. Compare its record with the disputed observation. This will not prove a universal cause, but it narrows the investigation without changing several variables at once.

Scale once those boundaries are visible. Related requests can share a deliberately created context; unrelated jobs can begin cleanly. Read our guide to product-price context for the record fields, or use the Crawlhub integration documentation to check your client's connection setup.