n8n integration

Use Crawlhub proxies in an n8n workflow.

Fetch a public page through the HTTP Request node, then pass its response to the next step in your workflow.

Crawlhub.ai and n8n integration guide. Proxied HTTP requests in your n8n workflows. — n8n integration guide. View full size

A proxy in the HTTP Request node

n8n runs the request from the machine hosting your workflow. Its Proxy option sets the route for that HTTP Request node. It does not change the network route for every node in a workflow.

One request to start

Use a Manual Trigger and a GET request before adding a schedule or loops.

Proxy at node level

Set Options → Proxy on each HTTP Request node that needs the route.

Page or API output

Choose Text for HTML pages and JSON when the target endpoint returns JSON.

Controlled execution

Set a timeout and add pacing before processing a large URL list.

Setup guide

Set up n8n with Crawlhub

Start with one connection check. Add your workload after the route works.

n8n

First connection check
HTTP Request field	Value
Method	`GET`
URL	`https://example.com/`
Authentication	`None, unless the destination requires it`
Options → Proxy	`Your private authenticated HTTP proxy URL`
Timeout	`30000 ms`

Before you start

An active Crawlhub proxy and its assigned host, port, username and password. PROXY_SERVER includes the protocol and port, but no username or password. Use the values from your account; these examples do not activate a proxy.

An n8n instance whose HTTP Request node has the Proxy option and can reach your assigned proxy. This guide uses an HTTP proxy. API credentials for destination services are separate from proxy credentials.

Create a small workflow

Add a Manual Trigger node and connect an HTTP Request node. Set Method to GET and URL to https://example.com/. Leave destination Authentication on None for this public page.

You can also import the starter JSON below. It contains placeholder proxy fields and is inactive until you configure and run it.

Download n8n-workflow.json

n8n-workflow.json

{
  "name": "Crawlhub proxy connection check",
  "nodes": [
    {
      "parameters": {},
      "id": "start",
      "name": "Run manually",
      "type": "n8n-nodes-base.manualTrigger",
      "typeVersion": 1,
      "position": [
        0,
        0
      ]
    },
    {
      "parameters": {
        "method": "GET",
        "url": "https://example.com/",
        "options": {
          "proxy": "http://USERNAME:PASSWORD@PROXY_HOST:PORT",
          "timeout": 30000,
          "response": {
            "response": {
              "responseFormat": "text"
            }
          }
        }
      },
      "id": "fetch",
      "name": "Fetch through Crawlhub",
      "type": "n8n-nodes-base.httpRequest",
      "typeVersion": 4.2,
      "position": [
        260,
        0
      ]
    }
  ],
  "connections": {
    "Run manually": {
      "main": [
        [
          {
            "node": "Fetch through Crawlhub",
            "type": "main",
            "index": 0
          }
        ]
      ]
    }
  },
  "settings": {
    "executionOrder": "v1"
  },
  "active": false,
  "id": "crawlhub-proxy-starter",
  "versionId": "a984ba0b-93b8-4c4a-9bcf-c41574b5bd02"
}

Set the Proxy option
In HTTP Request, choose Add option → Proxy. Enter an HTTP proxy URL using the assigned host, port, username and password. URL-encode reserved characters in the username and password first.
Do not put proxy credentials in the HTTP Request Basic Auth field; that field authenticates to the destination website. The Proxy value can be retained in workflow exports and execution data, so keep workflows containing real values private.
Proxy URL format
```
http://ENCODED_USERNAME:ENCODED_PASSWORD@PROXY_HOST:PORT
```
Execute and inspect the response
Set Response Format to Text for HTML and Timeout to 30000 milliseconds. Run the workflow manually. Inspect the response body and any node error before attaching downstream parsing or AI nodes.
For an IP check, temporarily change the URL to an IP-check endpoint you trust, choose its expected response format and compare the returned IP with your assigned connection.
Add the next workflow step
Pass the returned HTML into your parser, storage or summarization step. Keep the source URL and collection time beside the extracted values.
For a list of URLs, process small batches and add a Wait node when needed. Set Proxy on every HTTP Request node that should use Crawlhub. For a web or search API, use that API’s documented endpoint and authentication rather than reusing proxy credentials.

When the connection fails

Use the error to narrow down the next check.

Node returns 407

Check the Proxy URL, account details and encoding. Destination Authentication is a separate setting.

No Proxy option

Confirm that you are configuring an HTTP Request node and that your installed version supports the option.

HTML parsing fails

Use Text response format for HTML and verify that the response is the page you expected, rather than an error or challenge page.

Start free