Octoparse integration

Use Crawlhub proxies for local Octoparse runs.

Add an HTTP proxy to a local task. Use the included bridge when your upstream connection needs a username and password.

Octoparse supplied opening artwork — Supplied integration artwork. View full size

What is Octoparse?

Octoparse builds web scrapers through a point-and-click interface. Its current documentation limits custom proxies to local HTTP runs and does not support proxy usernames/passwords in the task’s proxy field.

Point-and-click extraction

Select page data and build the extraction workflow without writing scraper code.

Task execution

Run a configured task locally. Cloud proxy support is a separate built-in service.

Custom HTTP route

Use a compatible HTTP IP:port entry, or the local bridge supplied below for an authenticated upstream.

Supported exports

Export the results in the formats available in your version and plan, such as Excel or CSV.

Setup guide

Set up Octoparse with Crawlhub

Start with one connection check. Add your workload after the route works.

Octoparse

Local task proxy scope
Setting	Value in this recipe
Run location	`Your device / local run`
Proxy type	`Custom HTTP`
Octoparse proxy entry	`127.0.0.1:PORT_PRINTED_BY_BRIDGE`
Upstream login	`Kept in the local bridge, not the Octoparse field`

Before you start

Use an activated connection and its assigned hostname, port, protocol, full username and proxy password. The proxy login is separate from the account used to sign in to the tool.

Use a version with Task Settings → Anti-blocking → Use my own proxies. Custom proxies are for local runs, not Octoparse Cloud. Do not enter HOST:PORT:USERNAME:PASSWORD into a field whose documented format is IP:port.

For password-authenticated upstream connections, install a supported Node.js release and proxy-chain on the same device as Octoparse. The extraction workflow remains no-code; the proxy bridge runs as a local process.

Create a local task
Install and open Octoparse. Choose New → Custom Task, enter an allowed target URL and save.
Start with a small page and one extraction goal before adding pagination or schedules.
Document interface reference: create a custom task. View full size
Document interface reference: enter a task URL. View full size

Prepare the authenticated connection

For a proxy that needs login credentials, save both helper files below in one project folder. Install proxy-chain and start octoparse_bridge.mjs with your assigned server, username and password in the environment.

The helper prints a local URL such as http://127.0.0.1:49152. Leave it running. Its port can change on restart. The local endpoint has no login, listens only on this device and forwards to the authenticated upstream.

If your assigned HTTP connection does not require authentication, use its documented compatible numeric IP:port directly instead of this bridge.

Crawlhub v1 proxy parameters

Run in your terminal

npm init -y
npm install proxy-chain
export PROXY_SERVER='http://YOUR_ASSIGNED_HOST:YOUR_PORT'
export PROXY_USERNAME='YOUR_ASSIGNED_USERNAME'
export PROXY_PASSWORD='YOUR_PROXY_PASSWORD'
node octoparse_bridge.mjs

Download proxy_bridge.mjs

proxy_bridge.mjs

import { Server } from 'proxy-chain';

export async function createProxyBridge() {
  const { PROXY_SERVER, PROXY_USERNAME, PROXY_PASSWORD } = process.env;
  if (!PROXY_SERVER || !PROXY_USERNAME || !PROXY_PASSWORD) {
    throw new Error('Set PROXY_SERVER, PROXY_USERNAME and PROXY_PASSWORD.');
  }
  const upstream = new URL(PROXY_SERVER);
  if (!['http:', 'https:'].includes(upstream.protocol) ||
      upstream.username || upstream.password || upstream.search ||
      upstream.hash || (upstream.pathname && upstream.pathname !== '/')) {
    throw new Error('Use an HTTP(S) proxy server URL without credentials, a path or a query.');
  }
  upstream.username = PROXY_USERNAME;
  upstream.password = PROXY_PASSWORD;
  const server = new Server({
    host: '127.0.0.1',
    port: 0,
    prepareRequestFunction: () => ({ upstreamProxyUrl: upstream.href }),
  });
  await server.listen();
  return {
    url: `http://127.0.0.1:${server.port}`,
    close: () => server.close(true),
  };
}

Download octoparse_bridge.mjs

octoparse_bridge.mjs

import { createProxyBridge } from './proxy_bridge.mjs';

const bridge = await createProxyBridge();
console.log(bridge.url);
for (const signal of ['SIGINT', 'SIGTERM']) {
  process.once(signal, async () => { await bridge.close(); process.exit(0); });
}

Enable your own HTTP proxy
Open Task Settings → Anti-blocking. Enable Access website via proxies, choose Use my own proxies and open Configure.
For the bridge, enter 127.0.0.1 followed by the actual printed port, one entry per line. Do not include the http:// prefix, upstream username or password in Octoparse’s IP:port field. Confirm and save.
Switch every rotates entries in Octoparse’s list. With a single bridge entry, it does not change the upstream IP or sticky-session lifetime. Upstream session/rotation settings are controlled by the connection you supplied.
Task-editing proxy settings are separate in current releases. Enable the intended editing proxy as well if the page must load through it while you configure the task.
Octoparse IP proxy entry
```
127.0.0.1:PORT_PRINTED_BY_BRIDGE
```
The document’s proxy-entry screenshot shows a legacy credential-string example. Current Octoparse custom entries use only IP:port. For authenticated upstreams, enter the loopback bridge address described above; do not copy that pictured credential string.
Document interface reference: open Task Settings. View full size
Document interface reference: open Anti-blocking. View full size
Document interface reference: choose your own proxies. View full size
Document interface reference: the document’s legacy proxy-entry example. View full size
Build, run locally and export
Select the required page elements, use the appropriate text extraction action, and confirm the fields in Data Preview. Add the next-page workflow only after the first page extracts correctly.
Complete the task and choose Run on your device / local run. Select Standard Mode where your release offers it. Do not select Cloud for this custom-proxy recipe.
Inspect a small result and the proxy route before scaling, then export in a format supported by your edition. A proxy does not guarantee that a target will accept a request or remove CAPTCHAs. Stop the bridge with Ctrl+C when the run is finished.
The extraction and export screens are supplied examples, not measurements from a live Crawlhub run.
Document interface reference: select similar elements. View full size
Document interface reference: extract text into Data Preview. View full size
Document interface reference: configure next-page navigation. View full size
Document interface reference: complete the workflow. View full size
Document interface reference: choose a local Standard Mode run. View full size
Document interface reference: the document’s export-output example. View full size