HTML or JSON: choose the output that fits your pipeline

HTML and JSON solve different parts of a collection workflow. HTML preserves the page structure your parser needs. JSON carries named fields which another part of your application can consume directly.

The choice depends on who owns extraction and how much source context you need to retain. It is common for a pipeline to use both.

Choose HTML when you need control over parsing

HTML lets you select the elements your application needs: a product title, a price, a table, links or article text. It also gives you a source to re-parse when your field definitions change.

This is useful when a page contains more than one relevant record or when your application needs a custom extraction rule. Keep parsing code separate from the retrieval client so a selector change does not also change the way you connect to the page.

The Crawlhub Web Scraping API focuses on retrieving a public page and returning its HTML. Your code decides how to parse and use that content.

Choose JSON when you want a record contract

A JSON response is useful when the next step expects named fields. Search data, for example, can be represented as ranked organic results, ads and AI Overview content, with URLs attached to the entries.

The Crawlhub SERP API is organized around Google search results. An application can work with the result fields instead of maintaining its own search-page parser.

Not every query returns every type of result. Write client code that distinguishes an absent optional section from a failed request. An empty list of ads should not be treated as a broken organic-results response.

Keep the source with the fields

Structured output does not remove the need for provenance. Retain the requested URL or search query, location, device context where relevant and retrieval time. If an answer or alert depends on a record, those fields help you trace it back.

For AI workflows, send the useful text to the model together with its source URL. Keep your own record of the retrieval so a later answer can be checked against the version your application actually collected.

Handle failure explicitly

Use timeouts, check the HTTP status and validate the fields required by your application. A request error, malformed JSON and a missing price are three different failures. Give each one a state your code can inspect.

Retries should have a limit. Repeating an invalid request indefinitely does not repair a bad URL or a changed page structure. Capture enough context to diagnose the failure without logging credentials.

A practical starting point

Pick one page or one query. Write down the fields the next step needs and the context required to explain them. Choose HTML if you need to build that extraction yourself; choose JSON when the response already supplies the right record.

Once the first record is sound, add pagination, scheduling and storage. The output format should support that workflow, rather than dictate it.

Read the Crawlhub documentation for connection and client examples.