API Documentation

This API allows you to scrape web pages. You can send either a POST or GET request to retrieve the HTML content of a specified URL. The API supports both simple and recursive scraping.

Request Examples

cURL Example

# Simple scrape (POST)
curl -X POST https://scrape.rishwanth.me/api/scrape \
-H "Content-Type: application/json" \
-d '{"url": "https://example.com", "recursive": false}'

# Simple scrape (GET)
curl "https://scrape.rishwanth.me/api/scrape?url=https://example.com&recursive=false"

# Recursive scrape (POST)
curl -X POST https://scrape.rishwanth.me/api/scrape \
-H "Content-Type: application/json" \
-H "x-api-key: YOUR_API_KEY" \
-d '{"url": "https://example.com", "recursive": true}'

# Recursive scrape (GET)
curl "https://scrape.rishwanth.me/api/scrape?url=https://example.com&recursive=true" \
-H "x-api-key: YOUR_API_KEY"

API Endpoints

  • POST /api/scrape - Send a JSON body with 'url' and 'recursive' fields
  • GET /api/scrape - Send the 'url' and 'recursive' as query parameters

Parameters

  • url (required): The URL of the webpage to scrape
  • recursive (optional): Set to 'true' for recursive scraping, 'false' for simple scraping (default is 'false')

Authentication

Recursive scraping requires authentication. Include your API key in the 'x-api-key' header for recursive scraping requests.