Getting Started

Last updated 5 months ago

Getting Started

1. Welcome to Effortless Web Scraping

Welcome to the Rayobyte Web Scraping API! This powerful tool lets you scrape websites at scale without worrying about blocks, CAPTCHAs, or IP bans. It’s designed to be easy to use and flexible for both beginners and advanced developers.

➡️ Learn More:

2. What Makes This API a Game-Changer?

Say goodbye to web scraping headaches:

Reliable page loads even from challenging sites.
Built-in handling for CAPTCHAs, blocks, and rate limits.
Flexible data parsing options like XPath or CSS selectors.
Support for and .

Tip: For websites with specific structures or complex data needs, check out our Custom Modules for platforms like Google and Amazon. Contact support if you're interested.

3. Let’s Get You Started!

Step 1: Grab Your API Key

First, get your API key from your account dashboard. This key authenticates your requests and ensures they are processed correctly.

Step 2: Send Your First Request

Use the base URL format to request a page. Below are code samples in different languages:

curl 'https://api.scraping.rayobyte.com/?token=<API-KEY>&url=https://tools-api.rayobyte.com/free-ip-lookup/'

const axios = require('axios');

// Define the API URL and parameters
const url = 'https://api.scraping.rayobyte.com/';
const params = {
    token: '<API-KEY>',
    url: 'https://tools-api.rayobyte.com/free-ip-lookup/'
};

// Make the GET request
axios.get(url, { params })
    .then(response => {
        console.log("Response:", response.data);
    })
    .catch(error => {
        if (error.response) {
            console.error(`Error: ${error.response.status} - ${error.response.data}`);
        } else {
            console.error(`Error: ${error.message}`);
        }
    });

https://api.scraping.rayobyte.com/?token=<API-KEY>&url=https://tools-api.rayobyte.com/free-ip-lookup/

import requests

# Define the API URL and parameters
url = "https://api.scraping.rayobyte.com/"
params = {
    "token": "<API-KEY>",
    "url": "https://tools-api.rayobyte.com/free-ip-lookup/"
}

# Make the GET request
response = requests.get(url, params=params)

# Check and print the response
if response.status_code == 200:
    print("Response:", response.json())
else:
    print(f"Error: {response.status_code} - {response.text}")

Expected Result (JSON)

When you make a request, our API returns a structured JSON response containing all the key details you need. Here's a breakdown of the fields you'll see in the response:

{
  "status": "SUCCESS",
  "date": "Thu, 19 Dec 2024 07:56:04 GMT",
  "httpCode": 200,
  "headers": {
    "<Headers returned>"
  },
  "result": "<HTML content of the scraped page>"
}

Output from above API call

{
  "status": "SUCCESS",
  "date": "Thu, 19 Dec 2024 07:56:04 GMT",
  "httpCode": 200,
  "headers": {
    "server": "nginx/1.20.1",
    "content-length": "621",
    "access-control-max-age": "3600"
  },
  "result": {
    "ipVersion": 4,
    "ipAddress": "1.2.3.4",
    "latitude": -27.467541,
    "longitude": 153.028091,
    "countryName": "Australia",
    "countryCode": "AU",
    "timeZone": "+10:00",
    "zipCode": "4000",
    "cityName": "Brisbane",
    "regionName": "Queensland",
    "isProxy": false,
    "continent": "Oceania",
    "continentCode": "OC",
    "currency": {
      "code": "AUD",
      "name": "Australian Dollar"
    },
    "language": "English",
    "timeZones": [
      "Antarctica/Macquarie",
      "Australia/Adelaide",
      "Australia/Brisbane",
      "Australia/Broken_Hill",
      "Australia/Darwin",
      "Australia/Eucla",
      "Australia/Hobart",
      "Australia/Lindeman",
      "Australia/Lord_Howe",
      "Australia/Melbourne",
      "Australia/Perth",
      "Australia/Sydney"
    ],
    "tlds": [
      ".au"
    ]
  }

Explanation of the Fields:

status: Indicates whether the request was successful (SUCCESS) or encountered an issue (FAIL).
date: A timestamp of when the API processed your request.
httpCode: The HTTP status code of the target website, such as 200 for success or 404 for not found.
headers: A collection of HTTP headers returned by the target website, which may include details like server type, content length, or caching information.
result: The HTML content of the page you requested, which you can process further to extract the data you need.

4. Features You’ll Want to Explore

Make scraping smarter by adding these features to your requests:

5. Common Roadblocks? We’ve Got You.

Here’s how to solve some common issues:

Invalid API Key: Double-check your key or regenerate it in your dashboard.
Timeouts or Slow Responses: Adjust timeout settings (learn more here).

6. Ready to Level Up?

Take your scraping to the next level:

Google Module: Get structured SERP data for Google.
Amazon Module: Perfect for extracting product information.

➡️ Explore the Modules section for more advanced scraping modules. Contact support if you're interested.

Got a question?

NextFeatures

Last updated 5 months ago