API & Integrations

Automate URL submission from any CMS, workflow, or browser

Webhook Endpoint

POST URLs directly from any system

POSThttps://crawlfaster.com/api/webhook/submit
AuthorizationHeader

Bearer YOUR_API_KEY (generate in Settings)

Content-TypeHeader

application/json

urlsBody (array)

Array of URL strings to queue (required)

tagsBody (string)

Comma-separated tags for the URLs (optional)

curl
curl -X POST https://crawlfaster.com/api/webhook/submit \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"urls": ["https://example.com/new-article"]}'

Response format

json
// Success (200)
{ "queued": 1, "message": "1 URL queued" }

// Error (401)
{ "error": "Invalid or missing API key" }

// Error (400)
{ "error": "No valid URLs provided" }

Browser Bookmarklet

Submit the current page to CrawlFaster in one click

Drag the button below to your bookmarks bar. When you publish an eligible page, use the bookmark to queue it for your submission workflow—without copy-pasting.

Generate an API key in Settings → CMS Webhook Integration first — the bookmarklet uses it to authenticate.

Bookmarklet source code

javascript
javascript:(function(){var u=encodeURIComponent(location.href);fetch('https://crawlfaster.com/api/webhook/submit',{method:'POST',headers:{'Authorization':'Bearer YOUR_API_KEY','Content-Type':'application/json'},body:JSON.stringify({urls:[location.href]})}).then(r=>r.json()).then(d=>alert('CrawlFaster: '+(d.queued?d.queued+' URL queued':'Error: '+JSON.stringify(d)))).catch(e=>alert('CrawlFaster error: '+e.message));})();

WordPress Integration

Add this to your theme's functions.php or a custom plugin. Every time you publish or update a post, the URL is automatically queued in CrawlFaster.

php
// Add to your theme's functions.php or a plugin
add_action('publish_post', 'crawlfaster_notify', 10, 2);
function crawlfaster_notify($post_id, $post) {
    $url = get_permalink($post_id);
    wp_remote_post('https://crawlfaster.com/api/webhook/submit', [
        'headers' => [
            'Authorization' => 'Bearer YOUR_API_KEY',
            'Content-Type'  => 'application/json',
        ],
        'body' => json_encode(['urls' => [$url]]),
    ]);
}

Ghost CMS Integration

javascript
// In Ghost Admin → Settings → Integrations → Custom integrations
// Add a webhook for "Post published" pointing to:
// https://crawlfaster.com/api/webhook/submit
// Then use a Zapier/Make.com step or n8n to forward the URL with the Bearer token.

// Or use Ghost's custom code injection with fetch:
fetch('https://crawlfaster.com/api/webhook/submit', {
  method: 'POST',
  headers: {
    'Authorization': 'Bearer YOUR_API_KEY',
    'Content-Type': 'application/json'
  },
  body: JSON.stringify({ urls: [window.location.href] })
});

n8n / Make.com / Zapier

Use any automation platform with an HTTP Request node. Set the method to POST, add the Authorization header, and pass the URL from your trigger event.

n8n config
// n8n HTTP Request node
Method: POST
URL: https://crawlfaster.com/api/webhook/submit
Authentication: Header Auth
  Name: Authorization
  Value: Bearer YOUR_API_KEY
Body (JSON):
{
  "urls": ["{{ $json.url }}"],
  "tags": "n8n-auto"
}

Increase Your Daily Quota

Default is 200 URLs/day — you can request more

  1. Open Google Cloud Console → Indexing API → Quotas
  2. Find the "Publish/update URL requests per day" quota
  3. Click the pencil icon → enter your desired limit (e.g. 1,000)
  4. Submit the request — Google typically responds within 1–2 business days
  5. Once approved, update the limit in CrawlFaster's cron job settings if needed
Open Google Cloud Console Quota docs