CrawlFaster

What is CrawlFaster?

CrawlFaster is a workflow for eligible Google Indexing API notifications: sitemap import, batched URL_UPDATED requests, and submitted-request status tracking. A successful response is not an indexing guarantee.

What CrawlFaster does

CrawlFaster is the workflow tool at crawlfaster.com for site owners who send eligible Google Indexing API notifications and need a record of each request. The homepage describes it as a Google Indexing API tool for publishers and bloggers. The about page describes it as a technical workflow for service-account setup, request tracking, sitemap review, and status monitoring. It does not rank pages, and it does not tell Google that an ordinary article must be indexed.

You connect a Google Cloud service account. The homepage states that the private key is encrypted with AES-256-GCM before it is stored. CrawlFaster then mints an OAuth token and posts each URL to Google’s publish endpoint, https://indexing.googleapis.com/v3/urlNotifications:publish, with type URL_UPDATED. Google’s Using the Indexing API guide allows one URL in a single publish call, or up to 100 calls combined in one HTTP batch. The submitter in this codebase groups URLs into batches of 100, which matches that limit. Quota is still counted per URL: ten URLs in one batch still count as ten requests.

The queue is fed by sitemap import. The homepage says you can paste a sitemap.xml URL and CrawlFaster extracts the page URLs. You can also work from a URL list. A scheduled batch runs at 02:00 UTC and submits pending URLs for accounts that have a service account connected. A weekly re-inspection job runs Sunday at 06:00 UTC and calls Google’s URL Inspection API. The live pricing page says re-inspections do not count as URL submissions. A daily digest job is scheduled at 07:00 UTC.

Who it is for

The fit is a site owner who already knows which URLs are eligible for the Indexing API and wants the queue, the batch send, and the status table in one place. Google’s current guide says the Indexing API tells Google to update or remove job posting or livestreaming event pages, and that it can only be used to crawl pages with JobPosting structured data or BroadcastEvent embedded in a VideoObject. CrawlFaster’s about page repeats that limit and says each site owner must confirm eligibility, permissions, quota, and spam-policy compliance.

A newsroom, magazine, or blog that only publishes articles does not become eligible because it uses CrawlFaster. If you are checking whether a domain’s sitemap is reachable before you set up a service account, the homepage offers a free indexing health check at /health-check and says the report does not require an account. That check is a survey of the site. It is not an Indexing API submission.

Free accounts, as listed on the live pricing page, include the Google Indexing API workflow, Bing IndexNow, URL status tracking, and sitemap import. A Chrome extension and a WordPress plugin are not offered yet. Pro adds RSS feed monitoring, a syndication copy generator, CMS connectors for Webflow and Ghost, and priority email support. Email digest reports are not offered yet. Agency adds multi-site management, bulk CSV import and export, API access, and a priority support channel.

Plans and daily submission limits

Prices below are the figures published on https://crawlfaster.com/pricing and in the plan table in this repository. Free is $0 and includes 50 URL submissions per day. Pro is $19.99 per month, or $15.99 per month when annual billing is selected, and includes 200 URL submissions per day. Agency is $49.99 per month, or $39.99 per month when annual billing is selected, and includes 1,000 URL submissions per day. The pricing page shows an annual toggle marked save 20%. The pricing FAQ says Pro supports one primary site and Agency supports unlimited sites with multi-site management.

The same page says you can cancel from the billing portal and keep access until the end of the billing period. Refunds are described as processed within 48 to 72 hours depending on the bank, if you write within 7 days of the charge. Payment is by major credit and debit cards through Stripe. The FAQ says the Free plan is how you use core features, and that you upgrade to Pro when you need higher limits, RSS monitoring, or CMS connectors. It does not describe a separate Pro trial.

These caps are CrawlFaster plan limits. They are not Google’s quota. Google says the Indexing API provides a default quota for testing and that using the API requires you to request approval and quota. Google also says not to get around submission limits by using multiple accounts. Buying a higher CrawlFaster plan does not, by itself, raise the quota Google has approved for your cloud project. A URL submission, per the pricing FAQ, is each URL you submit to Google’s Indexing API.

What a successful request does not promise

Google answers a successful publish call with HTTP 200. For URL_UPDATED, that means Google may try to recrawl the URL soon. It does not mean the URL was crawled, added to the index, ranked, or shown in results on any clock. The notification-metadata endpoint reports the last time Google received an update or removal notification for that URL. It does not report when the URL was indexed. CrawlFaster’s about page states the same boundary: a successful API response records that Google received a notification.

If you remove a page, Google’s guide says the URL should already return 404 or 410, or the page should contain a noindex meta tag, before you send URL_DELETED. CrawlFaster’s publish path used for the queue sends URL_UPDATED. Removal still has to follow Google’s removal rules if you use that notification type.

Read the companion answers on how to get new pages noticed faster, CrawlFaster compared with typing URLs into Search Console, and when a publisher actually needs Indexing API access. Primary documentation is Using the Indexing API on Google Search Central.

Common questions

What is CrawlFaster?

CrawlFaster is a workflow for eligible Google Indexing API notifications. It imports sitemap URLs, sends batched publish requests with a Google Cloud service account, and tracks whether Google accepted each submission. It does not guarantee indexing or rankings.

How much does CrawlFaster cost?

The live pricing page lists Free at $0 with 50 URL submissions per day, Pro at $19.99 per month ($15.99 per month billed annually) with 200 per day, and Agency at $49.99 per month ($39.99 per month billed annually) with 1,000 per day.

Does an HTTP 200 mean Google indexed the URL?

No. Google says an HTTP 200 on an update notification means it may try to recrawl the URL soon. Indexing, ranking, and traffic are separate outcomes.

Related reading

Recommended on Amazon

As an Amazon Associate I earn from qualifying purchases.