✅ – extracts links from JS, CSS, PDF, images, and even onclick events. ✅ Custom extraction – use XPath or CSS selectors to pull specific link attributes. ✅ Link score – counts inbound/outbound links per URL, anchor text distribution. ✅ Redirect chain analysis – visualizes chains up to your specified limit. ✅ Hreflang & pagination link validation – checks alternate/prev/next links. ✅ Speed – 18.4 is 20–30% faster than 17.x on large sites (200k+ URLs).

He installed the "patched" 18.4. The frog icon turned green, and for a moment, Leo felt like an SEO god. He plugged in the client’s URL and hit "Start." The spider began its work, crawling through 10,000, 20,000, 30,000 pages. Leo watched the status codes roll in: 200s, 301s, and a sea of 404s he could finally report. But then, the "neverb" link’s true price was revealed.

With the proliferation of JavaScript frameworks like React and Angular, the source code of a page often differs significantly from the rendered DOM. Screaming Frog 18.4 utilizes a Chromium-based rendering engine to bridge this gap. It can identify links that exist only after JavaScript execution. This exposes a different type of "neverb" link: one that exists for users but is invisible to legacy crawlers, highlighting critical discrepancies in how search engines might index the site versus how users experience it.

Screaming Frog SEO Spider is a foundational desktop-based tool for technical SEO audits

To run this audit weekly:

When the Screaming Frog SEO Spider encounters a URL with a "184 NeverB Link" error, it means that the URL has a "nofollow" or "noreferrer" attribute in its HTML code. The "184" specifically refers to the HTTP status code, which indicates that the URL is not crawlable.

| Step | Action | |------|--------| | 1 | Click the tab | | 2 | Add filter: "Status Code" is "200 OK" | | 3 | Add filter: "Blocked by robots.txt" is "No" | | 4 | Add filter: "NoFollow" is "False" |