Supported setups
Crawlmend works on sites whose source code lives in a GitHub repository. It can crawl any public site, but a fix needs a repository to open a pull request against. This page is explicit about what is and is not covered so you can tell quickly whether Crawlmend fits.
How support is tiered
- Framework-aware fixes. Crawlmend carries conventions for how the framework sets page metadata and where page files live, so a generated change targets the right file and API.
- Detected, generic fixes. Crawlmend recognises the stack but has no framework-specific metadata conventions. Fixes are generated from the file contents and still go through the same review.
- Not supported. There is no repository to change, so Crawlmend can report problems but cannot open a pull request.
Framework-aware
| Framework | How metadata is handled |
|---|---|
| Next.js (App Router) | Edits the exported metadata object or generateMetadata(), and understands values inherited from a parent layout. Pages Router is not covered yet. |
| Nuxt | Uses useSeoMeta() (or useHead()) in the page's .vue file. |
| SvelteKit | Edits <svelte:head> in +page.svelte, and maps routes from the directory name. |
| Remix / React Router v7 | Edits the route's exported meta function. |
| Gatsby | Edits the exported Head component. |
| Astro | Knows that pages usually delegate their tags to a shared layout component. |
| Angular | Uses the Title and Meta services. |
Detected, generic fixes
Crawlmend also recognises WordPress, Shopify, Hugo, Jekyll, Eleventy, Docusaurus, VuePress, VitePress, Django, Flask, Laravel, Symfony, Rails, Drupal, Joomla, ExpressionEngine, Ghost, Craft, Statamic and .NET projects. For these there are no framework-specific metadata conventions yet, so expect more review effort, and expect some opportunities to come back as findings without an automatic fix.
What "compile-checked" means
Before a pull request is opened, Crawlmend applies the proposed change to a throwaway clone of your branch and checks that it does not break the build. What that check can be depends on the repository:
| Repository | Check that runs |
|---|---|
Has a tsconfig.json | tsc --noEmit before and after the change. Only errors the change introduces count; errors already in your repo are ignored. |
| PHP, Python or Ruby file edited | A real syntax check of the edited file with that language's own tooling. |
| Markdown or MDX edited | A front-matter syntax check. |
| Anything else | A lenient best-effort pass. This is not a compile guarantee. |
If the environment cannot run the check at all (for example the clone or install fails), Crawlmend blocks the change rather than letting an unverified one through. Dependencies are installed with install scripts disabled.
Not supported yet
- Sites with no code repository, including hosted site builders where you cannot edit source files.
- Repositories hosted outside GitHub. GitLab and Bitbucket are not supported.
- Next.js Pages Router metadata (
next/head) is not covered by framework conventions yet. - Fixes that need design or copywriting judgement, such as rewriting a page's body content, are findings you act on, not automatic changes.
Crawl limits
- A crawl covers up to 20,000 pages on the host you enter.
- A crawl stops after 15 minutes, or 20 minutes when JavaScript rendering is on. Pages reached by then are still reported.
- Each account can run two crawls at once.
- Individual page requests time out after 15 seconds.