One constant now decides the whole index question

The indexable constant governed robots.txt and nothing else. That turned out to be half a job, and the half it left undone was the harmful half.

A Disallow: / keeps a crawler off the page. It does not remove the address from an index, because a crawler that is kept away never reads the noindex it would have found there. Meanwhile the head of that same dev host was still rendering <link rel=canonical href="https://dev.example.com/">, which does not merely fail to say "ignore me": it claims the dev address as the official one for content that also lives in production. Every dev and stage host on this server was doing that, and it was invisible because the only thing anybody ever checked was robots.txt.

So the constant now answers the question everywhere it comes up, and one of the three places does not need the resource at all:

Where	Without indexable
Every response the app renders	X-Robots-Tag: noindex
robots.txt	User-agent: * and Disallow: /
%seo->head	noindex,follow, and the canonical link is dropped

The header is set by the response, not by the resource. An app that never loads seo is covered too, which is what most demos and internal tools are. It reaches as far as PHP does and no further: a stylesheet or an image the web server hands over itself never passes through the app.

The second change is that the engine now knows an error page when it renders one. view() used to set the response status after the head had already been assembled, so the resource could not see it; the assignment moved up, and anything from 400 upwards is noindex on its own. Six apps here were writing that rule by hand, in three different spellings, on the line right after if (app::route()) return. They no longer have to.

A page that genuinely wants to be crawled but not indexed, which is the only reliable way to get an address removed, says so by inviting the crawl and letting the header do the work:

method %seo.robots => 'User-agent: *'.lf.'Allow: /'.lf

Engine 1.0. See the SEO chapter.