bb4e8899c4
Google insists on indexing pages so it can figure out if they can be removed from the index. see: https://support.google.com/webmasters/answer/6332384?hl=en This change ensures the we have special behavior for Googlebot where we allow indexing, but block the actual indexing via X-Robots-Tag |
||
|---|---|---|
| .. | ||
| about | ||
| admin/backups | ||
| application | ||
| badges | ||
| categories | ||
| common | ||
| default | ||
| embed | ||
| exceptions | ||
| finish_installation | ||
| groups | ||
| invites | ||
| layouts | ||
| list | ||
| metadata | ||
| offline | ||
| posts | ||
| published_pages | ||
| qunit | ||
| robots_txt | ||
| safe_mode | ||
| search | ||
| session | ||
| static | ||
| tags | ||
| topics | ||
| user_api_keys | ||
| user_notifications | ||
| users | ||
| users_email | ||
| wizard | ||