bb4e8899c4
Google insists on indexing pages so it can figure out if they can be removed from the index. see: https://support.google.com/webmasters/answer/6332384?hl=en This change ensures the we have special behavior for Googlebot where we allow indexing, but block the actual indexing via X-Robots-Tag |
||
|---|---|---|
| .. | ||
| components | ||
| fabricators | ||
| fixtures | ||
| helpers | ||
| import_export | ||
| initializers | ||
| integration | ||
| integrity | ||
| jobs | ||
| lib | ||
| mailers | ||
| models | ||
| multisite | ||
| requests | ||
| script/import_scripts | ||
| serializers | ||
| services | ||
| support | ||
| tasks | ||
| views/omniauth_callbacks | ||
| rails_helper.rb | ||
| swagger_helper.rb | ||