CodeIgniter 404 路由 - 404 个断开的链接和 google 无法抓取站点但一切看起来都正常
CodeIgniter 404 Routing - 404 broken links and inability for google to crawl site BUT everything looks ok
我完全糊涂了。
我运行网站http://citylightstours.com
它建立在 CodeIgniter 平台上。
我在 Google Search Console 中注意到，我的网站只有 1 个页面在 Google 上编入索引。所有其他页面都有 404 错误，因此 google 没有列出它们。
因此我认为这是一个错误的站点地图，所以去 https://www.xml-sitemaps.com/ 生成了一个新站点地图。我输入了根 url，令我惊讶的是，生成的 xml 站点地图中只包含博客条目 - 我网站的主要页面 NONE 在那里！！
因此我去了另一个网站检查损坏的 links http://www.brokenlinkcheck.com/ 并且令我特别惊讶的是，我网站上的每个页面的状态都是 404 损坏 link。但是，单击那些 links 会显示一个有效页面。因此它们没有损坏 links，我可以很好地浏览网站。
因此，我不明白为什么自动机器人会附带一个 404 列表并且不会为网站编制索引，而当所有 link 似乎都有效时！？？？
有什么想法吗？
谢谢
更新：我也尝试从搜索控制台执行抓取和呈现，但在浏览器上显示的有效页面出现“未找到”错误！
更新 2：在 google 中执行 site:citylightstours.com 之后，我注意到唯一被索引的页面是博客页面。所有其他页面都已从索引中删除 - 知道为什么吗？？？
更新 3：其中一条评论表明它可能是 .htaccess 的问题，所以我 post 在这里希望有人发现一些东西。谢谢
更新 4：阅读此 post enter link description here 后，我认为可能是服务器 returns 一个 404 错误，实际页面代码作为客户 404 人类可读消息！ ！正如我所说，我使用 codeigniter，所以它一定与自定义 404 页面和路由有关。我不知道如何调试它，甚至不知道要看什么。谁能帮忙？...谢谢！
<IfModule mod_rewrite.c>
# Development
RewriteEngine On
RewriteBase /
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteCond !^(index\.php|images|scripts|styles|vendor|robots\.txt)
RewriteRule ^(.*)$ index.php/ [L]
</IfModule>
DirectoryIndex index.php
RewriteEngine on
RewriteCond !^(index\.php|images|css|js|robots\.txt|favicon\.ico)
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^(.*)$ ./index.php/ [L,QSA]
# ----------------------------------------------------------------------
# Better website experience for IE users
# ----------------------------------------------------------------------
<IfModule mod_setenvif.c>
<IfModule mod_headers.c>
BrowserMatch MSIE ie
Header set X-UA-Compatible "IE=Edge,chrome=1" env=ie
</IfModule>
</IfModule>
<IfModule mod_headers.c>
Header append Vary User-Agent
</IfModule>
# ----------------------------------------------------------------------
# Webfont access
# ----------------------------------------------------------------------
<FilesMatch "\.(ttf|otf|eot|woff|font.css)$">
<IfModule mod_headers.c>
Header set Access-Control-Allow-Origin "*"
</IfModule>
</FilesMatch>
# ----------------------------------------------------------------------
# Proper MIME type for all files
# ----------------------------------------------------------------------
# audio
AddType audio/ogg oga ogg
# video
AddType video/ogg .ogv
AddType video/mp4 .mp4
AddType video/webm .webm
# Proper svg serving. Required for svg webfonts on iPad
# twitter.com/FontSquirrel/status/14855840545
AddType image/svg+xml svg svgz
AddEncoding gzip svgz
# webfonts
AddType application/vnd.ms-fontobject eot
AddType font/truetype ttf
AddType font/opentype otf
AddType application/x-font-woff woff
# assorted types
AddType image/x-icon ico
AddType image/webp webp
AddType text/cache-manifest appcache manifest
AddType text/x-component htc
AddType application/x-chrome-extension crx
AddType application/x-xpinstall xpi
AddType application/octet-stream safariextz
# ----------------------------------------------------------------------
# gzip compression
# ----------------------------------------------------------------------
<IfModule mod_deflate.c>
<IfModule mod_setenvif.c>
<IfModule mod_headers.c>
SetEnvIfNoCase ^(Accept-EncodXng|X-cept-Encoding|X{15}|~{15}|-{15})$ ^((gzip|deflate)\s,?\s(gzip|deflate)?|X{4,13}|~{4,13}|-{4,13})$ HAVE_Accept-Encoding
RequestHeader append Accept-Encoding "gzip,deflate" env=HAVE_Accept-Encoding
</IfModule>
</IfModule>
<FilesMatch "^(?!.*\.ogg$|.*\.ogv$|.*\.mp4$).+" >
# html, txt, css, js, json, xml, htc:
<IfModule filter_module>
FilterDeclare COMPRESS
FilterProvider COMPRESS DEFLATE resp=Content-Type /text/(html|css|javascript|plain|x(ml|-component))/
FilterProvider COMPRESS DEFLATE resp=Content-Type /application/(javascript|json|xml|x-javascript)/
FilterChain COMPRESS
FilterProtocol COMPRESS change=yes;byteranges=no
</IfModule>
</FilesMatch>
# webfonts and svg:
<FilesMatch "\.(ttf|otf|eot|svg)$" >
SetOutputFilter DEFLATE
</FilesMatch>
</IfModule>
# ----------------------------------------------------------------------
# Expires headers (for better cache control)
# ----------------------------------------------------------------------
<IfModule mod_expires.c>
ExpiresActive on
# Perhaps better to whitelist expires rules? Perhaps.
ExpiresDefault "access plus 1 month"
# cache.appcache needs re-requests in FF 3.6 (thx Remy ~Introducing HTML5)
ExpiresByType text/cache-manifest "access plus 0 seconds"
# your document html
ExpiresByType text/html "access plus 0 seconds"
# data
ExpiresByType text/xml "access plus 0 seconds"
ExpiresByType application/xml "access plus 0 seconds"
ExpiresByType application/json "access plus 0 seconds"
# rss feed
ExpiresByType application/rss+xml "access plus 1 hour"
# favicon (cannot be renamed)
ExpiresByType image/x-icon "access plus 1 week"
# media: images, video, audio
ExpiresByType image/gif "access plus 1 month"
ExpiresByType image/png "access plus 1 month"
ExpiresByType image/jpg "access plus 1 month"
ExpiresByType image/jpeg "access plus 1 month"
ExpiresByType video/ogg "access plus 1 month"
ExpiresByType audio/ogg "access plus 1 month"
ExpiresByType video/mp4 "access plus 1 month"
ExpiresByType video/webm "access plus 1 month"
# htc files (css3pie)
ExpiresByType text/x-component "access plus 1 month"
# webfonts
ExpiresByType font/truetype "access plus 1 month"
ExpiresByType font/opentype "access plus 1 month"
ExpiresByType application/x-font-woff "access plus 1 month"
ExpiresByType image/svg+xml "access plus 1 month"
ExpiresByType application/vnd.ms-fontobject "access plus 1 month"
# css and javascript
ExpiresByType text/css "access plus 2 months"
ExpiresByType application/javascript "access plus 2 months"
ExpiresByType text/javascript "access plus 2 months"
<IfModule mod_headers.c>
Header append Cache-Control "public"
</IfModule>
</IfModule>
# ----------------------------------------------------------------------
# ETag removal
# ----------------------------------------------------------------------
FileETag None
# ----------------------------------------------------------------------
# Stop screen flicker in IE on CSS rollovers
# ----------------------------------------------------------------------
# The following directives stop screen flicker in IE on CSS rollovers - in
# combination with the "ExpiresByType" rules for images (see above). If
# needed, un-comment the following rules.
# BrowserMatch "MSIE" brokenvary=1
# BrowserMatch "Mozilla/4.[0-9]{2}" brokenvary=1
# BrowserMatch "Opera" !brokenvary
# SetEnvIf brokenvary 1 force-no-vary
RewriteEngine On
RewriteCond %{HTTP_HOST} !^citylightstours\.com$ [NC]
RewriteRule ^(.*)$ http://citylightstours.com/ [R=301,L]
RewriteCond %{HTTP_USER_AGENT} libwww-perl.*
RewriteRule .* ? [F,L]
已解决 - 站点中集成的 wordpress 博客正在为所有非 wordpress 页面（即 codeigniter 页面）设置 404 状态
index.php of CI 有以下代码需要注释掉
/*
*---------------------------------------------------------------
* WORDPRESS INTEGRATION
*---------------------------------------------------------------
* The ci_site_url function helps to avoid collision between WP & CI.
*/
//header("HTTP/1.0 200 OK");
define('WP_USE_THEMES', false);
require_once './blog/wp-blog-header.php';
add_filter('site_url', 'ci_site_url', 1);
function ci_site_url()
{
include(APPPATH.'/config/config.php');
return $config['base_url'];
}
我完全糊涂了。
我运行网站http://citylightstours.com
它建立在 CodeIgniter 平台上。
我在 Google Search Console 中注意到，我的网站只有 1 个页面在 Google 上编入索引。所有其他页面都有 404 错误，因此 google 没有列出它们。
因此我认为这是一个错误的站点地图，所以去 https://www.xml-sitemaps.com/ 生成了一个新站点地图。我输入了根 url，令我惊讶的是，生成的 xml 站点地图中只包含博客条目 - 我网站的主要页面 NONE 在那里！！
因此我去了另一个网站检查损坏的 links http://www.brokenlinkcheck.com/ 并且令我特别惊讶的是，我网站上的每个页面的状态都是 404 损坏 link。但是，单击那些 links 会显示一个有效页面。因此它们没有损坏 links，我可以很好地浏览网站。
因此，我不明白为什么自动机器人会附带一个 404 列表并且不会为网站编制索引，而当所有 link 似乎都有效时！？？？
有什么想法吗？
谢谢
更新：我也尝试从搜索控制台执行抓取和呈现，但在浏览器上显示的有效页面出现“未找到”错误！
更新 2：在 google 中执行 site:citylightstours.com 之后，我注意到唯一被索引的页面是博客页面。所有其他页面都已从索引中删除 - 知道为什么吗？？？
更新 3：其中一条评论表明它可能是 .htaccess 的问题，所以我 post 在这里希望有人发现一些东西。谢谢
更新 4：阅读此 post enter link description here 后，我认为可能是服务器 returns 一个 404 错误，实际页面代码作为客户 404 人类可读消息！ ！正如我所说，我使用 codeigniter，所以它一定与自定义 404 页面和路由有关。我不知道如何调试它，甚至不知道要看什么。谁能帮忙？...谢谢！
<IfModule mod_rewrite.c>
# Development
RewriteEngine On
RewriteBase /
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteCond !^(index\.php|images|scripts|styles|vendor|robots\.txt)
RewriteRule ^(.*)$ index.php/ [L]
</IfModule>
DirectoryIndex index.php
RewriteEngine on
RewriteCond !^(index\.php|images|css|js|robots\.txt|favicon\.ico)
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^(.*)$ ./index.php/ [L,QSA]
# ----------------------------------------------------------------------
# Better website experience for IE users
# ----------------------------------------------------------------------
<IfModule mod_setenvif.c>
<IfModule mod_headers.c>
BrowserMatch MSIE ie
Header set X-UA-Compatible "IE=Edge,chrome=1" env=ie
</IfModule>
</IfModule>
<IfModule mod_headers.c>
Header append Vary User-Agent
</IfModule>
# ----------------------------------------------------------------------
# Webfont access
# ----------------------------------------------------------------------
<FilesMatch "\.(ttf|otf|eot|woff|font.css)$">
<IfModule mod_headers.c>
Header set Access-Control-Allow-Origin "*"
</IfModule>
</FilesMatch>
# ----------------------------------------------------------------------
# Proper MIME type for all files
# ----------------------------------------------------------------------
# audio
AddType audio/ogg oga ogg
# video
AddType video/ogg .ogv
AddType video/mp4 .mp4
AddType video/webm .webm
# Proper svg serving. Required for svg webfonts on iPad
# twitter.com/FontSquirrel/status/14855840545
AddType image/svg+xml svg svgz
AddEncoding gzip svgz
# webfonts
AddType application/vnd.ms-fontobject eot
AddType font/truetype ttf
AddType font/opentype otf
AddType application/x-font-woff woff
# assorted types
AddType image/x-icon ico
AddType image/webp webp
AddType text/cache-manifest appcache manifest
AddType text/x-component htc
AddType application/x-chrome-extension crx
AddType application/x-xpinstall xpi
AddType application/octet-stream safariextz
# ----------------------------------------------------------------------
# gzip compression
# ----------------------------------------------------------------------
<IfModule mod_deflate.c>
<IfModule mod_setenvif.c>
<IfModule mod_headers.c>
SetEnvIfNoCase ^(Accept-EncodXng|X-cept-Encoding|X{15}|~{15}|-{15})$ ^((gzip|deflate)\s,?\s(gzip|deflate)?|X{4,13}|~{4,13}|-{4,13})$ HAVE_Accept-Encoding
RequestHeader append Accept-Encoding "gzip,deflate" env=HAVE_Accept-Encoding
</IfModule>
</IfModule>
<FilesMatch "^(?!.*\.ogg$|.*\.ogv$|.*\.mp4$).+" >
# html, txt, css, js, json, xml, htc:
<IfModule filter_module>
FilterDeclare COMPRESS
FilterProvider COMPRESS DEFLATE resp=Content-Type /text/(html|css|javascript|plain|x(ml|-component))/
FilterProvider COMPRESS DEFLATE resp=Content-Type /application/(javascript|json|xml|x-javascript)/
FilterChain COMPRESS
FilterProtocol COMPRESS change=yes;byteranges=no
</IfModule>
</FilesMatch>
# webfonts and svg:
<FilesMatch "\.(ttf|otf|eot|svg)$" >
SetOutputFilter DEFLATE
</FilesMatch>
</IfModule>
# ----------------------------------------------------------------------
# Expires headers (for better cache control)
# ----------------------------------------------------------------------
<IfModule mod_expires.c>
ExpiresActive on
# Perhaps better to whitelist expires rules? Perhaps.
ExpiresDefault "access plus 1 month"
# cache.appcache needs re-requests in FF 3.6 (thx Remy ~Introducing HTML5)
ExpiresByType text/cache-manifest "access plus 0 seconds"
# your document html
ExpiresByType text/html "access plus 0 seconds"
# data
ExpiresByType text/xml "access plus 0 seconds"
ExpiresByType application/xml "access plus 0 seconds"
ExpiresByType application/json "access plus 0 seconds"
# rss feed
ExpiresByType application/rss+xml "access plus 1 hour"
# favicon (cannot be renamed)
ExpiresByType image/x-icon "access plus 1 week"
# media: images, video, audio
ExpiresByType image/gif "access plus 1 month"
ExpiresByType image/png "access plus 1 month"
ExpiresByType image/jpg "access plus 1 month"
ExpiresByType image/jpeg "access plus 1 month"
ExpiresByType video/ogg "access plus 1 month"
ExpiresByType audio/ogg "access plus 1 month"
ExpiresByType video/mp4 "access plus 1 month"
ExpiresByType video/webm "access plus 1 month"
# htc files (css3pie)
ExpiresByType text/x-component "access plus 1 month"
# webfonts
ExpiresByType font/truetype "access plus 1 month"
ExpiresByType font/opentype "access plus 1 month"
ExpiresByType application/x-font-woff "access plus 1 month"
ExpiresByType image/svg+xml "access plus 1 month"
ExpiresByType application/vnd.ms-fontobject "access plus 1 month"
# css and javascript
ExpiresByType text/css "access plus 2 months"
ExpiresByType application/javascript "access plus 2 months"
ExpiresByType text/javascript "access plus 2 months"
<IfModule mod_headers.c>
Header append Cache-Control "public"
</IfModule>
</IfModule>
# ----------------------------------------------------------------------
# ETag removal
# ----------------------------------------------------------------------
FileETag None
# ----------------------------------------------------------------------
# Stop screen flicker in IE on CSS rollovers
# ----------------------------------------------------------------------
# The following directives stop screen flicker in IE on CSS rollovers - in
# combination with the "ExpiresByType" rules for images (see above). If
# needed, un-comment the following rules.
# BrowserMatch "MSIE" brokenvary=1
# BrowserMatch "Mozilla/4.[0-9]{2}" brokenvary=1
# BrowserMatch "Opera" !brokenvary
# SetEnvIf brokenvary 1 force-no-vary
RewriteEngine On
RewriteCond %{HTTP_HOST} !^citylightstours\.com$ [NC]
RewriteRule ^(.*)$ http://citylightstours.com/ [R=301,L]
RewriteCond %{HTTP_USER_AGENT} libwww-perl.*
RewriteRule .* ? [F,L]
已解决 - 站点中集成的 wordpress 博客正在为所有非 wordpress 页面（即 codeigniter 页面）设置 404 状态
index.php of CI 有以下代码需要注释掉
/*
*---------------------------------------------------------------
* WORDPRESS INTEGRATION
*---------------------------------------------------------------
* The ci_site_url function helps to avoid collision between WP & CI.
*/
//header("HTTP/1.0 200 OK");
define('WP_USE_THEMES', false);
require_once './blog/wp-blog-header.php';
add_filter('site_url', 'ci_site_url', 1);
function ci_site_url()
{
include(APPPATH.'/config/config.php');
return $config['base_url'];
}