SC7 升级后,Sitecore 重建索引失败,没有 error/log

Sitecore reindexing fails with no error/log after SC7 upgrade

我花了几天时间进行 SC7 升级(从 Sitecore 6.6 开始),我 运行 遇到重建索引的问题(Sitecore 桌面 > 控制面板 > 索引 > 索引管理器 > 网络> 重建)。

我已经停止了我们的各种计划任务,并且正在使用 SC 的私有实例,因此没有用户或进程在途中徘徊。在重建之前,我停止了 IIS/worker 个进程并删除了文件系统上所有现有的索引文件。

索引编制将开始,向导会将进度更新为大约 15,000(共 20,000 个项目)。那时它会开始越来越慢,然后停止。有时 CPU 达到最大值并停留在那里。有时 RAM 已满。有时 w3wp 会因堆损坏错误而崩溃。通常(4/5 次)它只是停下来不再前进。

日志和爬虫日志似乎没有任何异常。有几条消息是关于无法调整大小的大型 MP4 视频,以及一些无法将其内容编入索引的二进制文件。 Sitecore 似乎只是停止并重新启动(一个新的日志文件已启动并包含标准 SC 'boot up' 消息)。

日志:

ManagedPoolThread #9 17:23:44 INFO  Job started: Index_Update_IndexName=sitecore_web_index
ManagedPoolThread #11 17:23:50 INFO  Job ended: Sitecore.Tasks.DatabaseAgent (units processed: )
ManagedPoolThread #11 17:24:55 INFO  Job started: Sitecore.Tasks.DatabaseAgent
ManagedPoolThread #11 17:24:55 INFO  Job ended: Sitecore.Tasks.DatabaseAgent (units processed: )
ManagedPoolThread #4 17:25:55 INFO  Scheduling.DatabaseAgent started. Database: web
ManagedPoolThread #4 17:25:55 INFO  Job ended: Sitecore.Tasks.DatabaseAgent (units processed: )
ManagedPoolThread #1 17:27:00 INFO  Job started: Sitecore.Tasks.DatabaseAgent
ManagedPoolThread #1 17:27:00 INFO  Scheduling.DatabaseAgent started. Database: web
ManagedPoolThread #1 17:27:00 INFO  Examining schedules (count: 0)
ManagedPoolThread #1 17:27:00 INFO  Job ended: Sitecore.Tasks.DatabaseAgent (units processed: )
12088 17:27:54 ERROR Could not resize image as it was larger than the maximum size allowed for memory processing. Media item: {0} [/sitecore/media library/Files/[MP4 film]?sc_database=web]
ManagedPoolThread #16 17:28:05 INFO  Job started: Sitecore.Tasks.DatabaseAgent
ManagedPoolThread #16 17:28:05 INFO  Scheduling.DatabaseAgent started. Database: web
ManagedPoolThread #16 17:28:05 INFO  Examining schedules (count: 0)
ManagedPoolThread #16 17:28:05 INFO  Job ended: Sitecore.Tasks.DatabaseAgent (units processed: )
ManagedPoolThread #16 17:29:05 INFO  Job started: Sitecore.Tasks.DatabaseAgent
ManagedPoolThread #16 17:29:05 INFO  Scheduling.DatabaseAgent started. Database: web
ManagedPoolThread #16 17:29:05 INFO  Examining schedules (count: 0)
ManagedPoolThread #16 17:29:05 INFO  Job ended: Sitecore.Tasks.DatabaseAgent (units processed: )
1668 17:29:13 ERROR Could not resize image as it was larger than the maximum size allowed for memory processing. Media item: {0} [/sitecore/media library/Files/[MP4 film]?sc_database=web]
1668 17:29:33 ERROR Could not resize image as it was larger than the maximum size allowed for memory processing. Media item: {0} [/sitecore/media library/Files/[MP4 film]?sc_database=web]
1668 17:29:43 ERROR Could not resize image as it was larger than the maximum size allowed for memory processing. Media item: {0} [/sitecore/media library/Files/[MP4 film]?sc_database=web]
ManagedPoolThread #16 17:30:06 INFO  Job started: Sitecore.Tasks.DatabaseAgent
ManagedPoolThread #16 17:30:06 INFO  Scheduling.DatabaseAgent started. Database: web
ManagedPoolThread #16 17:30:06 INFO  Examining schedules (count: 0)
ManagedPoolThread #16 17:30:06 INFO  Job ended: Sitecore.Tasks.DatabaseAgent (units processed: )
7780 17:30:50 ERROR Could not resize image as it was larger than the maximum size allowed for memory processing. Media item: {0} [/sitecore/media library/Files/[MP4 film]?sc_database=web]
ManagedPoolThread #7 17:31:06 INFO  Job started: Sitecore.Tasks.DatabaseAgent
ManagedPoolThread #7 17:31:06 INFO  Scheduling.DatabaseAgent started. Database: web
ManagedPoolThread #7 17:31:06 INFO  Examining schedules (count: 0)
ManagedPoolThread #7 17:31:06 INFO  Job ended: Sitecore.Tasks.DatabaseAgent (units processed: )
ManagedPoolThread #2 17:32:06 INFO  Job started: Sitecore.Tasks.DatabaseAgent
ManagedPoolThread #2 17:32:06 INFO  Scheduling.DatabaseAgent started. Database: web
ManagedPoolThread #2 17:32:06 INFO  Examining schedules (count: 0)
ManagedPoolThread #2 17:32:06 INFO  Job ended: Sitecore.Tasks.DatabaseAgent (units processed: )
Heartbeat 17:32:49 WARN  Sitecore has no necessary permissions for reading/creating counters.
Heartbeat 17:32:49 INFO  Health.PrivateBytes: 0
Heartbeat 17:32:49 INFO  Health.CacheInstances: 95
Heartbeat 17:32:49 INFO  Health.CacheTotalCount: 70,442
Heartbeat 17:32:49 INFO  Health.CacheTotalSize: 78,936,852
Heartbeat 17:32:49 WARN  Sitecore has no necessary permissions for reading/creating counters.
Heartbeat 17:32:51 WARN  Sitecore has no necessary permissions for reading/creating counters.
ManagedPoolThread #19 17:32:51 INFO  Job started: Sitecore.Tasks.DatabaseAgent
ManagedPoolThread #19 17:32:51 INFO  Scheduling.DatabaseAgent started. Database: core
ManagedPoolThread #19 17:32:51 INFO  Examining schedules (count: 0)
ManagedPoolThread #19 17:32:51 INFO  Job ended: Sitecore.Tasks.DatabaseAgent (units processed: )
ManagedPoolThread #3 17:32:51 INFO  Job started: Sitecore.Tasks.TaskDatabaseAgent
ManagedPoolThread #3 17:32:51 INFO  Processing tasks (count: 1)
ManagedPoolThread #3 17:32:51 INFO  Executing email reminder task
ManagedPoolThread #3 17:32:56 INFO  Job ended: Sitecore.Tasks.TaskDatabaseAgent (units processed: )
ManagedPoolThread #14 17:33:06 INFO  Job started: Sitecore.Tasks.DatabaseAgent
ManagedPoolThread #14 17:33:06 INFO  Scheduling.DatabaseAgent started. Database: web
ManagedPoolThread #14 17:33:06 INFO  Examining schedules (count: 0)
ManagedPoolThread #14 17:33:06 INFO  Job ended: Sitecore.Tasks.DatabaseAgent (units processed: )
ManagedPoolThread #12 17:34:06 INFO  Job started: Sitecore.Tasks.DatabaseAgent
ManagedPoolThread #12 17:34:06 INFO  Scheduling.DatabaseAgent started. Database: web
ManagedPoolThread #12 17:34:06 INFO  Examining schedules (count: 0)
ManagedPoolThread #12 17:34:06 INFO  Job ended: Sitecore.Tasks.DatabaseAgent (units processed: )
ManagedPoolThread #7 17:35:07 INFO  Job started: Sitecore.Tasks.DatabaseAgent
ManagedPoolThread #7 17:35:07 INFO  Scheduling.DatabaseAgent started. Database: web
ManagedPoolThread #7 17:35:07 INFO  Examining schedules (count: 0)
ManagedPoolThread #7 17:35:07 INFO  Job ended: Sitecore.Tasks.DatabaseAgent (units processed: )
ManagedPoolThread #18 17:36:07 INFO  Job started: Sitecore.Tasks.DatabaseAgent
ManagedPoolThread #18 17:36:07 INFO  Scheduling.DatabaseAgent started. Database: web
ManagedPoolThread #18 17:36:07 INFO  Examining schedules (count: 0)
ManagedPoolThread #18 17:36:07 INFO  Job ended: Sitecore.Tasks.DatabaseAgent (units processed: )
ManagedPoolThread #16 17:37:07 INFO  Job started: Sitecore.Tasks.DatabaseAgent
ManagedPoolThread #16 17:37:07 INFO  Scheduling.DatabaseAgent started. Database: web
ManagedPoolThread #16 17:37:07 INFO  Examining schedules (count: 0)
ManagedPoolThread #16 17:37:07 INFO  Job ended: Sitecore.Tasks.DatabaseAgent (units processed: )
ManagedPoolThread #8 17:37:52 INFO  Job started: Sitecore.Tasks.UrlAgent
ManagedPoolThread #8 17:37:52 INFO  Scheduling.UrlAgent started. Url: http://local/sitecore/service/keepalive.aspx
Heartbeat 17:37:53 WARN  Sitecore has no necessary permissions for reading/creating counters.
2080 17:38:25 INFO  Cache created: 'GeoIp' (max size: 1MB, running total: 799MB)
ManagedPoolThread #8 17:38:26 INFO  Scheduling.UrlAgent done (received: 426 bytes)
ManagedPoolThread #8 17:38:26 INFO  Job ended: Sitecore.Tasks.UrlAgent (units processed: )
ManagedPoolThread #0 17:38:31 INFO  Job started: Sitecore.Tasks.DatabaseAgent
ManagedPoolThread #0 17:38:31 INFO  Scheduling.DatabaseAgent started. Database: web
ManagedPoolThread #0 17:38:31 INFO  Examining schedules (count: 0)
ManagedPoolThread #0 17:38:31 INFO  Job ended: Sitecore.Tasks.DatabaseAgent (units processed: )
ManagedPoolThread #4 17:39:31 INFO  Job started: Sitecore.Tasks.DatabaseAgent
ManagedPoolThread #4 17:39:31 INFO  Scheduling.DatabaseAgent started. Database: web
ManagedPoolThread #4 17:39:31 INFO  Examining schedules (count: 0)
ManagedPoolThread #4 17:39:31 INFO  Job ended: Sitecore.Tasks.DatabaseAgent (units processed: )
ManagedPoolThread #3 17:40:31 INFO  Job started: Sitecore.Tasks.DatabaseAgent
ManagedPoolThread #3 17:40:31 INFO  Scheduling.DatabaseAgent started. Database: web
ManagedPoolThread #3 17:40:31 INFO  Examining schedules (count: 0)
ManagedPoolThread #3 17:40:31 INFO  Job ended: Sitecore.Tasks.DatabaseAgent (units processed: )
ManagedPoolThread #2 17:41:32 INFO  Job started: Sitecore.Tasks.DatabaseAgent
ManagedPoolThread #2 17:41:32 INFO  Scheduling.DatabaseAgent started. Database: web
ManagedPoolThread #2 17:41:32 INFO  Examining schedules (count: 0)
ManagedPoolThread #2 17:41:32 INFO  Job ended: Sitecore.Tasks.DatabaseAgent (units processed: )

抓取工具:

12332 17:22:50 INFO  [Index=sitecore_web_index] Initializing OnPublishEndAsynchronousStrategy.
12332 17:22:50 INFO  [Index=sitecore_web_index] Initializing SitecoreItemCrawler. DB:web / Root:/sitecore
ManagedPoolThread #9 17:23:44 WARN  [Index=sitecore_web_index] Reset Started
ManagedPoolThread #9 17:23:44 WARN  [Index=sitecore_web_index] Reset Ended
ManagedPoolThread #9 17:23:44 WARN  [Index=sitecore_web_index] Full Rebuild Started
9828 17:23:57 ERROR Could not compute value for ComputedIndexField: _content for indexable: sitecore://web/{2199D684-1240-4702-B1AD-98CA54A482CD}?lang=en&ver=1
Exception: System.Runtime.InteropServices.COMException
Message: Error HRESULT E_FAIL has been returned from a call to a COM component.
Source: mscorlib
   at System.Runtime.InteropServices.ComTypes.IPersistFile.Load(String pszFileName, Int32 dwMode)
   at Sitecore.ContentSearch.Extracters.IFilterTextExtraction.FilterLoader.LoadAndInitIFilter(String fileName, String extension)
   at Sitecore.ContentSearch.Extracters.IFilterTextExtraction.FilterReader..ctor(String fileName)
   at Sitecore.ContentSearch.ComputedFields.MediaItemIFilterTextExtractor.ComputeFieldValue(IIndexable indexable)
   at Sitecore.ContentSearch.ComputedFields.MediaItemContentExtractor.ComputeFieldValue(IIndexable indexable)
   at Sitecore.ContentSearch.LuceneProvider.LuceneDocumentBuilder.AddComputedIndexFields()

11652 17:24:18 ERROR Could not compute value for ComputedIndexField: _content for indexable: sitecore://web/{87B6F986-53D4-4D87-9531-6CE90F684DC8}?lang=en&ver=1
Exception: System.Runtime.InteropServices.COMException
Message: Exception from HRESULT: 0x8004170C
Source: mscorlib
   at System.Runtime.InteropServices.ComTypes.IPersistFile.Load(String pszFileName, Int32 dwMode)
   at Sitecore.ContentSearch.Extracters.IFilterTextExtraction.FilterLoader.LoadAndInitIFilter(String fileName, String extension)
   at Sitecore.ContentSearch.Extracters.IFilterTextExtraction.FilterReader..ctor(String fileName)
   at Sitecore.ContentSearch.ComputedFields.MediaItemIFilterTextExtractor.ComputeFieldValue(IIndexable indexable)
   at Sitecore.ContentSearch.ComputedFields.MediaItemContentExtractor.ComputeFieldValue(IIndexable indexable)
   at Sitecore.ContentSearch.LuceneProvider.LuceneDocumentBuilder.AddComputedIndexFields()

6316 17:24:53 INFO  [Index=sitecore_core_index] IntervalAsynchronousUpdateStrategy executing.
6316 17:24:53 INFO  [Index=sitecore_core_index] History engine is empty. Incremental rebuild returns
6268 17:25:55 INFO  [Index=sitecore_core_index] IntervalAsynchronousUpdateStrategy executing.
6268 17:25:55 INFO  [Index=sitecore_core_index] History engine is empty. Incremental rebuild returns
9736 17:26:55 INFO  [Index=sitecore_core_index] IntervalAsynchronousUpdateStrategy executing.
9736 17:26:55 INFO  [Index=sitecore_core_index] History engine is empty. Incremental rebuild returns
6316 17:27:50 INFO  [Index=sitecore_web_index] TimeIntervalCommitPolicy.ShouldCommit - Time Limit Exceeded, lastCommit=20/02/2015 17:22:50, count=13635
6316 17:27:50 INFO  [Index=sitecore_web_index] Committing: Add: 13634; Update:0; DeleteUnique: 0; DeleteGroup: 0
6316 17:27:50 INFO  [Index=sitecore_web_index] Committed
1748 17:27:56 INFO  [Index=sitecore_core_index] IntervalAsynchronousUpdateStrategy executing.
1748 17:27:56 INFO  [Index=sitecore_core_index] History engine is empty. Incremental rebuild returns
9980 17:28:55 INFO  [Index=sitecore_core_index] IntervalAsynchronousUpdateStrategy executing.
9980 17:28:55 INFO  [Index=sitecore_core_index] History engine is empty. Incremental rebuild returns

我的 Sitecore.ContentSearch.Lucene.DefaultIndexConfiguration.config 文件:

<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
  <sitecore>
    <contentSearch>
      <indexUpdateStrategies>
        <intervalAsyncCore type="Sitecore.ContentSearch.Maintenance.Strategies.IntervalAsynchronousStrategy, Sitecore.ContentSearch">
          <param desc="database">core</param>
          <param desc="interval">00:01:00</param>
          <CheckForThreshold>true</CheckForThreshold>
        </intervalAsyncCore>
        <intervalAsyncMaster type="Sitecore.ContentSearch.Maintenance.Strategies.IntervalAsynchronousStrategy, Sitecore.ContentSearch">
          <param desc="database">master</param>
          <param desc="interval">00:00:05</param>
          <CheckForThreshold>true</CheckForThreshold>
        </intervalAsyncMaster>
        <manual type="Sitecore.ContentSearch.Maintenance.Strategies.ManualStrategy, Sitecore.ContentSearch" />
        <onPublishEndAsync type="Sitecore.ContentSearch.Maintenance.Strategies.OnPublishEndAsynchronousStrategy, Sitecore.ContentSearch">
          <param desc="database">web</param>
          <CheckForThreshold>true</CheckForThreshold>
        </onPublishEndAsync>
        <rebuildAfterFullPublish type="Sitecore.ContentSearch.Maintenance.Strategies.RebuildAfterFullPublishStrategy, Sitecore.ContentSearch" />
        <remoteRebuild type="Sitecore.ContentSearch.Maintenance.Strategies.RemoteRebuildStrategy, Sitecore.ContentSearch" />
        <syncMaster type="Sitecore.ContentSearch.Maintenance.Strategies.SynchronousStrategy, Sitecore.ContentSearch">
          <param desc="database">master</param>
        </syncMaster>
      </indexUpdateStrategies>
      <databasePropertyStore type="Sitecore.ContentSearch.Maintenance.IndexDatabasePropertyStore, Sitecore.ContentSearch">
        <Key>$(1)</Key>
        <Database>core</Database>
      </databasePropertyStore>
      <configuration type="Sitecore.ContentSearch.LuceneProvider.LuceneSearchConfiguration, Sitecore.ContentSearch.LuceneProvider">
        <defaultIndexConfiguration type="Sitecore.ContentSearch.LuceneProvider.LuceneIndexConfiguration, Sitecore.ContentSearch.LuceneProvider">
          <indexAllFields>true</indexAllFields>
          <analyzer type="Sitecore.ContentSearch.LuceneProvider.Analyzers.PerExecutionContextAnalyzer, Sitecore.ContentSearch.LuceneProvider">
            <param desc="defaultAnalyzer" type="Sitecore.ContentSearch.LuceneProvider.Analyzers.DefaultPerFieldAnalyzer, Sitecore.ContentSearch.LuceneProvider">
              <param desc="defaultAnalyzer" type="Lucene.Net.Analysis.Standard.StandardAnalyzer, Lucene.Net">
                <param hint="version">Lucene_30</param>
              </param>
            </param>

...剪断...

           </analyzer>

          <fieldMap             type="Sitecore.ContentSearch.FieldMap, Sitecore.ContentSearch">
          ...snip...
          </fieldMap>
          <virtualFieldProcessors hint="raw:AddVirtualFieldProcessor">
          ...snip...
          </virtualFieldProcessors>
          <exclude hint="list:ExcludeTemplate">
            <BucketFolderTemplate>{ADB6CA4F-03EF-4F47-B9AC-9CE2BA53FF97}</BucketFolderTemplate>
          </exclude>
          <include hint="list:IncludeField">
            <fieldId>{8CDC337E-A112-42FB-BBB4-4143751E123F}</fieldId>
          </include>-->
          <exclude hint="list:ExcludeField">
          ...snip...
          </exclude>
          <fields hint="raw:AddCustomField">
            <field luceneName="__smallCreatedDate" storageType="yes" indexType="tokenized" format="yyyyMMdd">__created</field>
            <field luceneName="__smallUpdatedDate" storageType="yes" indexType="tokenized" format="yyyyMMdd">__updated</field>
          </fields>
          <fields hint="raw:RemoveSpecialFields">
            <remove type="both">AllTemplates</remove>
            <remove type="both">Created</remove>
            <remove type="both">Editor</remove>
            <remove type="both">Hidden</remove>
            <remove type="both">Icon</remove>
            <remove type="both">Links</remove>
            <remove type="both">Updated</remove>
          </fields>
          <fields hint="raw:AddComputedIndexField">
          ...snip...
          </fields>
          <fieldReaders type="Sitecore.ContentSearch.FieldReaders.FieldReaderMap, Sitecore.ContentSearch">
          ...snip...
          </fieldReaders>
          <indexFieldStorageValueFormatter type="Sitecore.ContentSearch.LuceneProvider.Converters.LuceneIndexFieldStorageValueFormatter, Sitecore.ContentSearch.LuceneProvider">
          ...snip...
          </indexFieldStorageValueFormatter>
          <indexDocumentPropertyMapper type="Sitecore.ContentSearch.LuceneProvider.DefaultLuceneDocumentTypeMapper, Sitecore.ContentSearch.LuceneProvider" />
        </defaultIndexConfiguration>
      </configuration>
    </contentSearch>
    <settings>
      <setting name="ContentSearch.CalibrateSizeByDeletes" value="true" />
      <setting name="ContentSearch.ConcurrentMergeSchedulerThreads" value="25" />
      <setting name="ContentSearch.IndexMergeFactor" value="10" />
      <setting name="ContentSearch.LuceneQueryClauseCount" value="1024" />
      <setting name="ContentSearch.MaxDocumentBufferSize" value="10000" />
      <setting name="ContentSearch.MaxMergeDocs" value="10000" />
      <setting name="ContentSearch.MaxMergeMB" value="512" />
      <setting name="ContentSearch.MinMergeMB" value="10" />
      <setting name="ContentSearch.RamBufferSize" value="512" />
      <setting name="ContentSearch.TermIndexInterval" value="256" />
      <setting name="ContentSearch.UseCompoundFile" value="false" />
      <setting name="ContentSearch.WaitForMerges" value="true" />      
    </settings>
  </sitecore>
</configuration>

是否有人知道可能导致索引器停止的原因?我试过:

我完全不明白为什么索引在 SC7 和 SC6.6 中会这样消失。与我们的 SC6.6 索引设置相比,全新 SC7 安装没有明显的配置差​​异。

任何能帮助我查明真相的人都应该得到一枚奖章!提前谢谢你:)

编辑

我的是 Caching.DisableCacheSizeLimits 设置是:

<setting name="Caching.DisableCacheSizeLimits" value="false" />

检查项目 {2199D684-1240-4702-B1AD-98CA54A482CD} 和 {87B6F986-53D4-4D87-9531-6CE90F684DC8}

这些是媒体项目吗?

查看有关媒体索引的博客

http://www.sitecore.net/learn/blogs/technical-blogs/john-west-sitecore-blog/posts/2013/04/sitecore-7-indexing-media-with-ifilters