交换核心（重建索引 -> 上线）后我的从机损坏了

问题描述 投票：0回答：1

我在这里遇到了一个非常奇怪的问题，我已经按照我的设想设置了我的 solr 架构here 并且一切正常，除了我的 Solr 从机损坏。 （测试）设置如下：

Solr 设置

  • 一个 Solr master 有两个核心（reindex 和 live）
  • 一个 Solr slave 有两个核心（reindex 和 live）
  • 轮询间隔设置为 00:00:02
  • 优化或提交后复制
  • 在 Ubuntu (11.10) 版本 1.4.0 下运行 Solr。

过程

整个过程是这样的：

  1. 我们擦除 master reindex core 上的索引
  2. 我们在主服务器上为重建索引核心禁用复制
  3. 我们启动 DataImportHandler 以 10.000 个文档的形式导入 MySQL 数据（以防止长时间的 mysql 锁定）
  4. DataImportHandler 完成后（400 万条记录）我们在主服务器上启用复制
  5. slaves 将从 master 获取新的 reindex 核心
  6. Once fetching is done 通过查看 'isReplicating' 属性，我们发出 core swap

我们假设 live core 在完全重建后有 5000 个文档，reindex core 有 20000 个文档。

（错误的）结果

交换后的结果是：

Master: Live (20000), Reindex (5000) 从站：Live (20000)，Reindex (20000)

看起来slave将他的reindex core复制到live core但是不像master那样交换live core和reindex core。问题是我猜奴隶认为它比主人更新，结果是复制在重建索引核心上死了。

以下是发布 SWAP 后的一些日志消息：

13-mrt-2012 14:29:46 org.apache.solr.handler.SnapPuller fetchLatestIndex
INFO: Master's version: 1328526603592, generation: 332
13-mrt-2012 14:29:46 org.apache.solr.handler.SnapPuller fetchLatestIndex
INFO: Slave's version: 1328526603596, generation: 336
13-mrt-2012 14:29:46 org.apache.solr.handler.SnapPuller fetchLatestIndex
INFO: Starting replication process
13-mrt-2012 14:29:46 org.apache.solr.handler.SnapPuller fetchLatestIndex
INFO: Master's version: 1328526603596, generation: 336
13-mrt-2012 14:29:46 org.apache.solr.handler.SnapPuller fetchLatestIndex
INFO: Slave's version: 1328526603592, generation: 332
13-mrt-2012 14:29:46 org.apache.solr.handler.SnapPuller fetchLatestIndex
INFO: Starting replication process
13-mrt-2012 14:29:46 org.apache.solr.handler.SnapPuller fetchLatestIndex
INFO: Number of files in latest index in master: 1
13-mrt-2012 14:29:46 org.apache.solr.handler.SnapPuller fetchLatestIndex
INFO: Number of files in latest index in master: 11
13-mrt-2012 14:29:46 org.apache.solr.handler.SnapPuller downloadIndexFiles
INFO: Skipping download for /usr/share/solr/data/index/_ct.tvx
13-mrt-2012 14:29:46 org.apache.solr.handler.SnapPuller downloadIndexFiles
INFO: Skipping download for /usr/share/solr/data/index/_ct.fdt
13-mrt-2012 14:29:46 org.apache.solr.handler.SnapPuller downloadIndexFiles
INFO: Skipping download for /usr/share/solr/data/index/_ct.frq
13-mrt-2012 14:29:46 org.apache.solr.handler.SnapPuller downloadIndexFiles
INFO: Skipping download for /usr/share/solr/data/index/segments_9c
13-mrt-2012 14:29:46 org.apache.solr.handler.SnapPuller downloadIndexFiles
INFO: Skipping download for /usr/share/solr/data/index/_ct.fdx
13-mrt-2012 14:29:46 org.apache.solr.handler.SnapPuller downloadIndexFiles
INFO: Skipping download for /usr/share/solr/data/index/_ct.prx
13-mrt-2012 14:29:46 org.apache.solr.handler.SnapPuller downloadIndexFiles
INFO: Skipping download for /usr/share/solr/data/index/_ct.tii
13-mrt-2012 14:29:46 org.apache.solr.handler.SnapPuller downloadIndexFiles
INFO: Skipping download for /usr/share/solr/data/index/_ct.tvd
13-mrt-2012 14:29:46 org.apache.solr.handler.SnapPuller downloadIndexFiles
INFO: Skipping download for /usr/share/solr/data/index/_ct.tis
13-mrt-2012 14:29:46 org.apache.solr.handler.SnapPuller$FileFetcher fetchPackets
WARNING: No content recieved for file: {name=segments_98, lastmodified=0, size=0}
13-mrt-2012 14:29:46 org.apache.solr.handler.SnapPuller downloadIndexFiles
INFO: Skipping download for /usr/share/solr/data/index/_ct.tvf
13-mrt-2012 14:29:46 org.apache.solr.handler.SnapPuller downloadIndexFiles
INFO: Skipping download for /usr/share/solr/data/index/_ct.fnm
13-mrt-2012 14:29:46 org.apache.solr.handler.SnapPuller fetchLatestIndex
INFO: Total time taken for download : 0 secs
13-mrt-2012 14:29:46 org.apache.solr.update.DirectUpdateHandler2 commit
INFO: start commit(optimize=false,waitFlush=true,waitSearcher=true,expungeDeletes=false)
13-mrt-2012 14:29:46 org.apache.solr.handler.SnapPuller fetchLatestIndex
INFO: Total time taken for download : 0 secs
13-mrt-2012 14:29:46 org.apache.solr.handler.SnapPuller modifyIndexProps
INFO: New index installed. Updating index properties...
13-mrt-2012 14:29:46 org.apache.solr.update.DirectUpdateHandler2 commit
INFO: start commit(optimize=false,waitFlush=true,waitSearcher=true,expungeDeletes=false)
13-mrt-2012 14:29:46 org.apache.solr.search.SolrIndexSearcher <init>
INFO: Opening Searcher@598d00 main
13-mrt-2012 14:29:46 org.apache.solr.search.SolrIndexSearcher warm
INFO: autowarming Searcher@598d00 main from Searcher@189c12a main
    fieldValueCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
13-mrt-2012 14:29:46 org.apache.solr.search.SolrIndexSearcher warm
INFO: autowarming result for Searcher@598d00 main
    fieldValueCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
13-mrt-2012 14:29:46 org.apache.solr.search.SolrIndexSearcher warm
INFO: autowarming Searcher@598d00 main from Searcher@189c12a main
    filterCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=34,warmupTime=1,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
13-mrt-2012 14:29:46 org.apache.solr.update.DirectUpdateHandler2 commit
INFO: end_commit_flush
13-mrt-2012 14:29:46 org.apache.solr.handler.ReplicationHandler doFetch
SEVERE: SnapPull failed 
org.apache.solr.common.SolrException: Index fetch failed : 
    at org.apache.solr.handler.SnapPuller.fetchLatestIndex(SnapPuller.java:329)
    at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:264)
    at org.apache.solr.handler.SnapPuller$1.run(SnapPuller.java:159)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
    at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351)
    at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:165)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:267)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
    at java.lang.Thread.run(Thread.java:679)
Caused by: java.lang.RuntimeException: java.io.IOException: read past EOF
    at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1069)
    at org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.java:418)
    at org.apache.solr.handler.SnapPuller.doCommit(SnapPuller.java:467)
    at org.apache.solr.handler.SnapPuller.fetchLatestIndex(SnapPuller.java:319)
    ... 10 more
Caused by: java.io.IOException: read past EOF
    at org.apache.lucene.store.BufferedIndexInput.refill(BufferedIndexInput.java:151)
    at org.apache.lucene.store.BufferedIndexInput.readByte(BufferedIndexInput.java:38)
    at org.apache.lucene.store.ChecksumIndexInput.readByte(ChecksumIndexInput.java:36)
    at org.apache.lucene.store.IndexInput.readInt(IndexInput.java:68)
    at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:243)
    at org.apache.lucene.index.DirectoryReader$1.doBody(DirectoryReader.java:76)
    at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:683)
    at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:73)
    at org.apache.lucene.index.IndexReader.open(IndexReader.java:476)
    at org.apache.lucene.index.IndexReader.open(IndexReader.java:403)
    at org.apache.solr.core.StandardIndexReaderFactory.newReader(StandardIndexReaderFactory.java:38)
    at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1058)
    ... 13 more
13-mrt-2012 14:29:46 org.apache.solr.search.SolrIndexSearcher warm
INFO: autowarming result for Searcher@598d00 main
    filterCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=34,warmupTime=10,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
13-mrt-2012 14:29:46 org.apache.solr.core.SolrCore registerSearcher
INFO: [live] Registered new searcher Searcher@598d00 main
13-mrt-2012 14:29:46 org.apache.solr.search.SolrIndexSearcher close
INFO: Closing Searcher@189c12a main
    fieldValueCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
    filterCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=34,warmupTime=1,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
13-mrt-2012 14:29:46 org.apache.solr.handler.SnapPuller doCommit
INFO: Force open index writer to make sure older index files get deleted
13-mrt-2012 14:29:46 org.apache.solr.handler.ReplicationHandler doFetch
SEVERE: SnapPull failed 
org.apache.solr.common.SolrException: Index fetch failed : 
    at org.apache.solr.handler.SnapPuller.fetchLatestIndex(SnapPuller.java:329)
    at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:264)
    at org.apache.solr.handler.SnapPuller$1.run(SnapPuller.java:159)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
    at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351)
    at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:165)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:267)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
    at java.lang.Thread.run(Thread.java:679)
Caused by: java.io.IOException: read past EOF
    at org.apache.lucene.store.BufferedIndexInput.refill(BufferedIndexInput.java:151)
    at org.apache.lucene.store.BufferedIndexInput.readByte(BufferedIndexInput.java:38)
    at org.apache.lucene.store.ChecksumIndexInput.readByte(ChecksumIndexInput.java:36)
    at org.apache.lucene.store.IndexInput.readInt(IndexInput.java:68)
    at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:243)
    at org.apache.lucene.index.SegmentInfos$1.doBody(SegmentInfos.java:309)
    at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:683)
    at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:517)
    at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:306)
    at org.apache.lucene.index.IndexWriter.init(IndexWriter.java:1596)
    at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:1418)
    at org.apache.solr.update.SolrIndexWriter.<init>(SolrIndexWriter.java:191)
    at org.apache.solr.update.UpdateHandler.createMainIndexWriter(UpdateHandler.java:98)
    at org.apache.solr.update.DirectUpdateHandler2.openWriter(DirectUpdateHandler2.java:173)
    at org.apache.solr.update.DirectUpdateHandler2.forceOpenWriter(DirectUpdateHandler2.java:376)
    at org.apache.solr.handler.SnapPuller.doCommit(SnapPuller.java:471)
    at org.apache.solr.handler.SnapPuller.fetchLatestIndex(SnapPuller.java:319)
    ... 10 more

如果我手动完成所有操作，直到第 6 步一切正常。核心交换后一切都崩溃了。

我希望有人能帮助我或告诉我我做错了什么。当你进行核心交换时是否应该禁用复制？有什么建议吗？

architecture solr lucene high-availability dataimporthandler
1个回答
0
投票

我设法解决了这个问题，问题是我的两个核心都使用相同的 dataDir。通过将 solr.xml 文件中的 dataDir 设置为两个不同的目录解决了这个问题！

最新问题
© www.soinside.com 2019 - 2024. All rights reserved.