Failed to start Sonar Webservice

elasticsearch sonarqube sharding sonarqube5.1 elasticsearch-indices
sai picture sai · Aug 3, 2015 · Viewed 7k times · Source

I am working on Sonarqube 5.1. It used to work fine earlier, later i got this no space left on device error.So i have deleted some unwanted files and restarted the service. Now i can see that the service status is running, but i cannot access the webpage(since sonar webservice is not started) I have checked the logs and i find the below error.

2015.08.03 10:58:56 INFO   es[o.elasticsearch.node]  [sonar-1438592314778] started
2015.08.03 10:59:03 INFO   es[o.e.gateway]  [sonar-1438592314778] recovered [6] indices into cluster_state
2015.08.03 11:00:44 WARN   es[o.e.indices.cluster]  [sonar-1438592314778] [sourcelines][0] failed to start shard
org.elasticsearch.index.gateway.IndexShardGatewayRecoveryException: [sourcelines][0] failed to recover shard
at     org.elasticsearch.index.gateway.local.LocalIndexShardGateway.recover(LocalIndexShardGateway.java:287) ~[elasticsearch-1.4.4.jar:na]
at  org.elasticsearch.index.gateway.IndexShardGatewayService$1.run(IndexShardGatewayService.java:132) ~[elasticsearch-1.4.4.jar:na]
at  java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:114 2) [na:1.8.0_40]
at  java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_40]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_40]
Caused by: org.elasticsearch.index.translog.TranslogCorruptedException: translog corruption while reading from stream
at  org.elasticsearch.index.translog.ChecksummedTranslogStream.read(ChecksummedTranslogStream.java:70) ~[elasticsearch-1.4.4.jar:na]
at  org.elasticsearch.index.gateway.local.LocalIndexShardGateway.recover(LocalIndexShardGateway.java:257) ~[elasticsearch-1.4.4.jar:na]
... 4 common frames omitted
Caused by: org.elasticsearch.ElasticsearchException: failed to read [sourceline][22c2d1d1-8e73-47c0-b9fa-f8f5cc96b93f_796]
at org.elasticsearch.index.translog.Translog$Index.readFrom(Translog.java:520) ~[elasticsearch-1.4.4.jar:na]
at  org.elasticsearch.index.translog.ChecksummedTranslogStream.read(ChecksummedTranslogStream.java:68) ~[elasticsearch-1.4.4.jar:na]
... 5 common frames omitted
Caused by: org.elasticsearch.ElasticsearchIllegalArgumentException: No version type match [105]
at org.elasticsearch.index.VersionType.fromValue(VersionType.java:307) ~[elasticsearch-1.4.4.jar:na]
at org.elasticsearch.index.translog.Translog$Index.readFrom(Translog.java:517) ~[elasticsearch-1.4.4.jar:na]
... 6 common frames omitted
2015.08.03 11:00:44 WARN   es[o.e.c.action.shard]  [sonar-1438592314778] [sourcelines][0] sending failed shard for [sourcelines][0], node[5eRgYcVUTNCNLjhhzdiHGA], [P], s[INITIALIZING], indexUUID [JaH5lnRnRVOFD95Nw28W5Q], reason [Failed to start shard, message [IndexShardGatewayRecoveryException[[sourcelines][0] failed to recover shard]; nested: TranslogCorruptedException[translog corruption while reading from stream]; nested: ElasticsearchException[failed to read [sourceline][22c2d1d1-8e73-47c0-b9fa-f8f5cc96b93f_796]]; nested: ElasticsearchIllegalArgumentException[No version type match [105]]; ]]
2015.08.03 11:00:44 WARN   es[o.e.c.action.shard]  [sonar-1438592314778] [sourcelines][0] received shard failed for [sourcelines][0], node[5eRgYcVUTNCNLjhhzdiHGA], [P], s[INITIALIZING], indexUUID [JaH5lnRnRVOFD95Nw28W5Q], reason [Failed to start shard, message [IndexShardGatewayRecoveryException[[sourcelines][0] failed to recover shard]; nested: TranslogCorruptedException[translog corruption while reading from stream]; nested: ElasticsearchException[failed to read [sourceline][22c2d1d1-8e73-47c0-b9fa-f8f5cc96b93f_796]]; nested: ElasticsearchIllegalArgumentException[No version type match [105]]; ]]

do i have to delete shards? if yes how to do that? how to recover the shard? is indexing wrong? where do i find [sourcelines] on my machine ? help me with this.

Thankyou in advance.

Answer

Seb - SonarSource Team picture Seb - SonarSource Team · Aug 3, 2015

Your ES index seems to be corrupted.

All base information is stored in the DB. Do not delete or update it manually.

On the contrary, if you delete the ES indexes, they will simply be rebuilt at the next startup (which may take some time depending on the quantity of data you have).

To achieve that, delete the es directory in the data directory of your SQ instance.

Related questions

Getting elasticsearch "can not run as root" error after upgrading from SonarQube 6.5 to 6.6. Nothing else changed

Getting elasticsearch "can not run as root" error after upgrading from SonarQube 6.5 to 6.6. Nothing else changed. CentOS release 6.8 (Final) Java(TM) SE Runtime Environment (build 1.8.0_101-b13) java.lang.RuntimeException: can not run elasticsearch as root at org.elasticsearch.bootstrap.Bootstrap.…

Read More
Elasticsearch query to return all records

I have a small database in Elasticsearch and for testing purposes would like to pull all records back. I am attempting to use a URL of the form... http://localhost:9200/foo/_search?pretty=true&q={'matchAll':{''}} Can someone …

Read More
Making a request to a RESTful API using python

I have a RESTful API that I have exposed using an implementation of Elasticsearch on an EC2 instance to index a corpus of content. I can query the search by running the following from my terminal (MacOSX): curl -XGET 'http://…

Read More