mariadb galera - Error when a node shutdown ERROR 1047 WSREP has not yet prepared node for application use

namdt55555 picture namdt55555 · Nov 17, 2016 · Viewed 32.7k times · Source

I installed 2 Mariadb Galera nodes (mariadb-galera-10.0.27-linux-x86_64.tar.gz) on 2 CentOs 6.6 servers.

After installed, I start node1 with parameter --wsrep-new-cluster, then start node2 without this parameter. They work fine, data is synchronized successfully between 2 nodes.

But, when I shutdown node1. Node2 still running, but when I try to access database. It show this error:

use testdb;
ERROR 1047 (08S01): WSREP has not yet prepared node for application use 

What's happen in this case? Here is my configuration on 2 NODES (Just different IP address)

[galera] 
wsrep_on=ON
wsrep_cluster_name='mysql-cluster'
wsrep_provider='/home/mariadb/mariadb-galera/lib/galera/libgalera_smm.so'
wsrep_provider_options="gcache.size=1G"
wsrep_cluster_address="gcomm://10.211.26.116:4567?

pc.wait_prim=no"
wsrep_sst_method=rsync
binlog_format=row
default_storage_engine=InnoDB
innodb_autoinc_lock_mode=2
bind-address=0.0.0.0
wsrep_node_address=10.211.26.117:4567
wsrep_node_name='db2'

Answer

scarface_90 picture scarface_90 · Jun 21, 2017

TWO-NODE CLUSTERS

In a two-node cluster, a single-node failure causes the other to stop working.

Situation

You have a cluster composed of only two nodes. One of the nodes leaves the cluster ungracefully. That is, instead of being shut down through init or systemd, it crashes or suffers a loss of network connectivity. The node that remains becomes nonoperational. It remains so until some additional information is provided by a third party, such as a human operator or another node.

If the node remained operational after the other left the cluster ungracefully, there would be the risk that each of the two nodes will think itself as being the Primary Component. To prevent this, the node becomes nonoperational.

Solutions

There are two solutions available to you:

  • You can bootstrap the surviving node to form a new Primary Component, using the pc.boostrap wsrep Provider option. To do so, log into the database client and run the following command:

SET GLOBAL wsrep_provider_options='pc.bootstrap=YES';

This bootstraps the surviving node as a new Primary Component. When the other node comes back online or regains network connectivity with this node, it will initiate a state transfer and catch up with this node.

  • In the event that you want the node to continue to operate, you can use the pc.ignore_sb wsrep Provider option. To do so, log into the database client and run the following command:

SET GLOBAL wsrep_provider_options='pc.ignore_sb=TRUE';

The node resumes processing updates and it will continue to do so, even in the event that it suspects a split-brain situation.

Note Warning: Enabling pc.ignore_sb is dangerous in a multi-master setup, due to the aforementioned risk for split-brain situations. However, it does simplify things in master-slave clusters, (especially in cases where you only use two nodes).

In addition to the solutions provided above, you can avoid the situation entirely using Galera Arbitrator. Galera Arbitrator functions as an odd node in quorum calculations. Meaning that, if you enable Galera Arbitrator on one node in a two-node cluster, that node remains the Primary Component, even if the other node fails or loses network connectivity.

http://galeracluster.com/documentation-webpages/twonode.html