Agent mirrors can be used interchangeably when processing a search query. The Manticore instance hosting the distributed table where the mirrored agents are defined keeps track of mirror status (alive or dead) and response times, and performs automatic failover and load balancing based on this information.
agent = node1|node2|node3:9312:shard2The above example declares that node1:9312, node2:9312, and node3:9312 all have a table called shard2, and can be used as interchangeable mirrors. If any of these servers go down, the queries will be distributed between the remaining ones. When the server comes back online, the master will detect it and begin routing queries to all mirrors again.
A mirror may also include an individual table list, as follows:
agent = node1:9312:node1shard2|node2:9312:node2shard2This works similarly to the previous example, but different table names will be used when querying different servers. For example, node1shard2 will be used when querying node1:9312, and node2shard2 will be used when querying node2:9312.
By default, mirror selection uses the global or table-level ha_strategy. If you do not set it explicitly, the default strategy is random. The master stores metrics such as total query count, error count, and response time for each agent and groups them into time spans controlled by ha_period_karma. These statistics are then used by the balancing strategies described in Load balancing.
The karma period is in seconds and defaults to 60 seconds. The master stores up to 15 karma spans with per-agent statistics for instrumentation purposes (see SHOW AGENT STATUS). However, only the last two spans are used for HA/LB logic.
When there are no queries, the master sends a regular ping command every ha_ping_interval in order to collect statistics and check if the remote host is still alive. ha_ping_interval defaults to 1000 msec. Setting it to 0 disables pings, and statistics will only be accumulated based on actual queries. Together with ha_period_karma, this controls how quickly mirror health and latency changes influence load balancing.
This distinction is important:
- One
agent='host1|host2:table'entry means one remote shard with mirrored backends. - Multiple
agent='...'entries mean multiple remote shards.
For example:
# one shard, two mirrors
agent = node1|node2:9312:products
# two shards
agent = node1:9312:products_a
agent = node2:9312:products_bThe same mirroring concept can be configured either in a config file or with CREATE TABLE. For TCP connections, agent= must use the remote agent/API port (typically 9312), not the MySQL port (9306). If you want a specific balancing policy for just one distributed table, set ha_strategy on that table instead of relying only on the global default.
- Config
- SQL
table products_dist {
type = distributed
agent = 127.0.0.1:9312|127.0.0.1:9313:products
ha_strategy = roundrobin
}CREATE TABLE products_dist type='distributed'
agent='127.0.0.1:9312:products|127.0.0.1:9313:products'
ha_strategy='roundrobin';Assume the mirrored tables already exist, for example:
- SQL
CREATE TABLE products_dist type='distributed'
agent='127.0.0.1:9312:products|127.0.0.1:9313:products'
ha_strategy='roundrobin';Use the distributed table normally from the master, then verify how it was stored and how the mirrors are behaving.
- SQL
SELECT id, title, node FROM products_dist;
SHOW CREATE TABLE products_dist;
SHOW AGENT STATUS;+------+------------+-------+
| id | title | node |
+------+------------+-------+
| 1 | same title | node1 |
+------+------------+-------+
+---------------+----------------------------------------------------------------------------------+
| Table | Create Table |
+---------------+----------------------------------------------------------------------------------+
| products_dist | CREATE TABLE products_dist type='distributed' agent='127.0.0.1:9312:products|... |
+---------------+----------------------------------------------------------------------------------+Example of sharding a table over 4 servers total, in 2 shards with 2 mirrors for each shard.
- Config
# node1, node2 carry shard1 as local
# node3, node4 carry shard2 as local
# config on node1, node2
agent = node3:9312|node4:9312:shard2
# config on node3, node4
agent = node1:9312|node2:9312:shard1With Manticore, write transactions (such as INSERT, REPLACE, DELETE, TRUNCATE, UPDATE, COMMIT) can be replicated to other cluster nodes before the transaction is fully applied on the current node. Currently, replication is supported for percolate, rt and distributed tables in Linux and macOS.
Native Windows binaries for Manticore do not support replication. We recommend installing Manticore via WSL (Windows Subsystem for Linux).
On macOS, replication has limited support and is recommended only for development purposes.
Manticore's replication is powered by the Galera library and boasts several impressive features:
- True multi-master: read and write to any node at any time.
- Virtually synchronous replication no slave lag and no data loss after a node crash.
- Hot standby: no downtime during failover (since there is no failover).
- Tightly coupled: all nodes hold the same state and no diverged data between nodes is allowed.
- Automatic node provisioning: no need to manually backup the database and restore it on a new node.
- Easy to use and deploy.
- Detection and automatic eviction of unreliable nodes.
- Certification-based replication.
To set up replication in Manticore Search:
- The data_dir option must be set in the "searchd" section of the configuration file. Replication is not supported in plain mode.
- A listen directive must be specified, containing an IP address accessible by other nodes, or a node_address with an accessible IP address.
- Optionally, you can set unique values for server_id on each cluster node. If no value is set, the node will attempt to use the MAC address or a random number to generate the
server_id.
If authentication and authorization is enabled, cluster operations require the replication permission. Grant it to the user that should own replication operations for the cluster:
GRANT replication ON 'posts' TO 'repl_user';CREATE CLUSTER and JOIN CLUSTER can specify that user with '<user>' AS user. The user must exist with matching stored authentication data on the nodes that take part in the operation. Creating the same user name and password independently on each node is not enough, because the stored authentication data can differ. If you change auth data outside the daemon, run RELOAD AUTH on the affected nodes before using the cluster operation.
Later ALTER CLUSTER ... ADD, ALTER CLUSTER ... DROP, ALTER CLUSTER ... UPDATE nodes, and DELETE CLUSTER use the stored cluster user. When authentication is enabled, a successful JOIN CLUSTER replaces all local authentication data on the joining node with the donor cluster's authentication data.
If there is no replication listen directive set, Manticore will use the first two free ports in the range of 200 ports after the default protocol listening port for each created cluster. To set replication ports manually, the listen directive (of replication type) port range must be defined and the address/port range pairs must not intersect between different nodes on the same server. As a rule of thumb, the port range should specify at least two ports per cluster. When you define a replication listener with a port range (e.g., listen = 192.168.0.1:9320-9328:replication), Manticore doesn't immediately start listening on these ports. Instead, it will take random free ports from the specified range only when you start using replication.
A replication cluster is a group of nodes in which a write transaction is replicated. Replication is set up on a per-table basis, meaning that one table can only belong to one cluster. There is no limit on the number of tables that a cluster can have. All transactions such as INSERT, REPLACE, DELETE, TRUNCATE on any percolate or real-time table that belongs to a cluster are replicated to all the other nodes in that cluster. Distributed tables can also be part of the replication process. Replication is multi-master, so writes to any node or multiple nodes simultaneously will work just as well.
To create a cluster, you can typically use the command create cluster with CREATE CLUSTER <cluster name>, and to join a cluster, you can use join cluster with JOIN CLUSTER <cluster name> at 'host:port'. However, in some rare cases, you may want to fine-tune the behavior of CREATE/JOIN CLUSTER. The available options are:
This option specifies the name of the cluster. It should be unique among all the clusters in the system.
Note: The maximum allowable hostname length for the
JOINcommand is 253 characters. If you exceed this limit, searchd will generate an error.
The path option specifies the data directory for write-set cache replication and other cluster provider files. It does not change where replicated tables are stored. Incoming replicated tables are stored in their normal table directory under data_dir. This value should be unique among all the clusters in the system and should be specified as a relative path to the data_dir directory. By default, it is set to the value of data_dir.
Breaking change: Older versions also stored incoming replicated table files under the cluster path. If you used a custom cluster
path, upgrade carefully because replicated tables received by older versions might need to be moved or re-synchronized into the normaldata_dir/<table>layout.
The nodes option is a list of address:port pairs for all the nodes in the cluster, separated by commas. This list should be obtained using the node's API interface and can include the address of the current node as well. It is used to join the node to the cluster and to rejoin it after a restart.
The options option allows you to pass additional options directly to the Galera replication plugin, as described in the Galera Documentation Parameters
When working with a replication cluster, all write statements such as INSERT, REPLACE, DELETE, TRUNCATE, UPDATE that modify the content of a cluster's table must use thecluster_name:table_name expression instead of the table name. This ensures that the changes are propagated to all replicas in the cluster. If the correct expression is not used, an error will be triggered.
In the JSON interface, the cluster property must be set along with the table name for all write statements to a cluster's table. Failure to set the cluster property will result in an error.
The Auto ID for a table in a cluster should be valid as long as the server_id is correctly configured.
- SQL
- JSON
- PHP
- Python
- Python-asyncio
- Javascript
- Java
- C#
- Rust
INSERT INTO posts:weekly_index VALUES ( 'iphone case' )
TRUNCATE RTINDEX click_query:weekly_index
UPDATE INTO posts:rt_tags SET tags=(101, 302, 304) WHERE MATCH ('use') AND id IN (1,101,201)
DELETE FROM clicks:rt WHERE MATCH ('dumy') AND gid>206POST /insert -d '
{
"cluster":"posts",
"table":"weekly_index",
"doc":
{
"title" : "iphone case",
"price" : 19.85
}
}'
POST /delete -d '
{
"cluster":"posts",
"table": "weekly_index",
"id":1
}'$index->addDocuments([
1, ['title' => 'iphone case', 'price' => 19.85]
]);
$index->deleteDocument(1);indexApi.insert({"cluster":"posts","table":"weekly_index","doc":{"title":"iphone case","price":19.85}})
indexApi.delete({"cluster":"posts","table":"weekly_index","id":1})await indexApi.insert({"cluster":"posts","table":"weekly_index","doc":{"title":"iphone case","price":19.85}})
await indexApi.delete({"cluster":"posts","table":"weekly_index","id":1})res = await indexApi.insert({"cluster":"posts","table":"weekly_index","doc":{"title":"iphone case","price":19.85}});
res = await indexApi.delete({"cluster":"posts","table":"weekly_index","id":1});InsertDocumentRequest newdoc = new InsertDocumentRequest();
HashMap<String,Object> doc = new HashMap<String,Object>(){{
put("title","Crossbody Bag with Tassel");
put("price",19.85);
}};
newdoc.index("weekly_index").cluster("posts").id(1L).setDoc(doc);
sqlresult = indexApi.insert(newdoc);
DeleteDocumentRequest deleteRequest = new DeleteDocumentRequest();
deleteRequest.index("weekly_index").cluster("posts").setId(1L);
indexApi.delete(deleteRequest);Dictionary<string, Object> doc = new Dictionary<string, Object>();
doc.Add("title", "Crossbody Bag with Tassel");
doc.Add("price", 19.85);
InsertDocumentRequest newdoc = new InsertDocumentRequest(table: "weekly_index", cluster:posts, id: 1, doc: doc);
var sqlresult = indexApi.Insert(newdoc);
DeleteDocumentRequest deleteDocumentRequest = new DeleteDocumentRequest(table: "weekly_index", cluster: "posts", id: 1);
indexApi.Delete(deleteDocumentRequest);let mut doc = HashMap::new();
doc.insert("title".to_string(), serde_json::json!("Crossbody Bag with Tassel"));
doc.insert("price".to_string(), serde_json::json!(19.85));
let insert_req = InsertDocumentRequest {
table: serde_json::json!("weekly_index"),
doc: serde_json::json!(doc),
cluster: serde_json::json!("posts"),
id: serde_json::json!(1),
};
let insert_res = index_api.insert(insert_req).await;
let delete_req = DeleteDocumentRequest {
table: serde_json::json!("weekly_index"),
cluster: serde_json::json!("posts"),
id: serde_json::json!(1),
};
index_api.delete(delete_req).await;Read statements such as SELECT, CALL PQ, DESCRIBE can either use regular table names that are not prepended with a cluster name, or they can use the cluster_name:table_nameformat. If the latter is used, the cluster_name component is ignored.
When using the HTTP endpoint json/search, the cluster property can be specified if desired, but it can also be omitted.
- SQL
- JSON
SELECT * FROM weekly_index
CALL PQ('posts:weekly_index', 'document is here')POST /search -d '
{
"cluster":"posts",
"table":"weekly_index",
"query":{"match":{"title":"keyword"}}
}'
POST /search -d '
{
"table":"weekly_index",
"query":{"match":{"title":"keyword"}}
}'Replication plugin options can be adjusted using the SET statement.
A list of available options can be found in the Galera Documentation Parameters .
- SQL
- JSON
SET CLUSTER click_query GLOBAL 'pc.bootstrap' = 1POST /cli -d "
SET CLUSTER click_query GLOBAL 'pc.bootstrap' = 1
"It's possible for replicated nodes to diverge from one another, leading to a state where all nodes are labeled as non-primary. This can occur as a result of a network split between nodes, a cluster crash, or if the replication plugin experiences an exception when determining the primary component. In such a scenario, it's necessary to select a node and promote it to the role of primary component.
To identify the node that needs to be promoted, you should compare the last_committed cluster status variable value on all nodes. If all the servers are currently running, there's no need to restart the cluster. Instead, you can simply promote the node with the highest last_committed value to the primary component using the SET statement (as demonstrated in the example).
The other nodes will then reconnect to the primary component and resynchronize their data based on this node.
- SQL
- JSON
SET CLUSTER posts GLOBAL 'pc.bootstrap' = 1POST /cli -d "
SET CLUSTER posts GLOBAL 'pc.bootstrap' = 1
"To use replication, you need to define one listen port for SphinxAPI protocol and one listen for replication address and port range in the configuration file. Also, specify the data_dir folder to store incoming replicated tables.
- ini
searchd {
listen = 9312
listen = 192.168.1.101:9360-9370:replication
data_dir = /var/lib/manticore/
...
}To replicate tables, you must create a cluster on the server that has the local tables to be replicated.
- SQL
- JSON
- PHP
- Python
- Python-asyncio
- Javascript
- Java
- C#
- Rust
CREATE CLUSTER postsPOST /cli -d "
CREATE CLUSTER posts
"$params = [
'cluster' => 'posts'
]
];
$response = $client->cluster()->create($params);utilsApi.sql('CREATE CLUSTER posts')await utilsApi.sql('CREATE CLUSTER posts')res = await utilsApi.sql('CREATE CLUSTER posts');utilsApi.sql("CREATE CLUSTER posts");utilsApi.Sql("CREATE CLUSTER posts");utils_api.sql("CREATE CLUSTER posts", Some(true)).await;Add these local tables to the cluster
- SQL
- JSON
- PHP
- Python
- Python-asyncio
- Javascript
- Java
- C#
- Rust
ALTER CLUSTER posts ADD pq_title
ALTER CLUSTER posts ADD pq_clicksPOST /cli -d "
ALTER CLUSTER posts ADD pq_title
"
POST /cli -d "
ALTER CLUSTER posts ADD pq_clicks
"$params = [
'cluster' => 'posts',
'body' => [
'operation' => 'add',
'table' => 'pq_title'
]
];
$response = $client->cluster()->alter($params);
$params = [
'cluster' => 'posts',
'body' => [
'operation' => 'add',
'table' => 'pq_clicks'
]
];
$response = $client->cluster()->alter($params);utilsApi.sql('ALTER CLUSTER posts ADD pq_title')
utilsApi.sql('ALTER CLUSTER posts ADD pq_clicks')await utilsApi.sql('ALTER CLUSTER posts ADD pq_title')
await utilsApi.sql('ALTER CLUSTER posts ADD pq_clicks')res = await utilsApi.sql('ALTER CLUSTER posts ADD pq_title');
res = await utilsApi.sql('ALTER CLUSTER posts ADD pq_clicks');utilsApi.sql("ALTER CLUSTER posts ADD pq_title");
utilsApi.sql("ALTER CLUSTER posts ADD pq_clicks");utilsApi.Sql("ALTER CLUSTER posts ADD pq_title");
utilsApi.Sql("ALTER CLUSTER posts ADD pq_clicks");utils_api.sql("ALTER CLUSTER posts ADD pq_title", Some(true)).await;
utils_api.sql("ALTER CLUSTER posts ADD pq_clicks", Some(true)).await;All other nodes that wish to receive a replica of the cluster's tables should join the cluster as follows:
- SQL
- JSON
- PHP
- Python
- Python-asyncio
- Javascript
- Java
- C#
- Rust
JOIN CLUSTER posts AT '192.168.1.101:9312'POST /cli -d "
JOIN CLUSTER posts AT '192.168.1.101:9312'
"$params = [
'cluster' => 'posts',
'body' => [
'192.168.1.101:9312'
]
];
$response = $client->cluster->join($params);utilsApi.sql('JOIN CLUSTER posts AT \'192.168.1.101:9312\'')await utilsApi.sql('JOIN CLUSTER posts AT \'192.168.1.101:9312\'')res = await utilsApi.sql('JOIN CLUSTER posts AT \'192.168.1.101:9312\'');utilsApi.sql("JOIN CLUSTER posts AT '192.168.1.101:9312'");utilsApi.Sql("JOIN CLUSTER posts AT '192.168.1.101:9312'");utils_api.sql("JOIN CLUSTER posts AT '192.168.1.101:9312'", Some(true)).await;When running queries, prepend the table name with the cluster name posts: or use the cluster property for HTTP request object.
- SQL
- JSON
- PHP
- Python
- Python-asyncio
- Javascript
- Java
- C#
- Rust
INSERT INTO posts:pq_title VALUES ( 3, 'test me' )POST /insert -d '
{
"cluster":"posts",
"table":"pq_title",
"id": 3
"doc":
{
"title" : "test me"
}
}'$index->addDocuments([
3, ['title' => 'test me']
]);indexApi.insert({"cluster":"posts","table":"pq_title","id":3"doc":{"title":"test me"}})await indexApi.insert({"cluster":"posts","table":"pq_title","id":3"doc":{"title":"test me"}})res = await indexApi.insert({"cluster":"posts","table":"pq_title","id":3"doc":{"title":"test me"}});InsertDocumentRequest newdoc = new InsertDocumentRequest();
HashMap<String,Object> doc = new HashMap<String,Object>(){{
put("title","test me");
}};
newdoc.index("pq_title").cluster("posts").id(3L).setDoc(doc);
sqlresult = indexApi.insert(newdoc);Dictionary<string, Object> doc = new Dictionary<string, Object>();
doc.Add("title", "test me");
InsertDocumentRequest newdoc = new InsertDocumentRequest(index: "pq_title", cluster: "posts", id: 3, doc: doc);
var sqlresult = indexApi.Insert(newdoc);let mut doc = HashMap::new();
doc.insert("title".to_string(), serde_json::json!("test me"));
let insert_req = InsertDocumentRequest {
table: serde_json::json!("pq_title"),
doc: serde_json::json!(doc),
cluster: serde_json::json!("posts"),
id: serde_json::json!(3),
};
let insert_res = index_api.insert(insert_req).await;All queries that modify tables in the cluster are now replicated to all nodes in the cluster.
To create a replication cluster, you must set its name at a minimum.
If you are creating a single cluster or the first cluster, you may omit the path option. In this case, the data_dir option will be used as the cluster path. However, for all subsequent clusters, you must specify the path and the path must be available. The nodes option may also be set to list all nodes in the cluster.
- SQL
- JSON
- PHP
- Python
- Python-asyncio
- javascript
- Java
- C#
- Rust
CREATE CLUSTER posts
CREATE CLUSTER click_query '/var/data/click_query/' as path
CREATE CLUSTER click_query '/var/data/click_query/' as path, 'clicks_mirror1:9312,clicks_mirror2:9312,clicks_mirror3:9312' as nodesPOST /cli -d "
CREATE CLUSTER posts
"
POST /cli -d "
CREATE CLUSTER click_query '/var/data/click_query/' as path
"
POST /cli -d "
CREATE CLUSTER click_query '/var/data/click_query/' as path, 'clicks_mirror1:9312,clicks_mirror2:9312,clicks_mirror3:9312' as nodes
"$params = [
'cluster' => 'posts',
]
];
$response = $client->cluster()->create($params);
$params = [
'cluster' => 'click_query',
'body' => [
'path' => '/var/data/click_query/'
]
]
];
$response = $client->cluster()->create($params);
$params = [
'cluster' => 'click_query',
'body' => [
'path' => '/var/data/click_query/',
'nodes' => 'clicks_mirror1:9312,clicks_mirror2:9312,clicks_mirror3:9312'
]
]
];
$response = $client->cluster()->create($params);utilsApi.sql('CREATE CLUSTER posts')
utilsApi.sql('CREATE CLUSTER click_query \'/var/data/click_query/\' as path')
utilsApi.sql('CREATE CLUSTER click_query \'/var/data/click_query/\' as path, \'clicks_mirror1:9312,clicks_mirror2:9312,clicks_mirror3:9312\' as nodes')await utilsApi.sql('CREATE CLUSTER posts')
await utilsApi.sql('CREATE CLUSTER click_query \'/var/data/click_query/\' as path')
await utilsApi.sql('CREATE CLUSTER click_query \'/var/data/click_query/\' as path, \'clicks_mirror1:9312,clicks_mirror2:9312,clicks_mirror3:9312\' as nodes')res = await utilsApi.sql('CREATE CLUSTER posts');
res = await utilsApi.sql('CREATE CLUSTER click_query \'/var/data/click_query/\' as path');
res = await utilsApi.sql('CREATE CLUSTER click_query \'/var/data/click_query/\' as path, \'clicks_mirror1:9312,clicks_mirror2:9312,clicks_mirror3:9312\' as nodes');utilsApi.sql("CREATE CLUSTER posts");
utilsApi.sql("CREATE CLUSTER click_query '/var/data/click_query/' as path");
utilsApi.sql("CREATE CLUSTER click_query '/var/data/click_query/' as path, 'clicks_mirror1:9312,clicks_mirror2:9312,clicks_mirror3:9312' as nodes");utilsApi.Sql("CREATE CLUSTER posts");
utilsApi.Sql("CREATE CLUSTER click_query '/var/data/click_query/' as path");
utilsApi.Sql("CREATE CLUSTER click_query '/var/data/click_query/' as path, 'clicks_mirror1:9312,clicks_mirror2:9312,clicks_mirror3:9312' as nodes");utils_api.sql("CREATE CLUSTER posts", Some(true)).await;
utils_api.sql("CREATE CLUSTER click_query '/var/data/click_query/' as path", Some(true)).await;
utils_api.sql("CREATE CLUSTER click_query '/var/data/click_query/' as path, 'clicks_mirror1:9312,clicks_mirror2:9312,clicks_mirror3:9312' as nodes", Some(true)).await;If authentication and authorization is enabled, the effective replication user must have replication permission on the cluster target. You can specify that user in the CREATE CLUSTER statement:
GRANT replication ON 'posts' TO 'repl_user';
CREATE CLUSTER posts 'repl_user' AS user;If no user is specified, the current session user is used for the statement and stored as the cluster user after successful creation.
Before other auth-enabled nodes join or manage the cluster, provision the stored cluster user with matching authentication data and replication permission on those nodes.
If the nodes option is not specified when creating a cluster, the first node that joins the cluster will be saved as the nodes option.
To join an existing cluster, you must specify at least:
- The name of the cluster
- The
host:portof another node in the cluster you are joining
- SQL
- JSON
- PHP
- Python
- Python-asyncio
- javascript
- Java
- C#
- Rust
JOIN CLUSTER posts AT '10.12.1.35:9312'POST /cli -d "
JOIN CLUSTER posts AT '10.12.1.35:9312'
"$params = [
'cluster' => 'posts',
'body' => [
'10.12.1.35:9312'
]
];
$response = $client->cluster->join($params);utilsApi.sql('JOIN CLUSTER posts AT \'10.12.1.35:9312\'')await utilsApi.sql('JOIN CLUSTER posts AT \'10.12.1.35:9312\'')res = await utilsApi.sql('JOIN CLUSTER posts AT \'10.12.1.35:9312\'');utilsApi.sql("JOIN CLUSTER posts AT '10.12.1.35:9312'");utilsApi.Sql("JOIN CLUSTER posts AT '10.12.1.35:9312'");utils_api.sql("JOIN CLUSTER posts AT '10.12.1.35:9312'", Some(true)).await;{u'error': u'', u'total': 0, u'warning': u''}{u'error': u'', u'total': 0, u'warning': u''}{"total":0,"error":"","warning":""}If authentication and authorization is enabled, the effective replication user must have replication permission. You can specify that user in the JOIN CLUSTER statement:
GRANT replication ON 'posts' TO 'repl_user';
JOIN CLUSTER posts AT '10.12.1.35:9312' 'repl_user' AS user;The joining node and donor nodes must have the same user with matching stored authentication data. Creating the same user name and password independently on each node is not enough, because the stored authentication data can differ.
If no user is specified, the current session user is used for the join operation. After a successful join, the stored cluster user is taken from the donor cluster metadata.
NOTE: When authentication is enabled, a successful
JOIN CLUSTERreplaces all local authentication data on the joining node with the donor cluster's authentication data. If authentication logging is atinfoor higher, Manticore writes the previous local auth data tosearchd.log.authas a JSON backup before replacement. This backup includes salts and credential hashes, so keep the auth log private and redact it before sharing. IfJOIN CLUSTERcannot fetch the donor user, verify the replication user and matching auth data on donor nodes, and checksearchd.log.authon the donor nodes for API authentication failures.
In most cases, the above is sufficient when there is a single replication cluster. However, if you are creating multiple replication clusters, you must also set the path and ensure that the directory is available.
- SQL
- JSON
JOIN CLUSTER c2 at '127.0.0.1:10201' 'c2' as pathPOST /sql?mode=raw -d "JOIN CLUSTER c2 at '127.0.0.1:10201' 'c2' as path"A node joins a cluster by obtaining data from a specified node and, if successful, updates the node lists across all other cluster nodes in the same way as if it was done manually through ALTER CLUSTER ... UPDATE nodes. This list is used to re-join nodes to the cluster upon restart.
There are two lists of nodes:
1.cluster_<name>_nodes_set: used to re-join nodes to the cluster upon restart. It is updated across all nodes in the same way as ALTER CLUSTER ... UPDATE nodes does. JOIN CLUSTER command performs this update automatically. The Cluster status displays this list as cluster_<name>_nodes_set.
2. cluster_<name>_nodes_view: this list contains all active nodes used for replication and does not require manual management. ALTER CLUSTER ... UPDATE nodes actually copies this list of nodes to the list of nodes used to re-join upon restart. The Cluster status displays this list as cluster_<name>_nodes_view.
When nodes are located in different network segments or data centers, the nodes option may be set explicitly. This minimizes traffic between nodes and utilizes gateway nodes for intercommunication between data centers. The following code joins an existing cluster using the nodes option.
Note: The cluster
cluster_<name>_nodes_setlist is not updated automatically when this syntax is used. To update it, use ALTER CLUSTER ... UPDATE nodes.
- SQL
- JSON
- PHP
- Python
- Python-asyncio
- javascript
- Java
- C#
- Rust
JOIN CLUSTER click_query 'clicks_mirror1:9312;clicks_mirror2:9312;clicks_mirror3:9312' as nodesPOST /cli -d "
JOIN CLUSTER click_query 'clicks_mirror1:9312;clicks_mirror2:9312;clicks_mirror3:9312' as nodes
"$params = [
'cluster' => 'posts',
'body' => [
'nodes' => 'clicks_mirror1:9312;clicks_mirror2:9312;clicks_mirror3:9312'
]
];
$response = $client->cluster->join($params);utilsApi.sql('JOIN CLUSTER click_query \'clicks_mirror1:9312;clicks_mirror2:9312;clicks_mirror3:9312\' as nodes')await utilsApi.sql('JOIN CLUSTER click_query \'clicks_mirror1:9312;clicks_mirror2:9312;clicks_mirror3:9312\' as nodes')res = await utilsApi.sql('JOIN CLUSTER click_query \'clicks_mirror1:9312;clicks_mirror2:9312;clicks_mirror3:9312\' as nodes');utilsApi.sql("JOIN CLUSTER click_query 'clicks_mirror1:9312;clicks_mirror2:9312;clicks_mirror3:9312' as nodes");utilsApi.Sql("JOIN CLUSTER click_query 'clicks_mirror1:9312;clicks_mirror2:9312;clicks_mirror3:9312' as nodes");utils_api.sql("JOIN CLUSTER click_query 'clicks_mirror1:9312;clicks_mirror2:9312;clicks_mirror3:9312' as nodes", Some(true)).await;{u'error': u'', u'total': 0, u'warning': u''}{u'error': u'', u'total': 0, u'warning': u''}{"total":0,"error":"","warning":""}The JOIN CLUSTER command works synchronously and completes as soon as the node receives all data from the other nodes in the cluster and is in sync with them.
The JOIN CLUSTER operation can fail with an error message indicating a duplicate server_id. This occurs when the joining node has the same server_id as an existing node in the cluster. To resolve this issue, ensure that each node in the replication cluster has a unique server_id. You can change the default server_id in the "searchd" section of your configuration file to a unique value before attempting to join the cluster.