Manticore Search 最强大的功能之一是能够将全文搜索与地理位置结合起来。例如,零售商可以提供一种搜索,用户查找某个产品,结果集可以显示最近的有该产品库存的商店,这样用户就可以去店里取货。旅游网站可以基于限定区域的搜索提供结果,并按距离某点的远近排序(例如,“搜索酒店附近的博物馆”)。
要执行地理搜索,文档需要包含一对纬度/经度坐标。坐标可以存储为浮点属性。如果文档有多个位置,使用 JSON 属性存储坐标对可能更方便。
table myrt
{
...
rt_attr_float = lat
rt_attr_float = lon
...
}
坐标可以以度或弧度存储。
如果为纬度和经度属性生成了二级索引,且基于成本的优化器决定使用它们,则它们可能会自动用于加速地理搜索。
要计算两点之间的距离,可以使用GEODIST()函数。GEODIST 需要两个坐标对作为前四个参数。
第五个参数是简化的 JSON 格式,可以配置函数的某些方面。默认情况下,GEODIST 期望坐标为弧度,但可以添加 in=degrees 以允许使用度作为输入。用于计算地理距离的坐标必须与表中存储的坐标类型(度或弧度)相同,否则结果会产生误导。
计算出的距离默认单位为米,但通过 out 选项,可以转换为千米、英尺或英里。最后,默认使用一种称为 adaptive 的计算方法。还有一种基于 haversine 算法的替代方法,但该方法较慢且精度较低。
函数的结果——距离——可以用于 ORDER BY 子句对结果进行排序:
SELECT *, GEODIST(40.7643929, -73.9997683, lat, lon, {in=degrees, out=miles}) AS distance FROM myindex WHERE MATCH('...') ORDER BY distance ASC, WEIGHT() DESC;
或者限制结果在某点的半径范围内:
SELECT *,GEODIST(40.7643929, -73.9997683, lat,lon, {in=degrees, out=miles}) AS distance FROM myindex WHERE MATCH('...') AND distance <1000 ORDER BY WEIGHT(), DISTANCE ASC;
另一个地理搜索功能是判断某个位置是否在指定区域内。一个特殊函数构造多边形对象,另一个函数用来测试一组坐标是否包含在该多边形内。
有两个函数可用于创建多边形:
- GEOPOLY2D() - 创建考虑地球曲率的多边形
- POLY2D() - 创建平面空间中的简单多边形
POLY2D 适用于边长小于 500 公里的区域(对于 3-4 边的多边形;边数更多时应考虑更低的值)。对于边长较长的区域,必须使用 GEOPOLY2D 以保持结果准确。GEOPOLY2D 期望坐标为度的纬度/经度对;使用弧度将得到平面空间的结果(类似于 POLY2D)。
CONTAINS() 接受一个多边形和一组坐标作为输入,如果点在多边形内则输出 1,否则输出 0。
SELECT *,CONTAINS(GEOPOLY2D(40.76439, -73.9997, 42.21211, -73.999, 42.21211, -76.123, 40.76439, -76.123), 41.5445, -74.973) AS inside FROM myindex WHERE MATCH('...') AND inside=1;
Percolate 查询也称为持久查询、前瞻搜索、文档路由、反向搜索和逆向搜索。
传统的搜索方式是存储文档并对其执行搜索查询。然而,有些情况下我们希望将查询应用于新到达的文档以标记匹配。一些需要这种方式的场景包括监控系统收集数据并通知用户特定事件,例如达到某个指标阈值或监控数据中出现特定值。另一个例子是新闻聚合,用户可能只想收到某些类别或主题,甚至特定“关键词”的通知。
在这些情况下,传统搜索并不适用,因为它假设搜索是在整个集合上执行的。这个过程会随着用户数量的增加而成倍增长,导致大量查询在整个集合上运行,可能造成显著的额外负载。本节描述的替代方法是存储查询,然后将它们测试于新到达的文档或文档批次。
Google Alerts、AlertHN、Bloomberg Terminal 以及其他允许用户订阅特定内容的系统都使用类似技术。
- 参见 percolate 了解如何创建 PQ 表。
- 参见 向 percolate 表添加规则 学习如何添加 percolate 规则(也称为 PQ 规则)。这里有一个快速示例:
关于 percolate 查询,关键是你的搜索查询已经存储在表中。你需要提供的是文档,以检查它们是否匹配任何存储的规则。
你可以通过 SQL 或 JSON 接口执行 percolate 查询,也可以使用编程语言客户端。SQL 方法更灵活,而 HTTP 方法更简单,且提供了大部分所需功能。下表帮助你理解两者的区别。
| 期望行为 | SQL | HTTP |
|---|---|---|
| 提供单个文档 | CALL PQ('tbl', '{doc1}') |
query.percolate.document{doc1} |
| 提供单个文档(替代) | CALL PQ('tbl', 'doc1', 0 as docs_json) |
- |
| 提供多个文档 | CALL PQ('tbl', ('doc1', 'doc2'), 0 as docs_json) |
- |
| 提供多个文档(替代) | CALL PQ('tbl', ('{doc1}', '{doc2}')) |
- |
| 提供多个文档(替代) | CALL PQ('tbl', '[{doc1}, {doc2}]') |
- |
| 返回匹配的文档 ID | 0/1 作为 docs(默认禁用) | 默认启用 |
| 使用文档自身 ID 显示结果 | 'id field' 作为 docs_id(默认禁用) | 不可用 |
| 认为输入文档是 JSON | 1 作为 docs_json(默认 1) | 默认启用 |
| 认为输入文档是纯文本 | 0 作为 docs_json(默认 1) | 不可用 |
| 稀疏分布模式 | 默认 | 默认 |
| 分片分布模式 | sharded 作为 mode | 不可用 |
| 返回匹配查询的所有信息 | 1 作为 query(默认 0) | 默认启用 |
| 跳过无效 JSON | 1 作为 skip_bad_json(默认 0) | 不可用 |
| 在 SHOW META 中显示扩展信息 | 1 作为 verbose(默认 0) | 不可用 |
| 定义当未提供 docs_id 字段时加到文档 ID 的数值(主要用于分布式 PQ 模式) | 1 作为 shift(默认 0) | 不可用 |
为了演示其工作原理,以下是几个示例。我们创建一个包含两个字段的 PQ 表:
- title(文本)
- color(字符串)
以及其中的三条规则:
- 仅全文。查询:
@title bag - 全文加过滤。查询:
@title shoes。过滤条件:color='red' - 全文加更复杂的过滤。查询:
@title shoes。过滤条件:color IN('blue', 'green')
- SQL
- JSON
- PHP
- Python
- Python-asyncio
- javascript
- Java
- C#
- Rust
- TypeScript
- Go
CREATE TABLE products(title text, color string) type='pq';
INSERT INTO products(query) values('@title bag');
INSERT INTO products(query,filters) values('@title shoes', 'color=\'red\'');
INSERT INTO products(query,filters) values('@title shoes', 'color in (\'blue\', \'green\')');
select * from products;PUT /pq/products/doc/
{
"query": {
"match": {
"title": "bag"
}
},
"filters": ""
}
PUT /pq/products/doc/
{
"query": {
"match": {
"title": "shoes"
}
},
"filters": "color='red'"
}
PUT /pq/products/doc/
{
"query": {
"match": {
"title": "shoes"
}
},
"filters": "color IN ('blue', 'green')"
}$index = [
'table' => 'products',
'body' => [
'columns' => [
'title' => ['type' => 'text'],
'color' => ['type' => 'string']
],
'settings' => [
'type' => 'pq'
]
]
];
$client->indices()->create($index);
$query = [
'table' => 'products',
'body' => [ 'query'=>['match'=>['title'=>'bag']]]
];
$client->pq()->doc($query);
$query = [
'table' => 'products',
'body' => [ 'query'=>['match'=>['title'=>'shoes']],'filters'=>"color='red'"]
];
$client->pq()->doc($query);
$query = [
'table' => 'products',
'body' => [ 'query'=>['match'=>['title'=>'shoes']],'filters'=>"color IN ('blue', 'green')"]
];
$client->pq()->doc($query);utilsApi.sql('create table products(title text, color string) type=\'pq\'')
indexApi.insert({"table" : "products", "doc" : {"query" : "@title bag" }})
indexApi.insert({"table" : "products", "doc" : {"query" : "@title shoes", "filters": "color='red'" }})
indexApi.insert({"table" : "products", "doc" : {"query" : "@title shoes","filters": "color IN ('blue', 'green')" }})await utilsApi.sql('create table products(title text, color string) type=\'pq\'')
await indexApi.insert({"table" : "products", "doc" : {"query" : "@title bag" }})
await indexApi.insert({"table" : "products", "doc" : {"query" : "@title shoes", "filters": "color='red'" }})
await indexApi.insert({"table" : "products", "doc" : {"query" : "@title shoes","filters": "color IN ('blue', 'green')" }})res = await utilsApi.sql('create table products(title text, color string) type=\'pq\'');
res = indexApi.insert({"table" : "products", "doc" : {"query" : "@title bag" }});
res = indexApi.insert({"table" : "products", "doc" : {"query" : "@title shoes", "filters": "color='red'" }});
res = indexApi.insert({"table" : "products", "doc" : {"query" : "@title shoes","filters": "color IN ('blue', 'green')" }});utilsApi.sql("create table products(title text, color string) type='pq'", true);
doc = new HashMap<String,Object>(){{
put("query", "@title bag");
}};
newdoc = new InsertDocumentRequest();
newdoc.index("products").setDoc(doc);
indexApi.insert(newdoc);
doc = new HashMap<String,Object>(){{
put("query", "@title shoes");
put("filters", "color='red'");
}};
newdoc = new InsertDocumentRequest();
newdoc.index("products").setDoc(doc);
indexApi.insert(newdoc);
doc = new HashMap<String,Object>(){{
put("query", "@title shoes");
put("filters", "color IN ('blue', 'green')");
}};
newdoc = new InsertDocumentRequest();
newdoc.index("products").setDoc(doc);
indexApi.insert(newdoc);utilsApi.Sql("create table products(title text, color string) type='pq'", true);
Dictionary<string, Object> doc = new Dictionary<string, Object>();
doc.Add("query", "@title bag");
InsertDocumentRequest newdoc = new InsertDocumentRequest(index: "products", doc: doc);
indexApi.Insert(newdoc);
doc = new Dictionary<string, Object>();
doc.Add("query", "@title shoes");
doc.Add("filters", "color='red'");
newdoc = new InsertDocumentRequest(index: "products", doc: doc);
indexApi.Insert(newdoc);
doc = new Dictionary<string, Object>();
doc.Add("query", "@title bag");
doc.Add("filters", "color IN ('blue', 'green')");
newdoc = new InsertDocumentRequest(index: "products", doc: doc);
indexApi.Insert(newdoc);utils_api.sql("create table products(title text, color string) type='pq'", Some(true)).await;
let mut doc1 = HashMap::new();
doc1.insert("query".to_string(), serde_json::json!("@title bag"));
let insert_req1 = InsertDocumentRequest::new("products".to_string(), serde_json::json!(doc1));
index_api.insert(insert_req1).await;
let mut doc2 = HashMap::new();
doc2.insert("query".to_string(), serde_json::json!("@title shoes"));
doc2.insert("filters".to_string(), serde_json::json!("color='red'"));
let insert_req2 = InsertDocumentRequest::new("products".to_string(), serde_json::json!(doc2));
index_api.insert(insert_req2).await;
let mut doc3 = HashMap::new();
doc3.insert("query".to_string(), serde_json::json!("@title bag"));
doc3.insert("filters".to_string(), serde_json::json!("color IN ('blue', 'green')"));
let insert_req3 = InsertDocumentRequest::new("products".to_string(), serde_json::json!(doc3));
index_api.insert(insert_req3).await;res = await utilsApi.sql("create table test_pq(title text, color string) type='pq'");
res = indexApi.insert({
index: 'test_pq',
doc: { query : '@title bag' }
});
res = indexApi.insert(
index: 'test_pq',
doc: { query: '@title shoes', filters: "color='red'" }
});
res = indexApi.insert({
index: 'test_pq',
doc: { query : '@title shoes', filters: "color IN ('blue', 'green')" }
});apiClient.UtilsAPI.Sql(context.Background()).Body("create table test_pq(title text, color string) type='pq'").Execute()
indexDoc := map[string]interface{} {"query": "@title bag"}
indexReq := manticoreclient.NewInsertDocumentRequest("test_pq", indexDoc)
apiClient.IndexAPI.Insert(context.Background()).InsertDocumentRequest(*indexReq).Execute();
indexDoc = map[string]interface{} {"query": "@title shoes", "filters": "color='red'"}
indexReq = manticoreclient.NewInsertDocumentRequest("test_pq", indexDoc)
apiClient.IndexAPI.Insert(context.Background()).InsertDocumentRequest(*indexReq).Execute();
indexDoc = map[string]interface{} {"query": "@title shoes", "filters": "color IN ('blue', 'green')"}
indexReq = manticoreclient.NewInsertDocumentRequest("test_pq", indexDoc)
apiClient.IndexAPI.Insert(context.Background()).InsertDocumentRequest(*indexReq).Execute();+---------------------+--------------+------+---------------------------+
| id | query | tags | filters |
+---------------------+--------------+------+---------------------------+
| 1657852401006149635 | @title shoes | | color IN ('blue, 'green') |
| 1657852401006149636 | @title shoes | | color='red' |
| 1657852401006149637 | @title bag | | |
+---------------------+--------------+------+---------------------------+{
"table": "products",
"type": "doc",
"_id": 1657852401006149661,
"result": "created"
}
{
"table": "products",
"type": "doc",
"_id": 1657852401006149662,
"result": "created"
}
{
"table": "products",
"type": "doc",
"_id": 1657852401006149663,
"result": "created"
}Array(
[table] => products
[type] => doc
[_id] => 1657852401006149661
[result] => created
)
Array(
[table] => products
[type] => doc
[_id] => 1657852401006149662
[result] => created
)
Array(
[table] => products
[type] => doc
[_id] => 1657852401006149663
[result] => created
){'created': True,
'found': None,
'id': 0,
'table': 'products',
'result': 'created'}
{'created': True,
'found': None,
'id': 0,
'table': 'products',
'result': 'created'}
{'created': True,
'found': None,
'id': 0,
'table': 'products',
'result': 'created'}{'created': True,
'found': None,
'id': 0,
'table': 'products',
'result': 'created'}
{'created': True,
'found': None,
'id': 0,
'table': 'products',
'result': 'created'}
{'created': True,
'found': None,
'id': 0,
'table': 'products',
'result': 'created'}"table":"products","_id":0,"created":true,"result":"created"}
{"table":"products","_id":0,"created":true,"result":"created"}
{"table":"products","_id":0,"created":true,"result":"created"}{total=0, error=, warning=}
class SuccessResponse {
index: products
id: 0
created: true
result: created
found: null
}
class SuccessResponse {
index: products
id: 0
created: true
result: created
found: null
}
class SuccessResponse {
index: products
id: 0
created: true
result: created
found: null
}{total=0, error="", warning=""}
class SuccessResponse {
index: products
id: 0
created: true
result: created
found: null
}
class SuccessResponse {
index: products
id: 0
created: true
result: created
found: null
}
class SuccessResponse {
index: products
id: 0
created: true
result: created
found: null
}{total=0, error="", warning=""}
class SuccessResponse {
index: products
id: 0
created: true
result: created
found: null
}
class SuccessResponse {
index: products
id: 0
created: true
result: created
found: null
}
class SuccessResponse {
index: products
id: 0
created: true
result: created
found: null
}{
"table":"test_pq",
"_id":1657852401006149661,
"created":true,
"result":"created"
}
{
"table":"test_pq",
"_id":1657852401006149662,
"created":true,
"result":"created"
}
{
"table":"test_pq",
"_id":1657852401006149663,
"created":true,
"result":"created"
}{
"table":"test_pq",
"_id":1657852401006149661,
"created":true,
"result":"created"
}
{
"table":"test_pq",
"_id":1657852401006149662,
"created":true,
"result":"created"
}
{
"table":"test_pq",
"_id":1657852401006149663,
"created":true,
"result":"created"
}第一个文档不匹配任何规则。它可能匹配前两个,但它们需要额外的过滤条件。
第二个文档匹配一条规则。注意 CALL PQ 默认期望文档为 JSON,但如果使用 0 as docs_json,你可以传递纯字符串。
- SQL
- JSON
- PHP
- Python
- Python-asyncio
- javascript
- Java
- C#
- Rust
- TypeScript
- Go
CALL PQ('products', 'Beautiful shoes', 0 as docs_json);
CALL PQ('products', 'What a nice bag', 0 as docs_json);
CALL PQ('products', '{"title": "What a nice bag"}');POST /pq/products/search
{
"query": {
"percolate": {
"document": {
"title": "What a nice bag"
}
}
}
}$percolate = [
'table' => 'products',
'body' => [
'query' => [
'percolate' => [
'document' => [
'title' => 'What a nice bag'
]
]
]
]
];
$client->pq()->search($percolate);searchApi.percolate('products',{"query":{"percolate":{"document":{"title":"What a nice bag"}}}})await searchApi.percolate('products',{"query":{"percolate":{"document":{"title":"What a nice bag"}}}})res = await searchApi.percolate('products',{"query":{"percolate":{"document":{"title":"What a nice bag"}}}});PercolateRequest percolateRequest = new PercolateRequest();
query = new HashMap<String,Object>(){{
put("percolate",new HashMap<String,Object >(){{
put("document", new HashMap<String,Object >(){{
put("title","what a nice bag");
}});
}});
}};
percolateRequest.query(query);
searchApi.percolate("test_pq",percolateRequest);Dictionary<string, Object> percolateDoc = new Dictionary<string, Object>();
percolateDoc.Add("document", new Dictionary<string, Object> {{ "title", "what a nice bag" }});
Dictionary<string, Object> query = new Dictionary<string, Object> {{ "percolate", percolateDoc }};
PercolateRequest percolateRequest = new PercolateRequest(query=query);
searchApi.Percolate("test_pq",percolateRequest);let mut percolate_doc_fields = HashMap::new();
percolate_doc_fileds.insert("title".to_string(), "what a nice bag");
let mut percolate_doc = HashMap::new();
percolate_doc.insert("document".to_string(), percolate_doc_fields);
let percolate_query = PercolateRequestQuery::new(serde_json::json!(percolate_doc));
let percolate_req = PercolateRequest::new(percolate_query);
search_api.percolate("test_pq", percolate_req).await;res = await searchApi.percolate('test_pq', { query: { percolate: { document : { title : 'What a nice bag' } } } } );query := map[string]interface{} {"title": "what a nice bag"}
percolateRequestQuery := manticoreclient.NewPercolateQuery(query)
percolateRequest := manticoreclient.NewPercolateRequest(percolateRequestQuery)
res, _, _ := apiClient.SearchAPI.Percolate(context.Background(), "test_pq").PercolateRequest(*percolateRequest).Execute()+---------------------+
| id |
+---------------------+
| 1657852401006149637 |
+---------------------+
+---------------------+
| id |
+---------------------+
| 1657852401006149637 |
+---------------------+{
"took": 0,
"timed_out": false,
"hits": {
"total": 1,
"max_score": 1,
"hits": [
{
"table": "products",
"_type": "doc",
"_id": 1657852401006149644,
"_score": "1",
"_source": {
"query": {
"ql": "@title bag"
}
},
"fields": {
"_percolator_document_slot": [
1
]
}
}
]
}
}Array
(
[took] => 0
[timed_out] =>
[hits] => Array
(
[total] => 1
[max_score] => 1
[hits] => Array
(
[0] => Array
(
[_index] => products
[_type] => doc
[_id] => 1657852401006149644
[_score] => 1
[_source] => Array
(
[query] => Array
(
[match] => Array
(
[title] => bag
)
)
)
[fields] => Array
(
[_percolator_document_slot] => Array
(
[0] => 1
)
)
)
)
)
){'hits': {'hits': [{u'_id': u'2811025403043381480',
u'table': u'products',
u'_score': u'1',
u'_source': {u'query': {u'ql': u'@title bag'}},
u'_type': u'doc',
u'fields': {u'_percolator_document_slot': [1]}}],
'total': 1},
'profile': None,
'timed_out': False,
'took': 0}{'hits': {'hits': [{u'_id': u'2811025403043381480',
u'table': u'products',
u'_score': u'1',
u'_source': {u'query': {u'ql': u'@title bag'}},
u'_type': u'doc',
u'fields': {u'_percolator_document_slot': [1]}}],
'total': 1},
'profile': None,
'timed_out': False,
'took': 0}{
"took": 0,
"timed_out": false,
"hits": {
"total": 1,
"hits": [
{
"table": "products",
"_type": "doc",
"_id": 2811045522851233808,
"_score": "1",
"_source": {
"query": {
"ql": "@title bag"
}
},
"fields": {
"_percolator_document_slot": [
1
]
}
}
]
}
}class SearchResponse {
took: 0
timedOut: false
hits: class SearchResponseHits {
total: 1
maxScore: 1
hits: [{_index=products, _type=doc, _id=2811045522851234109, _score=1, _source={query={ql=@title bag}}, fields={_percolator_document_slot=[1]}}]
aggregations: null
}
profile: null
}class SearchResponse {
took: 0
timedOut: false
hits: class SearchResponseHits {
total: 1
maxScore: 1
hits: [{_index=products, _type=doc, _id=2811045522851234109, _score=1, _source={query={ql=@title bag}}, fields={_percolator_document_slot=[1]}}]
aggregations: null
}
profile: null
}class SearchResponse {
took: 0
timedOut: false
hits: class SearchResponseHits {
total: 1
maxScore: 1
hits: [{_index=products, _type=doc, _id=2811045522851234109, _score=1, _source={query={ql=@title bag}}, fields={_percolator_document_slot=[1]}}]
aggregations: null
}
profile: null
}{
"took": 0,
"timed_out": false,
"hits": {
"total": 1,
"hits": [
{
"table": "test_pq",
"_type": "doc",
"_id": 1657852401006149661,
"_score": "1",
"_source": {
"query": {
"ql": "@title bag"
}
},
"fields": {
"_percolator_document_slot": [
1
]
}
}
]
}
}{
"took": 0,
"timed_out": false,
"hits": {
"total": 1,
"hits": [
{
"table": "test_pq",
"_type": "doc",
"_id": 1657852401006149661,
"_score": "1",
"_source": {
"query": {
"ql": "@title bag"
}
},
"fields": {
"_percolator_document_slot": [
1
]
}
}
]
}
}- SQL
- JSON
- PHP
- Python
- Python-asyncio
- javascript
- Java
- C#
- Rust
- TypeScript
- Go
CALL PQ('products', '{"title": "What a nice bag"}', 1 as query);POST /pq/products/search
{
"query": {
"percolate": {
"document": {
"title": "What a nice bag"
}
}
}
}$percolate = [
'table' => 'products',
'body' => [
'query' => [
'percolate' => [
'document' => [
'title' => 'What a nice bag'
]
]
]
]
];
$client->pq()->search($percolate);searchApi.percolate('products',{"query":{"percolate":{"document":{"title":"What a nice bag"}}}})await searchApi.percolate('products',{"query":{"percolate":{"document":{"title":"What a nice bag"}}}})res = await searchApi.percolate('products',{"query":{"percolate":{"document":{"title":"What a nice bag"}}}});PercolateRequest percolateRequest = new PercolateRequest();
query = new HashMap<String,Object>(){{
put("percolate",new HashMap<String,Object >(){{
put("document", new HashMap<String,Object >(){{
put("title","what a nice bag");
}});
}});
}};
percolateRequest.query(query);
searchApi.percolate("test_pq",percolateRequest);Dictionary<string, Object> percolateDoc = new Dictionary<string, Object>();
percolateDoc.Add("document", new Dictionary<string, Object> {{ "title", "what a nice bag" }});
Dictionary<string, Object> query = new Dictionary<string, Object> {{ "percolate", percolateDoc }};
PercolateRequest percolateRequest = new PercolateRequest(query=query);
searchApi.Percolate("test_pq",percolateRequest);let mut percolate_doc_fields = HashMap::new();
percolate_doc_fileds.insert("title".to_string(), "what a nice bag");
let mut percolate_doc = HashMap::new();
percolate_doc.insert("document".to_string(), percolate_doc_fields);
let percolate_query = PercolateRequestQuery::new(serde_json::json!(percolate_doc));
let percolate_req = PercolateRequest::new(percolate_query);
search_api.percolate("test_pq", percolate_req).await;res = await searchApi.percolate('test_pq', { query: { percolate: { document : { title : 'What a nice bag' } } } } );query := map[string]interface{} {"title": "what a nice bag"}
percolateRequestQuery := manticoreclient.NewPercolateQuery(query)
percolateRequest := manticoreclient.NewPercolateRequest(percolateRequestQuery)
res, _, _ := apiClient.SearchAPI.Percolate(context.Background(), "test_pq").PercolateRequest(*percolateRequest).Execute()+---------------------+------------+------+---------+
| id | query | tags | filters |
+---------------------+------------+------+---------+
| 1657852401006149637 | @title bag | | |
+---------------------+------------+------+---------+{
"took": 0,
"timed_out": false,
"hits": {
"total": 1,
"max_score": 1,
"hits": [
{
"table": "products",
"_type": "doc",
"_id": 1657852401006149644,
"_score": "1",
"_source": {
"query": {
"ql": "@title bag"
}
},
"fields": {
"_percolator_document_slot": [
1
]
}
}
]
}
}Array
(
[took] => 0
[timed_out] =>
[hits] => Array
(
[total] => 1
[max_score] => 1
[hits] => Array
(
[0] => Array
(
[_index] => products
[_type] => doc
[_id] => 1657852401006149644
[_score] => 1
[_source] => Array
(
[query] => Array
(
[match] => Array
(
[title] => bag
)
)
)
[fields] => Array
(
[_percolator_document_slot] => Array
(
[0] => 1
)
)
)
)
)
){'hits': {'hits': [{u'_id': u'2811025403043381480',
u'table': u'products',
u'_score': u'1',
u'_source': {u'query': {u'ql': u'@title bag'}},
u'_type': u'doc',
u'fields': {u'_percolator_document_slot': [1]}}],
'total': 1},
'profile': None,
'timed_out': False,
'took': 0}{'hits': {'hits': [{u'_id': u'2811025403043381480',
u'table': u'products',
u'_score': u'1',
u'_source': {u'query': {u'ql': u'@title bag'}},
u'_type': u'doc',
u'fields': {u'_percolator_document_slot': [1]}}],
'total': 1},
'profile': None,
'timed_out': False,
'took': 0}{
"took": 0,
"timed_out": false,
"hits": {
"total": 1,
"hits": [
{
"table": "products",
"_type": "doc",
"_id": 2811045522851233808,
"_score": "1",
"_source": {
"query": {
"ql": "@title bag"
}
},
"fields": {
"_percolator_document_slot": [
1
]
}
}
]
}
}class SearchResponse {
took: 0
timedOut: false
hits: class SearchResponseHits {
total: 1
maxScore: 1
hits: [{_index=products, _type=doc, _id=2811045522851234109, _score=1, _source={query={ql=@title bag}}, fields={_percolator_document_slot=[1]}}]
aggregations: null
}
profile: null
}class SearchResponse {
took: 0
timedOut: false
hits: class SearchResponseHits {
total: 1
maxScore: 1
hits: [{_index=products, _type=doc, _id=2811045522851234109, _score=1, _source={query={ql=@title bag}}, fields={_percolator_document_slot=[1]}}]
aggregations: null
}
profile: null
}class SearchResponse {
took: 0
timedOut: false
hits: class SearchResponseHits {
total: 1
maxScore: 1
hits: [{_index=products, _type=doc, _id=2811045522851234109, _score=1, _source={query={ql=@title bag}}, fields={_percolator_document_slot=[1]}}]
aggregations: null
}
profile: null
}{
"took": 0,
"timed_out": false,
"hits": {
"total": 1,
"hits": [
{
"table": "test_pq",
"_type": "doc",
"_id": 1657852401006149661,
"_score": "1",
"_source": {
"query": {
"ql": "@title bag"
}
},
"fields": {
"_percolator_document_slot": [
1
]
}
}
]
}
}{
"took": 0,
"timed_out": false,
"hits": {
"total": 1,
"hits": [
{
"table": "test_pq",
"_type": "doc",
"_id": 1657852401006149661,
"_score": "1",
"_source": {
"query": {
"ql": "@title bag"
}
},
"fields": {
"_percolator_document_slot": [
1
]
}
}
]
}
}请注意,使用 CALL PQ,您可以通过不同方式提供多个文档:
- 作为圆括号中的普通文档数组
('doc1', 'doc2')。这需要0 as docs_json - 作为圆括号中的 JSON 数组
('{doc1}', '{doc2}') - 或作为标准 JSON 数组
'[{doc1}, {doc2}]'
- SQL
- JSON
- PHP
- Python
- Python-asyncio
- javascript
- Java
- C#
- Rust
- TypeScript
- Go
CALL PQ('products', ('nice pair of shoes', 'beautiful bag'), 1 as query, 0 as docs_json);
CALL PQ('products', ('{"title": "nice pair of shoes", "color": "red"}', '{"title": "beautiful bag"}'), 1 as query);
CALL PQ('products', '[{"title": "nice pair of shoes", "color": "blue"}, {"title": "beautiful bag"}]', 1 as query);POST /pq/products/search
{
"query": {
"percolate": {
"documents": [
{"title": "nice pair of shoes", "color": "blue"},
{"title": "beautiful bag"}
]
}
}
}$percolate = [
'table' => 'products',
'body' => [
'query' => [
'percolate' => [
'documents' => [
['title' => 'nice pair of shoes','color'=>'blue'],
['title' => 'beautiful bag']
]
]
]
]
];
$client->pq()->search($percolate);searchApi.percolate('products',{"query":{"percolate":{"documents":[{"title":"nice pair of shoes","color":"blue"},{"title":"beautiful bag"}]}}})await searchApi.percolate('products',{"query":{"percolate":{"documents":[{"title":"nice pair of shoes","color":"blue"},{"title":"beautiful bag"}]}}})res = await searchApi.percolate('products',{"query":{"percolate":{"documents":[{"title":"nice pair of shoes","color":"blue"},{"title":"beautiful bag"}]}}});percolateRequest = new PercolateRequest();
query = new HashMap<String,Object>(){{
put("percolate",new HashMap<String,Object >(){{
put("documents", new ArrayList<Object>(){{
add(new HashMap<String,Object >(){{
put("title","nice pair of shoes");
put("color","blue");
}});
add(new HashMap<String,Object >(){{
put("title","beautiful bag");
}});
}});
}});
}};
percolateRequest.query(query);
searchApi.percolate("products",percolateRequest);var doc1 = new Dictionary<string, Object>();
doc1.Add("title","nice pair of shoes");
doc1.Add("color","blue");
var doc2 = new Dictionary<string, Object>();
doc2.Add("title","beautiful bag");
var docs = new List<Object> {doc1, doc2};
Dictionary<string, Object> percolateDoc = new Dictionary<string, Object> {{ "documents", docs }};
Dictionary<string, Object> query = new Dictionary<string, Object> {{ "percolate", percolateDoc }};
PercolateRequest percolateRequest = new PercolateRequest(query=query);
searchApi.Percolate("products",percolateRequest);let mut percolate_doc_fields1 = HashMap::new();
percolate_doc_fields1.insert("title".to_string(), "nice pair of shoes");
percolate_doc_fields1.insert("color".to_string(), "blue");
let mut percolate_doc_fields2 = HashMap::new();
percolate_doc_fields2.insert("title".to_string(), "beautiful bag");
let mut percolate_doc_fields_list: [HashMap; 2] = [percolate_doc_fields1, percolate_doc_fields2];
let mut percolate_doc = HashMap::new();
percolate_doc.insert("documents".to_string(), percolate_doc_fields_list);
let percolate_query = PercolateRequestQuery::new(serde_json::json!(percolate_doc));
let percolate_req = PercolateRequest::new(percolate_query);
search_api.percolate("products", percolate_req).await;docs = [ {title : 'What a nice bag'}, {title : 'Really nice shoes'} ];
res = await searchApi.percolate('test_pq', { query: { percolate: { documents : docs } } } );doc1 := map[string]interface{} {"title": "What a nice bag"}
doc2 := map[string]interface{} {"title": "Really nice shoes"}
query := []interface{} {doc1, doc2}
percolateRequestQuery := manticoreclient.NewPercolateQuery(query)
percolateRequest := manticoreclient.NewPercolateRequest(percolateRequestQuery)
res, _, _ := apiClient.SearchAPI.Percolate(context.Background(), "test_pq").PercolateRequest(*percolateRequest).Execute()+---------------------+------------+------+---------+
| id | query | tags | filters |
+---------------------+------------+------+---------+
| 1657852401006149637 | @title bag | | |
+---------------------+------------+------+---------+
+---------------------+--------------+------+-------------+
| id | query | tags | filters |
+---------------------+--------------+------+-------------+
| 1657852401006149636 | @title shoes | | color='red' |
| 1657852401006149637 | @title bag | | |
+---------------------+--------------+------+-------------+
+---------------------+--------------+------+---------------------------+
| id | query | tags | filters |
+---------------------+--------------+------+---------------------------+
| 1657852401006149635 | @title shoes | | color IN ('blue, 'green') |
| 1657852401006149637 | @title bag | | |
+---------------------+--------------+------+---------------------------+{
"took": 0,
"timed_out": false,
"hits": {
"total": 2,
"max_score": 1,
"hits": [
{
"table": "products",
"_type": "doc",
"_id": 1657852401006149644,
"_score": "1",
"_source": {
"query": {
"ql": "@title bag"
}
},
"fields": {
"_percolator_document_slot": [
2
]
}
},
{
"table": "products",
"_type": "doc",
"_id": 1657852401006149646,
"_score": "1",
"_source": {
"query": {
"ql": "@title shoes"
}
},
"fields": {
"_percolator_document_slot": [
1
]
}
}
]
}
}Array
(
[took] => 23
[timed_out] =>
[hits] => Array
(
[total] => 2
[max_score] => 1
[hits] => Array
(
[0] => Array
(
[_index] => products
[_type] => doc
[_id] => 2810781492890828819
[_score] => 1
[_source] => Array
(
[query] => Array
(
[match] => Array
(
[title] => bag
)
)
)
[fields] => Array
(
[_percolator_document_slot] => Array
(
[0] => 2
)
)
)
[1] => Array
(
[_index] => products
[_type] => doc
[_id] => 2810781492890828821
[_score] => 1
[_source] => Array
(
[query] => Array
(
[match] => Array
(
[title] => shoes
)
)
)
[fields] => Array
(
[_percolator_document_slot] => Array
(
[0] => 1
)
)
)
)
)
){'hits': {'hits': [{u'_id': u'2811025403043381494',
u'table': u'products',
u'_score': u'1',
u'_source': {u'query': {u'ql': u'@title bag'}},
u'_type': u'doc',
u'fields': {u'_percolator_document_slot': [2]}},
{u'_id': u'2811025403043381496',
u'table': u'products',
u'_score': u'1',
u'_source': {u'query': {u'ql': u'@title shoes'}},
u'_type': u'doc',
u'fields': {u'_percolator_document_slot': [1]}}],
'total': 2},
'profile': None,
'timed_out': False,
'took': 0}{'hits': {'hits': [{u'_id': u'2811025403043381494',
u'table': u'products',
u'_score': u'1',
u'_source': {u'query': {u'ql': u'@title bag'}},
u'_type': u'doc',
u'fields': {u'_percolator_document_slot': [2]}},
{u'_id': u'2811025403043381496',
u'table': u'products',
u'_score': u'1',
u'_source': {u'query': {u'ql': u'@title shoes'}},
u'_type': u'doc',
u'fields': {u'_percolator_document_slot': [1]}}],
'total': 2},
'profile': None,
'timed_out': False,
'took': 0}{
"took": 6,
"timed_out": false,
"hits": {
"total": 2,
"hits": [
{
"table": "products",
"_type": "doc",
"_id": 2811045522851233808,
"_score": "1",
"_source": {
"query": {
"ql": "@title bag"
}
},
"fields": {
"_percolator_document_slot": [
2
]
}
},
{
"table": "products",
"_type": "doc",
"_id": 2811045522851233810,
"_score": "1",
"_source": {
"query": {
"ql": "@title shoes"
}
},
"fields": {
"_percolator_document_slot": [
1
]
}
}
]
}
}class SearchResponse {
took: 0
timedOut: false
hits: class SearchResponseHits {
total: 2
maxScore: 1
hits: [{_index=products, _type=doc, _id=2811045522851234133, _score=1, _source={query={ql=@title bag}}, fields={_percolator_document_slot=[2]}}, {_index=products, _type=doc, _id=2811045522851234135, _score=1, _source={query={ql=@title shoes}}, fields={_percolator_document_slot=[1]}}]
aggregations: null
}
profile: null
}class SearchResponse {
took: 0
timedOut: false
hits: class SearchResponseHits {
total: 2
maxScore: 1
hits: [{_index=products, _type=doc, _id=2811045522851234133, _score=1, _source={query={ql=@title bag}}, fields={_percolator_document_slot=[2]}}, {_index=products, _type=doc, _id=2811045522851234135, _score=1, _source={query={ql=@title shoes}}, fields={_percolator_document_slot=[1]}}]
aggregations: null
}
profile: null
}class SearchResponse {
took: 0
timedOut: false
hits: class SearchResponseHits {
total: 2
maxScore: 1
hits: [{_index=products, _type=doc, _id=2811045522851234133, _score=1, _source={query={ql=@title bag}}, fields={_percolator_document_slot=[2]}}, {_index=products, _type=doc, _id=2811045522851234135, _score=1, _source={query={ql=@title shoes}}, fields={_percolator_document_slot=[1]}}]
aggregations: null
}
profile: null
}{
"took": 0,
"timed_out": false,
"hits": {
"total": 2,
"hits": [
{
"table": "test_pq",
"_type": "doc",
"_id": 1657852401006149661,
"_score": "1",
"_source": {
"query": {
"ql": "@title bag"
}
},
"fields": {
"_percolator_document_slot": [
1
]
}
},
{
"table": "test_pq",
"_type": "doc",
"_id": 1657852401006149662,
"_score": "1",
"_source": {
"query": {
"ql": "@title shoes"
}
},
"fields": {
"_percolator_document_slot": [
1
]
}
}
]
}
}{
"took": 0,
"timed_out": false,
"hits": {
"total": 2,
"hits": [
{
"table": "test_pq",
"_type": "doc",
"_id": 1657852401006149661,
"_score": "1",
"_source": {
"query": {
"ql": "@title bag"
}
},
"fields": {
"_percolator_document_slot": [
1
]
}
},
{
"table": "test_pq",
"_type": "doc",
"_id": 1657852401006149662,
"_score": "1",
"_source": {
"query": {
"ql": "@title shoes"
}
},
"fields": {
"_percolator_document_slot": [
1
]
}
}
]
}
}- SQL
- JSON
- PHP
- Python
- Python-asyncio
- javascript
- Java
- C#
- Rust
- TypeScript
- Go
CALL PQ('products', '[{"title": "nice pair of shoes", "color": "blue"}, {"title": "beautiful bag"}]', 1 as query, 1 as docs);POST /pq/products/search
{
"query": {
"percolate": {
"documents": [
{"title": "nice pair of shoes", "color": "blue"},
{"title": "beautiful bag"}
]
}
}
}$percolate = [
'table' => 'products',
'body' => [
'query' => [
'percolate' => [
'documents' => [
['title' => 'nice pair of shoes','color'=>'blue'],
['title' => 'beautiful bag']
]
]
]
]
];
$client->pq()->search($percolate);searchApi.percolate('products',{"query":{"percolate":{"documents":[{"title":"nice pair of shoes","color":"blue"},{"title":"beautiful bag"}]}}})await searchApi.percolate('products',{"query":{"percolate":{"documents":[{"title":"nice pair of shoes","color":"blue"},{"title":"beautiful bag"}]}}})res = await searchApi.percolate('products',{"query":{"percolate":{"documents":[{"title":"nice pair of shoes","color":"blue"},{"title":"beautiful bag"}]}}});percolateRequest = new PercolateRequest();
query = new HashMap<String,Object>(){{
put("percolate",new HashMap<String,Object >(){{
put("documents", new ArrayList<Object>(){{
add(new HashMap<String,Object >(){{
put("title","nice pair of shoes");
put("color","blue");
}});
add(new HashMap<String,Object >(){{
put("title","beautiful bag");
}});
}});
}});
}};
percolateRequest.query(query);
searchApi.percolate("products",percolateRequest);var doc1 = new Dictionary<string, Object>();
doc1.Add("title","nice pair of shoes");
doc1.Add("color","blue");
var doc2 = new Dictionary<string, Object>();
doc2.Add("title","beautiful bag");
var docs = new List<Object> {doc1, doc2};
Dictionary<string, Object> percolateDoc = new Dictionary<string, Object> {{ "documents", docs }};
Dictionary<string, Object> query = new Dictionary<string, Object> {{ "percolate", percolateDoc }};
PercolateRequest percolateRequest = new PercolateRequest(query=query);
searchApi.Percolate("products",percolateRequest);let mut percolate_doc_fields1 = HashMap::new();
percolate_doc_fields1.insert("title".to_string(), "nice pair of shoes");
percolate_doc_fields1.insert("color".to_string(), "blue");
let mut percolate_doc_fields2 = HashMap::new();
percolate_doc_fields2.insert("title".to_string(), "beautiful bag");
let mut percolate_doc_fields_list: [HashMap; 2] = [percolate_doc_fields1, percolate_doc_fields2];
let mut percolate_doc = HashMap::new();
percolate_doc.insert("documents".to_string(), percolate_doc_fields_list);
let percolate_query = PercolateRequestQuery::new(serde_json::json!(percolate_doc));
let percolate_req = PercolateRequest::new(percolate_query);
search_api.percolate("products", percolate_req).await;docs = [ {title : 'What a nice bag'}, {title : 'Really nice shoes'} ];
res = await searchApi.percolate('test_pq', { query: { percolate: { documents : docs } } } );doc1 := map[string]interface{} {"title": "What a nice bag"}
doc2 := map[string]interface{} {"title": "Really nice shoes"}
query := []interface{} {doc1, doc2}
percolateRequestQuery := manticoreclient.NewPercolateQuery(query)
percolateRequest := manticoreclient.NewPercolateRequest(percolateRequestQuery)
res, _, _ := apiClient.SearchAPI.Percolate(context.Background(), "test_pq").PercolateRequest(*percolateRequest).Execute()+---------------------+-----------+--------------+------+---------------------------+
| id | documents | query | tags | filters |
+---------------------+-----------+--------------+------+---------------------------+
| 1657852401006149635 | 1 | @title shoes | | color IN ('blue, 'green') |
| 1657852401006149637 | 2 | @title bag | | |
+---------------------+-----------+--------------+------+---------------------------+{
"took": 0,
"timed_out": false,
"hits": {
"total": 2,
"max_score": 1,
"hits": [
{
"table": "products",
"_type": "doc",
"_id": 1657852401006149644,
"_score": "1",
"_source": {
"query": {
"ql": "@title bag"
}
},
"fields": {
"_percolator_document_slot": [
2
]
}
},
{
"table": "products",
"_type": "doc",
"_id": 1657852401006149646,
"_score": "1",
"_source": {
"query": {
"ql": "@title shoes"
}
},
"fields": {
"_percolator_document_slot": [
1
]
}
}
]
}
}Array
(
[took] => 23
[timed_out] =>
[hits] => Array
(
[total] => 2
[max_score] => 1
[hits] => Array
(
[0] => Array
(
[_index] => products
[_type] => doc
[_id] => 2810781492890828819
[_score] => 1
[_source] => Array
(
[query] => Array
(
[match] => Array
(
[title] => bag
)
)
)
[fields] => Array
(
[_percolator_document_slot] => Array
(
[0] => 2
)
)
)
[1] => Array
(
[_index] => products
[_type] => doc
[_id] => 2810781492890828821
[_score] => 1
[_source] => Array
(
[query] => Array
(
[match] => Array
(
[title] => shoes
)
)
)
[fields] => Array
(
[_percolator_document_slot] => Array
(
[0] => 1
)
)
)
)
)
){'hits': {'hits': [{u'_id': u'2811025403043381494',
u'table': u'products',
u'_score': u'1',
u'_source': {u'query': {u'ql': u'@title bag'}},
u'_type': u'doc',
u'fields': {u'_percolator_document_slot': [2]}},
{u'_id': u'2811025403043381496',
u'table': u'products',
u'_score': u'1',
u'_source': {u'query': {u'ql': u'@title shoes'}},
u'_type': u'doc',
u'fields': {u'_percolator_document_slot': [1]}}],
'total': 2},
'profile': None,
'timed_out': False,
'took': 0}{'hits': {'hits': [{u'_id': u'2811025403043381494',
u'table': u'products',
u'_score': u'1',
u'_source': {u'query': {u'ql': u'@title bag'}},
u'_type': u'doc',
u'fields': {u'_percolator_document_slot': [2]}},
{u'_id': u'2811025403043381496',
u'table': u'products',
u'_score': u'1',
u'_source': {u'query': {u'ql': u'@title shoes'}},
u'_type': u'doc',
u'fields': {u'_percolator_document_slot': [1]}}],
'total': 2},
'profile': None,
'timed_out': False,
'took': 0}{
"took": 6,
"timed_out": false,
"hits": {
"total": 2,
"hits": [
{
"table": "products",
"_type": "doc",
"_id": 2811045522851233808,
"_score": "1",
"_source": {
"query": {
"ql": "@title bag"
}
},
"fields": {
"_percolator_document_slot": [
2
]
}
},
{
"table": "products",
"_type": "doc",
"_id": 2811045522851233810,
"_score": "1",
"_source": {
"query": {
"ql": "@title shoes"
}
},
"fields": {
"_percolator_document_slot": [
1
]
}
}
]
}
}class SearchResponse {
took: 0
timedOut: false
hits: class SearchResponseHits {
total: 2
maxScore: 1
hits: [{_index=products, _type=doc, _id=2811045522851234133, _score=1, _source={query={ql=@title bag}}, fields={_percolator_document_slot=[2]}}, {_index=products, _type=doc, _id=2811045522851234135, _score=1, _source={query={ql=@title shoes}}, fields={_percolator_document_slot=[1]}}]
aggregations: null
}
profile: null
}class SearchResponse {
took: 0
timedOut: false
hits: class SearchResponseHits {
total: 2
maxScore: 1
hits: [{_index=products, _type=doc, _id=2811045522851234133, _score=1, _source={query={ql=@title bag}}, fields={_percolator_document_slot=[2]}}, {_index=products, _type=doc, _id=2811045522851234135, _score=1, _source={query={ql=@title shoes}}, fields={_percolator_document_slot=[1]}}]
aggregations: null
}
profile: null
}class SearchResponse {
took: 0
timedOut: false
hits: class SearchResponseHits {
total: 2
maxScore: 1
hits: [{_index=products, _type=doc, _id=2811045522851234133, _score=1, _source={query={ql=@title bag}}, fields={_percolator_document_slot=[2]}}, {_index=products, _type=doc, _id=2811045522851234135, _score=1, _source={query={ql=@title shoes}}, fields={_percolator_document_slot=[1]}}]
aggregations: null
}
profile: null
}{
"took": 0,
"timed_out": false,
"hits": {
"total": 2,
"hits": [
{
"table": "test_pq",
"_type": "doc",
"_id": 1657852401006149661,
"_score": "1",
"_source": {
"query": {
"ql": "@title bag"
}
},
"fields": {
"_percolator_document_slot": [
1
]
}
},
{
"table": "test_pq",
"_type": "doc",
"_id": 1657852401006149662,
"_score": "1",
"_source": {
"query": {
"ql": "@title shoes"
}
},
"fields": {
"_percolator_document_slot": [
1
]
}
}
]
}
}{
"took": 0,
"timed_out": false,
"hits": {
"total": 2,
"hits": [
{
"table": "test_pq",
"_type": "doc",
"_id": 1657852401006149661,
"_score": "1",
"_source": {
"query": {
"ql": "@title bag"
}
},
"fields": {
"_percolator_document_slot": [
1
]
}
},
{
"table": "test_pq",
"_type": "doc",
"_id": 1657852401006149662,
"_score": "1",
"_source": {
"query": {
"ql": "@title shoes"
}
},
"fields": {
"_percolator_document_slot": [
1
]
}
}
]
}
}默认情况下,匹配的文档 id 对应于您提供列表中的相对编号。但是,在某些情况下,每个文档已经有自己的 id。对于这种情况,CALL PQ 有一个选项 'id field name' as docs_id。
请注意,如果通过提供的字段名找不到 id,则 PQ 规则不会显示在结果中。
此选项仅适用于通过 SQL 使用 CALL PQ。
- SQL
CALL PQ('products', '[{"id": 123, "title": "nice pair of shoes", "color": "blue"}, {"id": 456, "title": "beautiful bag"}]', 1 as query, 'id' as docs_id, 1 as docs);+---------------------+-----------+--------------+------+---------------------------+
| id | documents | query | tags | filters |
+---------------------+-----------+--------------+------+---------------------------+
| 1657852401006149664 | 456 | @title bag | | |
| 1657852401006149666 | 123 | @title shoes | | color IN ('blue, 'green') |
+---------------------+-----------+--------------+------+---------------------------+当使用带有独立 JSON 的 CALL PQ 时,您可以使用选项 1 作为 skip_bad_json 来跳过输入中的任何无效 JSON。在下面的示例中,第 2 个查询由于无效的 JSON 而失败,但第 3 个查询通过使用 1 作为 skip_bad_json 避免了错误。请记住,当通过 HTTP 发送 JSON 查询时,此选项不可用,因为在这种情况下整个 JSON 查询必须是有效的。
- SQL
CALL PQ('products', ('{"title": "nice pair of shoes", "color": "blue"}', '{"title": "beautiful bag"}'));
CALL PQ('products', ('{"title": "nice pair of shoes", "color": "blue"}', '{"title": "beautiful bag}'));
CALL PQ('products', ('{"title": "nice pair of shoes", "color": "blue"}', '{"title": "beautiful bag}'), 1 as skip_bad_json);+---------------------+
| id |
+---------------------+
| 1657852401006149635 |
| 1657852401006149637 |
+---------------------+
ERROR 1064 (42000): Bad JSON objects in strings: 2
+---------------------+
| id |
+---------------------+
| 1657852401006149635 |
+---------------------+Percolate 查询是为高吞吐量和大数据量设计的。为了优化性能以实现更低的延迟和更高的吞吐量,请考虑以下内容。
percolate 表有两种分布模式,以及 percolate 查询如何针对它工作:
- 稀疏(默认)。 适用于:大量文档,镜像 PQ 表。当您的文档集很大但存储在 PQ 表中的查询集较小时,稀疏模式是有益的。在此模式下,您传递的文档批次将被分配给多个代理,因此每个节点只处理请求中文档的一部分。Manticore 会拆分您的文档集并在镜像之间分发块。一旦代理完成查询处理,Manticore 会收集并合并结果,返回一个最终的查询集,就像它来自单个表一样。使用复制来辅助此过程。
- 分片。 适用于:大量 PQ 规则,规则分布在多个 PQ 表中。在此模式下,整个文档集会广播到分布式 PQ 表的所有表,而不会初始拆分文档。当推送的文档集相对较小时,但存储的查询数量很大时,这种方式是有益的。在这种情况下,更适合在每个节点上只存储部分 PQ 规则,然后合并从处理相同文档集但针对不同 PQ 规则集的节点返回的结果。此模式必须显式设置,因为它意味着网络负载增加,并且期望表具有不同的 PQ,这一点是复制无法开箱即用实现的。
假设您有定义为:
table pq_d2
{
type = distributed
agent = 127.0.0.1:6712:pq
agent = 127.0.0.1:6712:ptitle
}
“pq”和“ptitle”各自包含:
- SQL
- JSON
- PHP
- Python
- Python-asyncio
- javascript
- Java
- C#
- Rust
- TypeScript
- Go
SELECT * FROM pq;POST /pq/pq/search$params = [
'table' => 'pq',
'body' => [
]
];
$response = $client->pq()->search($params);searchApi.search({"table":"pq","query":{"match_all":{}}})await searchApi.search({"table":"pq","query":{"match_all":{}}})res = await searchApi.search({"table":"pq","query":{"match_all":{}}});Map<String,Object> query = new HashMap<String,Object>();
query.put("match_all",null);
SearchRequest searchRequest = new SearchRequest();
searchRequest.setIndex("pq");
searchRequest.setQuery(query);
SearchResponse searchResponse = searchApi.search(searchRequest);object query = new { match_all=null };
SearchRequest searchRequest = new SearchRequest("pq", query);
SearchResponse searchResponse = searchApi.Search(searchRequest);let query = SearchQuery::new();
let search_req = SearchRequest {
table: "pq".to_string(),
query: Some(Box::new(query)),
..Default::default(),
};
let search_res = search_api.search(search_req).await;res = await searchApi.search({"table":"test_pq","query":{"match_all":{}}});query := map[string]interface{} {}
percolateRequestQuery := manticoreclient.NewPercolateRequestQuery(query)
percolateRequest := manticoreclient.NewPercolateRequest(percolateRequestQuery)
res, _, _ := apiClient.SearchAPI.Percolate(context.Background(), "test_pq").PercolateRequest(*percolateRequest).Execute()+------+-------------+------+-------------------+
| id | query | tags | filters |
+------+-------------+------+-------------------+
| 1 | filter test | | gid>=10 |
| 2 | angry | | gid>=10 OR gid<=3 |
+------+-------------+------+-------------------+
2 rows in set (0.01 sec){
"took":0,
"timed_out":false,
"hits":{
"total":2,
"hits":[
{
"_id": 1,
"_score":1,
"_source":{
"query":{ "ql":"filter test" },
"tags":"",
"filters":"gid>=10"
}
},
{
"_id": 2,
"_score":1,
"_source":{
"query":{"ql":"angry"},
"tags":"",
"filters":"gid>=10 OR gid<=3"
}
}
]
}
}(
[took] => 0
[timed_out] =>
[hits] =>
(
[total] => 2
[hits] =>
(
[0] =>
(
[_id] => 1
[_score] => 1
[_source] =>
(
[query] =>
(
[ql] => filter test
)
[tags] =>
[filters] => gid>=10
)
),
[1] =>
(
[_id] => 1
[_score] => 1
[_source] =>
(
[query] =>
(
[ql] => angry
)
[tags] =>
[filters] => gid>=10 OR gid<=3
)
)
)
)
){'hits': {'hits': [{u'_id': u'2811025403043381501',
u'_score': 1,
u'_source': {u'filters': u"gid>=10",
u'query': u'filter test',
u'tags': u''}},
{u'_id': u'2811025403043381502',
u'_score': 1,
u'_source': {u'filters': u"gid>=10 OR gid<=3",
u'query': u'angry',
u'tags': u''}}],
'total': 2},
'profile': None,
'timed_out': False,
'took': 0}{'hits': {'hits': [{u'_id': u'2811025403043381501',
u'_score': 1,
u'_source': {u'filters': u"gid>=10",
u'query': u'filter test',
u'tags': u''}},
{u'_id': u'2811025403043381502',
u'_score': 1,
u'_source': {u'filters': u"gid>=10 OR gid<=3",
u'query': u'angry',
u'tags': u''}}],
'total': 2},
'profile': None,
'timed_out': False,
'took': 0}{"hits": {"hits": [{"_id": 2811025403043381501,
"_score": 1,
"_source": {"filters": u"gid>=10",
"query": "filter test",
"tags": ""}},
{"_id": 2811025403043381502,
"_score": 1,
"_source": {"filters": u"gid>=10 OR gid<=3",
"query": "angry",
"tags": ""}}],
"total": 2},
"timed_out": false,
"took": 0}class SearchResponse {
took: 0
timedOut: false
hits: class SearchResponseHits {
total: 2
maxScore: null
hits: [{_id=2811045522851233962, _score=1, _source={filters=gid>=10, query=filter test, tags=}}, {_id=2811045522851233951, _score=1, _source={filters=gid>=10 OR gid<=3, query=angry,tags=}}]
aggregations: null
}
profile: null
}class SearchResponse {
took: 0
timedOut: false
hits: class SearchResponseHits {
total: 2
maxScore: null
hits: [{_id=2811045522851233962, _score=1, _source={filters=gid>=10, query=filter test, tags=}}, {_id=2811045522851233951, _score=1, _source={filters=gid>=10 OR gid<=3, query=angry,tags=}}]
aggregations: null
}
profile: null
}class SearchResponse {
took: 0
timedOut: false
hits: class SearchResponseHits {
total: 2
maxScore: null
hits: [{_id=2811045522851233962, _score=1, _source={filters=gid>=10, query=filter test, tags=}}, {_id=2811045522851233951, _score=1, _source={filters=gid>=10 OR gid<=3, query=angry,tags=}}]
aggregations: null
}
profile: null
}{
'hits':
{
'hits':
[{
'_id': '2811025403043381501',
'_score': 1,
'_source':
{
'filters': "gid>=10",
'query': 'filter test',
'tags': ''
}
},
{
'_id':
'2811025403043381502',
'_score': 1,
'_source':
{
'filters': "gid>=10 OR gid<=3",
'query': 'angry',
'tags': ''
}
}],
'total': 2
},
'profile': None,
'timed_out': False,
'took': 0
}{
'hits':
{
'hits':
[{
'_id': '2811025403043381501',
'_score': 1,
'_source':
{
'filters': "gid>=10",
'query': 'filter test',
'tags': ''
}
},
{
'_id':
'2811025403043381502',
'_score': 1,
'_source':
{
'filters': "gid>=10 OR gid<=3",
'query': 'angry',
'tags': ''
}
}],
'total': 2
},
'profile': None,
'timed_out': False,
'took': 0
}然后您在分布式表上执行 CALL PQ,并传入几个文档。
- SQL
- JSON
- PHP
- Python
- Python-asyncio
- javascript
- Java
- C#
- Rust
- TypeScript
- Go
CALL PQ ('pq_d2', ('{"title":"angry test", "gid":3 }', '{"title":"filter test doc2", "gid":13}'), 1 AS docs);POST /pq/pq/search -d '
"query":
{
"percolate":
{
"documents" : [
{ "title": "angry test", "gid": 3 },
{ "title": "filter test doc2", "gid": 13 }
]
}
}
'$params = [
'table' => 'pq',
'body' => [
'query' => [
'percolate' => [
'documents' => [
[
'title'=>'angry test',
'gid' => 3
],
[
'title'=>'filter test doc2',
'gid' => 13
],
]
]
]
]
];
$response = $client->pq()->search($params);searchApi.percolate('pq',{"percolate":{"documents":[{"title":"angry test","gid":3},{"title":"filter test doc2","gid":13}]}})await searchApi.percolate('pq',{"percolate":{"documents":[{"title":"angry test","gid":3},{"title":"filter test doc2","gid":13}]}})res = await searchApi.percolate('pq',{"percolate":{"documents":[{"title":"angry test","gid":3},{"title":"filter test doc2","gid":13}]}});percolateRequest = new PercolateRequest();
query = new HashMap<String,Object>(){{
put("percolate",new HashMap<String,Object >(){{
put("documents", new ArrayList<Object>(){{
add(new HashMap<String,Object >(){{
put("title","angry test");
put("gid",3);
}});
add(new HashMap<String,Object >(){{
put("title","filter test doc2");
put("gid",13);
}});
}});
}});
}};
percolateRequest.query(query);
searchApi.percolate("pq",percolateRequest);var doc1 = new Dictionary<string, Object>();
doc1.Add("title","angry test");
doc1.Add("gid",3);
var doc2 = new Dictionary<string, Object>();
doc2.Add("title","filter test doc2");
doc2.Add("gid",13);
var docs = new List<Object> {doc1, doc2};
Dictionary<string, Object> percolateDoc = new Dictionary<string, Object> {{ "documents", docs }};
Dictionary<string, Object> query = new Dictionary<string, Object> {{ "percolate", percolateDoc }};
PercolateRequest percolateRequest = new PercolateRequest(query=query);
searchApi.Percolate("pq",percolateRequest);let mut percolate_doc_fields1 = HashMap::new();
percolate_doc_fields1.insert("title".to_string(), "angry test");
percolate_doc_fields1.insert("gid".to_string(), 3);
let mut percolate_doc_fields2 = HashMap::new();
percolate_doc_fields2.insert("title".to_string(), "filter test doc2");
percolate_doc_fields2.insert("gid".to_string(), 13);
let mut percolate_doc_fields_list: [HashMap; 2] = [percolate_doc_fields1, percolate_doc_fields2];
let mut percolate_doc = HashMap::new();
percolate_doc.insert("documents".to_string(), percolate_doc_fields_list);
let percolate_query = PercolateRequestQuery::new(serde_json::json!(percolate_doc));
let percolate_req = PercolateRequest::new(percolate_query);
search_api.percolate("pq", percolate_req).await;docs = [ {title : 'What a nice bag'}, {title : 'Really nice shoes'} ];
res = await searchApi.percolate('test_pq', { query: { percolate: { documents : docs } } } );doc1 := map[string]interface{} {"title": "What a nice bag"}
doc2 := map[string]interface{} {"title": "Really nice shoes"}
query := []interface{} {doc1, doc2}
percolateRequestQuery := manticoreclient.NewPercolateQuery(query)
percolateRequest := manticoreclient.NewPercolateRequest(percolateRequestQuery)
res, _, _ := apiClient.SearchAPI.Percolate(context.Background(), "test_pq").PercolateRequest(*percolateRequest).Execute()+------+-----------+
| id | documents |
+------+-----------+
| 1 | 2 |
| 2 | 1 |
+------+-----------+{
"took":0,
"timed_out":false,
"hits":{
"total":2,"hits":[
{
"_id": 2,
"_score":1,
"_source":{
"query":{"title":"angry"},
"tags":"",
"filters":"gid>=10 OR gid<=3"
}
}
{
"_id": 1,
"_score":1,
"_source":{
"query":{"ql":"filter test"},
"tags":"",
"filters":"gid>=10"
}
},
]
}
}(
[took] => 0
[timed_out] =>
[hits] =>
(
[total] => 2
[hits] =>
(
[0] =>
(
[_index] => pq
[_type] => doc
[_id] => 2
[_score] => 1
[_source] =>
(
[query] =>
(
[ql] => angry
)
[tags] =>
[filters] => gid>=10 OR gid<=3
),
[fields] =>
(
[_percolator_document_slot] =>
(
[0] => 1
)
)
),
[1] =>
(
[_index] => pq
[_id] => 1
[_score] => 1
[_source] =>
(
[query] =>
(
[ql] => filter test
)
[tags] =>
[filters] => gid>=10
)
[fields] =>
(
[_percolator_document_slot] =>
(
[0] => 0
)
)
)
)
)
){'hits': {'hits': [{u'_id': u'2811025403043381480',
u'table': u'pq',
u'_score': u'1',
u'_source': {u'query': {u'ql': u'angry'},u'tags':u'',u'filters':u"gid>=10 OR gid<=3"},
u'_type': u'doc',
u'fields': {u'_percolator_document_slot': [1]}},
{u'_id': u'2811025403043381501',
u'table': u'pq',
u'_score': u'1',
u'_source': {u'query': {u'ql': u'filter test'},u'tags':u'',u'filters':u"gid>=10"},
u'_type': u'doc',
u'fields': {u'_percolator_document_slot': [1]}}],
'total': 2},
'profile': None,
'timed_out': False,
'took': 0}{'hits': {'hits': [{u'_id': u'2811025403043381480',
u'table': u'pq',
u'_score': u'1',
u'_source': {u'query': {u'ql': u'angry'},u'tags':u'',u'filters':u"gid>=10 OR gid<=3"},
u'_type': u'doc',
u'fields': {u'_percolator_document_slot': [1]}},
{u'_id': u'2811025403043381501',
u'table': u'pq',
u'_score': u'1',
u'_source': {u'query': {u'ql': u'filter test'},u'tags':u'',u'filters':u"gid>=10"},
u'_type': u'doc',
u'fields': {u'_percolator_document_slot': [1]}}],
'total': 2},
'profile': None,
'timed_out': False,
'took': 0}{'hits': {'hits': [{u'_id': u'2811025403043381480',
u'table': u'pq',
u'_score': u'1',
u'_source': {u'query': {u'ql': u'angry'},u'tags':u'',u'filters':u"gid>=10 OR gid<=3"},
u'_type': u'doc',
u'fields': {u'_percolator_document_slot': [1]}},
{u'_id': u'2811025403043381501',
u'table': u'pq',
u'_score': u'1',
u'_source': {u'query': {u'ql': u'filter test'},u'tags':u'',u'filters':u"gid>=10"},
u'_type': u'doc',
u'fields': {u'_percolator_document_slot': [1]}}],
'total': 2},
'profile': None,
'timed_out': False,
'took': 0}class SearchResponse {
took: 10
timedOut: false
hits: class SearchResponseHits {
total: 2
maxScore: 1
hits: [{_index=pq, _type=doc, _id=2811045522851234165, _score=1, _source={query={ql=@title angry}}, fields={_percolator_document_slot=[1]}}, {_index=pq, _type=doc, _id=2811045522851234166, _score=1, _source={query={ql=@title filter test doc2}}, fields={_percolator_document_slot=[2]}}]
aggregations: null
}
profile: null
}class SearchResponse {
took: 10
timedOut: false
hits: class SearchResponseHits {
total: 2
maxScore: 1
hits: [{_index=pq, _type=doc, _id=2811045522851234165, _score=1, _source={query={ql=@title angry}}, fields={_percolator_document_slot=[1]}}, {_index=pq, _type=doc, _id=2811045522851234166, _score=1, _source={query={ql=@title filter test doc2}}, fields={_percolator_document_slot=[2]}}]
aggregations: null
}
profile: null
}class SearchResponse {
took: 10
timedOut: false
hits: class SearchResponseHits {
total: 2
maxScore: 1
hits: [{_index=pq, _type=doc, _id=2811045522851234165, _score=1, _source={query={ql=@title angry}}, fields={_percolator_document_slot=[1]}}, {_index=pq, _type=doc, _id=2811045522851234166, _score=1, _source={query={ql=@title filter test doc2}}, fields={_percolator_document_slot=[2]}}]
aggregations: null
}
profile: null
}{
"took": 0,
"timed_out": false,
"hits": {
"total": 2,
"hits": [
{
"table": "test_pq",
"_type": "doc",
"_id": 1657852401006149661,
"_score": "1",
"_source": {
"query": {
"ql": "@title bag"
}
},
"fields": {
"_percolator_document_slot": [
1
]
}
},
{
"table": "test_pq",
"_type": "doc",
"_id": 1657852401006149662,
"_score": "1",
"_source": {
"query": {
"ql": "@title shoes"
}
},
"fields": {
"_percolator_document_slot": [
1
]
}
}
]
}
}{
"took": 0,
"timed_out": false,
"hits": {
"total": 2,
"hits": [
{
"table": "test_pq",
"_type": "doc",
"_id": 1657852401006149661,
"_score": "1",
"_source": {
"query": {
"ql": "@title bag"
}
},
"fields": {
"_percolator_document_slot": [
1
]
}
},
{
"table": "test_pq",
"_type": "doc",
"_id": 1657852401006149662,
"_score": "1",
"_source": {
"query": {
"ql": "@title shoes"
}
},
"fields": {
"_percolator_document_slot": [
1
]
}
}
]
}
}在前面的示例中,我们使用了默认的 稀疏 模式。为了演示 分片 模式,让我们创建一个由 2 个本地 PQ 表组成的分布式 PQ 表,并向 "products1" 添加 2 个文档,向 "products2" 添加 1 个文档:
create table products1(title text, color string) type='pq';
create table products2(title text, color string) type='pq';
create table products_distributed type='distributed' local='products1' local='products2';
INSERT INTO products1(query) values('@title bag');
INSERT INTO products1(query,filters) values('@title shoes', 'color=\'red\'');
INSERT INTO products2(query,filters) values('@title shoes', 'color in (\'blue\', \'green\')');
现在,如果您向 CALL PQ 添加 'sharded' as mode,它将把文档发送到所有代理的表(在此情况下仅本地表,但它们可以是远程的以利用外部硬件)。此模式通过 JSON 接口不可用。
- SQL
CALL PQ('products_distributed', ('{"title": "nice pair of shoes", "color": "blue"}', '{"title": "beautiful bag"}'), 'sharded' as mode, 1 as query);+---------------------+--------------+------+---------------------------+
| id | query | tags | filters |
+---------------------+--------------+------+---------------------------+
| 1657852401006149639 | @title bag | | |
| 1657852401006149643 | @title shoes | | color IN ('blue, 'green') |
+---------------------+--------------+------+---------------------------+请注意,配置中代理镜像的语法(当一个 agent 行分配了多个主机,用 | 分隔)与 CALL PQ 查询模式无关。每个 agent 始终代表一个节点,无论为该代理指定了多少 HA 镜像。
在某些情况下,您可能想要获取有关 percolate 查询性能的更多详细信息。为此,有一个选项 1 as verbose,该选项仅通过 SQL 可用,允许您保存更多性能指标。您可以使用 SHOW META 查询查看它们,该查询可以在 CALL PQ 之后运行。有关更多信息,请参见 SHOW META。
- 1 as verbose
- 0 as verbose
CALL PQ('products', ('{"title": "nice pair of shoes", "color": "blue"}', '{"title": "beautiful bag"}'), 1 as verbose); show meta;CALL PQ('products', ('{"title": "nice pair of shoes", "color": "blue"}', '{"title": "beautiful bag"}'), 0 as verbose); show meta;+---------------------+
| id |
+---------------------+
| 1657852401006149644 |
| 1657852401006149646 |
+---------------------+
+-------------------------+-----------+
| Variable name | Value |
+-------------------------+-----------+
| total | 0.000 sec |
| setup | 0.000 sec |
| queries_matched | 2 |
| queries_failed | 0 |
| document_matched | 2 |
| total_queries_stored | 3 |
| term_only_queries | 3 |
| fast_rejected_queries | 0 |
| time_per_query | 27, 10 |
| time_of_matched_queries | 37 |
+-------------------------+-----------++---------------------+
| id |
+---------------------+
| 1657852401006149644 |
| 1657852401006149646 |
+---------------------+
+-----------------------+-----------+
| Variable name | Value |
+-----------------------+-----------+
| total | 0.000 sec |
| queries_matched | 2 |
| queries_failed | 0 |
| document_matched | 2 |
| total_queries_stored | 3 |
| term_only_queries | 3 |
| fast_rejected_queries | 0 |
+-----------------------+-----------+自动完成,或称词语补全,是在你输入时预测并建议单词或短语的结尾。它常用于:
- 网站上的搜索框
- 搜索引擎中的建议
- 应用程序中的文本字段
Manticore 提供了一个高级的自动完成功能,在你输入时给出建议,类似于知名搜索引擎中的功能。这有助于加快搜索速度,让用户更快找到所需内容。
除了基本的自动完成功能外,Manticore 还包含增强用户体验的高级功能:
- 拼写纠正(模糊匹配): Manticore 的自动完成通过使用识别和修正常见错误的算法,帮助纠正拼写错误。这意味着即使你输入错误,也能找到你想要的内容。
- 键盘布局自动检测: Manticore 能够识别你使用的键盘布局。这在多语言环境中非常有用,或者当你不小心使用了错误的语言输入时。例如,如果你错误地输入了“ghbdtn”,Manticore 会知道你想输入的是“привет”(俄语中的“你好”),并建议正确的单词。
Manticore 的自动完成可以根据不同需求和设置进行定制,是许多应用的灵活工具。

注意:
CALL AUTOCOMPLETE和/autocomplete需要 Manticore Buddy。如果无法使用,请确保已安装 Buddy。
要在 Manticore 中使用自动完成,请使用 CALL AUTOCOMPLETE SQL 语句或其 JSON 等价的 /autocomplete。此功能基于你的索引数据提供词语补全建议。
在继续之前,请确保你打算用于自动完成的表已启用 infixes。
注意: 表设置中有一个针对 min_infix_len 的自动检查,使用 30 秒缓存以提升 CALL AUTOCOMPLETE 的性能。修改表后,首次使用 CALL AUTOCOMPLETE 可能会有短暂延迟(通常不明显)。只有成功的结果会被缓存,因此如果你删除表或禁用 min_infix_len,CALL AUTOCOMPLETE 可能暂时返回错误结果,直到最终显示与 min_infix_len 相关的错误。
CALL AUTOCOMPLETE('query_beginning', 'table', [...options]);
POST /autocomplete
{
"table":"table_name",
"query":"query_beginning"
[,"options": {<autocomplete options>}]
}
layouts:逗号分隔的键盘布局代码字符串,用于检测因键盘布局不匹配导致的输入错误(例如使用错误布局时输入“ghbdtn”而非“привет”)。Manticore 会比较不同布局中字符的位置以建议纠正。至少需要 2 个布局才能有效检测不匹配。可用选项:us, ru, ua, se, pt, no, it, gr, uk, fr, es, dk, de, ch, br, bg, be(更多详情见这里)。默认:无fuzziness:0、1或2(默认:2)。查找拼写错误的最大 Levenshtein 距离。设置为0禁用模糊匹配preserve:0或1(默认:0)。设置为1时,保留搜索结果中没有模糊匹配的词(例如,“hello wrld”返回“hello wrld”和“hello world”)。设置为0时,仅返回有成功模糊匹配的词(例如,“hello wrld”仅返回“hello world”)。对保留短词或可能不存在于 Manticore Search 中的专有名词特别有用prepend:布尔值(SQL 中为 0/1)。若为 true(1),在最后一个词前添加星号以进行前缀扩展(例如,*word)append:布尔值(SQL 中为 0/1)。若为 true(1),在最后一个词后添加星号以进行后缀扩展(例如,word*)expansion_len:最后一个词扩展的字符数。默认:10force_bigrams:布尔值(SQL 中为 0/1)。强制对所有词长使用二元组(2 字符 n-gram)而非三元组,这可以改善对字符调换错误的匹配。默认:0(对长度≥6的词使用三元组)
- SQL
- SQL with no fuzzy search
- JSON
- SQL with preserve option
- JSON with preserve option
mysql> CALL AUTOCOMPLETE('hello', 'comment');
+------------+
| query |
+------------+
| hello |
| helio |
| hell |
| shell |
| nushell |
| powershell |
| well |
| help |
+------------+mysql> CALL AUTOCOMPLETE('hello', 'comment', 0 as fuzziness);
+-------+
| query |
+-------+
| hello |
+-------+POST /autocomplete
{
"table":"comment",
"query":"hello"
}mysql> CALL AUTOCOMPLETE('hello wrld', 'comment', 1 as preserve);
+------------+
| query |
+------------+
| hello wrld |
| hello world|
+------------+POST /autocomplete
{
"table":"comment",
"query":"hello wrld",
"options": {
"preserve": 1
}
}[
{
"total": 8,
"error": "",
"warning": "",
"columns": [
{
"query": {
"type": "string"
}
}
],
"data": [
{
"query": "hello"
},
{
"query": "helio"
},
{
"query": "hell"
},
{
"query": "shell"
},
{
"query": "nushell"
},
{
"query": "powershell"
},
{
"query": "well"
},
{
"query": "help"
}
]
}
][
{
"total": 2,
"error": "",
"warning": "",
"columns": [
{
"query": {
"type": "string"
}
}
],
"data": [
{
"query": "hello wrld"
},
{
"query": "hello world"
}
]
}
]- SQL
- JSON
mysql> CALL AUTOCOMPLETE('ipohne', 'products', 1 as force_bigrams);POST /autocomplete
{
"table":"products",
"query":"ipohne",
"options": {
"force_bigrams": 1
}
}+--------+
| query |
+--------+
| iphone |
+--------+[
{
"total": 1,
"error": "",
"warning": "",
"columns": [
{
"query": {
"type": "string"
}
}
],
"data": [
{
"query": "iphone"
}
]
}
]- 此演示 展示了自动完成功能:

- 关于模糊搜索和自动完成的博客文章 - https://manticoresearch.com/blog/new-fuzzy-search-and-autocomplete/
虽然 CALL AUTOCOMPLETE 是大多数用例推荐的方法,Manticore 还支持其他可控且可定制的方式来实现自动完成功能:
要自动完成句子,可以使用 中缀搜索。你可以通过提供文档字段的开头并:
我们的博客中有一篇相关文章,还有一个互动课程。一个快速示例如下:
- 假设你有一篇文档:
My cat loves my dog. The cat (Felis catus) is a domestic species of small carnivorous mammal. - 然后你可以使用
^、""和*,当用户输入时,你可以发起类似这样的查询:^"m*"、^"my *"、^"my c*"、^"my ca*"等等 - 它会找到该文档,如果你还使用了高亮,你将得到类似这样的结果:
<b>My cat</b> loves my dog. The cat ( ...
在某些情况下,你只需要自动补全一个单词或几个单词。这时,你可以使用 CALL KEYWORDS。
CALL KEYWORDS 通过 SQL 接口提供,可以用来检查关键词是如何被分词的,或者获取特定关键词的分词形式。如果表启用了infixes,它允许你快速找到给定关键词的可能结尾,非常适合自动补全功能。
这是通用中缀搜索的一个很好的替代方案,因为它性能更高,只需要表的字典,而不需要文档本身。
CALL KEYWORDS(text, table [, options])
CALL KEYWORDS 语句将文本拆分为关键词。它返回关键词的分词和规范化形式,如果需要,还会返回关键词统计信息。此外,当表启用了词形还原器时,还会提供查询中每个关键词的位置以及所有分词关键词的形式。
| 参数 | 描述 |
|---|---|
| text | 要拆分成关键词的文本 |
| table | 用于获取文本处理设置的表名 |
| 0/1 as stats | 是否显示关键词统计,默认是 0 |
| 0/1 as fold_wildcards | 是否折叠通配符,默认是 0 |
| 0/1 as fold_lemmas | 是否折叠形态词形,默认是 0 |
| 0/1 as fold_blended | 是否折叠混合词,默认是 0 |
| N as expansion_limit | 覆盖服务器配置中定义的expansion_limit,默认是 0(使用配置中的值) |
| docs/hits as sort_mode | 按 'docs' 或 'hits' 对输出结果排序。默认不排序。 |
| jieba_mode | 查询的结巴分词模式。详情见jieba_mode |
示例展示了如果假设用户想要获取“my cat ...”的自动补全结果时的工作方式。因此,在应用端你只需要为每个新词从“normalized”列建议用户可能的结尾。通常使用 'hits' as sort_mode 或 'docs' as sort_mode 按点击数或文档数排序是有意义的。
- Examples
MySQL [(none)]> CALL KEYWORDS('m*', 't', 1 as stats);
+------+-----------+------------+------+------+
| qpos | tokenized | normalized | docs | hits |
+------+-----------+------------+------+------+
| 1 | m* | my | 1 | 2 |
| 1 | m* | mammal | 1 | 1 |
+------+-----------+------------+------+------+
MySQL [(none)]> CALL KEYWORDS('my*', 't', 1 as stats);
+------+-----------+------------+------+------+
| qpos | tokenized | normalized | docs | hits |
+------+-----------+------------+------+------+
| 1 | my* | my | 1 | 2 |
+------+-----------+------------+------+------+
MySQL [(none)]> CALL KEYWORDS('c*', 't', 1 as stats, 'hits' as sort_mode);
+------+-----------+-------------+------+------+
| qpos | tokenized | normalized | docs | hits |
+------+-----------+-------------+------+------+
| 1 | c* | cat | 1 | 2 |
| 1 | c* | carnivorous | 1 | 1 |
| 1 | c* | catus | 1 | 1 |
+------+-----------+-------------+------+------+
MySQL [(none)]> CALL KEYWORDS('ca*', 't', 1 as stats, 'hits' as sort_mode);
+------+-----------+-------------+------+------+
| qpos | tokenized | normalized | docs | hits |
+------+-----------+-------------+------+------+
| 1 | ca* | cat | 1 | 2 |
| 1 | ca* | carnivorous | 1 | 1 |
| 1 | ca* | catus | 1 | 1 |
+------+-----------+-------------+------+------+
MySQL [(none)]> CALL KEYWORDS('cat*', 't', 1 as stats, 'hits' as sort_mode);
+------+-----------+------------+------+------+
| qpos | tokenized | normalized | docs | hits |
+------+-----------+------------+------+------+
| 1 | cat* | cat | 1 | 2 |
| 1 | cat* | catus | 1 | 1 |
+------+-----------+------------+------+------+这里有一个很好的技巧可以改进上述算法——使用bigram_index。当你为表启用它时,索引中不仅包含单个词,还包含相邻的每对词作为单独的标记。
这不仅可以预测当前词的结尾,还可以预测下一个词,这对于自动补全功能尤其有益。
- Examples
MySQL [(none)]> CALL KEYWORDS('m*', 't', 1 as stats, 'hits' as sort_mode);
+------+-----------+------------+------+------+
| qpos | tokenized | normalized | docs | hits |
+------+-----------+------------+------+------+
| 1 | m* | my | 1 | 2 |
| 1 | m* | mammal | 1 | 1 |
| 1 | m* | my cat | 1 | 1 |
| 1 | m* | my dog | 1 | 1 |
+------+-----------+------------+------+------+
MySQL [(none)]> CALL KEYWORDS('my*', 't', 1 as stats, 'hits' as sort_mode);
+------+-----------+------------+------+------+
| qpos | tokenized | normalized | docs | hits |
+------+-----------+------------+------+------+
| 1 | my* | my | 1 | 2 |
| 1 | my* | my cat | 1 | 1 |
| 1 | my* | my dog | 1 | 1 |
+------+-----------+------------+------+------+
MySQL [(none)]> CALL KEYWORDS('c*', 't', 1 as stats, 'hits' as sort_mode);
+------+-----------+--------------------+------+------+
| qpos | tokenized | normalized | docs | hits |
+------+-----------+--------------------+------+------+
| 1 | c* | cat | 1 | 2 |
| 1 | c* | carnivorous | 1 | 1 |
| 1 | c* | carnivorous mammal | 1 | 1 |
| 1 | c* | cat felis | 1 | 1 |
| 1 | c* | cat loves | 1 | 1 |
| 1 | c* | catus | 1 | 1 |
| 1 | c* | catus is | 1 | 1 |
+------+-----------+--------------------+------+------+
MySQL [(none)]> CALL KEYWORDS('ca*', 't', 1 as stats, 'hits' as sort_mode);
+------+-----------+--------------------+------+------+
| qpos | tokenized | normalized | docs | hits |
+------+-----------+--------------------+------+------+
| 1 | ca* | cat | 1 | 2 |
| 1 | ca* | carnivorous | 1 | 1 |
| 1 | ca* | carnivorous mammal | 1 | 1 |
| 1 | ca* | cat felis | 1 | 1 |
| 1 | ca* | cat loves | 1 | 1 |
| 1 | ca* | catus | 1 | 1 |
| 1 | ca* | catus is | 1 | 1 |
+------+-----------+--------------------+------+------+
MySQL [(none)]> CALL KEYWORDS('cat*', 't', 1 as stats, 'hits' as sort_mode);
+------+-----------+------------+------+------+
| qpos | tokenized | normalized | docs | hits |
+------+-----------+------------+------+------+
| 1 | cat* | cat | 1 | 2 |
| 1 | cat* | cat felis | 1 | 1 |
| 1 | cat* | cat loves | 1 | 1 |
| 1 | cat* | catus | 1 | 1 |
| 1 | cat* | catus is | 1 | 1 |
+------+-----------+------------+------+------+CALL KEYWORDS 支持分布式表,因此无论你的数据集多大,都可以从中受益。
拼写纠正,也称为:
- 自动纠正
- 文本纠正
- 修正拼写错误
- 容错拼写
- “你的意思是?”
等等,是一种软件功能,用于建议替代词或自动纠正您输入的文本。纠正输入文本的概念可以追溯到20世纪60年代,当时计算机科学家Warren Teitelman(他还发明了“撤销”命令)提出了一种称为D.W.I.M.(“做我意思的事”)的计算哲学。Teitelman认为,计算机不应只接受格式完全正确的指令,而应被编程为识别明显的错误。
第一个提供拼写纠正功能的知名产品是1993年发布的Microsoft Word 6.0。
拼写纠正可以通过几种方式实现,但重要的是要注意,没有纯粹的程序化方法能以较高质量将您错误输入的“ipone”转换为“iphone”。大多数情况下,系统必须基于某个数据集。该数据集可以是:
- 一个正确拼写单词的词典,词典又可以是:
- 基于您的真实数据。这里的想法是,词典中由您的数据组成的拼写大部分是正确的,系统尝试找到与输入单词最相似的单词(我们稍后将讨论如何用Manticore实现)。
- 或者基于与您的数据无关的外部词典。这里可能出现的问题是您的数据和外部词典差异过大:词典中可能缺少某些单词,而您的数据中可能缺少其他单词。
- 不仅基于词典,还基于上下文,例如,“white ber”会被纠正为“white bear”,而“dark ber”会被纠正为“dark beer”。上下文不仅可能是查询中的相邻单词,还可能是您的位置、时间、当前句子的语法(是否将“there”改为“their”)、您的搜索历史,以及几乎任何可能影响您意图的因素。
- 另一种经典方法是使用之前的搜索查询作为拼写纠正的数据集。这在自动补全功能中使用得更多,但对自动纠正也有意义。其思想是用户大多数拼写是正确的,因此我们可以使用他们的搜索历史中的单词作为事实来源,即使我们在文档中没有这些单词或不使用外部词典。这里也可以实现上下文感知。
Manticore提供了模糊搜索选项以及可用于自动拼写纠正的命令CALL QSUGGEST和CALL SUGGEST。
模糊搜索功能允许通过考虑搜索查询中的轻微变体或拼写错误,实现更灵活的匹配。它的工作方式类似于普通的SELECT SQL语句或/search JSON请求,但提供了额外的参数来控制模糊匹配行为。
注意:
fuzzy选项需要Manticore Buddy。如果不起作用,请确保已安装Buddy。
注意:
fuzzy选项不适用于多查询。
SELECT
...
MATCH('...')
...
OPTION fuzzy={0|1}
[, distance=N]
[, preserve={0|1}]
[, layouts='{be,bg,br,ch,de,dk,es,fr,uk,gr,it,no,pt,ru,se,ua,us}']
}
注意:通过SQL进行模糊搜索时,MATCH子句中不应包含除短语搜索操作符之外的任何全文操作符,并且应仅包含您希望匹配的单词。
- SQL
- SQL with additional filters
- JSON
- SQL with preserve option
- JSON with preserve option
SELECT * FROM mytable WHERE MATCH('someting') OPTION fuzzy=1, layouts='us,ua', distance=2;带有额外过滤条件的更复杂模糊搜索查询示例:
SELECT * FROM mytable WHERE MATCH('someting') OPTION fuzzy=1 AND (category='books' AND price < 20);POST /search
{
"table": "test",
"query": {
"bool": {
"must": [
{
"match": {
"*": "ghbdtn"
}
}
]
}
},
"options": {
"fuzzy": true,
"layouts": ["us", "ru"],
"distance": 2
}
}SELECT * FROM mytable WHERE MATCH('hello wrld') OPTION fuzzy=1, preserve=1;POST /search
{
"table": "test",
"query": {
"bool": {
"must": [
{
"match": {
"*": "hello wrld"
}
}
]
}
},
"options": {
"fuzzy": true,
"preserve": 1
}
}+------+-------------+
| id | content |
+------+-------------+
| 1 | something |
| 2 | some thing |
+------+-------------+
2 rows in set (0.00 sec)+------+-------------+
| id | content |
+------+-------------+
| 1 | hello wrld |
| 2 | hello world |
+------+-------------+
2 rows in set (0.00 sec)POST /search
{
"table": "table_name",
"query": {
<full-text query>
},
"options": {
"fuzzy": {true|false}
[,"layouts": ["be","bg","br","ch","de","dk","es","fr","uk","gr","it","no","pt","ru","se","ua","us"]]
[,"distance": N]
[,"preserve": {0|1}]
}
}
注意:如果您使用query_string,请注意它不支持除短语搜索操作符之外的全文操作符。查询字符串应仅由您希望匹配的单词组成。
fuzzy:开启或关闭模糊搜索。distance:设置匹配的Levenshtein距离。默认值为2。preserve:0或1(默认:0)。设置为1时,保留搜索结果中没有模糊匹配的单词(例如,“hello wrld”返回“hello wrld”和“hello world”)。设置为0时,仅返回成功模糊匹配的单词(例如,“hello wrld”仅返回“hello world”)。对于保留可能不存在于Manticore Search中的短词或专有名词特别有用。layouts:用于检测因键盘布局不匹配导致的输入错误的键盘布局(例如,使用错误布局时输入“ghbdtn”代替“привет”)。Manticore比较不同布局中字符的位置以建议纠正。至少需要2个布局才能有效检测不匹配。默认不使用任何布局。使用空字符串''(SQL)或空数组[](JSON)关闭此功能。支持的布局包括:be- 比利时AZERTY布局bg- 标准保加利亚布局br- 巴西QWERTY布局ch- 瑞士QWERTZ布局de- 德国QWERTZ布局dk- 丹麦QWERTY布局es- 西班牙QWERTY布局fr- 法国AZERTY布局uk- 英国QWERTY布局gr- 希腊QWERTY布局it- 意大利QWERTY布局no- 挪威QWERTY布局pt- 葡萄牙QWERTY布局ru- 俄罗斯JCUKEN布局se- 瑞典QWERTY布局ua- 乌克兰JCUKEN布局us- 美国QWERTY布局
- 此演示展示了模糊搜索功能:

- 关于模糊搜索和自动补全的博客文章 - https://manticoresearch.com/blog/new-fuzzy-search-and-autocomplete/
这两个命令都可以通过 SQL 访问,支持查询本地(普通和实时)以及分布式表。语法如下:
CALL QSUGGEST(<word or words>, <table name> [,options])
CALL SUGGEST(<word or words>, <table name> [,options])
options: N as option_name[, M as another_option, ...]
这些命令为给定单词提供字典中的所有建议。它们仅在启用了infixing且设置了dict=keywords的表上工作。它们返回建议的关键词、建议关键词与原始关键词之间的 Levenshtein 距离,以及建议关键词的文档统计信息。
如果第一个参数包含多个单词,则:
CALL QSUGGEST只返回最后一个单词的建议,忽略其余部分。CALL SUGGEST只返回第一个单词的建议。
这就是它们之间唯一的区别。支持多种选项进行自定义:
| 选项 | 描述 | 默认值 |
|---|---|---|
| limit | 返回前 N 个匹配项 | 5 |
| max_edits | 仅保留 Levenshtein 距离小于或等于 N 的字典词 | 4 |
| result_stats | 提供找到词的 Levenshtein 距离和文档计数 | 1(启用) |
| delta_len | 仅保留长度差小于 N 的字典词 | 3 |
| max_matches | 保留的匹配数量 | 25 |
| reject | 被拒绝的词是那些不比匹配队列中已有词更好的匹配。它们被放入一个拒绝队列,如果有词能进入匹配队列,拒绝队列会被重置。此参数定义拒绝队列的大小(为 reject*max(max_matched,limit))。如果拒绝队列满了,搜索潜在匹配的过程会停止 | 4 |
| result_line | 另一种显示模式,按行返回所有建议、距离和文档数 | 0 |
| non_char | 不跳过包含非字母符号的字典词 | 0(跳过此类词) |
| sentence | 返回原始句子,并将最后一个单词替换为匹配词 | 0(不返回完整句子) |
| force_bigrams | 强制对所有单词长度使用二元组(2字符 n-gram)而非三元组,这可以改善对字符调换错误的匹配 | 0(对长度≥6的单词使用三元组) |
为了展示其工作原理,我们创建一个表并添加几个文档。
create table products(title text) min_infix_len='2';
insert into products values (0,'Crossbody Bag with Tassel'), (0,'microfiber sheet set'), (0,'Pet Hair Remover Glove');
如你所见,拼写错误的单词 "crossbUdy" 被纠正为 "crossbody"。默认情况下,CALL SUGGEST/QSUGGEST 返回:
distance- Levenshtein 距离,表示将给定单词转换为建议词所需的编辑次数docs- 包含建议词的文档数量
要禁用这些统计信息的显示,可以使用选项 0 as result_stats。
- Example
call suggest('crossbudy', 'products');+-----------+----------+------+
| suggest | distance | docs |
+-----------+----------+------+
| crossbody | 1 | 1 |
+-----------+----------+------+- Example
call suggest('bagg with tasel', 'products');+---------+----------+------+
| suggest | distance | docs |
+---------+----------+------+
| bag | 1 | 1 |
+---------+----------+------+- Example
CALL QSUGGEST('bagg with tasel', 'products');+---------+----------+------+
| suggest | distance | docs |
+---------+----------+------+
| tassel | 1 | 1 |
+---------+----------+------+添加 1 as sentence 使 CALL QSUGGEST 返回整个句子,并将最后一个单词纠正。
- Example
CALL QSUGGEST('bag with tasel', 'products', 1 as sentence);+-------------------+----------+------+
| suggest | distance | docs |
+-------------------+----------+------+
| bag with tassel | 1 | 1 |
+-------------------+----------+------+1 as result_line 选项改变了建议在输出中的显示方式。它不再将每个建议显示在单独的行中,而是将所有建议、距离和文档数显示在一行中。以下示例演示了这一点:
CALL QSUGGEST('bagg with tasel', 'products', 1 as result_line);
+----------+--------+
| name | value |
+----------+--------+
| suggests | tassel |
| distance | 1 |
| docs | 1 |
+----------+--------+
force_bigrams 选项可以帮助处理字符调换错误的单词,例如 "ipohne" 与 "iphone"。通过使用二元组而非三元组,算法能更好地处理字符调换。
CALL SUGGEST('ipohne', 'products', 1 as force_bigrams);
+--------+----------+------+
| suggest| distance | docs |
+--------+----------+------+
| iphone | 2 | 1 |
+--------+----------+------+
- 此交互式课程 展示了
CALL SUGGEST在一个小型网页应用中的工作方式。
