ElasticSearch

Tue, 10 May 2016 07:46:00 GMT

ElasticSearch各个版本的apo

https://www.elastic.co/guide/en/elasticsearch/client/java-api/2.2/index.html

猪眼�?/a> 2016-05-10 15:46 发表评论

Solr中的group与facet的区�?

Tue, 03 May 2016 08:34:00 GMT

facet 自己理解��是分组聚合用的�Q?如下说明

http://blog.csdn.net/a925907195/article/details/47257243

Solr中的group与facet的区�?/p>

如果是简单的使用的话�Q�那么Facet与group都可以用来进行数据的聚合查询�Q�但是他们还是有很大的区别的�?/p>

首先上facet跟group的操作：

Facet的例子：

public voidFacetFieldQuery() throws Exception {

solrServer = createSolrServer();

SolrQueryquery = newSolrQuery();//建立一个新的查�?/span>

query.setQuery("jobsName:计算机维�?/span>");

query.setFacet(true);//讄��facet=on

// 分类信息分�ؓ�Q�薪��_��发布旉��Q�教育背景，工作�l�验�Q�公司类型，工作�c�d��

query.addFacetField(new String[] {"salary","publishDate",

"educateBackground","jobExperience","companytype","jobsType" });//讄��需�?/span>facet的字�D?/span>

query.setFacetLimit(10);// 限制facet�q�回的数�?/span>

query.setFacetMissing(false);//不统�?/span>null的�?/span>

query.setFacetMinCount(1);// 讄��q�回的数据中每个分组的数据最��|��比如讄��?/span>1�Q�则�l�计数量最��ؓ1�Q�不然不昄��

//query.addFacetQuery("publishDate:[2014-04-11T00:00:00Z TO2014-04-13T00:00:00Z]");

QueryResponseresponse = solrServer.query(query);

System.out.println("查询旉��Q?/span>" + response.getQTime());

Listfacets = response.getFacetFields();//�q�回�?/span>facet列表

for (FacetField facet :facets) {

System.out.println(facet.getName());

System.out.println("----------------");

Listcounts = facet.getValues();

for (Count count : counts){

System.out.println(count.getName()+":"+ count.getCount());

}

System.out.println();

}

�q�行�l�果如下�Q?/p>

查询旉��Q?6

salary

----------------

面议:6882

2001-4000:1508

其他:671

4001-6000:536

3000-4499:224

2000-2999:181

6001-8000:179

3000-5000:82

1000-2000:81

4500-5999:75

publishDate

----------------

2014-08-05T00:00:00Z:793

2014-08-04T00:00:00Z:775

2014-07-30T00:00:00Z:601

2014-08-07T00:00:00Z:548

2014-08-06T00:00:00Z:539

2014-08-11T00:00:00Z:472

2014-08-20T00:00:00Z:439

2014-08-12T00:00:00Z:438

2014-08-01T00:00:00Z:405

2014-08-03T00:00:00Z:376

educateBackground

----------------

大专:4486

本科:1872

其他:1344

不限:1147

中专:680

高中:472

薪水范围�Q?430

中技:161

初中:140

��士:94

jobExperience

----------------

其他:2623

不限:2249

1-3�q?1770

1�q?1301

2�q?773

3-4�q?528

3-5�q?379

应届毕业�?309

5-7�q?162

1�q�以�?136

companytype

----------------

民营公司:3702

民营:2605

国企:835

股䆾制企�?729

其他:707

合资:632

外资�Q�非�Ƨ美�Q?377

外商独资:350

外资�Q�欧��）:271

上市公司:228

jobsType

----------------

全职:10734

��D��:59

实习:39

Group查询�Q?/p>

/**group查询

* @throws Exception

public void GroupFieldQuery() throws Exception {

solrServer = createSolrServer();

SolrQuery query = new SolrQuery("jobsName:计算机维�?/span>");

// 讄��通过facet查询�?/span>true�Q�表�C�查询时使用facet机制

query.setParam(GroupParams.GROUP,true);

query.setParam(GroupParams.GROUP_FIELD,"salary");

// 讄��每个quality对应�?/span>

query.setParam(GroupParams.GROUP_LIMIT,"1");

// 讄��q�回doc文档数据�Q�因只需要数量，故设�|��ؓ0

query.setRows(10);

QueryResponse response = solrServer.query(query);

if (response !=null) {

GroupResponse groupResponse =response.getGroupResponse();

if(groupResponse !=null) {

List groupList =groupResponse.getValues();

for(GroupCommand groupCommand : groupList){

List groups =groupCommand.getValues();

for(Group group : groups) {

System.out.println("group查询..."+group.getGroupValue()+"数量为："+group.getResult().getNumFound());

}

group查询...面议数量为：6882

group查询...4500-5999数量为：75

group查询...2001-4000数量为：1508

group查询...其他数量为：671

group查询...2000-2999数量为：181

group查询...4001-6000数量为：536

group查询...2000-4000数量为：19

group查询...2000-3000数量为：34

group查询...3000-4499数量为：224

group查询...3000-5000数量为：82

facet的查询结果主要是分组信息�Q�有什么分�l�，每个分组包括多少记录�Q�但是分�l�中有哪些数据是不可知道的，只有�q�一步搜索�?br /> group则类��g��关系数据库的group by�Q�可以用于一个或者几个字�D�去重、显�C�Z��个group的前几条记录�{��?/p>

The Grouping feature only works if groups are inthe same shard. You must use the custom sharding feature to use the Groupingfeature.

两者其实用��h��q�是有比较大的区别的�Q�但是如果说区别的话可以看下wiki上的�q�段

Field Collapsing and Result Grouping aredifferent ways to think about the same Solr feature.

Field Collapsing collapsesa group of results with the same field value down to a single (or fixed number)of entries. For example, most search engines such as Google collapse on site soonly one or two entries are shown, along with a link to click to see moreresults from that site. Field collapsing can also be used to suppress duplicatedocuments.

Result Grouping groupsdocuments with a common field value into groups, returning the top documentsper group, and the top groups based on what documents are in the groups. Oneexample is a search at Best Buy for a common term such as DVD, that shows thetop 3 results for each category ("TVs &Video","Movies","Computers", etc)

下面�q�两个查询语句一个是facet的一个是group�?/p>

http://localhost:8080/solr/JobsOtherWeb0/select?q=jobsName%3A%E8%AE%A1%E7%AE%97%E6%9C%BA%E7%BB%B4%E6%8A%A4&group=true&group.field=salary&group.limit=1&rows=10

http://localhost:8080/solr/JobsOtherWeb0/select?q=jobsName%3A%E8%AE%A1%E7%AE%97%E6%9C%BA%E7%BB%B4%E6%8A%A4&facet=true&facet.field=salary&facet.field=publishDate&facet.field=educateBackground&facet.field=jobExperience&facet.field=companytype&facet.field=jobsType&facet.limit=10&facet.missing=false&facet.mincount=1

其中facet查询出的如下�Q�（只截取部分结果）

�Ҏ��条�g查询出的是查询结果，facet是聚�c�d��的信息跟查询条�g是分开的，查询�l�果也跟facet没关�p�R�?/p>

但是下面看group查询�?/p>

也就是你的查询条件是跟group相关的，�q�回的查询结果也是跟group相关的，比如说你惌��查询的结果在每个分组�?都有数据采集�Q�那么就最好用group�Q�这样出来的数据跟group也是相关的，但是有个问题�Q�比如说你要查询group每个采集1个，ok那么你查询的时候的条�grows��无效了�Q�也不能说无效，主要是看你怎么使用�Q�，��是最多每个分�l�给你返回一个，多了没有了�?/p>

再细说点��是如果你想查询归查询聚�c�d��聚类�Q�那么��用facet�Q�如果想使用�c�M��采集的效果，每个group分组采集多少个，那么使用group查询�?/p>

猪眼�?/a> 2016-05-03 16:34 发表评论

Tue, 24 Sep 2013 08:34:00 GMT

lucene查询�l�果需要先分组后排序，看了相关文档�Q�lucene提供了分�l?排序功能�Q�但是比较消耗内容，实际情况�?我们一般只需要前面的1000条，所以我自己实现了排序算法，��量��占用内宏V�?br />�Ҏ��如下�Q?br />

// 分组�l�计方式
   /**
   *
   * @param topnumber 取分�l�后的前几条数据
   * @return
   */
   public static List cachegroup(int topnumber) {
       try {
           int page = 30000;
           String filePath = "D://lucene/index2";
           File dirfile = new File(filePath);
           Directory dir = FSDirectory.open(dirfile);
           IndexReader reader = DirectoryReader.open(dir);
           IndexSearcher indexSearcher = new IndexSearcher(reader);
           String searchTerm = "aaa";
           boolean getScores = true;
           boolean getMaxScores = true;
           boolean fillFields = true;
           GroupingSearch groupingSearch = new GroupingSearch("author");
           groupingSearch.setGroupSort(Sort.RELEVANCE);
           groupingSearch.setFillSortFields(true);
           groupingSearch.setCachingInMB(40.0, true);
           groupingSearch.setAllGroups(true);
           TermQuery query = new TermQuery(new Term("content", searchTerm));
           TopDocs hits = indexSearcher.search(query, 100);
           int count = hits.totalHits;
           int mode = count % page;
           int times = count / page;
           if (mode != 0)
               times = times + 1;
           List indexvalue = new LinkedList();
           for (int i = 0; i < times; i++) {
               int groupoffset = i * page;
               TopGroups result = groupingSearch.search(
                       indexSearcher, query, groupoffset, page);
               GroupDocs[] gds = result.groups;
               for (GroupDocs gd : gds) {
                   indexvalue.add(new FieldSort(gd.totalHits, gd.groupValue
                           .utf8ToString()));
               }
               Comparator comp = Collections.reverseOrder();
               Collections.sort(indexvalue, comp);
               indexvalue = indexvalue.subList(0, topnumber);
           }
           return indexvalue;
       } catch (Exception ex) {
           ex.printStackTrace();
       }
       return null;
   }


   public static void readList(List list)
   {
       Iterator it=list.iterator();
       while(it.hasNext())
       {
           FieldSort fieldsort=(FieldSort)it.next();
           System.out.println(fieldsort.getFieldvalue()+"/"+fieldsort.getFieldname());
       }
   }

package speed;

import java.util.Comparator;

public class FieldSort implements Comparable{

   private int    fieldvalue;
   private String fieldname;
   public FieldSort() {
       super();
   }
   public FieldSort(int fieldvalue,String fieldname) {
       super();
       this.setFieldvalue(fieldvalue);
       this.setFieldname(fieldname);
   }

   public int getFieldvalue() {
       return fieldvalue;
   }
   public void setFieldvalue(int fieldvalue) {
       this.fieldvalue = fieldvalue;
   }
   public String getFieldname() {
       return fieldname;
   }
   public void setFieldname(String fieldname) {
       this.fieldname = fieldname;
   }
   @Override
   public String toString(){
       return fieldname+"/"+fieldvalue;
   }
   public int compareTo(FieldSort t){
       if(t==null)return 0;
       else{
           return this.fieldvalue-t.fieldvalue;
       }
   }



}

猪眼�?/a> 2013-09-24 16:34 发表评论

欧美亚洲精品一区二区,国产亚洲福利一区二区免费看,亚洲国产成人九九综合

ElasticSearch

Solr中的group与facet的区�?