solrのcache
5904 ワード
pythonスクリプトでsolrをcacheテストしました.
1.同じリクエストでキャッシュされますが、プロセスが終了するとcacheは無効になります.
2.時間範囲フィルタリングは性能に大きな影響を及ぼす:6日200 ms、60日800 ms、4倍の差.
3.テストされたnot id:0と通常のキーワードの性能に差はありません.
テストコード:
要求パラメータ:
Jul 12, 2012 5:46:00 PM org.apache.solr.core.SolrCore execute INFO:[feed]webapp=/solr path=/selectparams={fl= id&sort=last_replied_datetime+desc&start=0&q=NOT+id=0&wt=standard&fq=category:9+OR+category:1&fq=-pop_score:0&fq=-status:6&fq=-photo_id:0&fq=buyable:0+OR+buyable:11+OR+buyable:14&fq+ buyable:14&fqqqq= 14&fqqqqqq= 14&=favorite_count:[1+TO+*]&fq=add_datetime:[2012-05-13 T 17:45:59 Z+TO+*]&rows=24&version=2.2}hits=11738 status=0 QTime=11232(初回リクエスト) Jul 12, 2012 5:46:00 PM org.apache.solr.core.SolrCore execute INFO: [feed] webapp=/solr path=/select params={fl=id&sort=last_replied_datetime+desc&start=0 &q=NOT+id:0&wt=standard&fq=category:9+OR+category:1&fq=-pop_score:0&fq=-status:6&fq=-photo_id:0&fq=buyable:0+OR+buyable:11+OR+buyable:14&fq=favorite_count:[1+TO+*]&fq=add_datetime:[2012-05-13T17:45:59Z+TO+*]&rows=24&version=2.2} hits=111738 status=0 QTime=0 (cache ) Jul 12, 2012 5:46:00 PM org.apache.solr.core.SolrCore execute INFO: [feed] webapp=/solr path=/select params={fl=id&sort=last_replied_datetime+desc&start=24 &q=NOT+id:0&wt=standard&fq=category:9+OR+category:1&fq=-pop_score:0&fq=-status:6&fq=-photo_id:0&fq=buyable:0+OR+buyable:11+OR+buyable:14&fq=favorite_count:[1+TO+*]&fq=add_datetime:[2012-05-13T17:45:59Z+TO+*]&rows=24&version=2.2} hits=111738 status=0 QTime=1 (cache ) Jul 12, 2012 5:46:00 PM org.apache.solr.core.SolrCore execute INFO: [feed] webapp=/solr path=/select params={fl=id&sort=last_replied_datetime+desc&start=48 &q=NOT+id:0&wt=standard&fq=category:9+OR+category:1&fq=-pop_score:0&fq=-status:6&fq=-photo_id:0&fq=buyable:0+OR+buyable:11+OR+buyable:14&fq=favorite_count:[1+TO+*]&fq=add_datetime:[2012-05-13T17:45:59Z+TO+*]&rows=24&version=2.2} hits=111738 status=0 QTime=0 (cache ) Jul 12, 2012 5:46:00 PM org.apache.solr.core.SolrCore execute INFO: [feed] webapp=/solr path=/select params={fl=id&sort=last_replied_datetime+desc&start=72 &q=NOT+id:0&wt=standard&fq=category:9+OR+category:1&fq=-pop_score:0&fq=-status:6&fq=-photo_id:0&fq=buyable:0+OR+buyable:11+OR+buyable:14&fq=favorite_count:[1+TO+*]&fq=add_datetime:[2012-05-13T17:45:59Z+TO+*]&rows=24&version=2.2} hits=111738 status=0 QTime=0 (cache ) Jul 12, 2012 5:46:00 PM org.apache.solr.core.SolrCore execute INFO: [feed] webapp=/solr path=/select params={fl=id&sort=last_replied_datetime+desc&start=96 &q=NOT+id:0&wt=standard&fq=category:9+OR+category:1&fq=-pop_score:0&fq=-status:6&fq=-photo_id:0&fq=buyable:0+OR+buyable:11+OR+buyable:14&fq=favorite_count:[1+TO+*]&fq=add_datetime:[2012-05-13T17:45:59Z+TO+*]&rows=24&version=2.2} hits=111738 status=0 QTime=0 (cache )
これらの要求も行かなかった.
INFO: [feed] webapp=/solr path=/select params={fl=id&sort=last_replied_datetime+desc&start=90&q=*&wt=standard&fq=category:9+OR+category:1&fq=-pop_score:0&fq=-status:6&fq=-photo_id:0&fq=buyable:0+OR+buyable:11+OR+buyable:14&fq=add_datetime:[2012-06-18T21:03:33Z+TO+*]&fq=favorite_count:[1+TO+*]&rows=30&version=2.2} hits=38600 status=0 QTime=561 INFO: [feed] webapp=/solr path=/select params={fl=id&sort=last_replied_datetime+desc&start=1380&q=*&wt=standard&fq=category:9+OR+category:1&fq=-pop_score:0&fq=-status:6&fq=-photo_id:0&fq=buyable:0+OR+buyable:11+OR+buyable:14&fq=add_datetime:[2012-06-18T21:11:58Z+TO+*]&fq=favorite_count:[1+TO+*]&rows=30&version=2.2} hits=38589 status=0 QTime=656 INFO: [feed] webapp=/solr path=/select params={fl=id&sort=last_replied_datetime+desc&start=90&q=*&wt=standard&fq=category:9+OR+category:1&fq=-pop_score:0&fq=-status:6&fq=-photo_id:0&fq=buyable:0+OR+buyable:11+OR+buyable:14&fq=add_datetime:[2012-06-18T21:11:58Z+TO+*]&fq=favorite_count:[1+TO+*]&rows=30&version=2.2} hits=38589 status=0 QTime=701 INFO: [feed] webapp=/solr path=/select params={fl=id&sort=last_replied_datetime+desc&start=120&q=*&wt=standard&fq=category:9+OR+category:1&fq=-pop_score:0&fq=-status:6&fq=-photo_id:0&fq=buyable:0+OR+buyable:11+OR+buyable:14&fq=add_datetime:[2012-06-18T21:11:58Z+TO+*]&fq=favorite_count:[1+TO+*]&rows=24&version=2.2} hits=38589 status=0 QTime=634
http://hankesi2000.iteye.com/blog/1083377
1.同じリクエストでキャッシュされますが、プロセスが終了するとcacheは無効になります.
2.時間範囲フィルタリングは性能に大きな影響を及ぼす:6日200 ms、60日800 ms、4倍の差.
3.テストされたnot id:0と通常のキーワードの性能に差はありません.
テストコード:
fq = ['category:9 OR category:1', '-pop_score:0', '-status:6', '-photo_id:0', 'buyable:0 OR buyable:11 OR buyable:14', 'favorite_count:[1 TO *]']
from_date = datetime.datetime.now() - datetime.timedelta(days=60)
fq.append("add_datetime:[%s TO *]" % from_date.strftime("%Y-%m-%dT%H:%M:%SZ"))
tag_set = ['NOT id:0']
kw = tag_set and " OR ".join(tag_set) or "*:*"
query(kw, sort=sort, fq=fq)
count = 0
while count<5:
query(kw, start=count*24,sort=sort, fq=fq)
count= count+1要求パラメータ:
NOT id:0 {'fq': ['category:9 OR category:1', '-pop_score:0', '-status:6', '-photo_id:0', 'buyable:0 OR buyable:11 OR buyable:14', 'favorite_count:[1 TO *]', 'add_datetime:[2012-05-13T17:48:31Z TO *]'], 'rows': 24, 'start': 0}
Jul 12, 2012 5:46:00 PM org.apache.solr.core.SolrCore execute INFO:[feed]webapp=/solr path=/selectparams={fl= id&sort=last_replied_datetime+desc&start=0&q=NOT+id=0&wt=standard&fq=category:9+OR+category:1&fq=-pop_score:0&fq=-status:6&fq=-photo_id:0&fq=buyable:0+OR+buyable:11+OR+buyable:14&fq+ buyable:14&fqqqq= 14&fqqqqqq= 14&=favorite_count:[1+TO+*]&fq=add_datetime:[2012-05-13 T 17:45:59 Z+TO+*]&rows=24&version=2.2}hits=11738 status=0 QTime=11232(初回リクエスト) Jul 12, 2012 5:46:00 PM org.apache.solr.core.SolrCore execute INFO: [feed] webapp=/solr path=/select params={fl=id&sort=last_replied_datetime+desc&start=0 &q=NOT+id:0&wt=standard&fq=category:9+OR+category:1&fq=-pop_score:0&fq=-status:6&fq=-photo_id:0&fq=buyable:0+OR+buyable:11+OR+buyable:14&fq=favorite_count:[1+TO+*]&fq=add_datetime:[2012-05-13T17:45:59Z+TO+*]&rows=24&version=2.2} hits=111738 status=0 QTime=0 (cache ) Jul 12, 2012 5:46:00 PM org.apache.solr.core.SolrCore execute INFO: [feed] webapp=/solr path=/select params={fl=id&sort=last_replied_datetime+desc&start=24 &q=NOT+id:0&wt=standard&fq=category:9+OR+category:1&fq=-pop_score:0&fq=-status:6&fq=-photo_id:0&fq=buyable:0+OR+buyable:11+OR+buyable:14&fq=favorite_count:[1+TO+*]&fq=add_datetime:[2012-05-13T17:45:59Z+TO+*]&rows=24&version=2.2} hits=111738 status=0 QTime=1 (cache ) Jul 12, 2012 5:46:00 PM org.apache.solr.core.SolrCore execute INFO: [feed] webapp=/solr path=/select params={fl=id&sort=last_replied_datetime+desc&start=48 &q=NOT+id:0&wt=standard&fq=category:9+OR+category:1&fq=-pop_score:0&fq=-status:6&fq=-photo_id:0&fq=buyable:0+OR+buyable:11+OR+buyable:14&fq=favorite_count:[1+TO+*]&fq=add_datetime:[2012-05-13T17:45:59Z+TO+*]&rows=24&version=2.2} hits=111738 status=0 QTime=0 (cache ) Jul 12, 2012 5:46:00 PM org.apache.solr.core.SolrCore execute INFO: [feed] webapp=/solr path=/select params={fl=id&sort=last_replied_datetime+desc&start=72 &q=NOT+id:0&wt=standard&fq=category:9+OR+category:1&fq=-pop_score:0&fq=-status:6&fq=-photo_id:0&fq=buyable:0+OR+buyable:11+OR+buyable:14&fq=favorite_count:[1+TO+*]&fq=add_datetime:[2012-05-13T17:45:59Z+TO+*]&rows=24&version=2.2} hits=111738 status=0 QTime=0 (cache ) Jul 12, 2012 5:46:00 PM org.apache.solr.core.SolrCore execute INFO: [feed] webapp=/solr path=/select params={fl=id&sort=last_replied_datetime+desc&start=96 &q=NOT+id:0&wt=standard&fq=category:9+OR+category:1&fq=-pop_score:0&fq=-status:6&fq=-photo_id:0&fq=buyable:0+OR+buyable:11+OR+buyable:14&fq=favorite_count:[1+TO+*]&fq=add_datetime:[2012-05-13T17:45:59Z+TO+*]&rows=24&version=2.2} hits=111738 status=0 QTime=0 (cache )
これらの要求も行かなかった.
INFO: [feed] webapp=/solr path=/select params={fl=id&sort=last_replied_datetime+desc&start=90&q=*&wt=standard&fq=category:9+OR+category:1&fq=-pop_score:0&fq=-status:6&fq=-photo_id:0&fq=buyable:0+OR+buyable:11+OR+buyable:14&fq=add_datetime:[2012-06-18T21:03:33Z+TO+*]&fq=favorite_count:[1+TO+*]&rows=30&version=2.2} hits=38600 status=0 QTime=561 INFO: [feed] webapp=/solr path=/select params={fl=id&sort=last_replied_datetime+desc&start=1380&q=*&wt=standard&fq=category:9+OR+category:1&fq=-pop_score:0&fq=-status:6&fq=-photo_id:0&fq=buyable:0+OR+buyable:11+OR+buyable:14&fq=add_datetime:[2012-06-18T21:11:58Z+TO+*]&fq=favorite_count:[1+TO+*]&rows=30&version=2.2} hits=38589 status=0 QTime=656 INFO: [feed] webapp=/solr path=/select params={fl=id&sort=last_replied_datetime+desc&start=90&q=*&wt=standard&fq=category:9+OR+category:1&fq=-pop_score:0&fq=-status:6&fq=-photo_id:0&fq=buyable:0+OR+buyable:11+OR+buyable:14&fq=add_datetime:[2012-06-18T21:11:58Z+TO+*]&fq=favorite_count:[1+TO+*]&rows=30&version=2.2} hits=38589 status=0 QTime=701 INFO: [feed] webapp=/solr path=/select params={fl=id&sort=last_replied_datetime+desc&start=120&q=*&wt=standard&fq=category:9+OR+category:1&fq=-pop_score:0&fq=-status:6&fq=-photo_id:0&fq=buyable:0+OR+buyable:11+OR+buyable:14&fq=add_datetime:[2012-06-18T21:11:58Z+TO+*]&fq=favorite_count:[1+TO+*]&rows=24&version=2.2} hits=38589 status=0 QTime=634
http://hankesi2000.iteye.com/blog/1083377