Ambari Infra Solr - Tuning for ranger_audits collection

Summary:-

By default when using solr cloud for ranger audit, it creates the ranger_audits with 1 shard by default.

As the audit data grows, we might hit scale problem in a single node.  Also too many OOM errors, to avoid that we create multiple shards for ranger collection.

Will walk-through the steps to do this.  If your cluster is kerberized, you have to kinit as infra-solr keytab.

 “Ranger_Audits” Collection solr UI view after installing ranger audits to solr

Step1:-

Delete the existing ranger_audits collection:- (For fresh install)

[root@c149-node2 ~]# su - infra-solr
[infra-solr@c149-node2 ~]$
[infra-solr@c149-node2 ~]$ curl -i --negotiate -u : "http://c149-node2.example.com:8886/solr/admin/collections?action=delete&name=ranger_audits"

HTTP/1.1 200 OK
Content-Type: application/xml; charset=UTF-8
Transfer-Encoding: chunked

<?xml version="1.0" encoding="UTF-8"?>
<response>
<lst name="responseHeader"><int name="status">0</int><int name="QTime">892</int></lst><lst name="success"><lst name="172.25.33.4:8886_solr"><lst name="responseHeader"><int name="status">0</int><int name="QTime">32</int></lst></lst></lst>
</response>

Step2:-

Create the ranger_audits collections with 3 shards (have 3 infra hosts in our cluster)

[infra-solr@c149-node2 ~]$ curl -i --negotiate -u : "c149-node2.example.com:8886/solr/admin/collections?action=create&name=ranger_audits&numShards=3&replicationFactor=1&collection.configName=ranger_audits"

HTTP/1.1 200 OK
Content-Type: application/xml; charset=UTF-8
Transfer-Encoding: chunked

<?xml version="1.0" encoding="UTF-8"?>
<response>
<lst name="responseHeader"><int name="status">0</int><int name="QTime">2562</int></lst><lst name="success"><lst name="172.25.34.130:8886_solr"><lst name="responseHeader"><int name="status">0</int><int name="QTime">1404</int></lst><str name="core">ranger_audits_shard1_replica1</str></lst><lst name="172.25.33.133:8886_solr"><lst name="responseHeader"><int name="status">0</int><int name="QTime">1430</int></lst><str name="core">ranger_audits_shard2_replica1</str></lst><lst name="172.25.33.4:8886_solr"><lst name="responseHeader"><int name="status">0</int><int name="QTime">1438</int></lst><str name="core">ranger_audits_shard3_replica1</str></lst></lst>
</response>

Step 3:-

Add additional replica’s for each shard.

Shard1:

curl -i --negotiate -u : "c149-node2.example.com:8886/solr/admin/collections?action=ADDREPLICA&collection=ranger_audits&shard=shard1&node=172.25.33.133:8886_solr”

Shard2:

curl -i --negotiate -u : "c149-node2.example.com:8886/solr/admin/collections?action=ADDREPLICA&collection=ranger_audits&shard=shard2&node=172.25.34.130:8886_solr"

Shard3:

curl -i --negotiate -u : "c149-node2.example.com:8886/solr/admin/collections?action=ADDREPLICA&collection=ranger_audits&shard=shard3&node=172.25.33.4:8886_solr"