This article is for giving list of fields used for storing Audits to various sources (DB / HDFS / Solr).
Audit to HDFS
| Audit to HDFS | Description | Sample Value | Data Type | Introduced in Version |
|---|
| id | Unique Id or Row id of audit log event | 85f0f6d7-2415-44e6-b277-6751d6c86ac7-3 | Number/String | 0.5 |
| policy version | Policy Version which is used in the authorization | numeric value | Number/String | 0.6 |
| result | Access result | 1 (Allowed) or 0 (Denied) | Number | 0.5 |
| access | Access type of executed event | READ/WRITE/SELECT etc. | String | 0.5 |
| cliType | Client Type | HiveServer, HiveMetaStore | String | 0.6 |
| agent | plugin involved in authorization | hdfs, hiveserver2, hbase..etc | String | 0.5 |
| enforcer | Access enforcer | hadoop-acl/ranger-acl | String | 0.5 |
| sess | Session Id | 606b0764-7914-4f32-8343-04d8be6e5bd5 | String | 0.5 |
| cliIP | Ip address of machine from where event was performed | 10.0.0.1 | String | 0.5 |
| policy | Policy id of the resource on which access event was executed | 1 | Number | 0.5 |
| repo | Repository Name | hadoopdev | String | 0.5 |
| repoType | Repository Type | HDFS/HIVE/HBase | Number | 0.5 |
| reason |
| testdb/testtable/column1 | String | 0.5 |
| evtTime | event request timestamp | 2016-10-12 6:11:45 | datetime | 0.5 |
| reqUser | user who requested the access | ranger | String | 0.5 |
| action | operation performed | QUERY/write | String | 0.6 onwards |
| resource | resource path | testdb/testtable/column1 | String | 0.5 |
| resType | Type of accessed resource | @column | String | 0.5 |
| seq_num | sequence number of audit log | 1 | Number | 0.5 |
| event_count | no of similar event executed in specific interval | 3 | Number | 0.5 |
| event_dur_ms | event execution time in ms | 10 | Number | 0.5 |
| tags | tag details associated with respective resource/policy | PCI | array[string] | 0.6 onwards |
| additional_info | additional informations are stored in this field. | like forwarded address, remote address, accessType list etc. | Map<String,String> | 0.6 |
| cluster_name | cluster name where the request came from | Cluster 1 | String | 0.6 |
| zone_name | Zone name when zone policy authorized the request |
| String | 0.6 |
| agentHost | hostname of agent | test-hbase-0710-1.openstacklocal |
| 0.5 |
| logType |
| RangerAudit |
| 0.5 |
Audit to Solr
| Audit to SOLR | Description | Sample Value | Data Type | Introduced in Version |
|---|
| id | Unique Id or Row id of audit log event | 85f0f6d7-2415-44e6-b277-6751d6c86ac7-3 | Number/String | 0.5 |
| policy version | Policy Version which is used in the authorization | numeric value | Number/String | 0.6 |
| result | Access result | 1(Allowed) or 0 (Denied) | Number | 0.5 |
| access | Access type of executed event | READ/WRITE/SELECT etc. | String | 0.5 |
| cliType | Client Type | HiveServer, HiveMetaStore | String | 0.6 |
| agent | plugin involved in authorization | hdfs, hiveserver2, hbase..etc | String | 0.5 |
| enforcer | Access enforcer | hadoop-acl/ranger-acl | String | 0.5 |
| sess | Session Id | 606b0764-7914-4f32-8343-04d8be6e5bd5 | String | 0.5 |
| cliIP | Ip address of machine from where event was performed | 10.0.0.1 | String | 0.5 |
| policy | Policy id of the resource on which access event was executed | 1 | Number | 0.5 |
| repo | Repository Name | hadoopdev | String | 0.5 |
| repoType | Repository Type | HDFS/HIVE/HBase | Number | 0.5 |
| reason |
| testdb/testtable/column1 | String | 0.5 |
| evtTime | event request timestamp | 2016-10-12 6:11:45 | datetime | 0.5 |
| reqUser | user who requested the access | ranger | String | 0.5 |
| action | operation performed | QUERY/write | String | 0.6 onwards |
| resource | resource path | testdb/testtable/column1 | String | 0.5 |
| resType | Type of accessed resource | @column | String | 0.5 |
| seq_num | sequence number of audit log | 1 | Number | 0.5 |
| event_count | no of similar event executed in specific interval | 3 | Number | 0.5 |
| event_dur_ms | event execution time in ms | 10 | Number | 0.5 |
| tags | tag details associated with respective resource/policy | PCI | array[string] | 0.6 onwards |
| additional_info | additional informations are stored in this field. | like forwarded address, remote address etc. | Map<String,String> | 0.6 |
| cluster_name | cluster name where the request came from | Cluster 1 | String | 0.6 |
| zone_name | Zone name when zone policy authorized the request |
| String | 0.6 |
| agentHost | hostname of agent | test-hbase-0710-1.openstacklocal | String | 0.5 |
| logType | Log Type | RangerAudit | String | 0.5 |
| _ttl_ | Time to live | +90DAYS | String | 0.5 |
| _expire_at_ | Expiry Time Stamp of Audit Event | 2017-02-12T11:39:44.839Z | String | 0.5 |
| _version_ | Version | 1550973492097187800 | Number | 0.5 |
5 Comments
Pradeep Bhadani
Mehul Parikh Gautam Borad Under "Audit to HDFS", data type of "tags" is "Array of String (array[string])" not "String".
"Tags" field will have the number of tags attached to a policy and sample value looks like :
"tags":["tg1","tg2"]
Pradeep Bhadani
Mehul ParikhGautam Borad For HDFS, data type of coulmn "access" is not "number" . It is string as it has value like "READ" , "USE" etc..
Mehul Parikh
I will update the schema, compiling the schema with latest changes.
hariprasad tandur
Mehul Parikh Gautam Borad For HDFS looks like access and result columns were interchanged in the above table, access should be a string and result should be a number
Gautam Borad
Thanks hariprasad tandur I have made the changes.