At Agira, Technology Simplified, Innovation Delivered, and Empowering Business is what we are passionate about. We always strive to build solutions that boost your productivity.

Scroll

, ,

Apache Ranger 0.7.1 Installation

  • By Saravana
  • September 21, 2017
  • 1494 Views

Apache Ranger 0.7.1 is the current version of Apache Ranger. This blog will help you to Install Apache Ranger 0.7.1 on CentOS operating system. I have explained the entire process in simple and easy steps.

Prerequisites

The following components are required to process ranger installation.

  • JDK – For Running RangerAdmin, RangerKMS
  • RDBMS – Storing policies, Ranger Users, and Groups (MySQL/Oracle/Postgres/SQL Server)
  • Solr – Store Audit Logs (Apache Solr – 5.2.1+)
  • HDFS – Store Audit Logs
  • Kerberos – Enabling Kerberos will ensure that the requests get authenticated.

Preparing to install

You need to log in as root or will need “sudo” access to install some of the build tools.

Step 1: Install Maven 3.3.9
$ cd /usr/local
# Download maven latest distribution tar from apache maven site
$ wget http://www-eu.apache.org/dist/maven/maven-3/3.3.9/binaries/apache-maven-3.3.9-bin.tar.gz
$ tar -xvf apache-maven-3.3.9-bin.tar.gz
# Add environment varialves in .bashrc
export M2_HOME=/usr/local/apache-maven-3.3.9
export M2=$M2_HOME/bin
export PATH=$M2:$PATH
#Now to test your install of Maven,
$ mvn -version

 

Step 2: Install git
$ yum install git

 

Step 3: Install gcc
$ yum install gcc

 

Step 4: Install MySQL
$ wget http://dev.mysql.com/get/mysql57-community-release-el6-7.noarch.rpm
$ sudo rpm -Uvh mysql57-community-release-el6-7.noarch.rpm
$ sudo yum install mysql-community-server
$ service mysqld start

 
 

  • To get the temp mysql password
$ grep 'temporary password' /var/log/mysqld.log

 
 

  • Change default settings
mysql_secure_installation

 
 

  • Download the MySQL JDBC and place it into shared director
$ wget https://dev.mysql.com/get/Downloads/Connector-J/mysql-connector-java-5.1.40.tar.gz
$ tar -xvf mysql-connector-java-5.1.40.tar.gz
$ mv mysql-connector-java-5.1.40/mysql-connector-java-5.1.40-bin.jar /usr/share/java/mysql-connector-java.jar

 

Build Ranger Admin from source

Add environment varialves in .bashrc

export MAVEN_OPTS="-Xmx2048m"
export _JAVA_OPTIONS="-Xms1024m -Xmx1024m -XX:MaxPermSize=512m"
export JAVA_HOME=/usr/lib/jvm/java-openjdk
export PATH=$JAVA_HOME/bin:$PATH

 

Clone ranger source code

$ mkdir ~/dev
$ cd ~/dev
$ git clone https://github.com/apache/incubator-ranger.git
$ cd incubator-ranger
$ git checkout tags/release-0.7.1 -b ranger-0.7.1

 

Building the source

$ cd ~/dev/incubator-ranger

Run Maven to build ranger packages

$ mvn clean compile package assembly:assembly install

 

After maven build succeeds, Verify all tar files under target directory

$ ls target/*.tar.gz

 

Sample output:

ranger-0.7.1-admin.tar.gz ranger-0.7.1-kms.tar.gz ranger-0.7.1-storm-plugin.tar.gz ranger-0.7.1-hbase-plugin.tar.gz ranger-0.7.1-knox-plugin.tar.gz ranger-0.7.1-usersync.tar.gz ranger-0.7.1-hdfs-plugin.tar.gz ranger-0.7.1-migration-util.tar.gz ranger-0.7.1-yarn-plugin.tar.gz ranger-0.7.1-hive-plugin.tar.gz ranger-0.7.1-solr-plugin.tar.gz ranger-0.7.1-kafka-plugin.tar.gz ranger-0.7.1-src.tar.gz

 

Install Solr

Audit logs are stored in Solr. (Note: if don’t want Solr, we can use DB to store the log https://community.hortonworks.com/questions/27957/does-ranger-05-work-without-solr.html) (we will try for store logs in HDFS)

$ cd ~/dev/incubator-ranger/security-admin/contrib/solr_for_audit_setup

 

Edit install.properties and add the following values,

SOLR_INSTALL = true
SOLR_DOWNLOAD_URL = http://archive.apache.org/dist/lucene/solr/5.2.1/solr-5.2.1.tgz
SOLR_INSTALL_FOLDER = /opt/solr
JAVA_HOME = /usr/lib/jvm/java-openjdk
SOLR_USER = solr
SOLR_RANGER_HOME = /opt/solr/ranger_audit_server
SOLR_RANGER_PORT = 6083
SOLR_DEPLOYMENT = standalone
SOLR_RANGER_DATA_FOLDER = /opt/solr/ranger_audit_server/data
SOLR_LOG_FOLDER = /var/log/solr/ranger_audits
SOLR_MAX_MEM = 2g

 

Run setup script to install solr

$ ./setup.sh

 

Open /opt/solr/ranger_audit_server/install_notes.txt for instructions to start and stop Solr.

Check Solr by accessing http://${SOLR_HOST}:6083 from your browser.

Install Ranger Policy Admin

$ cd /usr/local
$ sudo tar zxvf ~/dev/incubator-ranger/target/ranger-0.7.1-admin.tar.gz
$ sudo ln -s ranger-0.7.1-admin/ ranger-admin
$ cd /usr/local/ranger-admin/

 

Configure Solr

  • Update install.properties
$ cd /usr/local/ranger-admin
$ vi install.properties
# Mysql root
db_root_user=root
db_root_password=root
db_host=localhost
# DB UserId used for the XASecure schema
db_name=ranger
db_user=rangeradmin
db_password=rangeradmin
# audit log
audit_store=solr
audit_solr_urls=http://HOST_ADDRESS:6083/solr/ranger_audits
policymgr_external_url=http://localhost:6080

 
 

  • After updating the required properties, run setup.sh
$ ./setup.sh

 
 

  • Now th following command starts Ranger Admin
$ ranger-admin start

 
The ranger-admin script supports the following options,

  • Starting Ranger Admin service: ranger-admin start
  • Stopping Ranger Admin service: ranger-admin stop
  • Restarting Ranger Admin service: ranger-admin restart

Verification can be done visiting the external URL of the server using browser, for instance

http://HOST_ADDRESS:6080 (Username: admin & PAssword: admin)

 

Configuring Ranger Admin Authentication Modes

  • Active Directory, LDAP and Unix are the authentication methods supported by Ranger Admin.

UNIX

authentication_method = UNIX
remoteLoginEnabled = true
authServiceHostName = localhost
authentication_method = 5151

Installing the Ranger UserSync Process

  • Extract Ranger plugin build at /usr/local
$ cd /usr/local
$ tar zxvf ~/dev/incubator-ranger/target/ranger-0.7.1-usersync.tar.gz
$ ln -s ranger-0.7.1-usersync/ ranger-usersync
$ cd ranger-usersync
$ mkdir -p /var/log/ranger-­usersync
$ chown ranger /var/log/ranger-usersync
$ chgrp ranger /var/log/ranger-usersync

 
 

  • Edit the install.properties file
$ POLICY_MGR_URL = http://HOST_ADDRESS:6080
$ SYNC_SOURCE = unix
$ logdir = /var/log/ranger/usersync

 
 

  • Install usersync by running ./setup.sh command
$ cd /usr/local/ranger-usersync
$ ./setup.sh

 
 

  • After installing ranger ­usersync, Use start/stop services of usersync.
./ranger­-usersync-services.sh start
./ranger­-usersync-services.sh stop

 

Apache Ranger HDFS Plugins

Step 1: Create new REPOSITORY_NAME service

If the REPOSITORY_NAME service doesn’t exist in Ranger Admin, the hdfs-plugin cannot communicate with it. So we have to make sure that it exists.

  • Go to http://HOST_ADDRESS:6080
  • Open Resource based policies link
  • Click ‘+’ under HDFS Grid to add new service
  • Update Service and Config Properties with the following

Service Name : hadoopdev
Active Status : Enabled
Select Tag Service : HDFS (Service name created under Tag based policies)
Username : hadoop
Password : hadoop@123
Namenode URL : http://HOST_ADDRESS:50070

 

  • Leave all other field as defaults and save

Step 2: Install and configure HDFS Plugins

  • Extract Ranger plugin build at /usr/local
$ cd /usr/local
$ tar zxvf ~/dev/incubator-ranger/target/ranger-0.7.1-hdfs-plugin.tar.gz
$ ln -s ranger-0.7.1-hdfs-plugin/ ranger-hdfs-plugin
$ cd ranger-hdfs-plugin

 
 

  • Edit the install.properties file
POLICY_MGR_URL = http://HOST_ADDRESS:6080
REPOSITORY_NAME = hadoopdev
XAAUDIT.SOLR.ENABLE = true
XAAUDIT.SOLR.URL = http://HOST_ADDRESS:6083/solr/ranger_audits
XAAUDIT.SOLR.USER = NONE
XAAUDIT.SOLR.PASSWORD = NONE
XAAUDIT.SOLR.ZOOKEEPER = NONE
XAAUDIT.SOLR.FILE_SPOOL_DIR = /var/log/hadoop/hdfs/audit/solr/spool

 
 

  • Enable hdfs­-plugin by running ./enable-hdfs-plugin.sh command
$ cd /usr/local/ranger-hdfs-plugin
$ ./enable-hdfs-plugin.sh

 
 

Ranger hdfs plugin installation might fail. To resolve this issue create a symlink as conf dir of hadoop linking to hadoop conf dir.

$ cd /usr/local/hadoop/
$ ln -s /home/hadoop/hadoop/etc/hadoop/ /usr/local/hadoop/conf
$ ln -s /home/hadoop/hadoop/share/hadoop/hdfs/lib/ /usr/local/hadoop/lib
  • Add HADOOP_HOME env value into .bashrc file
export HADOOP_HOME=/home/hadoop/hadoop

 

  • Once the above changes are done run enable plugin script
$ cd /usr/local/ranger-hdfs-plugin
 $ ./enable-hdfs-plugin.sh

 
 

Ranger Hive Plugin

Step 1: Create new REPOSITORY_NAME service

Make sure the REPOSITORY_NAME service exists in Ranger Admin. If not, the hive-plugin will not be able to communicate with Ranger admin

 

  • Go to http://HOST_ADDRESS:6080
  • Open Resource based policies link
  • Click + under HIVE Grid to add new service
  • Update following Service Details and Config Properties

Service Name : hivedev
Active Status : Enabled
Select Tag Service : HIVE (Service name created under Tag based policies)
Username : hadoop
Password : hadoop@123
jdbc.driverClassName : org.apache.hive.jdbc.HiveDriver
jdbc.url : jdbc:hive2://sandbox:10000/

 

  • Leave all other field as default and save

Step 2: Install and configure HIVE Plugins

  • Extract Ranger plugin build at /usr/local
$ cd /usr/local
$ tar zxvf ~/dev/incubator-ranger/target/ranger-0.7.1-hive-plugin.tar.gz
$ ln -s ranger-0.7.1-hive-plugin/ ranger-hive-plugin
$ cd ranger-hive-plugin

 
 

  • Edit the install.properties file
POLICY_MGR_URL = http://HOST_ADDRESS:6080
REPOSITORY_NAME = hivedev
XAAUDIT.SOLR.ENABLE = true
XAAUDIT.SOLR.URL = http://HOST_ADDRESS:6083/solr/ranger_audits
XAAUDIT.SOLR.USER = NONE
XAAUDIT.SOLR.PASSWORD = NONE
XAAUDIT.SOLR.ZOOKEEPER = NONE
XAAUDIT.SOLR.FILE_SPOOL_DIR = /var/log/hive/audit/solr/spool

 
 

  • Create log directory
$ mkdir /var/log/hive
$ chown -R hadoop:hadoop /var/log/hive

 
 

  • Enable hive-plugin by running enable-hive-plugin.sh command
$ cd /usr/local/ranger-hive-plugin
 $ ./enable-hive-plugin.sh

 

That’s all now the latest version of Apache Ranger is installed in your system. Please comment below if you have any queries regarding the installation process. To read more on web development follow Agiratechnologies.

 

0Likes

Saravana

An enthusiastic Tech Lead with 7 plus years of experience in Web development arena. Owns legitimate experience in Ruby, Ruby On Rails, AngularJs, DevOps. Golang, Another add on, This young tech freak never miss a chance to get his hands on planting and Gardening even in his busy weekends.

Related Blogs

Web Development
Top 4 Reasons To Integrate An Athleisure Shopify Store To Your Fitness App.
Ruby on Rails
Hire A Freelance RoR Developer Or Work With A ROR Development Company
Web Development
Modern Best Practices in the CMS Development and Integration
Web Development
How Do Large Companies Hire The Best Web Developers?
Web Development
Java Vs .Net: Who Will Reign In The Future?
Web Development
Loyalty Programs for eCommerce: A Comprehensive Guide