After a while it can get tedious to access and review server logs via the command line. There are several tools available that can provide the same information in a graphical manner. Recently I’ve migrated to Splunk as there are both Enterprise and Free versions available.
- Of course, you’ll need a Splunk server installed first, as the forwarder is really just another (lighter) instance that will forward the log information to a central location.
- Download the system appropriate installer from:
http://www.splunk.com/download/universalforwarder - Check to see if you are running 32 or 64 bit OS.
uname -a
If you see i686 you are 32 bit, if x86_64 you are 64 bit! - Download, you’ll likely need a different version:
sudo dpkg -i splunkforwarder-6.1.3-220630-linux-2.6-intel.deb
or
sudo dpkg -i splunkforwarder-6.1.3-220630-linux-2.6-amd64.deb
- Enable auto-start on reboot:
cd /opt/splunkforwarder/bin/
sudo ./splunk enable boot-start
-
- Start the server:
sudo service splunk start
- Set the password:
The default ‘
admin
‘ password is ‘changeme
‘ so we need to change it immediately to do anything else, or we will see errors in future steps.sudo /opt/splunkforwarder/bin/splunk edit user admin -password YOUR_NEW_PASSWORD -auth admin:changeme
- Set the server:
sudo /opt/splunkforwarder/bin/splunk add forward-server YOUR_SERVER_ADDRESS:9997
NOTE: if you get prompted for a splunk username/password you likely skipped the above step. Remember – the forwarder is a new ‘light’ installation of the server and as such has it’s own users!
- Enable some monitors on the box:Some common services and log locations to get you started…
- Apache2 HTTPd
sudo /opt/splunkforwarder/bin/splunk add monitor /var/log/apache2 -index main -sourcetype Apache2
- Tomcat7
sudo /opt/splunkforwarder/bin/splunk add monitor /opt/tomcat7/logs -index main -sourcetype Tomcat7
- MySQL
sudo /opt/splunkforwarder/bin/splunk add monitor /var/log/mysql -index main -sourcetype MySQL
- Postfix (SMTP)
sudo /opt/splunkforwarder/bin/splunk add monitor /var/log/mail.log -index main -sourcetype Postfix
- Squid3 (Proxy)
sudo /opt/splunkforwarder/bin/splunk add monitor /var/log/squid/access.log -index main -sourcetype Squid3
sudo /opt/splunkforwarder/bin/splunk add monitor /var/log/squid/cache.log -index main -sourcetype Squid3
SonarQubesudo /opt/splunkforwarder/bin/splunk add monitor /opt/sonar/logs -index main -sourcetype Sonar
- PM2
sudo /opt/splunkforwarder/bin/splunk add monitor /home/{user}/.pm2/logs -index main -sourcetype PM2
- NPM
sudo /opt/splunkforwarder/bin/splunk add monitor /home/scott/.npm/_logs -index main -sourcetype NPM
- Start the server:
- (OPTIONAL) Verify configuration by opening file at the following:
sudo su
vi /opt/splunkforwarder/etc/apps/search/local/inputs.conf
exit
- You now should be able to log into your server and see new data flowing from the forwarder.
NOTE: this requires you to enable ‘receiving’ of data on the port specified above, usually 9997.
REFERENCES:
- http://answers.splunk.com/answers/50082/how-do-i-configure-a-splunk-forwarder-on-linux
- http://docs.splunk.com/Documentation/Splunk/latest/Data/Configureyourinputs
- http://answers.splunk.com/answers/12638/prompt-for-splunk-user-when-configuring-universal-forwarder
- http://answers.splunk.com/answers/41677/some-splunk-command-lines-ask-for-username-forwarders-dont-have-user