Using a personal proxy server can be helpful for a variety of reasons, such as:
- Performance – network speed and bandwidth
- Security – filtering and monitoring
- Debugging – to trace activity
Here are some simple steps to get you started, obviously you will need to further “harden” security to make it production ready!
sudo apt-get install squid3
sudo mv squid.conf squid.orig
sudo vi squid.conf
NOTE: the following configuration works, but will likely need to be adapted for your specific usage.
auth_param digest program /usr/lib/squid3/digest_file_auth -c /etc/squid3/passwords
#auth_param digest program /usr/lib/squid3/digest_pw_auth -c /etc/squid3/passwords
auth_param digest realm proxy
auth_param basic credentialsttl 4 hours
acl authenticated proxy_auth REQUIRED
acl localnet src 10.0.0.0/8 # RFC 1918 possible internal network
acl localnet src 172.16.0.0/12 # RFC 1918 possible internal network
acl localnet src 192.168.0.0/16 # RFC 1918 possible internal network
acl localnet src fc00::/7 # RFC 4193 local private network range
acl localnet src fe80::/10 # RFC 4291 link-local (directly plugged) machines
#acl SSL_ports port 443
#http_access deny to_localhost
#http_access deny CONNECT !SSL_ports
http_access allow localnet
http_access allow localhost
http_access allow authenticated
Create the users and passwords:
sudo apt-get install apache2-utils (required for htdigest)
sudo htdigest -c /etc/squid3/passwords proxy user1
sudo htdigest /etc/squid3/passwords proxy user2
Open up firewall port (if enabled):
sudo ufw allow 3128
Restart the server and tail the logs:
sudo service squid3 restart
sudo tail -f /var/log/squid3/access.log
OTHER FILE LOCATIONS:
MONITORING with Splunk…
sudo /opt/splunkforwarder/bin/splunk add monitor /var/log/squid3/access.log -index main -sourcetype Squid3
sudo /opt/splunkforwarder/bin/splunk add monitor /var/log/squid3/cache.log -index main -sourcetype Squid3
Splunk is a popular enterprise level tool for log collection, analysis and management. While you can obtain an enterprise license, most functions are available in the free community edition.
Setup is very easy:
- Download and move the .tar.gz file to the appropriate server (i386 vs. amd64)
sudo dpkg -i splunk*.deb
- Start the server:
sudo /opt/splunk/bin/splunk start
The first time you run after installation or update you will have to accept terms.
Access the admin screen:
Go to Settings/Forwarding * Receiving
– add new (port 9997)
- Open firewall port (if enabled):
sudo ufw allow 8000
Now to start as a service…
sudo /opt/splunk/bin/splunk enable boot-start
Subversion is a commonly used central version control system for software development. There are currently still a large number of organizations that rely upon it, many have since moved on to Git.
sudo apt-get install apache2 apache2-utils
sudo apt-get install subversion subversion-tools libapache2-svn
sudo mkdir /home/svn
svnadmin create /home/svn/test
Create a group for subversion users:
sudo groupadd subversion
sudo adduser USERNAME
Add a user to the group:
sudo useradd -G USERNAME subversion
sudo chown -R www-data:subversion /home/svn/test
sudo chmod -R g+rws /home/svn/test
sudo a2enmod dav_svn
To create/clobber a new file for the first user:
sudo htpasswd -c /etc/apache2/.htpasswd YOURUSER
To add additional users:
sudo htpasswd /etc/apache2/.htpasswd YOURUSER
(repeat for new users without the -c as that creates/clobbers the file)
sudo vi /etc/apache2/sites-available/000-default.conf
Then add to the bottom:
(NOTE1: the LimitExcept can be enabled to allow anonymous access):
(NOTE2: the LimitXMLRequestBody can be uncomment to allow large commits)
AuthName "Subversion Repository"
# AuthUserFile /etc/svn-auth
#<LimitExcept GET PROPFIND OPTIONS REPORT>
sudo service apache2 reload
sudo service apache2 restart
NOTE: At this point you should be able to browse and do a remote checkout of the code from another machine….
svn co http://YOUR-IP-OR-HOSTNAME/svn/test --username YOURUSER --password YOURPASS
sudo vi /etc/init/svnserve.conf
Add the following:
# svnserve - Subversion server
description "Subversion server"
start on (local-filesystems and net-device-up IFACE=lo and started udev-finish)
stop on runlevel 
respawn limit 2 3600
exec /usr/bin/svnserve --foreground --daemon --config-file /home/svn/repos/conf/svnserve.conf --root /home/svn/repos/
sudo initctl start svnserve
Back on the client side…
Create a new folder inside your user folder:
Check out the project into this folder:
svn checkout http://YOUR-IP-OR-HOSTNAME/svn/test
Let us just add a new HTML index file to the folder:
Add it to version control:
svn add index.html
Commit the new file:
svn commit -m "commit message"
That should cover most cases for you…
After a while it can get tedious to access and review server logs via the command line. There are several tools available that can provide the same information in a graphical manner. Recently I’ve migrated to Splunk as there are both Enterprise and Free versions available.
- Of course, you’ll need a Splunk server installed first, as the forwarder is really just another (lighter) instance that will forward the log information to a central location.
- Download the system appropriate installer from:
- Check to see if you are running 32 or 64 bit OS.
uname -aIf you see i686 you are 32 bit, if x86_64 you are 64 bit!
- Download, you’ll likely need a different version:
sudo dpkg -i splunkforwarder-6.1.3-220630-linux-2.6-intel.deb
sudo dpkg -i splunkforwarder-6.1.3-220630-linux-2.6-amd64.deb
- Enable auto-start on reboot:
sudo ./splunk enable boot-start
- Start the server:
sudo service splunk start
- Set the password:
The default ‘
admin‘ password is ‘
changeme‘ so we need to change it immediately to do anything else, or we will see errors in future steps.
sudo /opt/splunkforwarder/bin/splunk edit user admin -password YOUR_NEW_PASSWORD -auth admin:changeme
- Set the server:
sudo /opt/splunkforwarder/bin/splunk add forward-server YOUR_SERVER_ADDRESS:9997
NOTE: if you get prompted for a splunk username/password you likely skipped the above step. Remember – the forwarder is a new ‘light’ installation of the server and as such has it’s own users!
- Enable some monitors on the box:Some common services and log locations to get you started…
- Apache2 HTTPd
sudo /opt/splunkforwarder/bin/splunk add monitor /var/log/apache2 -index main -sourcetype Apache2
sudo /opt/splunkforwarder/bin/splunk add monitor /opt/tomcat7/logs -index main -sourcetype Tomcat7
sudo /opt/splunkforwarder/bin/splunk add monitor /var/log/mysql -index main -sourcetype MySQL
- Postfix (SMTP)
sudo /opt/splunkforwarder/bin/splunk add monitor /var/log/mail.log -index main -sourcetype Postfix
- Squid3 (Proxy)
sudo /opt/splunkforwarder/bin/splunk add monitor /var/log/squid/access.log -index main -sourcetype Squid3
sudo /opt/splunkforwarder/bin/splunk add monitor /var/log/squid/cache.log -index main -sourcetype Squid3
sudo /opt/splunkforwarder/bin/splunk add monitor /opt/sonar/logs -index main -sourcetype Sonar
- (OPTIONAL) Verify configuration by opening file at the following:
- You now should be able to log into your server and see new data flowing from the forwarder.
NOTE: this requires you to enable ‘receiving’ of data on the port specified above, usually 9997.
Installation of Sonar requires but a few simple steps, though they can be rather obscure to many developers.
Connect to MySQL:
CREATE DATABASE sonar CHARACTER SET utf8 COLLATE utf8_general_ci;
grant all privileges on sonar.* to '[email protected]' identified by 'sonar';
Easiest method, if you are on Ubuntu:
sudo vi /etc/apt/sources.list
- Then add the following line:
deb http://downloads.sourceforge.net/project/sonar-pkg/deb binary/
sudo apt-get update
sudo apt-get install sonar
- You will then have to stop/start as above to edit the configuration, generally to change database config:
sudo vi /opt/sonar/conf/sonar.properties
Manual installation (and upgrade?) if you prefer to do things the hard way 🙂
- Download and unzip the release:
sudo mv sonar-3.6.1 /opt/sonar
sudo rm -r /opt/sonar (to remove old link)
sudo ln -s /opt/sonar-3.6.1/ /opt/sonar
(uncomment lines for MySQL, comment out H2)
sudo cp /opt/sonar/bin/linux-x86-64/sonar.sh /etc/init.d/sonar
sudo vi /etc/init.d/sonar
add the following 2 lines:
sudo update-rc.d -f sonar remove
sudo chmod 755 /etc/init.d/sonar
sudo update-rc.d sonar defaults
Upgrades can be a little odd, if you see the maintenance page at http://localhost:9000/maintenance, go to http://localhost:9000/setup and do the required steps.
I’ve installed and managed dozens of MySQL installations for several years, occasionally it seems that an install just doesn’t run like it has in the past.
Recently I had a problem where the service would not start (Error 1067) on Windows Server 2003 (R2)… which is running under VMWare. After checking the obvious places and turning up nothing I started down the list of potential solutions exposed by Google search.
The ultimate solution it seems is that the ‘my.ini’ file needed to include the specific path information required by the service…. without it the service would not start.
Here’s my current file (c:\windows\my.ini) for reference:
For the really observant readers of this entry… you will notice that this is for MySQL 4.0 (which is no longer officially supported). It’s mostly used as it is widely compatible across various host systems that are sometimes problematic with newer releases.
Let’s review this for just one moment… all ‘sworn’ government officials take a similar oath when they assume office. This is the wording used for enlisted members of the U.S. military.
I,______________, do solemnly swear (or affirm) that I will support and defend the Constitution of the United States against all enemies, foreign and domestic; that I will bear true faith and allegiance to the same; and that I will obey the orders of the President of the United States and the orders of the officers appointed over me, according to regulations and the Uniform Code of Military Justice. So help me God.
Without delving into the details, I personally feel that many of our elected officials chose to ignore their oaths, instincts, and the people they represent. Let’s not forget these facts come election time as voting along “traditional” party lines doesn’t ever seem to fix matters, it’s time for a change!
Disturbed, especially by H1B and illegal immigration.