- [Kei Kurono] (https://perufw.wordpress.com) <[email protected]>
Proxy HaarpCache is a fork of ThunderCache.
HaarpCache is a robust static and dynamic cache that provides support to the DASH technology.
-
You can to install HaarpCache, in distributions Debian/Ubuntu, using a bash script:
wget https://raw.githubusercontent.com/keikurono/haarpcache/master/installHaarp.sh chmod +x installHaarp.sh bash installHaarp.shVideo installation:
http://www.youtube.com/watch?v=ZdSrxnjY2OAMore information (Installation with Mikrotik, etc):
https://perufw.wordpress.com/2015/10/28/new-version-haarpcache-v2-0b/ -
For manual installation:
su apt-get update apt-get upgrade apt-get install build-essential mysql-server mysql-client php5 apache2 php5-mysql \ libblkid-dev libcurl4-gnutls-dev libmysqlclient15-dev libapache2-mod-auth-mysql \ libapache2-mod-php5 sharutils curl autoconf squid3 git g++-4.4 cd /usr/src/ git clone https://github.com/keikurono/haarpcache.git cd haarpcache ./configure ./configure CXX=g++-4.4 # <--- for ubuntu make make install mysql -u root -p < haarp.sql cd /etc/init.d update-rc.d haarp defaults 98 -
Create a user for database of haarpcache:
mysql -u root -p grant all privileges on haarp.* to haarp_user@localhost identified by 'haarpcache_password';Change the username ('haarp_user') and password ('haarpcache_password') for which you want.
-
Configure your /etc/haarp/haarp.conf
CACHEDIR <dir_1>|<dir_2>|<dir_3> ... MYSQL_USER <haarp_user> MYSQL_PASS <haarpcache_password><dir_1>,<dir_2>,... are the directories where cache files will be stored, for default '/haarp/'.<haarp_user>and<haarpcache_password>are the user and password for the database of haarpcache defined above. -
Copy and paste the line above at the end of the file 'squid.conf' (or 'squid3.conf' which is in operation):
acl haarp_lst url_regex -i "/etc/haarp/haarp.lst" never_direct allow haarp_lst cache deny haarp_lst cache_peer 127.0.0.1 parent 8080 0 proxy-only no-digest dead_peer_timeout 2 seconds cache_peer_access 127.0.0.1 allow haarp_lst cache_peer_access 127.0.0.1 deny all -
Below the line of the
http_portconfiguration, add:acl google url_regex -i (googlevideo.com|www.youtube.com) acl iphone browser -i regexp (iPhone|iPad) acl BB browser -i regexp (BlackBerry|PlayBook) acl Winphone browser -i regexp (Windows.*Phone|Trident|IEMobile) acl Android browser -i regexp Android request_header_access User-Agent deny google !iphone !BB !Winphone !Android request_header_replace User-Agent Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)with this we avoid the redirects
HTTPtoHTTPSinyoutube.com. Only works for PC's, and when you enter tohttp://www.youtube.comfrom the address bar (of your browser) or fromwww.google.com. -
On squid.conf Comment the line:
#hierarchy_stoplist cgi-bin ? -
On crontab add the line:
0 0 * * * root /etc/init.d/haarpclean -
View reports of your cache system:
cp /usr/src/haarpcache/etc/haarp/haarp.php /var/www/Change the line 24 of the file haarp.php with your user and password for your database haarpcache.
if (!( $db = new PDO('mysql:host=localhost;dbname=haarp', 'haarp_user','haarpcache_password') ) ) -
Your viewer:
http://<IP_Your_Server>/haarp.php -
Security:
iptables -I INPUT 1 -i <eth_wan_1> -m tcp -p tcp --dport 8080 -m state --state NEW -j DROP iptables -I INPUT 1 -i <eth_wan_2> -m tcp -p tcp --dport 8080 -m state --state NEW -j DROP ...The port 8080 is of your haarpcache server. You can change this editing the file /etc/haarp/haarp.conf.
-
Avoid the
QUICprotocol connections onwww.youtube.comiptables -A FORWARD -i <eth_lan> -p udp -m udp --dport 80 -j REJECT --reject-with icmp-port-unreachable iptables -A FORWARD -i <eth_lan> -p udp -m udp --dport 443 -j REJECT --reject-with icmp-port-unreachableadd the following line in
squid.conf:# Disable alternate protocols reply_header_access Alternate-Protocol deny all -
All iptables rules should be placed in a *.sh file to be executed whenever the computer is started.
-
Finally:
/etc/init.d/haarp restart squid -k reconfigure -
Handling QoS can be done by making a marking of packages based on the search for the string "X-Cache: HIT from HAARP" or "HAARP: HIT from" in the packet header. Thus a possible handling may carried out as follows:
IF_LAN=eth0 MAX_DOWN=1300kbps MIN_CACHE_DOWN=1000kbps MAX_CACHE_DOWN=1100kbps iptables -A OUTPUT -t mangle -o $IF_LAN -p tcp -m string --string "X-Cache: HIT from Haarp" --algo kmp -j MARK --set-mark 666 tc qdisc add dev $IF_LAN root handle 1:0 htb default 10 r2q 15 tc class add dev $IF_LAN parent 1:0 classid 1:1 htb rate $MAX_DOWN ceil $MAX_DOWN tc class add dev $IF_LAN parent 1:1 classid 1:66 htb rate $MIN_CACHE_DOWN ceil $MAX_CACHE_DOWN tc qdisc add dev $IF_LAN parent 1:66 handle 66:0 sfq perturb 30 tc filter add dev $IF_LAN protocol ip parent 1:0 handle 666 fw classid 1:66
For a common update:
su
cd /usr/src/
rm -fr haarpcache
killall haarpclean 2>/dev/null
git clone https://github.com/keikurono/haarpcache.git
cd haarpcache
./configure
# IF you have problems with the last command, then (for debian/ubuntu systems) do:
apt-get install g++-4.4
./configure CXX=g++-4.4
#
make
make install
mysql -u root -p < haarp.sql
# ... and enter your password of Mysql.
cp -b etc/haarp/haarp.lst /etc/haarp/haarp.lst
/etc/init.d/haarp restart
squid -k reconfigure
Depending on the location of the logs defined in the file /etc/haarp/haarp.conf:
tail -f /var/log/haarp/access.log
tail -f /var/log/haarp/error.log
Level information in the logs: change the LOGLEVEL on /etc/haarp/haarp.conf.
-
Currently Haarp supports:
Youtube (https not supported), Netflix (Video and Images), DoubleClick (Block of ADS), Googlevideo, Dailymotion, AOL, Vimeo, Metacafe, Modovideo, Bitgravity, Globo, Terra, Viddler, Justin, Edgecastcdn, Youku, Wrzuta, 5min, Viddler, Tumblr, Facebook (https not supported);
Bomusica, Bullafina, Coqui, Cutmu, Domusica, Domusicas, Dungeonbusters, Goobeo, Gooveo, Miniclip, Molenillo, Mp3-Buscador, Musicaveo, Ning, Nofeehost, Sonicomusica, Turner, Vevos;
4shared, Mediafire, Etrustdownloads, Ziddu;
Akamaihd, Amazonaws, Blogspot, Imageshack, Issuu, Mercadolibre, Submanga, Ytimg;
Avast, Avgate, Eset, Mcafee, Avira, Bitdefender;
Windowsupdate, Adobe;
Porntube, Xvideos, Pornhub, Redtube, YouPorn, Serviporno, Tube8;
Appspot, Cloudfront, Juegosdiarios, Juegosjuegos, Netpocket, Nordeus, Popcap, Socialpointgames, Telaxo, Tetrisfb, Vostucdn, Wooga, Zgncdn;
Disneylatino, Friv, Geewa, Hulkshare, Kixeye, Llnwd, Maguinamotors, Manabar;
SpeedTest (some servers);
Configure your file /etc/haarp/haarp.lst to disable or enable the plugins.
Version 2.1.2
2016/01/04
- New plugins: Nvidia (drivers), Blazzers (World Of Wordcraft, Diablo 3, etc), League of Legends.
- Update plugin akamaihd.
- Change of format in some columns of the databases: support for cache of files of great size.
- Disabled for default the general plugin (in BETA).
Version 2.1.1
2015/12/08
- Update of the eset plugin.
- Reducing the workload on the haarpclean.
- Limit of count of users in the database.
Version 2.1
2015/12/01
- Improve performance for the use of the Mysql.
- Fixed the problem: 'general protection ip:7fc4e8d35b9f sp:7fff604fde10 error:0 in libmysqlclient.so'.
- Update regular expression (in the haarp.lst file) for netflix, globo and amazonwas. Delete plugin for amazonwas domain.
Version 2.0b
2015/10/28
- Plugin general.
- Adding script for check status of haarp; update process installation from a script
- Update reg. expressions for multiples plugins; reedit some messages of errors for Debug mode.
- Fixed problem with reader of header HTTP.
Version 1.5.1
2015/09/26
- Fix problem: "segfault at 10 ip 000000000042ee66 sp 00007fffb74d2270 error 4 in haarp[400000+45000]".
- Manager of multiples ip in the header "X-forwarded-for".
- Update bash script of installation: force the redirect to haarp from squid in squid.conf.
Version 1.5
2015/09/14
- Storing the address IP of the user who uses the cache.
- Adding user history for each file in cache.
- Update the regexp for some plugins in haarp.lst.
- By default, the 75% of the hard disk is used.
Version 1.4.1 Thanks to Hernan Dario Nacimiento.
2015/07/31
- Update the plugins for edgecastcdn, socialpointgames and akamaihd.
- Update plugin of Youtube: not cached of videos on live.
- Using the parameter ‘clen’ of the url from youtube as file size.
- Update the HaarpClean program.
- Fixed the problem of bad replacement string in the SQL requests.
- Update the file of configuration for logrotate.
- Fix problem: ‘general protection ip:7f39b6cdd44e sp:7ffc278771f0 error:0 in libmysqlclient.so.18.0.0‘.
Version 1.4
2015/07/07
- Adding 50 new plugins.
- Improved support for DASH technology.
- Update the file of configuration for logrotate.
- Fix problem: 'segfault at 6 ip 00007f850c2cef36 sp 00007fff19f22d00 error 4 in libmysqlclient.so.18.1.0'.
- Fix problem: 'Error in /usr/local/sbin/haarp: free(): invalid pointer: 0x00007fb5e166d618'.
Version 1.3
2015/06/13
- Fix Problem with concurrent caching of files.
- Update the plugin Youtube.
Version 1.2.1 Thanks to Samuel Espinoza, Oscar Vaquero y Fernando Maniglia.
2015/05/29
- Fix concurrent caching of files.
- Update haarpclean script.
- Update some plugins: Youtube, Dailymotion, 4shared, Issuu, Avgate, Submanga, YourPorn, PornHub, Xvideos, PornTube, Tube8.
- Files of youtube are cached with the watch ID from the URL.
Version 1.2
2013/12/24
- Improved ad-blocking plugin for Youtube.
- Update plugins: globo.com, youtube.com.
- Differentiation of the header for the chrome browser.
- Update of haarpclean script for the cleaning the cache.
- Synchronous to the storage of the cache in an only file.
Version 1.1
2013/02/09
- Cache dynamic and intelligent.
- Update of plugins for the smart operation.
- Multidisc.
- Blocking advertising to Youtube.
- Cache of videos in html5 for Youtube.
Fork created by
Manolo Canales Cuba
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent