Caching proxy possible?

General discussion about Gargoyle, OpenWrt or anything else even remotely related to the project

Moderator: Moderators

Post Reply
b0rg
Posts: 11
Joined: Mon Aug 22, 2011 2:42 am

Caching proxy possible?

Post by b0rg »

I have just started using Gargoyle and am far from an expert so not sure if this is possible. I would like to use the usb on my router to connect one of the spare thumb drives I have lying around and have it use that space for a caching web proxy, in an attempt to lessen the bandwidth used by our household with normal surfing and stretch how long it takes us to reach our monthly caps. Ideally I'd like something that didn't count against the user's quota if it came from the cache, but did count if it had to be downloaded from the internet. If that's not possible then something that counts against their quota either way would be acceptable.

Currently I have 5 constantly used pc's on my network so there is quite a lot of duplicated pages and a cache inside the network should do a lot to lower what actually has to be downloaded, my problem lies in wanting to set individual quotas as well. If I setup the cache on a separate linux box then as long as any of the machines here are using that as proxy the router would see everyone's traffic as coming from it so quotas would be out. Even if I find a proxy that lets me do quotas on it that would only handle web traffic and I need to monitor all of it, though web is all I really see a need to cache.

Is this possible via an addon or something?

pbix
Developer
Posts: 1373
Joined: Fri Aug 21, 2009 5:09 pm

Re: Caching proxy possible?

Post by pbix »

Interesting. I have never heard this subject come up before and there is no ready made solution in Gargoyle. I think it is possible to get close to your goal but it will require hours of effort on your part. If you are ready to work and learn and spend a few bucks here is the direction I suggest you strike out.

I see that squid has a package for OpenWRT so you can install it on your Gargoyle router if your router has sufficient RAM and FLASH (maybe some buck required). Once that is installed I would think some strategically placed iptables entries could transparently redirect your HTTP/HTTPS traffic to squid and not break the quota accounting that Gargoyle uses. Gargoyle quotas use iptable chains and so would the transparent proxy redirect so with the proper order it should work.

With this solution you would get caching and quotas at the same time. It is not exactly what you want because your users would not get credit for data they retrieved from cache. But it would lower your overall WAN load so perhaps you could administratively increase everyones quota based on the average cache hit rate you are seeing.

To go all the way and reach nerdvana would require you to modify squid such that it can interact with Gargoyle's quota system and credit users when they hit the cache rather than the WAN.

I was looking at http://wiki.squid-cache.org/Features/Tproxy4 this is an optional component of squid which seems relevant. Perhaps it could be installed and configured to allow you to reach nerdvana.

Good luck and keep us informed on your progress.
Linksys WRT1900ACv2
Netgear WNDR3700v2
TP Link 1043ND v3
TP-Link TL-WDR3600 v1
Buffalo WZR-HP-G300NH2
WRT54G-TM

b0rg
Posts: 11
Joined: Mon Aug 22, 2011 2:42 am

Re: Caching proxy possible?

Post by b0rg »

Thank you for the nudge in the right direction. I will read up on squid and iptables and see what I can figure out with it. I only have the most basic of linux knowledge so this will likely take a while but until ATT drops their quotas or there's another option in my area I need to find some way to drop traffic without totally changing how we use the net, and if I can get a caching server up that should make a nice dent considering how many of the same sites are visited several times daily by all 5 of us here.

The router I'm running (TL-WR1043N) has a usb port I can use to have considerable cache space, and it had the highest listed ram/flash from the compatible routers list, which is why I chose it when shopping for a router to use gargoyle with for the quota management. So if any router will work with it, it should be possible on this one.

ispyisail
Moderator
Posts: 5194
Joined: Mon Apr 06, 2009 3:15 am
Location: New Zealand

Re: Caching proxy possible?

Post by ispyisail »

and it had the highest listed ram/flash from the compatible routers list
maybe the ubnt routerstation pro should be listed

802.3af 48V compatible
Gigabit Ethernet Switch (4 ports)
128MB RAM
16MB Flash
USB 2.0 Support
SDIO Support
680MHz CPU (Option to overclock to 800)
Robust ESD/Surge Protection for Industrial Applications
Support for 3 Ubiquiti SR,XR,or SR71 radios
Built in dB9 connector
Easy Access to full GPIO signals

b0rg
Posts: 11
Joined: Mon Aug 22, 2011 2:42 am

Re: Caching proxy possible?

Post by b0rg »

Ok that does have some impressive specs for a home router. :) The fact that it's just a bare circuit board is a bit off putting for me, making some sort of case would be a pain.

gargoyle fan
Posts: 8
Joined: Sat Sep 01, 2012 12:16 pm

Re: Caching proxy possible?

Post by gargoyle fan »

I am also very interested in this. Has any progress been made with using squid cahce with gargoyle?

I have a wzr-hp-g300nh (64 MB ram 32 memory MB) running Gargoyle Version:1.4.7

It has a USB port for nas and i was hoping to attach a drive and use it as squid cache to reduce data quota and increase speed a little.

Im a noob to linux and code, so hold my hand a little? ;)

Thanks

Also looking for help in my DNS cache thread too thanks :) http://www.gargoyle-router.com/phpbb/vi ... f=5&t=3164

User avatar
ericwong
Posts: 430
Joined: Sat Aug 25, 2012 6:15 am
Location: Melbourne, Australia
Contact:

Re: Caching proxy possible?

Post by ericwong »

I wonder how much traffic you will be able to save with a local transparent proxy server setup...
Eric Wong

PM me if you need to buy Gargoyle router in Australia/NZ, willing to pay me to help you on your Gargoyle configurations or build custom configured ROM with pre-installed app or try to fix your bricked router. Yes, I am looking for job/work.

gargoyle fan
Posts: 8
Joined: Sat Sep 01, 2012 12:16 pm

Re: Caching proxy possible?

Post by gargoyle fan »

ericwong wrote:I wonder how much traffic you will be able to save with a local transparent proxy server setup...
Are you suggesting it would be pointless?

we have 23 devices often accessing the same content. The data saving might now be huge, but it would also make the net more snappy which would be really great considering its been split so many ways.

Any help implementing this please?

User avatar
ericwong
Posts: 430
Joined: Sat Aug 25, 2012 6:15 am
Location: Melbourne, Australia
Contact:

Re: Caching proxy possible?

Post by ericwong »

gargoyle fan wrote: Are you suggesting it would be pointless?
Don't know, never tried. At least I know it won't work in my environment because the data killer here is someone doing BT downloads or online video watching which caching won't help because I am pretty sure the chance that anyone watching the same content more than once is not likely...
gargoyle fan wrote: we have 23 devices often accessing the same content. The data saving might now be huge, but it would also make the net more snappy which would be really great considering its been split so many ways.
Are you suggesting a classroom sort of environment where you have dozens of users accessing exactly the same page? I would say this sort of environment would probably work well then..
gargoyle fan wrote: Any help implementing this please?
All the help you need is already well documented.
E.g. I was looking at using Tiny Proxy before for a private remote proxy but it will work well as a transparent proxy too.
http://wiki.openwrt.org/doku.php?id=old ... .tinyproxy
Eric Wong

PM me if you need to buy Gargoyle router in Australia/NZ, willing to pay me to help you on your Gargoyle configurations or build custom configured ROM with pre-installed app or try to fix your bricked router. Yes, I am looking for job/work.

gargoyle fan
Posts: 8
Joined: Sat Sep 01, 2012 12:16 pm

Re: Caching proxy possible?

Post by gargoyle fan »

Thanks :) i will do some reading and post back if i need help.

Perhaphs you could cache BT traffic too? sure you would need a big drive 2TB should do it though. Just an idea :)

Post Reply