Re: [squid-users] squid deployment in 6Gbit network with tproxy as L2 bridge.

On 24/08/2013 12:10 a.m., Pawel Mojski wrote:
> W dniu 2013-08-23 14:08, Pawel Mojski pisze:
>> W dniu 2013-08-23 12:12, Amos Jeffries pisze:
>>> On 22/08/2013 8:47 p.m., Pawel Mojski wrote:
>>>> Hi Guys;
>>>>
>>>> I have some intresting deployment scenario. I have to install squid
>>>> box(es) as L2 bridge in 10Gbit network with 6Gbit amonunt of traffic in
>>>> peak.
>>>> Squid is used to forward traffic to our ecap adapter. Ofcourse it's
>>>> impossible to handle that traffic amount on one box.
>>>> So, how to deploy it? I imagine such scenario. The first will be
>>>> "balancer", it will be a linux box with 2 10gigs cards. Then, the
>>>> "balancer" have to somehow redirect traffic to a squid boxes.
>>>> My first thought was to use wccpv2 protocol, but I have figured that
>>>> wccp router mode is very weakly supported on linux. I've found only two
>>>> projects, one writen in C from 2002 and one in python from 2011.
>>>> So, do you have any suggestions how to forward traffic to squid boxes?
>>>> The main thing is to provide source-ip spoofing functionality and have
>>>> only one bridge in 10gig network.
>>>> Squid boxes will be connected to the balancer seperate interfaces over
>>>> separate switch.
>>>>
>>>> Thanks in advance for further ideas.
>>> Yes WCCP may let you down here. There are quite a few limits to Squid
>>> support for it as well, not least of which is weak support for
>>> multiple caches per router.
>>>
>>> Doing this with only 1 bridge is probaly going to bite you as well, it
>>> will need one massive beast of a machine. Each Squid process will
>>> easily handle 100-150Mbps or so of traffic so you are looking at an
>>> estimate of around 45-60 Squid with 1 CPU core each.
>>>
>>> If you can find a box with enough CPU to split the traffic that way SM
>>> should serve you okay although it may have a lower bps capacity than
>>> standalone Squid due to the accept() races and UDS traffic the workers
>>> have to manage. Otherwise I suggest looking in the direction of policy
>>> routing the traffic using the iptables model for splitting traffic
>>> over ports
>>> (http://wiki.squid-cache.org/ConfigExamples/ExtremeCarpFrontend#Frontend_Balancer_Alternative_1:_iptables)
>>> but possibly splitting the traffic over routes to several cache boxes.
>>> Each of which doing regualar TPROXY on a smaller segment of the
>>> traffic before being aggregated back into the line upstream by another
>>> splitter/joiner.
>>>
>>> Amos
>> Thanks a lot Amos.
>>
>> I've already solved the problem yesterday.
>> I made a "balancer" machine which split the traffic over 8 squid boxes
>> (via ip rule) and then join it into one stream.
>> Disadvantage of the solution is require 16 (2 per each scanning box) on
>> the balancer, but I can deal with it.
>> Works like a charm.
>>
>> Regards;
>> Pawel Mojski
> require 16 additional vlan interfaces ofcourse, sorry for convinient.

Any reason that is not 16 boxen on a single vlan?

Amos
Received on Fri Aug 23 2013 - 12:25:30 MDT