Re: Squid 2 - ?huh? why two ICP ports?

Hi

> I would be really disappointed if this is true, because Cache-Digests
> are expensive if you have multiple peers and a low-bandwidth link.
> I moved away from Cache-Digests, because they stall client connections
> and are TCP based (retransmits). I can live with lost ICP requests,
> but updating digests every n minutes from peers with large caches is
> not the way to go if you have limited bandwidth.
>
> Consider I have five peers and they have different update times for
> their digests. I would request new digests at 13:10, 13:20, 13:30,
> 13:40 and 13:50 for example.

I guess that it would be more efficient if a "diff" equivalent
(perhaps two RLE encoded masks? or was this done already?). This is
certain to be smaller.

> This mail is longer than I wanted. If anything above sounds offending,
> take my word that it's not intentional. I just want to clearly state
> what I think about moving away from ICP.

The problem here is that there are essentially two types of people
using caches:

1) people that want to reduce latency
2) people that want to save bandwidth

(1) are almost always in the USA. These people have huge caches, huge
amounts of bandwidth and very speedy networks. But they want faster
networks: specifically for the pages that are commonly accessed. These
people don't really care if their peering uses a moderate amount of
bandwidth, as long as the latency is low. These people will use cache
digests: once you have the digest locally your 'lookup latency' is
close to zero.

In the USA almost nobody peers with siblings, since ICP almost always
adds latency, defeating the cache's purpose.

(2) are almost always overseas. Generally (we) have slow, overloaded
links. We don't want to waste bandwidth with large data transfers
every few minutes, we would rather send smaller packets when we need
them.

It's important to realise, though, that cache digests are almost
certainly smaller than ICP requests if you have a heavily loaded
cache. If you send an ICP requests for each URL, each ICP packet is
about 60 bytes long (40 bytes for the request, with at least 20 bytes
for the requested URL).

Assumptions: a cache digest is 200 000 bytes (a large peer may exceed
this quite easily, though). This is equivalent to 3333 ICP packets
(200000/60). If you retrieve the cache digest every 10 minutes
(600 seconds), you need to be sending 5 1/2 icp queries a second to
make cache digests worthwhile (3333/600). For ISP caches, cache
digests make sense. For ISP customers, you almost certainly want to
use ICP.

For (2), multicast ICP may be best for ISP customers. I haven't played
with it, though.

Oskar
Received on Fri Mar 26 1999 - 05:38:47 MST