Re: storeGet() -> storeGetPublic() ?

Robert Collins wrote:

> Hang on, I must have missed something.
> Correct me if I'm wrong but to create a digest you set a bit x on in a large array if 1 or more cached URI's put through filter(URI)
> match that bit number.

Sort of, except that there are more then one bit involved, and for any
given bit there are many matching URLs.

> if we have a small ref count on each bit, and an event when a URI
> gets deleted, we can decrement the refcount for bit filter(URI).

Will increase the memory requirement for the digest by quite a bit
compared to the gains in amount of information..

> That should be a lot smaller.

Than having a full index yes.

> As for rebuilding the digest, what about a schedulable background
> process to create a new digest, and then swap the active digest
> across when it completes?

That is how it is typically done when it is time to clean up the digest.
However, if you cannot retreive the index of stored objects then this
gets a bit problematic.

> Ie for a period of time you have two digests, one active, one being
> built, and they are both registered to recieve URI deletion
> messages until the new one becomes active.

Or you could simply update the digest you have and hope that it is
reasonably correct. But unless you can read the store index the digest
will detoriate by time, typically forgetting older objects, and you
might also end up with overpopulated digest which is not of any use to
anyone, or a underpopulated one which uses to much space/bandwidth.

The good news is that digest sizes tends to stabilize once populated, so
with some cleverness we might actually get away with a digest that is
continously updated with additions/deletetions without having to rebuild
it if your goal is to keep an approximation of what objects we have in
the cache.

For the published digest there is one more criteria which is quite
important and which AFAICT requires one to have access to the store
index: We do not wish to publish information about expired objects to
our neighbours even if in the cache. This because they are not allowed
to initiate refreshes/revalidations in our cache.

/Henrik
Received on Sun Jan 07 2001 - 08:21:26 MST