RE: [squid-users] Wondering Why This Isn't Caching

Here the back-ported patch to 3.4.1, which we will be testing internally - until it gets incorporated into the 3.4 release tree...

Initial tests show that it works as expected - the vary-response-header is now handled correctly...

Thanks,
        Martin

Here the modified patch based on http://bugs.squid-cache.org/show_bug.cgi?id=3806 which applies against the clean tar ball of 3.4.1:

=== modified file 'src/store.cc'
--- src/store.cc 2013-02-16 02:26:31 +0000
+++ src/store.cc 2013-03-25 10:51:54 +0000
@@ -750,7 +750,7 @@
             StoreEntry *pe = storeCreateEntry(mem_obj->url, mem_obj->log_url, request->flags, request->method);
             /* We are allowed to do this typecast */
             HttpReply *rep = new HttpReply;
- rep->setHeaders(Http::scOkay, "Internal marker object", "x-squid-internal/vary", -1, -1, squid_curtime + 100000);
+ rep->setHeaders(Http::scOkay, "Internal marker object", "x-squid-internal/vary", 0, -1, squid_curtime + 100000);
             vary = mem_obj->getReply()->header.getList(HDR_VARY);
 
             if (vary.size()) {

Various fixes making Vary caching work better.
More work is needed to re-enable shared memory caching of Vary responses.

bag5s r12741: Do not start storing the vary marker object until its key becomes public.
bag5s r12742: Log failed (due to "Vary object loop" or "URL mismatch") hits as TCP_MISSes.
bag5s r12743: Refuse to cache Vary-controlled objects in shared memory (for now).

=== modified file 'src/MemStore.cc'
--- src/MemStore.cc 2012-10-16 00:18:09 +0000
+++ src/MemStore.cc 2013-11-08 22:02:09 +0000
@@ -293,40 +293,46 @@ MemStore::considerKeeping(StoreEntry &e)
     }
 
     assert(e.mem_obj);
 
     const int64_t loadedSize = e.mem_obj->endOffset();
     const int64_t expectedSize = e.mem_obj->expectedReplySize();
 
     // objects of unknown size are not allowed into memory cache, for now
     if (expectedSize < 0) {
         debugs(20, 5, HERE << "Unknown expected size: " << e);
         return;
     }
 
     // since we copy everything at once, we can only keep fully loaded entries
     if (loadedSize != expectedSize) {
         debugs(20, 7, HERE << "partially loaded: " << loadedSize << " != " <<
                expectedSize);
         return;
     }
 
+ if (e.mem_obj->vary_headers) {
+ // XXX: We must store/load SerialisedMetaData to cache Vary in RAM
+ debugs(20, 5, "Vary not yet supported: " << e.mem_obj->vary_headers);
+ return;
+ }
+
     keep(e); // may still fail
 }
 
 bool
 MemStore::willFit(int64_t need) const
 {
     return need <= static_cast<int64_t>(Ipc::Mem::PageSize());
 }
 
 /// allocates map slot and calls copyToShm to store the entry in shared memory
 void
 MemStore::keep(StoreEntry &e)
 {
     if (!map) {
         debugs(20, 5, HERE << "No map to mem-cache " << e);
         return;
     }
 
     sfileno index = 0;
     Ipc::StoreMapSlot *slot = map->openForWriting(reinterpret_cast<const cache_key *>(e.key), index);

=== modified file 'src/client_side_reply.cc'
--- src/client_side_reply.cc 2013-10-01 23:21:17 +0000
+++ src/client_side_reply.cc 2013-11-08 22:02:09 +0000
@@ -470,71 +470,73 @@ clientReplyContext::cacheHit(StoreIOBuff
         /* the store couldn't get enough data from the file for us to id the
          * object
          */
         /* treat as a miss */
         http->logType = LOG_TCP_MISS;
         processMiss();
         return;
     }
 
     assert(!EBIT_TEST(e->flags, ENTRY_ABORTED));
     /* update size of the request */
     reqsize = result.length + reqofs;
 
     /*
      * Got the headers, now grok them
      */
     assert(http->logType == LOG_TCP_HIT);
 
     if (strcmp(e->mem_obj->url, http->request->storeId()) != 0) {
         debugs(33, DBG_IMPORTANT, "clientProcessHit: URL mismatch, '" << e->mem_obj->url << "' != '" << http->request->storeId() << "'");
+ http->logType = LOG_TCP_MISS; // we lack a more precise LOG_*_MISS code
         processMiss();
         return;
     }
 
     switch (varyEvaluateMatch(e, r)) {
 
     case VARY_NONE:
         /* No variance detected. Continue as normal */
         break;
 
     case VARY_MATCH:
         /* This is the correct entity for this request. Continue */
         debugs(88, 2, "clientProcessHit: Vary MATCH!");
         break;
 
     case VARY_OTHER:
         /* This is not the correct entity for this request. We need
          * to requery the cache.
          */
         removeClientStoreReference(&sc, http);
         e = NULL;
         /* Note: varyEvalyateMatch updates the request with vary information
          * so we only get here once. (it also takes care of cancelling loops)
          */
         debugs(88, 2, "clientProcessHit: Vary detected!");
         clientGetMoreData(ourNode, http);
         return;
 
     case VARY_CANCEL:
         /* varyEvaluateMatch found a object loop. Process as miss */
         debugs(88, DBG_IMPORTANT, "clientProcessHit: Vary object loop!");
+ http->logType = LOG_TCP_MISS; // we lack a more precise LOG_*_MISS code
         processMiss();
         return;
     }
 
     if (r->method == Http::METHOD_PURGE) {
         removeClientStoreReference(&sc, http);
         e = NULL;
         purgeRequest();
         return;
     }
 
     if (e->checkNegativeHit()
             && !r->flags.noCacheHack()
        ) {
         http->logType = LOG_TCP_NEGATIVE_HIT;
         sendMoreData(result);
     } else if (!http->flags.internal && refreshCheckHTTP(e, r)) {
         debugs(88, 5, "clientCacheHit: in refreshCheck() block");
         /*
          * We hold a stale copy; it needs to be validated

--- src/store.cc.caching 2013-12-08 17:20:54.000000000 -0800
+++ src/store.cc 2013-12-13 02:29:00.101025671 -0800
@@ -795,12 +795,14 @@
             }
 
 #endif
- pe->replaceHttpReply(rep);
+ pe->replaceHttpReply(rep,false); // no write until key is public
 
             pe->timestampsSet();
 
             pe->makePublic();
 
+ pe->startWriting(); // after makePublic()
+
             pe->complete();
 
             pe->unlock();

-----Original Message-----
From: Amos Jeffries [mailto:squid3_at_treenet.co.nz]
Sent: Donnerstag, 12. Dezember 2013 21:03
To: squid-users_at_squid-cache.org
Subject: RE: [squid-users] Wondering Why This Isn't Caching

On 2013-12-12 23:57, Martin Sperl wrote:
> Ok - I have just tried the latest version from trunk and it seems to
> do what it is expected to do now!
>
> So the question is: will this patch also go into 3.4Beta or do we have
> to wait for 3.5 to get this resolved?

Well yes and no. 3.4 beta is done with. I am okay with it going into the
3.4 stables' though.

Amos

This message and the information contained herein is proprietary and confidential and subject to the Amdocs policy statement,
you may review at http://www.amdocs.com/email_disclaimer.asp
Received on Fri Dec 13 2013 - 11:51:15 MST