Multi-level cache having overlapping congruence groups of associativity sets in different cache levels

A computer cache memory having at least two levels includes associativity sets allocated to congruence groups, each congruence group having multiple associativity sets (preferably two) in the higher level cache and multiple associativity sets (preferably three) in the lower level cache. The address range of an associativity set in the higher level cache is distributed among all the associativity sets in the lower level cache within the same congruence group, so that these lower level associativity sets are effectively shared by all associativity sets in the same congruence group in the higher level. The lower level cache is preferably a victim cache of the higher level cache. This sharing of lower level associativity sets by different associativity sets in the higher level effectively increases the associativity of the lower level to hold cast-outs of a hot associativity set in the upper level.