Apparatus and method to reduce bandwidth and latency overheads of probabilistic caches

An apparatus and method to reduce bandwidth and latency associated with probabilistic caches. For example, one embodiment of a processor comprises: a plurality of cores to execute instructions and process data, one or more of the cores to generate a request for a first cache line; a cache controller comprising cache lookup logic to determine a first way of a cache in which to search for the first cache line based on a first set of tag bits comprising one or more bits associated with the first cache line; the cache lookup logic to compare a second set of tag bits of the first cache line with a third set of tag bits of an existing cache line stored in the first way, wherein if the second set of tag bits and the third set of tag bits to not match, then the cache lookup logic to determine that the first cache line is not in the first way and to compare a fourth set of tag bits of the first cache line with a fifth set of tag bits of the existing cache line, wherein responsive to a match between the fourth set of tag bits and the fifth set of tag bits, the cache lookup logic to determine that the first cache line is stored in a second way and to responsively read the first cache line from the second way.