Method and apparatus for determining the errors of a multi-valued data signal that are outside the limits of an eye mask

Disclosed herein is a method and apparatus used to measure the number of time a multi-valued data signal transmitted from either a communication device of subsystem deviates across and into one or more bounded areas or zones as defined by an eye mask that is overlaid onto an eye diagram. The present invention employs an iterative process to accumulate and display mask violation that might result from a data signal transmitted from a target device or communications subsystem that deviates across the boundaries either above or below or into the center of the eye diagram. In addition, the present invention also has the ability to isolate particular threshold voltage-delay points along the boundaries above or below and around the perimeter of the mask polygon of the eye diagram where mask violations have occurred. This provides the ability to supply additional information and feedback about the behavior and performance of the targeted device or subsystem being tested.