Information handling system low latency touch rejection buffer

A portable information handling system having a touchscreen display selectively rejects touch inputs by removing images from the display associated with the rejected touch inputs after presentation of the images. A touch buffer temporarily stores detected touches, such as by removing touches as the age in the buffer exceeds a threshold. If a trigger event is detected, such as detection of a palm touch or writing stylus, then all or selected of the touch inputs remaining in the buffer are removed as images presented at the display.