Order preservation in data parallel operations

Various technologies and techniques are disclosed for preserving input element ordering in data parallel operations. This ordering may be based on element ordinal position in the input or a programmer-specified key-selection routine that generates sortable keys for each input element. Complex data parallel operations are re-written to contain individual data parallel operations that introduce partitioning and merging. Each partition is then processed independently in parallel. The system ensures that downstream operations remember ordering information established by certain other operations, using techniques that vary depending upon which categories the consumer operations are in. Data is merged back into one output stream using a final merge process that is aware of the ordering established among data elements.