Automatic schema upgrade in a big data pipeline

Disclosed are embodiments for providing batch performance using a stream processor. In one embodiment, a method is disclosed comprising receiving an event that includes a plurality of fields and extracting needed fields from the plurality of fields. The method then serializes the plurality of fields and generates a new event that includes the set of needed fields and a hidden field, the value of the hidden field comprising the serialized fields. The method then transmits the new event for processing using at least one processing stage of a stream processor. In response, the method reserializes a processed event generated by the stream processor and outputs the reserialized event to a downstream consumer.