Method and apparatus for managing data object size in a multi-user environment

One or more embodiments of the invention enable improved communication with a database comprising multiple clients utilizing multiple large data objects concurrently. For example when a client system interacts with a server with respect to a data object that is over a threshold size, the system may utilizing a communication methodology that minimizes system resource usage such as CPU utilization and network utilization. In one embodiment of the invention when a client request for an object falls within the relevant size threshold, one or more embodiments of the invention segment the object into smaller size chunks. Hence the server is not required to assemble all data associated with a request at once, but is instead able to immediately start transmitting smaller segments of data. Allowing for the transmission of smaller data chunks prevents the server from allocating large blocks of memory to one object and although the server may be required to handle more memory allocations, each allocation is smaller in size and can therefore be processed much faster. The determination of the chunk size is dependent on inherent system resources such as the amount of server memory, and the available bandwidth of the network. In addition, the determination of chunk size is dependent on environmental factors such as the time of day, the day of the week, the number of users, the number of predicted users for a given time and day based on historical logging, and the current and predicted network utilization for a given time and day. One or more embodiments of the invention obtain the chunk size and optionally obtain a chunk transfer size from a server that may alter these quantities dynamically in order to minimize resource utilization.