PHP: insert a short string into a long one without allocating much memory?

Question

Consider this code:

$position = ...
$data = substr($data, 0, $position) . $insertion . substr($data, $position);

Normally $data in the case of the script being discussed is no larger than 2-3 MB. But it happened that a user who would like to use the script tried to use it for $data with the size of about 17 MB and got

PHP Fatal error:  Allowed memory size of 33554432 bytes exhausted (tried to allocate 17687776 bytes) in //index.php on line 179'

where line 179 is the second one shown above ($data = ...), 33554432 bytes = 32 MB and 17687776 > 16 MB. Now I'm assuming that the problem is memory is allocated for both $data and substr($data, 0, $position) and substr($data, $position). I'm aware of the memory limit and suggested them to try to increase it. However, I wonder if I can optimize this so that we don't need the memory limit to be more than twice larger than $data.

So my question is: is there some clever way to insert $insertion to minimize memory usage? (not to store $data in memory twice) How about the case where substr($data, $position) is relatively small?

Answer 1

Your current implementation depends on target position. It most memory-efficient if the position near to end of data. Worst case is inserting into begin of the data.

Mini-benchmark for example:

$data      = str_repeat('abcde', 500000);
$insertion = 'abc';
$position  = 0;

$data = substr($data, 0, $position) . $insertion . substr($data, $position);

echo memory_get_peak_usage(), "\n";

Script above show peak usage is 7868728 bytes (on my machine). If I set the $position = strlen($data) / 2 then it prints 6619424. If append the string at the end of data ($position = strlen($data)) then it prints 5365720.

(Update: memory consumption of code above also depends on the PHP version)

But PHP already has efficient function for replace text within a portion of a string: substr_replace():

$data = substr_replace($data, $insertion, $position, 0);

In all cases, this requires as much memory as the implementation above (through concatenation of substr() results) in best case.