PHP: Buffered and Unbuffered queries

Buffered and Unbuffered queries

Queries are using the buffered mode by default. This means that query results are immediately transferred from the MySQL Server to PHP and then are kept in the memory of the PHP process. This allows additional operations like counting the number of rows, and moving (seeking) the current result pointer. It also allows issuing further queries on the same connection while working on the result set. The downside of the buffered mode is that larger result sets might require quite a lot memory. The memory will be kept occupied till all references to the result set are unset or the result set was explicitly freed, which will automatically happen during request end at the latest. The terminology "store result" is also used for buffered mode, as the whole result set is stored at once.

Note:
When using libmysqlclient as library PHP's memory limit won't count the memory used for result sets unless the data is fetched into PHP variables. With mysqlnd the memory accounted for will include the full result set.

Unbuffered MySQL queries execute the query and then wait for the data from the MySQL server to be fetched. This uses less memory on the PHP-side, but can increase the load on the server. Unless the full result set was fetched from the server no further queries can be sent over the same connection. Unbuffered queries can also be referred to as "use result". Once all rows in the result set are fetched, the result set is gone and it cannot be iterated again.

Following these characteristics, unbuffered queries should be used only when a large result set is expected that will be processed sequentially. Unbuffered queries contain a number of pitfalls that makes it more difficult to use them, e.g. the number of rows in the result set is unknown until the last row is fetched. Buffered queries are the easier and more flexible way to process result sets.

Because buffered queries are the default, the examples below will demonstrate how to execute unbuffered queries with each API.

Example #1 Unbuffered query example: mysqli

<?php
$mysqli  = new mysqli("localhost", "my_user", "my_password", "world");
$unbufferedResult = $mysqli->query("SELECT Name FROM City", MYSQLI_USE_RESULT);

foreach ($unbufferedResult as $row) {
    echo $row['Name'] . PHP_EOL;
}
?>

Example #2 Unbuffered query example: pdo_mysql

<?php
$pdo = new PDO("mysql:host=localhost;dbname=world", 'my_user', 'my_password');
$pdo->setAttribute(PDO::MYSQL_ATTR_USE_BUFFERED_QUERY, false);

$unbufferedResult = $pdo->query("SELECT Name FROM City");
foreach ($unbufferedResult as $row) {
    echo $row['Name'] . PHP_EOL;
}
?>

Found A Problem?

Learn How To Improve This Page • Submit a Pull Request • Report a Bug

＋add a note

User Contributed Notes 1 note

down

polygon dot co dot in at gmail dot com ¶

1 year ago

The bufferred and unbuffered queries can be used for a limited amount of records.

For example; while implementing download CSV for a query using buffered way, memory limit issues comes up above 30,000 records to be buffered.

Similarly, For unbuffered the load switched to database server.

This load on both the web (buffered) and MySQL (unbuffered) servers can be reduced as below supporting download CSV for 30,000+ records.

<?php
// Shell command. 
$shellCommand = 'mysql '
    . '--host='.escapeshellarg($hostname).' '
    . '--user='.escapeshellarg($username).' ' 
    . '--password='.escapeshellarg($password).' '
    . '--database='.escapeshellarg($database).' ' 
    . '--execute='.escapeshellarg($sql).' '
    . '| sed -e \'s/"/""/g ; s/\t/","/g ; s/^/"/g ; s/$/"/g\'';

// CSV headers 
header("Content-type: text/csv"); 
header("Content-Disposition: attachment; filename={$csvFilename}"); 
header("Pragma: no-cache"); 
header("Expires: 0");

// Execute command via shell and echo the complete output as a string
echo shell_exec($shellCommand);
?>

There will be a bit of CPU consumption for the sed regex.

＋add a note