Image may be NSFW.
Clik here to view.
I wrote an blog entry about caching sql with php before, since then I re-wrote the php functions that I’ve been using. The main difference betweek the older functions, and what I will describe here is storing in memory versus the disk.
The old routines stored the results on disk and in doing so I assumed that I will not create a mess of files. However, about two months ago I found that my server is almost running out of disk space. By testing, I found that more than 500GBs are being used by temporary and small files thrown in the tmp directory, all of which are used to store SQL queries.
I tried to delete those files, and it didn’t work since the files were too many for the rm command. I had to run find, pipe the result to awk and write an rm command for every file. I also had to nice the process as I didn’t want to block apache or the database servers running. Finally, after approximately 60 hours the delete process was done.
I had to change the way the sql results are being stored to avoid future complications and to save the disk space. I found a PHP module called APC, which remains running with apache and can store PHP data in a way similary to assocative arrays.
After testing, I found that around 200-300 Megs of ram is sufficient to generate more than 99% hit ratio, which is excellent. What remained is how to store the data.
I had to insert a set of functions that perform update, delete and insert. Those functions will invalidate the cache of the corresponding rows. In my older website versions, I used to connect to the database for every user hit. Now, I do a lazy check in a function called dbh() which is not listed here. In some instances, if the same user keeps on browsing the website and reads pages whose rows are already cached, the website will respond without actually running any queries against the database, and it can run toally 100% out of memory. I found that in some cases those simple routines can lead to speedups of approximately 30x. Only if the user requests a row which is not cached, the code will then query the database (or update), cache the results and return it.
The function cache_query_row takes in the table name, and the condition that will be used to query. The condition is an associative array containing the field names in the keys, and the required values as data. It will then check the cache, if that field is not in the cache, it will query the database and store the result. The function expects to find at most one row. If more than one row exists, the first is returned and stored.
In case there was more than one row in the database, then start by first doing a query for the keys, and calling the cache storage functions to store the row keys, then fetch every single row using cache_query_row.
-
/* This code is released under the latest version of the GNU Public License.
-
* It is provided AS IS in the hope it will be useful, without any garantees or waranties
-
* of any type. If you do not have a copy of the license, you may download one from:
-
* http://www.gnu.org/copyleft/gpl.html .
-
*
-
* NOTE: Some parts of the code references other functions in other libraries that I built,
-
* specifically the function query_row, is assumed to return the first row of multiple
-
* matches in a database query.
-
**/
-
$cache_prefix=‘/tmp/clkercache/’;
-
-
$cacheenabled=1;
-
-
function makewhere($array)
-
{
-
$where=”;
-
-
foreach($array as $key => $value){
-
}
-
-
return $where;
-
}
-
-
function cache_fetch($filename, $expiration=3600) // One hour cache
-
{
-
$filename=$cache_prefix.$filename;
-
-
if (!$cacheenabled) return false;
-
-
// See if it exists in APC
-
$content=apc_fetch($filename);
-
-
if (!$content) return false;
-
-
return $data;
-
}
-
-
function cache_store($filename,$data)
-
{
-
global $cache_prefix;
-
$filename=$cache_prefix.$filename;
-
-
apc_add($filename,$coded,3600); // One hour
-
}
-
-
function cache_query_row($tablename, $condition)
-
{
-
$row=cache_fetch("clker-$tablename-$coded");
-
-
if (!$row){
-
$row=query_row("select * from $tablename where ".makewhere($condition));
-
if ($row!=FALSE) cache_store("clker-$tablename-$coded",$row);
-
}
-
-
return $row;
-
}
-
-
function cache_delete($filename)
-
{
-
global $cache_prefix;
-
$filename=$cache_prefix.$filename;
-
-
apc_delete($filename);
-
}
-
-
function cache_update_row($tablename, $condition, $row)
-
{
-
global $dbh;
-
-
// Force requery next time
-
cache_delete("clker-$tablename-$coded");
-
-
// Update the database
-
dbcon(); // Lazy test to connect to the database if needed
-
-
$tablename,
-
$row,
-
$condition);
-
-
return $res;
-
}
-
-
function cache_insert_row($tablename, $row)
-
{
-
global $dbh;
-
-
// Update the cache
-
cache_store("clker-$tablename-$coded",$row);
-
-
// Update the database
-
dbcon();
-
$oldsqltime=$sqltime;
-
$sqltime-=getmicrotime();
-
$tablename,
-
$row);
-
-
return $res;
-
}
-
-
function cache_delete_row($tablename, $condition)
-
{
-
global $dbh;
-
-
// Update the cache
-
cache_delete("clker-$tablename-$coded");
-
-
// Update the database
-
dbcon();
-
$oldsqltime=$sqltime;
-
$sqltime-=getmicrotime();
-
$tablename,
-
$condition);
-
-
return $res;
-
}