Static Variables and the efficiency of mainfile

fkelly · Posted: Thu Oct 27, 2005 9:52 am

I've been plaguing a Sentinel Forum with questions about includes and how things are executed for a while, as well as reading my F manual and trying experiments on my test system. All this was initiated when 64bitguy asked about whether some filters in mainfile.php and some tests for file_exists were needed if you KNEW that you were going to run sentinel.php. I'll move my questions here where they are more appropriate and won't distract from the main thrust of that thread.

Montego, Raven, Evader99 have all tried to help me but I have to say that I'm not satisfied that I've gotten to the bottom of this yet. The big question that I have is whether code in mainfile isn't being included or executed unnecessarily often resulting in reduced efficiency. I wrote a little function in mainfile to try to set a static variable and increment it every time mainfile is called. Here it is:

Code:

function mfcounter() {
if (!isset($mfcounter)) {
echo '<table bgcolor="green"><tr><td>setting mfcounter</td></tr></table>';
static $mfcounter;
$mfcounter = 0; }
echo '<table bgcolor="green"><tr><td>mfcounter= ' . $mfcounter . '</td></tr></table>';
$mfcounter = $mfcounter + 1;
return $mfcounter;
}

I don't really need the return at this point since the echoes are done in the routine. I call this routine from the point in mainfile (this is a 7.4 system) where the config table is queried. Every time I call it I get the message "setting mfcounter" and the value of mfcounter is 0. So what is it that I don't understand. My reading of the manual is that the static variable should "persist".

Figuring that maybe my coding was an issue I looked down in the is_user function that real coders wrote. There is a static variable there that I could test. So I put some echoes into is_user as per the following:

Code:

Doing this I wind up with "is user" being echoed in my funky green background probably a dozen times when I load a page. So obviously Nuke is calling this routine multiple times and the static variable isn't being set right. Is this just because of what I'm doing or is there a real problem here?

Also, with respect to the "how often mainfile is included or parsed issue" I tried to get around using the static variable counting method by adding a field to the config table called mfcount. I set it to 0 by default and wrote a little query to read the value into a variable, increment it and write it back to the config table. Sure enough, despite the require_once for mainfile in all the modules this value is being incremented every time you go from screen to screen in Nuke.

I guess that's enough questions for now and I sure will appreciate any enlightenment anyone can provide.

montego · Posted: Thu Oct 27, 2005 10:14 am

First issue: Your original function included in mainfile.php is being called only once. That is why you are seeing it only say "0". You are setting it to zero, then displaying zero, and then incrementing it. If the function was called again, you would be seeing "1" as you would expect. Bottom line is that mainfile.php is being parsed only once.

With regards to the call to is_user, this function is called many, many times -- as you are seeing -- throughout nuke and add-on modules. Therefore, it is not a valid test of what you are trying to understand Your original test is an accurate reflection of mainfile.php being "parsed" once.

The issue really is just because mainfile.php is being parsed once, that doesn't mean the functions aren't called many times. What 64bit was getting at in the other thread was that there are functions / code being called in mainfile.php that are doing similar things as to what Sentinel does, therefore, why have the duplication.

Hope that helps,
montego

fkelly · Posted: Thu Oct 27, 2005 10:30 am

Thanks Montego but I don't think that's right.

I just changed the code to do the following:

Code:

mfcounter();
mfcounter();
define('NUKE_FILE', true);
$dbi = $db->db_connect_id;

So now the function is called twice, no? Here's what the output says:

Quote:

setting mfcounter
mfcounter= 0
setting mfcounter
mfcounter= 0

I really believe there is something keeping my static variables from getting set. Likewise, if you look at the code in is_user that I provided, the tests for the static variable $userSave are never being met. I'm running PHP version 4.3.4 here at home on my windows XP system.

montego · Posted: Thu Oct 27, 2005 12:22 pm

fkelly,

Sorry, at work now, so not alot of time to think this through. Try changing the name of your function to something like fn_mfcounter().

fkelly · Posted: Thu Oct 27, 2005 2:18 pm

No rush. I tried your suggestion, no joy. I copied and pasted the test program from the PHP manual for static variables into a new php file and ran that and saw that it worked, so it's nothing related to my system. Probably it is just related to my understanding of the "scope" of the variables.

I modified my function to test isset as follows:

Code:

Then I modifed the function calls as follows:

Code:

fn_mfcounter();
if (isset($mfcounter)) {
echo '<table bgcolor="green"><tr><td>mfcounter is set after first function call </td></tr></table>'; }
else {
echo '<table bgcolor="green"><tr><td>mfcounter is not set after first function call </td></tr></table>'; }
fn_mfcounter();
fn_mfcounter();

The results I get are:

Quote:

setting mfcounter
mfcounter is finally set
mfcounter= 0
mfcounter is not set after first function call
setting mfcounter
mfcounter is finally set
mfcounter= 1
setting mfcounter
mfcounter is finally set
mfcounter= 2

What this says to me is that each time the function is called, the isset condition is not satisfied and $mfcounter is set. But somehow it is getting incremented the way it should. Maybe isset doesn't work with static variables?

Later: okay I read the manual some more. Here's the part that explains it:

Quote:

A static variable exists only in a local function scope, but it does not lose its value when program execution leaves this scope.

This is a bit counter intuitive. The variable stops existing when the function is done but it retains it's value? I guess that's the way it is.

So if this is true what's the point of using static variables in functions such as is_admin later in mainfile?

Code:

Would there ever be a situation in which the isset test at line 3 of the is_admin function would be met? It doesn't look that way to me.

evaders99 · Moderator Joined: Apr 30, 2004 Posts: 2856

The way I stop mainfile.php from being included more than ones is use a definition

Code:

if (!defined('MAIN_FILE')) {
define('MAIN_FILE',1);

...
}

That way, the code is only executed once

Raven · Posted: Thu Oct 27, 2005 4:16 pm

fkelly wrote:

The big question that I have is whether code in mainfile isn't being included or executed unnecessarily often resulting in reduced efficiency

Two very different questions! Being included unecessarily can be avoided by changing all calls to require_once('mainfile.php'); See

Only registered users can see links on this board!
Get registered or login to the forums!

for a better understanding. Being executed unecessarily is more of a coding issue. But, Evaders code will not help anymore than the require_once. Actually require_once is more efficient. You need to understand that require_once and/or Evader's code will only be in effect for the duration of the script that is being executed. In other words if you have a script like
script1.php
-- include('mainfile.php');
-- include('script2.php'); // assume script2.php has an include('mainfile.php');
-- SCRIPT1 ERROR!

script1.php
-- include('mainfile.php');
-- include('script2.php'); // assume script2.php has an include_once('mainfile.php');
-- script1 continues on its merry way NOT attempting to include mainfile.php again

script1.php
-- include('mainfile.php'); //mainfile is called and executed
-- script1.php continues until it ends or goes back to a menu
script2.php
-- include_once('mainfile.php'); // mainfile.php will be reloaded because script2 is a unique script at this point.

If nuke were set up to have a Main program that simply includes all subprograms, then mainfile would operate like what you want/expect. But, that would be a fundamental logic rewrite.

montego · Posted: Thu Oct 27, 2005 7:23 pm

I am sure that I am missing something here, but is it not true that all key clicks of pages (at least valid ones and not direct access hack attempts) for nuke go through one of three main "driver" scripts: root/index.php, root/modules.php, or root/admin.php? In each of these mainfile is loaded up-front. All other scripts are included with either include/include_once/require/require_once. Based on the rules laid out by Raven above, wouldn't that mean that mainfile.php would only ever be parsed once because if anyone in subsequent scripts used either the directives include() or require(), PHP would throw a parse error?

Therefore, can we not conclude that mainfile.php is "parsed"/"loaded" only once, but various functions defined within it could be called numerous times???

What am I missing guys?

montego

Raven · Posted: Thu Oct 27, 2005 7:31 pm

module.php is called, afresh, every time. So, mainfile.php is called afresh every time. Is that enough to shoot holes?

montego · Posted: Thu Oct 27, 2005 7:38 pm

Raven,

I am actually in agreement with you. I am not trying to say that somehow mainfile.php or any other scripts is somehow "cached" at the server level between key-clicks (request from browser to server). Yes, every request back to the web server and PHP script will re-invoke all of what we're talking about up above.

If I am reading fkelly right, he/she is suggesting (or trying to understand if) that somehow mainfile.php is called ("parsed") multiple times within a given request. Of which, I am trying to explain, as you, that that is not the case. Right?

Regards,
montego

Raven · Posted: Thu Oct 27, 2005 7:46 pm

If by a given request you mean a browser refresh/reload, then yes. In other words, if the "master" script calls 100 sub-scripts and each of those sub-scripts calls mainfile.php, mainfile.php will only be parsed one time assuming it's coded correctly.

fkelly · Posted: Fri Oct 28, 2005 10:01 am

Quote:

If I am reading fkelly right, he/she is suggesting (or trying to understand if) that somehow mainfile.php is called ("parsed") multiple times within a given request. Of which, I am trying to explain, as you, that that is not the case. Right?

Thanks to you folks and some experimenting I have learned that mainfile is not parsed multiple times for say a given index.php in a given module. However, I believe it is fair to say that it is executed multiple times. In fact I'm sure of that because I've added a field to the config table and a sql statement to mainfile and updated a counter in that field and watched it get incremented each time I clicked on a module. For the parts of mainfile that are functions of course, they won't get executed until they are called. And the parts that include other programs such as nukesentinel.php we would want to be executed each time to keep the demons at bay. However, what I would call the "initialization" parts such as reading in values from the config table into variables I would question whether there might be another approach. I'm going to see if I can figure anything out on that and I will post if I do. Given the problems with static variables I outline below I don't think that finding a solution that doesn't involve massive changes to Nuke is a slam dunk by any means.

In the course of this experimenting I believe that I've found an error with the way static variables are being used in the functions in mainfile. I believe that historically a test to see whether a static variable was "set" was added to several of these functions, such as is_admin or is_user. The idea was a good one: if the variable is already set then just return the value of the variable (in most cases a 1) and skip the rest of the processing. The only problem is that it doesn't work. Try the following code out in a "standalone" program and you will see:

Code:

<?PHP
function Test()
{
if (isset($a)) {
echo '<table bgcolor="green"><tr><td>$a is set ' . $a . '</td></tr></table>';
}
else {
echo '<table bgcolor="green"><tr><td>$a is not set ' . $a . '</td></tr></table>';
}
static $a = 0;
$a++;
echo $a;
if (isset($a)) {
echo '<table bgcolor="green"><tr><td>$a is set after initialization in function test' . $a . '</td></tr></table>';
}
}
Test();
Test();
Test();
?>

The problem is that the value of the static variable is incremented but the variable itself is not actually set when you invoke the function. I quoted something from the PHP manual previously about this and I've spent hours puzzling over it because it is so counter-intuitive. But I don't believe that in is_user, for instance, the test:

Code:

if (isset($userSave)) {

will ever be positive. Here is a version of is_user with my experiments shown that you could try (on a test system of course):

Code:

From my experiments there are a couple of keys to this. First, you need to declare $userSave at the top of the routine or else your tests won't work. Second you need to test the value and not whether it is set. Right now I think the test for isset($userSave) fails every time and the routine is executed as if the test weren't even there.

Raven · Posted: Fri Oct 28, 2005 10:20 am

In all of my mainfile code the variable is always at the top and I have tested mine and it does work (I helped develop the original process). I still use a modified v6.9 so if those aren't at the top then you would be correct. I don't know, right off hand, if that's FB or Chat's fixes.

fkelly · Posted: Fri Oct 28, 2005 10:54 am

Here's the first few lines of code from the 7.4 patched mainfile that I downloaded about 6 weeks ago (from the is_user routine):

Code:

function is_user($user) {
if (!$user) { return 0; }
if (isset($userSave)) return $userSave;
if (!is_array($user)) {
$user = base64_decode($user);
$user = addslashes($user);
$user = explode(":", $user); ... and so on
}

I can wait for the "patchers" to reply. This is nothing "fatal" -- you just don't get any efficiencies this way unless I am badly mistaken (which has happened).

Raven · Posted: Fri Oct 28, 2005 1:58 pm

Here is what I use

Code:

fkelly · Posted: Fri Oct 28, 2005 2:20 pm

Thanks Raven. Just to be sure I just went to nukefixes.com and re-downloaded the latest patched version for Nuke7.4. The code is as I quoted before and I don't believe it will work properly.

I suspect that someone from the nukefixes group will pay a visit here soon, they have been very helpful and responsive in the past, and if not I'll post something on their site.

The whole way that so-called static variables work is way too subtle (or obscure) for my liking, not that that matters a tinker's d***ed.

montego · Posted: Fri Oct 28, 2005 2:32 pm

Quote:

In fact I'm sure of that because I've added a field to the config table and a sql statement to mainfile and updated a counter in that field and watched it get incremented each time I clicked on a module.

fkelly, we are still not completely "connecting" here. We need to have a common language and understanding here in order to make sure we get this right. Let us dissect this from ONE click to a given News article (for example).

Upon that click, the browser has made its request to the web server using script modules.php. This starts the whole "process" if you will. That ONE request, now starts a whole lot of script parsing and execution. As you have found, mainfile.php script itself is parsed the first (and should be only, if as Raven has said that things were coded properly) time from modules.php.

Once PHP has parsed mainfile.php for syntax and defined functions, constants, etc., it executes the statements that are NOT functions (unless a given statement calls that function). So, yes, the call is made to get the config data and variables are set. Some of these statements may in fact include/require additional php scripts and the process continues sequentially from there.

Once each file is completely parsed and executed, control is returned to the scripts which included it. Hopefully by the time its done, all of these scripts have produced the intended output back to the browser and that specific request is complete.

Most certainly, once the user sitting at the browser clicks on another link, such as another news article or post a comment or something, all of the above happens all over again whether variables are static or not. Therefore, having the configuration values as static will not help anything.

Static variables become helpful when used in functions that are called multiple times within the whole process defined above, such as is_user(). They can also be a curse if you don't know what you are doing becomes the value is retained once the function scope is "lost". Especially an issue with counters or other mathematical calculations, concatenations, etc.

Remember, the web is normally "stateless", meaning that unless you as a programmer do something to retain data between "requests", nothing is retained automatically. Common ways of introducing statefulness are:

1) Cookies
2) Session objects
3) Hidden POST type variables
4) Database
5) File

As you can see, static variables is not in this list. In your example, if I am reading it correctly, you say that "between clicks" you are seeing the count go up in the database. Well, that is because you are saving the "state" of the count in the database so, yes, it will be incremented each time you submit another request from the browser to the server which in turn eventually includes and executes mainfile.php.

PHP is not like Java Servlets or JSPs where the code is compiled on the fly ONCE and then retained to service future requests. ALL of the PHP scripts in question get re-parsed and re-executed every time. That is the nature of scripting. Now, who is to say that the PHP developers won't introduce something along the same lines as JSP or Servlets, but remember that the web is "stateless", so you will still need to go out and grab key data from somewhere, such as is done in mainfile.php to get the site configuration data, or possibly a session object where it was cached for a given user's session.

That is just the way it is.

Regards,
montego

fkelly · Posted: Fri Oct 28, 2005 3:13 pm

Quote:

fkelly, we are still not completely "connecting" here. We need to have a common language and understanding here in order to make sure we get this right. Let us dissect this from ONE click to a given News article (for example).

Montego, I'm all for that and I appreciate your help. I think we are substantively in agreement but I agree that we are not completely connecting. I think that, with Raven's help, we are in agreement that in most cases (or modules) mainfile is just parsed once (because of the require_once syntax) but the statements in it are executed each time, at least those statements that are not inside functions. So, like you say, if you go to news several times the config values will be read into variables several times.

What I was thinking of experimenting with is to take the code that reads the config file values into variables and encapsulate it into a function. Then make the variables in that function static. Then have one static variable that serves as a "flag" for whether the config table needs to be read or not. The first time in the flag variable is 0, after the config values are put into static variables it is set to 1.

Then where the current nuke modules rely on mainfile reading and rereading the config table, they would just call the function. The function would return the variables e.g., return ($sitename, $nukeurl, $site_logo ... etc)

I know this may be a dead end. And I know that it may not save much if any computer processing cycles. But I'll learn something and if anyone is interested I'll report the results back.

By the way I have two custom modules on my site that I wrote before I knew about Nuke (and later converted). They use sessions to retain persistent data about the user and his privileges and they work like a charm without requiring me to keep rereading the tables to see what the user is entitled to. But I also know that making Nuke sessions based would be a total nightmare and that sessions have their own problems in terms of security. So I'm not even proposing to go there.

montego · Posted: Fri Oct 28, 2005 6:47 pm

Once a given request has completed sending the output back to the browser, without saving the variables using one of the methods discussed in my previous post, these variable values will not be retained across subsequent "key clicks".

It'll be a good learning process for you to go through.

Regards,
montego

fkelly · Posted: Sat Oct 29, 2005 9:37 am

Well before I tackled Nuke and mainfile.php I thought I would do a proof of concept that I could create any "persistence" in variable values in a couple of standalone programs. Last night I was fully convinced you were right, Montego, but I thought I would learn something. Now I have learned something but I'm even more perplexed. The static variables seem to "persist" even more than I thought they would. If anyone is "into" this just copy the following two segments of code into new php programs and run them. They are short.

Number 1 I call "test_includes.php" and follows: