Need to Monitor Dblog file

Posted: Fri Feb 22, 2008 10:57 am

As documented in rnconfig and the HowToInstall manual, we included a database logging facility in RN2.20.00. I'm going to post the key part of the instructions from the manual here in case anyone "glided" past then.

Quote:

Rnconfig now also contains a variable called $loglevel which can be used to generate a text file with all sql queries that are executed or just sql queries that generate errors. The default setting $loglevel=0 will not generate any log entries. $loglevel=1 will generate entries only for SQL errors. $loglevel=2 will generate entries for all SQL queries.
To set up logging you need to create a directory called rnlogs beneath your nuke root directory. Then create a file called dblog (no extension). You also need to be using the mysql.php file in the /db directory. If you have the $dbtype variable set to "MySQL" in your config.php then you are using mysql.php and can proceed. (Most users are using this but you should check). Finally, set the permissions on the dblog file to 666 on unix systems.
If you elect to utilize the logging feature, then we strongly recommend that you regularly review the dblog file using your CPANEL or VDECK file manager or any text editor that has access to the file in Windows. Use $loglevel=2 only in a single user development environment . Loglevel=2 is not a set it and forget it type of setting. Loglevel=1 can be used in most settings but you should still periodically review the file. If you are generating SQL errors it is definitely something that requires looking into or you can post them on Ravenphpscripts forums to seek assistance.

Now the reason I post this is to give an additional warning that if you turn this on as an admin, you need to remember to monitor the file periodically. I think the documentation is clear enough that you can't just turn loglevel 2 (to capture all SQL) and leave it. However, you might think that if you just turn level 1 on (just capture errors) you should be okay. This morning however on my test system I found the error file had grown to 269k. "What the heck" I exclaimed and went in to look. The problem was that at some point a couple of weeks ago the MYSQL server on the host had gone down. So you get a MYSQL server has gone away message for every SQL statement. With 100 or so SQL statements for every attempted page load this can build up quickly. So I just want to emphasize, if you turn this on then you need to monitor it.

In the future, as time permits, I'd like to build this in as an ACP "applet" that would monitor and even truncate the file size as needed every time the ACP is loaded. But that's not done yet and there's a million other priorities so just keep your eyes on dblog periodically for now. Or just turn it on when you are actively tracking an error or to monitor your system for errors for a specified period and then turn it off when you aren't looking. Also, you can periodically delete any records in the file (after capturing them off to a text file for further analysis if wanted) without doing any harm.

registered · Site Admin Joined: Jun 04, 2004 Posts: 6437

Thanks, fkelly - that's a good point. And...a good idea for improving RN.

Posted: Tue Mar 04, 2008 8:59 am

Idea for future - in the admin area, perhaps a notification that checks the file size of the log itself -

"Size of error log: 296K"

Would suggest comparing the returned file size and if greater than x, make the font blink red. .

-sting
(yes, I am still out here lurking. . .)

Posted: Tue Mar 04, 2008 10:28 am

Sting, that's a perfectly good idea and one I intend to incorporate. Basically I'm planning a logging applet just like the other applets on the control panel. Anytime you go into the ACP it will show the size of the file, we should probably also put the flags to turn it on or off in there and offer the option to display the warnings right on the screen, with some ability to page thru them the way NS does with tracked IP's. Also give the admin the ability to truncate the file right from the ACP.

LOL, there's my todo list, now send me some time to do it.

registered · Posted: Tue Mar 04, 2008 9:58 pm

sting, definitely good to see you are still out there. So, management life hasn't killed you off yet... I'm working on it... lol.

fkelly, I like the idea. With regards to logging, I wonder if we should then change it to be db based instead. Just a thought. Be careful with the paging through the file to consider PHP memory... i.e., read in chunks. Wink

(I know, I should have to say it, but I just shake my head sometimes with some of my software vendors as to what they come up with and I've had this problem with one of their J2EE-based apps. So frustrating.)

Posted: Tue Mar 04, 2008 11:23 pm

M. Yes, Sentinel stores the IP tracking data in the database and that could be a good model to follow. Maybe we should just wait a couple of months and see how the current implementation works out and then decide how much effort to put into this. If few people ever even turn on the logging or look at it it would be hard to justify a lot of programming to make it even more sophisticated.

Posted: Wed Mar 05, 2008 11:09 am

fkelly - I just emailed you 15 hours so you should be able to get that done with plenty to spare. Hopefully the attachment won't be too large for your inbox.

Let me know if you need more, I have no use for the hours I have piled up here, I am too busy to use them.

montego - As for management, I prefer to call it "running a day care center for software developers".

Wink

-sting
is glad to be back in an active forum post again....