To fork or not to fork - that is the question

PHP-Portal Project Joined: Mar 05, 2004 Posts: 12

I've been going around and kind of "scouting the competition" as it were...

Almost any of the portals using php and MySQL have been hit by the SQL injection exploit. Would it be possible to have a "watchdog" type of script? I mean such that any time a url is being passed to the site, it first goes through watchdog.php and if it's ok it gets passed through?

This way you'd be working with ONE file when (and if) a security issue came up involving passing unexpected parameters through the URL. (which seems to be the bulk majority of the vulnerability reports)

Site Admin/Owner Joined: Aug 27, 2002 Posts: 17088

Exactly what my design will do. All scripts written for this framework will pass through the SQL Object, if you will, where certain security methodology will be applied to all. I will allow for the passing of parameters to "influence" based upon other security criteria being met.

Posted: Thu Mar 18, 2004 2:21 pm

OK, you need to provide a brief synopsis of the overall aims of the project

Give me some sound material to work with or alternatively give me grounds to make it up.

Vapourware has always been here.

The issue is: As I see it.

I installed the "protector" script a couple of weeks ago without really knowing what it would do. It has saved my site from several hacking attempts.

Actually my biggest fear is that someone makes a "magic" add on, everyone downloads this and "bang" you have been done.

You download a "must have" add on that is a trojan.

I have done it loads of times. Without really giving it too much thought.

Ohhh!

I am such a wally.

It's so easy to do is my point.

Posted: Thu Mar 18, 2004 3:50 pm

Gee, so much time and so little to do Laughing

This thread has pretty much chronicled the overall aims. I know it's long and I fully intend to get more out there as soon as I can. But, in brief, it is to be a framework that will allow you to plug stuff into. The functionality will be broken down into Core and Other. The Core functionality will be a type of API/Classes that will allow for plugins that all use the same base calls to access the high-risk tables, if you will. In other words, to verify an admin's password, right now you just "select * from nuke_authors where aid=5;"; In my design, you will never get there that way. You will do something more along the lines of

$adminInfo = new adminObject();
$adminInfoA() = $adminInfo->getAdminInfo('name','pass');
or
$adminName = $adminInfo->getAdminInfo('name');

Then, inside of the adminObject Class will be a method/function that calls the SQL Object AFTER the Security Object has cleared the call, etc. You know of the recent exploit of adding the UNION statement, or adding the "' or 1" to a where clause, etc. By containing the calls this absolutely can't happen. One of the main reasons that nuke keeps getting hacked is because it sends SQL logic via the get and post methods. This is much easier to protect from happening that you are led to believe. But, present nuke development keeps attempting to fix the symptom and not the cause. And, the author keeps releasing more 'features' with the same bad coding style, and reintroduces the same errors!

Now you will rightfully ask, "Well, I can still write my own routine and do whatever I want". That's true. However, I don't have to use it Smile

. So, the choice will be up to the webmaster as it is now. But at least I will be providing a safety valve if you want to use it. Security or not - the Webmaster's call. There will be a more generic call to allow for Admins to allow for 'exceptions'.

Hope this is enough for now Smile

Posted: Thu Mar 18, 2004 3:55 pm

I guess what I am pointing out is that YOU might design a great script, but how are you going to control the add on's?

Which is the point of open source right? So that everyone else can contribute?

I politely asked Michael at burnwave tonight if I could have permission to forward his script to Audioslaved to have it googletapped.

"I suppose" was his answer.

I understand the reasoning behind open source but this is my first foray into really get involved with this.

Quite frankly.

If I have purchased an "add on", in this case a shopping cart module.

If I have then offered to have it "tapped" for the benefit of the community.

Then I should have a better response than this?

I understand that there is a difference between "open source" and "commercial" but really Raven, how are you going to define "customer service" In the way that I would like it viewed?

So Mr Programmer - I have given you some programming view points, and I have tried to put my point of view.

I can sell your "vapourware" Mr Raven. Can you make the outcome become the reality?

Posted: Thu Mar 18, 2004 5:04 pm

I can only do what I can do. Ever hear the expression "you can lead a horse to water ..."? Coppermine and Gallery are 2 good examples, I think, if I understand your question. They were written as stand alone applications. Then, someone, maybe the original authors maybe not, decided there was a need to 'nuke' it, and it was. I wrote KISGB as a stand-alone guestbook and was literally diluged with requests for Nuke, PostNuke, and Phpwebsite modules. I explored what would have to happen and what the best approach would be - rewrite or alter or don't. My point is that if an application is written that people want to use in PHP-Portal, then someone will convert it.

It is not up to me, per se, to control the add-ons (and I know you're not saying it is). What I am saying is that I will provide a framework that has built in safety and other things that right now is left up to each person/module to reproduce. It is redundant, it bloats the system, etc. I intend on designing a highly efficient and solid framework that developers will WANT to code to. There will not be the need to figure out the difference between Sections, Content, whatever. Think of the savings in development time and cost when you simplify the database calls alone.

Yes, at this point it is VW. Can I make the outcome reality? I don't know. I can try and I have no ego to bruise. If I have to bail, then it hasn't cost anyone anything. I could just as easily have a heart attack and be taken out of the picture too Surprised

.Will it persuade anyone to leave nuke? Or PostNuke? Don't know. That's not my intent on going in. I am not intending to compete for the sake of winning. I'm just intending on offering a choice. One that I haven't seen yet.

I SO much appreciate your challenges and directness Smile

. It is very refreshing for me to have people challenge me with the right spirit and motivation. Now, if I could just find the time .....

Posted: Fri Mar 19, 2004 7:16 am

I came to nuke because I realised that a lot of the stuff that I wanted to do in html was already inherent in the nuke coding.

Things like "membership" and "newsletters" can take a typical beginner at internet marketing a month or longer to solve. With nuke it is not simple, but it is there if you look hard enough. Fancy newsletters et al.

I have decided to stick with the version of nuke that I have until such time as something supremely better emerges to do the task for me. It has been hacked and modded to such an extent that I no longer consider it "nuke". It's my own personal marketing monster machine. I could never have achieved this without open source.

As an aside, the credits to nuke remain. However I seriously consider removing them as people are using these to hunt down nuke sites and hack them. Proof is in your server logs. Eggs and suck I know Very Happy

I know that you will have considered these things, however here is my wish list. This is MY wish list. I am (or was Laughing

) a non programmer.

I think we have solved the search engine problem by using mod_rewrite. I understand your concern about system overhead. If anyone knows of a better way then please step forward. There are optimisation libraries available from Zend to help.

The theming system is a complete nightmare. You must include a system whereby one can modify the HTML code easily. The way that the "echo" statement is used means that when a php page is opened in Dreamweaver - the page is blank, because Dreamweaver does not understand the code. I know that programmers do not use dreamweaver - but people want things to look nice - and this is a good tool to use, along with imageready.

All graphic designers and web development agencies use Adobe and Macromedia products. It's a fact. There are open source alternatives, but if you want to produce something that is going to be EASILY used by the masses then make the output readable by these tools.

Whilst I am at it do we have to go down the road of "header", "footer" and side blocks? A single updateable HTML template would be great.

Next, you must not allow the core files to be overwritten in any way shape or form. This goes against everything that open source stands for I know, however here is my reasoning:

Using the current CMS, X produces a great system that everyone loves and uses. A security breach is discovered, Y patches it.

Z produces a module that everybody wants - it overwrites a file that Y patched. Crying or Very sad

Complete insanity.

Don't allow anyone to produce modules/add on's (if that's what they are to be called) that overwrite core files.

I was going to go on, However I'll stop there, because in a rare moment of clarity, I think I just discovered that you are way ahead of me in your thinking. Shocked

.

Yes, I know what I want you to make.

I want you to make a whore that will accept all comers for free.

I think you got here a long time before me though Very Happy

Posted: Fri Mar 19, 2004 8:05 am

As I mused on this last night I remembered that I had already considered the 'add-ons' issue. In my design, the MCP, along with other responsibilites, is a traffic cop. MCP determines who plays in his sandbox; who drives on his roads. Addons (and this is a generic term) will have to be recognized by Core before being allowed to execute. That "recognition" process has not been designed (obviously) but will be designed to not allow execution of malformed structure, if you will.

I believe I said early on that updates must NOT and will NOT overwrite core code. Even Core updates will not overwrite Core code. Instead, the concept of runtime libraries will be used, if at all possible. I am still exploring this as the PHP developers, in their infinite wisdom Laughing

, have determined that thou shalt not restate/reregister an already registered function. In any event, the intent is to have (conceptually) the following structure:

Folders:
Core (as delivered)
index.php
whatever.php
images
somepic.jpg

CoreUpdates
index.php
images
somepic.php

The MCP will always look in CoreUpdates before Core and will use files there (or wherever you direct him to) instead. Kind of a reverse-lookup scheme. Conceptually, you could have infinite releases available. Obviously, this is only for code as the database updates are not worked like this. So, bringing this down to a 'nuke' methodology, assuming there were no database issues to content with, you could have this structure and the MCP would know which to use:

origphpnuke v6.8
html
index.php
mainfile.php
config.php

patchedphpnuke v6.8
html
mainfile.php

patchedphpnuke v6.8 will have been mapped to phpnuke v6.8 so the MCP will know to look there first. Great way to "try" stuff! If overriding mainfile.php just produces a blank page, just remove it from the update path or tell the MCP not to override, or whatever.

Any clearer?

Posted: Fri Mar 19, 2004 8:41 am

<?php

/************************************************************************/
/* PHP-Portal: Your Title here*/
/* ============================================ */
/* */
/* PHP - Portal system Copyright (c) 2004 by Ravenphpscripts */
/* http://www.ravenphpscripts.com */
/* Your license information here */

I made a start for you Laughing

Hangin' Around Joined: Mar 19, 2004 Posts: 36

I agree whole-heartedly that there is a major need for a more secure phpnuke. All the bells and whistles in the world won't do any good if the site is so vulnerable that any want to be hacker can break in. This has been a large problem I have had with the current phpnukes.
But I also wonder about the wisdom in forking off from the current phpnuke, as you stated this causes divisions in the community and most of the time the new little guy is the one that fails. And it all becomes wasted effort if it fails. So I think a careful plan as to what is needed, and what is wanted. How it can be achieved. And how long will it take to have a working solution. Are all questions that need to be figured out before venturing forward on this path.

Also what would the chances be for cross compatibility? Would it be possible for unaltered phpnuke modules/blocks/ad-ons to remain compatible with this fork. I am guessing that keeping compatibility would be a bit of a challenge, and obviously an unaltered module/block/ad-on would not be guaranteed safe. But it would provide an immense pool of resources to help keep the fork afloat until its community can create a sufficient number of new secure modules/blocks/ad-ons and grow in size to become the new main content management system.

As scary as the thought of possibly heading down a dead end road is, security is a big enough issue for me, that I will follow Mr. Green

And I offer my full services in testing things on my server, programming (where I can I am no security or programming expert but I do understand most of the basics and don't mind monotonous work), planning and management, and perhaps even some financial support when I have it Rolling Eyes

You have provided an excellent service so far with all your security updates, support in installation, and help recoving after being hacked. Thanks a million Very Happy

Hangin' Around Joined: Feb 20, 2004 Posts: 29

Let me add some ideas about security - but they apply to virtually everything else and reflect the way I would like to see this new CMS handle things:

Today PHP-Nuke handles security as follows: in mainfile.php, it starts a loop through all GET and POST variables and checks if they contain "malicious code". How do those checks look like? They are calls to the eregi() function, using a regular expression to match the supposedly malicious code. But what does it match?

The word "<script>" for example. But what about " <script> ", i.e. a few blanks at the start or the end? What about "< script >", i.e. a few blanks following the starting "tag delimiters"?

What about "<script color="red">, i.e. "<script>" with some unknown, non-standard attribute?

Oh yes, we could "tweak" that regular expression. But what if we have to do 3 more tests? Or 4, or 5?

Such a situation "cries" for a table. A table with, say, "tests". A "test", in this context, consists of a regular expression and an "action". The idea is that all variables (GET, POST,...), or only some of them (?), have to "pass" all those "tests".

And now the clou: the administrator enters the desirable tests into that table.

The administrator entering regular expressions and actions? That looks like the awk programming model. Yes, indeed. Very Happy

But we will equip that table with a whole range of "sensible" tests. Should a security hole arise, we would then say "enter this regexp and this action into that table from the admin panel".

No more hacking code. Customizing tables is the way to go.

Let's think about it a little. Wink

Posted: Sat Mar 20, 2004 4:30 pm

It would have to be done so that it stays in memory or something so its not consuming all available connections. Sounds like an interesting approach. Cache the query results or use an XML cache of results that is only updated when something new is added.

Posted: Sat Mar 20, 2004 5:06 pm

I have been giving serious thoughts as to the object models. I'm inclined to go with an object model that is bound to the SQL. Originally I was thinking of separating them. In other words, for instance, there would be a USER object and a separate SQL object to read a table. So you would instantiate the USER object and the USER object would call the SQL table object to access the USER table. Instead, I think maybe the USER class should have a method bound that accesses the USER table. My thinking is that you should never be accessing the USER table unless you want the USER object. That pattern would then follow for the authors table and authors object. That, by inference, should stop the exploits like the UNION problem. Why? Because the MCP would not allow access to the authors table except that the author's Class/Object was calling it. Comments?

Posted: Sat Mar 20, 2004 5:59 pm

I think the user table itself should be limited to the actual ID#, username, email, pass, user_group. Leave all that other stuff in a seperate users_info table or something. Not a big deal to duplicate ID# ect... in a the user_info table. Checking one against the other would take and extra step but I think it would be worth it in the end. Then once user is logged in user can be granted access to user_info or not Very Happy

We can keep all access to user so sterile it would be most secure. But loosen up on stuff like siggies allow some basic html and attributes ect...

Going along with what your saying because I can't think of any reason it would be much of a hassle. Once user and user level are established by logging in the rest should fall into place nicely.

Posted: Sun Mar 21, 2004 2:27 am

sixonetonoffun, what you are proposing is the classic notion of normalization and the "extra step" is a JOIN of the tables through the foreign key (the user ID). Makes perfectly sense, as well as the idea of some kind of "SQL caching".

What I don't understand is Raven's prediction that an MCP with a user object that encapsulates all user relevant methods would eliminate SQL injections of the UNION type.

The problem I see is that, in order to be able to offer all possible information on users, you would need many, many methods - one for each JOIN of the users table with any other "normaliized" table containing user relevant information:

- a method for a JOIN with the address data.
- a method for a JOIN with the statistics data.
- a method for a JOIN with the user preferences data.
- ...and so on.

Now, if the MCP would (and it should) allow all those JOINs (which are implemented through the UNION SQL clause), then where is the added security? You still have to check if a GET or POST variable creates one of those JOINs which are allowed by the MCP, but are still unacceptable from the security point of view at that specific situation.

In my table of security tests, we would probably have to add a column "hook point". The meaning would be that at "hook point" 61, to say a number, only that and that regular expressions for the variables are acceptable. One could think of this "hook point" as a "workflow event", meaning that at workflow event "sending activation mail" only a JOIN with the address data is allowed.

Maybe we should "outsource" the MD5 password to a table with only two columns, user ID and MD5 password. Then we only allow access to this information at very specific moments. Like access to /etc/passwd in Unix systems and the PAM (Pluggable Authentication Modules) framework.

Ideas over ideas... Smile

Posted: Sun Mar 21, 2004 2:33 am

The UNION injections are a result of passing invalidated sql in the GET/POST values of variables. The added security is in the fact that the method would disallow any and all sql verbs in DATA. The method/object would be the only one allowed to do that. See what I mean?

Posted: Sun Mar 21, 2004 10:26 am

karakas wrote:

sixonetonoffun, what you are proposing is the classic notion of normalization and the "extra step" is a JOIN of the tables through the foreign key (the user ID).

Normalizing is the activity of placing data that is common to a column, not unique, in separate tables to avoid redundant storage of the same values. Or, more technically, "A process of removing alternate representations of equivalent sequences from textual data, to convert the data into a form which can be binary-compared for equivalence.". The table, under the present setup is just called "users". As such it contains user information. Now if there is data in the users table that is common, in value, to all users, then we would want to extract it and place it in a separate table, if it is an efficiency issue. Organization is really what I think is being discussed here. Since the data is pretty much unique to each user, then whether to split it out into separate tables, which in turn will mean more overhead and calls to MySQL, will have to be measured. OTOH, if a USER object is bound to the USER table, then a generic method can be built that will only return "stat" information when stat information is required. This does not mean that every table has to have a unique object bound to it. There could still be a generic one that is isolated form the individual ones. Different approaches to solve the same issue. Great discussion!

Posted: Sun Mar 21, 2004 10:38 am

Raven wrote:

The UNION injections are a result of passing invalidated sql in the GET/POST values of variables. The added security is in the fact that the method would disallow any and all sql verbs in DATA. The method/object would be the only one allowed to do that. See what I mean?

Yes, makes sense.

However, it *will* boil down to a check through a table that contains regular expressions of all those SQL verbs. I hope this can be done efficiently and also that we will not have to forbid URLs like

http://www.mysite.com/mcp.php?text="state of the union address"

just because there is the word "union" in some parameter.

edited by chris: added quote to avoid misunderstandings.

Posted: Sun Mar 21, 2004 10:55 am

Raven wrote:

The table, under the present setup is just called "users". As such it contains user information. Now if there is data in the users table that is common, in value, to all users, then we would want to extract it and place it in a separate table, if it is an efficiency issue. Organization is really what I think is being discussed here. Since the data is pretty much unique to each user, then whether to split it out into separate tables, which in turn will mean more overhead and calls to MySQL, will have to be measured.

Yes, we will have to make measurements. But what I was thinking of is not subject to measurements, but is rather a matter of "good practices": don't mix master data with dynamic (or transaction) data.

"Address" is master data, it doesn't change that often. "Statistics", like "number of posts" is dynamic data and changes every time a user posts something.

Today, in PHP-Nuke, master and dynamic data are all thrown into the users table. We should not do this. We should separate master from dynamic data.

Posted: Sun Mar 21, 2004 11:00 am

I would agree with that approach.

Posted: Sun Mar 21, 2004 11:13 am

karakas wrote:

Raven wrote:

The UNION injections are a result of passing invalidated sql in the GET/POST values of variables. The added security is in the fact that the method would disallow any and all sql verbs in DATA. The method/object would be the only one allowed to do that. See what I mean?

Yes, makes sense.

However, it *will* boil down to a check through a table that contains regular expressions of all those SQL verbs. I hope this can be done efficiently and also that we will not have to forbid URLs like

http://www.mysite.com/mcp.php?text="state of the union address"

just because there is the word "union" in some parameter.

edited by chris: added quote to avoid misunderstandings.

Agreed, of course. Since we are only concerned with SQL queries, I don't see that this will be an issue. The MCP (concept/reality) only passes control to some other object/action. If a user object is being accessed, the user object may have methods like
findByID()
findByUserName()
findByRegDate()

Those methods will cleanse the incoming data for the query, based on rules as you have mentioned (possibly table driven), knowing that it should not have verbs. A username might be union. So what do we do with that? Maybe we disallow certain names in the interest of safety. Now, let's say we have a "search" routine. That SQL might have a method called findByText(). The corresponding sql might be like
$comments = 'state of the union address';
$sql = "where comments like '%$comments%'";

which could be altered to
$comments = 'state of the union address'."%' union select blah";

so that
$sql = "where comments like '%$comments%'";

would be something like
where comments like '%state of the union address%' union select blah%'

(not complete but you get the picture). In this case we would see a 'like' pattern that could be checked for, and so on. I really do believe that between the 2 mentioned approaches, we could tighten the screws down CORRECTLY to secure the sql. Yes, hacking attempts will continue and will happen and will be patched. IBM, UNIX, LINUX, MS, ORACLE,ADOBE, ad nauseum - it continues with all of them. We will attempt to do our best and to respond quickly when something happens. Is any of this making sense or not?

Useless Joined: Aug 23, 2002 Posts: 213 Location: Chicago

I have 100% faith on Raven and Chatserve, they are one of the few guys who have brought the PHP-Nuke to what it is now... I would also like to mention NSN and NukeCops team, you all have been a great help to the community.

I canceled my club membership the first day when FB announced that club members will get the version before the public, I wanted to support him because he was supporting the community but all these new announcements wont do him any good.

We do need a fork that is more reliable and secure but at the same time still can support all the modules for PHP-Nuke.
There are 1000's of people who have wrote outstanding script for the community and that's what makes this community better than anyone.

As of right now, we have 3 forks that are outstanding and people can look up to.
1: NukeCops bundle: Most security fixes come from Chatserve, many other addons by Raven and few other coders. Most commonly used.

2: NSN-Nuke: almost a complete rewrite of core code, forced credit links to the authors, heard most of the modules aren't open source and you can't edit most part of it. You get what you have, nothing more, nothing less. Not a huge supporting team, most users can't help each other because of encrypted code. All PHP-Nuke modules are worthless for it so not too many choices when it comes to finding an addon.

3: CPG-Nuke: I just loooooove the concept of having BB code used everywhere, outstanding speed and you are able to use most modules for PHP-Nuke with a little or no code editing. Support team is VERY small but is very active and has way less issues when it comes to running it, layout for admin area and normal user is just classic. A little bigger number of people are using it than NSN-Nuke or so it seems.

My personal suggestion will be either work with CPG-Nuke or do add the lead coder of that CMS in your team and start a fork that can actually give ALL types of PHP-Nuke users a new home. This is exactly why NC bundle is so popular.

My 2 cents.

Posted: Sun Mar 21, 2004 2:34 pm

Raven wrote:

In this case we would see a 'like' pattern that could be checked for, and so on.

I think, at the end, we will have to "parse" the resulting SQL query, make some DOM (Document Object Model) of it and search it with XPath. Shocked

Just kidding. Very Happy

We have a "preprocessing" and a "postprocessing" step too, which I haven't mentioned yet: at preprocessing, we consult the "data dictionary" for the "type" of the variable we are expecting from GET or POST. If our GET/POST variable is, for example, "user", then we check the type of the "user data element" in the data dictionary. If it tells us it must be "max. 30 chars" (where the maximum may well be adjustable in the admin panel), then we can truncate the URL "user" variable to 30 chars. If it tells us that it cannot contain %$?#'" etc. we can check that. If it contains the name of some external test script, we can run it.

At postprocessing, our "test table" may contain information on whether the expected result is "0 or 1" lines, or more lines. Thus, if we get more than 1 lines due to a (possibly naive) UNION, but we know from our test table that for this object the result should be at most 1 line, then we can catch this and issue an error.

Regular Joined: Aug 22, 2003 Posts: 84

I’m using 7.2 and its faster!

I stripped right blocks out...
Slimmed it down.

It’s FASTER! Some users have complained to the walls that my 6.9 had issues.

Because I cant fix issues my self.
I want to support a "Guru" and for the truth... My last several versions have come from this page.

I want a distro:

1. Stable
2. Minor bugs removed.
3. Support and make donations
4. Community
5. Fixes

But users are always going to install things and break it and come back with a HELP request.

I am very happy with 7.2 but I had a day and 1/2 of bugs to work out.
(The red-x, and I removed guest avatars all together)

I vote for solid secure version. But then some time down the road they will release "Nuke 10" and o man...that sounds to cool. I would have to install it. LOL..

No, sometime something out there will blow us away! I don’t know...
What else could FB or others install? I like the Beta nuke cops automatic upgrade system.

What ever you do, people will eventually follow the "Flock" of people following FB.

Then at that point you will have to call your 6.9 RV call it 7.0 RV.
and start over fresh with a gang of bugs!

Posted: Tue Mar 23, 2004 11:17 pm

This is NOT going to be a fork of nuke. Conceptually it will be a framework for many things - a Portal. A CMS can live in a portal. My envisionment is that the portal will accomodate 'contexts' or templates, if you will. The main template/context that we will design first will be a phpnuke template. But, regardless of what template is plugged in, the underlying infrastructure/api will be the same. The name is PHP Portal.