Programming a new interface for NSN groups

Posted: Sat Oct 27, 2007 2:43 pm

NSN groups is a vital component of Ravennuke and the efforts of its authors are much appreciated. However, I believe that we need to make groups even more central to Ravennuke over time -- for instance I think we need to have a way to base administrative privileges off of groups rather than assigning admins to the authors table (a separate topic). To do that we first need to improve the groups administrative interface. If you don't know what I mean go try to add users to a group and page thru the extensive listing that results.

What I think we need is a screen that shows for a group, in columns:

existing users in the group
a list of all users with a select option
a list of users that will be added based on selections as they are made (with the ability to deselect)
possibly a list of all users that will be in the group after submit is hit and the groups table is updated

However, this design becomes problematic for large sites with thousands of users. In particular we may need to segment the listing of all users to find them in batches. This would mean submitting the form via some kind of "continue" or "get more" button until your final list is built, then submitting it to update the groups tables. Likewise, to add interactivity to the form so that we could keep tabs on what users have already been selected to join the group we'd need to use Jabascript -- I think.

We've all seen this kind of interface on other systems. And following St. Augustine's dictum "man is as lazy as he dares to be" I'm not interested in reinventing any programming wheels. Does anyone know of any code we could adapt to this situation? Does anyone have any insights on how this could be accomplished with the least pain? Personally I hate programming in Jabascript but I really don't see how it's going to be avoided here.

registered · Posted: Sat Oct 27, 2007 4:46 pm

Using AJAX techniques to refresh a list of users with "get more" or "continue" buttons to keep you on the same page would be nice here.

I also like the idea of basing access rights on groups and getting that more ingrained in nuke.

Posted: Sat Oct 27, 2007 5:26 pm

I have a book on Ajax setting right here. I'm not sure that's even needed and I'd love to avoid Javascript but that might not be possible.

I keep intending to do a longer post on the administration system in Nuke and how it could be improved by using groups but that is just not something I can sit down and "bang out" in a single session. But yeah, if you look at the current admin system, it is crazy ... especially where you want to give certain admins limited rights within a given module or two. The granularity just isn't there plus the overhead of going back thru admin.php looking for case and links files every time is just nuts. But I need to spell that out better. Meanwhile I was hoping to figure out something for the mid term with nsngroups.

registered · Posted: Sat Oct 27, 2007 9:09 pm

Yeah, I've been waiting for CNBYA 5.0 to come up for way too long. Might be worth taking a look at what NukeEvo has done with 4.4.2 or fade2grey and starting with that.

I agree. Its a mess. I am used to far more sophisticated user and permission management tools that this is a "joke" (standard PHP-Nuke). I applaud BobM as well for putting together NSN Groups long back because at least we finally had some means for managing user group permissions.

I'd like to see a user's "object" with permission model be matched up against the permission model for the particular module function... allowed or not... etc.... Total re-write anyone? lol.

Posted: Sat Oct 27, 2007 9:34 pm

montego wrote:

I'd like to see a user's "object" with permission model be matched up against the permission model for the particular module function... allowed or not... etc.... Total re-write anyone? lol.

Yes that is how I would do it too. Rewrite?....Sure!!!! Very Happy

Posted: Sat Oct 27, 2007 9:35 pm

fkelly wrote:

I have a book on Ajax setting right here. I'm not sure that's even needed and I'd love to avoid Javascript but that might not be possible.

I think the J in AJAX is Javascript. Smile

Posted: Sun Oct 28, 2007 7:04 am

Yes Gremmie it is. And I think that AJAX requires a pretty high speed connection and some other prerequisites that many Nuke sites (or at least users) may not have. Heck, right now you can see in many places where Nuke avoids using even plain old Javascript where it would be logical to do so. Instead it will validate the data (e.g., does the confirm password input match the change password input) in the receiving script and give you an error message and a GOBACK link if you mistype.

What I was thinking of for starters was maybe an "advanced user ad" function for the index.php of the nsngroups admin. That way the old method could stay in place for those who like it while we work the rough edges out of the new approach. I'm cautious and an incrementalist at heart and I think that having an easily managed way of putting users into groups and managing them there is the core we need before we go hog wild on an user / group permissions system.

It's a separate issue but I'd almost like to break the links between NSNgroups and Forums groups and not have nsngroups do any updating of the forums groups (and vice versa). There have been some recent posts about problems caused by having the two partially tied together. Forums can and does work by itself and we might be better off just leaving it alone.

Posted: Sun Oct 28, 2007 8:06 am

Just a little nit to preserve it for my own memory. There are two programs that handle adding users to groups. They are both in the admin/modules/nsngroups directory. The first, NSNGroupsUsersAdd.php, displays all the users for a group on a screen. This is "paged" with up to 50 users on a page (you select the number in Groups Admin). There is a checkbox for each user who can be added. If the user is already in the group then the checkbox is disabled.

When you click submit the NSNGroupsUsersAddSave.php program is called (it goes thru the whole admin.php and ops thingy). The user is added to the group AND to any corresponding PHPBB group, then the page is redisplayed with the checkbox for any users you have selected disabled.

Note a couple of issues. First, the page is not automatically incremented. You have to go down and select the next page. Basically you have to "guess" where in your list of users alphabetically any user you are looking for will fall ... or else page thru the whole list (I have 24 pages of users) to find what you are looking for. Second, if you check someone on the first page, then increment the page and select someone on the second, then submit ... only the person from the second page is is added to the group. I'm thinking that short term we might want to put a warning about this behavior on the screen as it's caught me short a number of times.

I know the authors had good reason for doing the application this way. You don't want a select box for users with 10,000 users in it like some sites might have. You may not want to use Javascript to store an array of users to be updated when the final "submit to update" is hit. And you might not want a temporary table to hold the users to add to the group either. Again maybe as a temporary fix we can look at updating the groups tables with entries from the old page if a new page is selected. Don't know: this will change the behavior of the application a bit and anyone who is used to it the old way might be surprised.

As I say, I'm just using the Forums here for some note taking, but anyone who wants to comment is welcome.

Posted: Sun Oct 28, 2007 1:35 pm

More notes. Some recent threads have discussed problems occurring "between" NSNgroups and forum groups. Here's what happens:

When you go to add a group thru nsn administration, the nsngroupsaddsave program also adds records into the bbgroups and bbuser_group tables with a group_id equal to the phpbb field of the nsngr_groups table. That is generated by a sql_nextid against the table bbgroups. So, essentially a linkage is set up between the nsngr groups tables and the forums groups tables.

Likewise when you add a user to a nsn group he is added to the corresponding forums group, matching the forums group_id field against the phpbb field from the nsn table.

Great so far. And if you delete a nsn group the corresponding forums groups gets deleted too as do all users in that group in the bbuser_group tables. Okay, sort of.

But, suppose your forum admin goes in and sees a group and starts adding users to it thru the forums usergroup facilities. Forums doesn't know anything about nsngroups so now you have "extra" users in the forums "version" of a group who don't exist in the nsn version of the group. And, if your nsn group admin goes in later and adds one of the users who has already been added to the forums group, that user will be added to the forums group twice. I think that can cause problems.

I'm also thinking it could be problematic if the forums admin went in and added a bunch of users to a group that had initially been created thru nsngroups. Then the nsngroups admin would only see the original users he created and maybe think "I can blow that group away" and do so and the forums admin would all of a sudden be missing a whole group.

In short I don't like this whole relationship. I think nsngroups should be totally separate from forums groups. I think we should dump all the updating of forums groups out from a revised nsngroups.

Consider also, suppose PHPBB changed their groups table structure. Now we try to update to it. Oops nsngroups is generating SQL errors.

Posted: Tue Oct 30, 2007 6:47 am

fkelly, you kill me. Laughing

Just too much to read right now... sorry... but, did latch onto this

Quote:

It's a separate issue but I'd almost like to break the links between NSNgroups and Forums groups and not have nsngroups do any updating of the forums groups (and vice versa). There have been some recent posts about problems caused by having the two partially tied together. Forums can and does work by itself and we might be better off just leaving it alone.

I am not in agreement with that. I would rather not have to manage a separate set of groups for phpBB. Maybe we should try and fix the integration there better. I'd rather see on the phpBB side where it uses the groups and users for those groups that were previously set up using NSN Groups and all that has to be done on the phpBB side is to assign forums to groups. This use model is the same with modules, blocks, and messages (would be nice in news too) and I would think it be good to also simplify it as such for phpBB.

I'll have to read through the other two posts another time and comment.

Posted: Tue Oct 30, 2007 7:15 am

Take your time M. I am posting this stuff partly to remind myself as I move thru this. I recently found a fix in these forums that I had written just a few months ago and totally forgotten so it helps to record things where they can be searched.

Short term I am looking at improving the add users to a group screen in NSN without changing anything on the relationship between NSN groups and Forum groups. The issue that I have with what you wrote is that to make the relationship "two way" -- in other words every time you add or change a forum group you'd make the corresponding change to a NSN group -- means that you'd have to mess with the forum code. Then, when there's a forum update you are stuck trying to integrate your changes into the new set of forum code. If you look at what they are doing with PHPBB3 it promises to be fairly major.

There would also be a problem with "legacy" groups in the forums that don't have any equivalent NSN group. You'd have to detect which ones they are and leave the NSN side alone for those. It really could be quite a mess.

It's a different thread and issue but I'm thinking that we might really be better off long run NOT integrating Forums into Nuke AT ALL with the possible exception of some import/export filters between user tables. Just run, starting with PHPBB3, forums as a standalone. You'd get rid of all the add-ons to Nuke themes you have to do to have Forums templates and you'd be able to do PHPBB updates without worrying about what it does to integration. But that's off track for this thread as I say.

Posted: Fri Nov 02, 2007 1:38 am

In an ideal world, modules should be just that - modular.
With the forums for examle, if you removed the 'Forums' folder it shouldn't break anything (though we all know it would result in catastrophic failure of the CMS).

Personally, I think the key to handling groups is to firstly get modules completely modular. Forums have always been a pain in this respect due to inter dependencies with themes, includes/. etc

Again, personally, I wouldn't be wanting to use two seperate 'group'ing functions, one for nuke, the other for the forums. The integration of the forums should cater for the scenario that nuke groups will affects its behaviour/access rights but also that there should be no real inter-dependency between them - you should still be able to remove the forum module without breaking the core 'groups' functionality.

I'm more than happy to add 'modularize' the forums to my TO DO list but don't expect work to start until 2008

Posted: Fri Nov 02, 2007 7:31 am

Quote:

Again, personally, I wouldn't be wanting to use two seperate 'group'ing functions, one for nuke, the other for the forums. The integration of the forums should cater for the scenario that nuke groups will affects its behaviour/access rights but also that there should be no real inter-dependency between them - you should still be able to remove the forum module without breaking the core 'groups' functionality.

The devil is in the details Guardian. Smile

I can agree with you theoretically but then when I really look at your statement and think about how the code works ... Let me give an example. When you add a group thru NSN groups it also adds a corresponding Forums group. These are separate groups in separate tables in the database. If you "remove the forum module" -- which in my mind also means removing the Forums tables -- you break NSN groups. The SQL won't work. Perhaps "breaking" is not the right word but at a minimum there will be all sorts of SQL errors in the background as tables don't get updated and can't even be found.

Now let's take an even more likely scenario. If you Google PHPBB3 and find their pages describing their release candidate, you will see that they say they are making significant changes (improvements) to groups functionality. I haven't looked to see if that means table structure changes but it very well could. And if it does that means we've got to change nsngroups as part of any integration with PHPBB3. Oops.

In my mind there is a very big difference between the integration of modules that are written to be part of Nuke and those that are written completely independently and whose authors would probably prefer that Nuke die a violent and painful death. In the first case we can probably count on some degree of cooperation and communication and in the second case we can pretty much count on them making whatever changes they see desirable for their standalone product and Nuke can go to h*ll.

Now maybe we can ingeniously code around this: have some kind of check for the existence of Forums and the version in the nsngroups code for instance, but it sure adds a lot of complexity. Somewhere on my todo list is to download a copy of PHPBB3 and install it and take a look but I'm afraid it's not something I'm likely to get to right away. It's another topic entirely but I'm thinking: just run the darn thing (PHPBB3) as a standalone product with some kind of import/export between user tables at most and leave integration and all it entails totally out of the picture. But then of course we'd have to deal with the fact that integration is built into the core of the Nuke code now and we'd want to start tearing that out, which would create a whole new set of problems.

Things are so simple ...

Posted: Fri Nov 02, 2007 9:16 am

The forum *should* be stand alone. The 'module' component should merely be there as a transition layer or wrapper to pull the forums into the nuke header/footer/side blocks and of course synch and relevant data between core nuke tables and those required by the forums itself.

As you rightly point out Smile

if you remove the forums, everything falls over because at the moment there are hard coded (and other) inter-dependencies and it is those we need to get rid of/work around IMHO.

Easier said than done I know, especially when you have two lots of scripts sending headers and goodness knows what else to the browser Sad

but in *theory* at least, a module should be just that - modular - plug it in, pull it out at will with no adverse affects on anything else.

Of course there is the alternative of integrating the forums and nuke closer together so that they are truly part of the same distro but that would mean doing our own updates, fixes and moving away from thrid party code completely.

All good stuff though and food for thought. I have not tried the phpBB 3.x branch for quite some time so I may go back and re visit that soon.

Posted: Sat Nov 03, 2007 8:55 am

IMO, people want integration (at least I do) because I want my "customers" to have a consistent visual and navigational experience and I do not want double registration and other type nightmares. We are partly there with the current integration; notice I said "partly" - its not bad, but definitely not 100%.

There is no free lunch. My vote is that if we're going to include it, it should be fully integrated. Heck, we've already been mucking with the phpBB 2.0.x code now anyways. But, the good news is that phpBB 2.0.x branch updates a few and far between now and quite stable, so really, how much work really would it be to employ a 2.0.23 patch if one ever came out? Not much IMO.

Posted: Sat Nov 03, 2007 9:35 am

Agreed on the no free lunch M. Agreed also that fully integrated is BEST and avoiding double registration etc. is BEST too. However, in my first two years running a Nuke site (like 3 years ago to 2 years ago) I went thru a number of painful Forum upgrades ... running Approve Membership made it worse ... from like 2.0.7 to 2.0.22. It's been relatively stable for the last year and that's a good thing.

However, PHPBB3 could be painful also and then when they get there I'd guess there will be more frequent minor release upgrades. Whether they'd play havoc with any integration we did for PHPBB3 is a big unknown ... heck PHPBB3 is a big unknown.

I've seen sites, www.expresso.net is one, where they have dual registration. It is not THAT BAD of a thing. They just have a "take me to forums" link on their main pages and then a link back. The advantage is that you can run addons and hacks for PHPBB without having to worry about what they do to your integration.

But this is getting off track, which I tend to do.

registered · Posted: Sun Nov 04, 2007 2:10 am

Forums should be standalone as much as possible, really phpNuke needs to integrate some easy hooks to share registration... Mediawiki does that pretty well.

I would work on phpBB3, but the quick release of RC6 followed by RC7 doesn't give me much confidence in a final (relatively) finished release right now.

Posted: Tue Dec 18, 2007 1:53 pm

I usually know the nick of the user I want to add. So, would it be very difficult to add a Search feature to the Add User page?
Also, the high limit for rows per page is 50. Would it require a lot of work to change that limit to 100, 200, 300, 500 even 1000?

Posted: Tue Dec 18, 2007 3:19 pm

We have talked about having a "generic" search feature in Your Account and should probably design this so that it can be shared in Groups maintenance also. There is a very rudimentary (and optional, you will turn it on in rnconfig) dropdown box in YA admin coming in RN2.20. It searches on any string you put in in both the email address and username fields. The same type of logic could be applied to nsngroups. The search is done in Javascript and the select box is narrowed down to those users who match the search string.

At some point, hopefully for the next release after 2.20 (assuming such takes place) it would be desirable to integrate groups and users maintenance. If you look at a user you should be able to see his/her groups membership. If you look at a group, you should be able to see what members are in it. There should also be a pick list type of functionality where you could scroll thru the user list and drag selected ones to a group (or groups), see the effects and hit submit when you are all done. I think that technically this will need to be done in Javascript so we don't have to keep going back to the server for refreshes every time there is a change.

The limit on rows per page could be lifted pretty easily but I'd rather see us rethink and redesign the whole groups interface. While we are at it we should also develop an alternative admin interface that relies on groups privileges rather stuffing a comma delimited (eeks!) admin field in the modules table and rummaging thru case and links files just to see where to go next within admin (and whether you have sufficient privileges to go there). Within the admin for a given module just set the action on forms to other programs within that admin directory and skip going back thru admin.php every time and checking all the case files and ops for the whole system. It's easy enough to program if (!is_group($blat)) echo die(you dont have privileges).

registered · Site Admin Joined: Jun 04, 2004 Posts: 6437

Admittedly, I haven't read the entire volume of info here, but I will chip in that the current method (pun intended) of determining admin capability - which requires SQL on EVERY module to determine if the user is an admin for that module - is EXTREMELY inefficient. I'd much prefer a method (or, in the absence of a true user / admin user object, a function) that uses a cache to determine one's administrator access.

I realize that's just a drop in the bucket in what you're talking about, but once you have that common method / function in place, it's MUCH easier to make changes, wrap around phpBB2, phpBB3, etc.

Posted: Tue Dec 18, 2007 8:12 pm

We are departing from the thread a bit but I will just add, it is not just the SQL overhead. There are also file system reads of all case and links files every time you go from function to function in admin. So if you are in function X in say downloads and you want to go to function Y, you send an op back to admin.php which goes running thru all the case files for every module until it finds a matching op and then it calls back downloads, which looks at the ops again and finally gets around to calling function Y. Why function X can't just call function Y is beyond me. Or if function X is a form have the action of that form be a program that contains function Y and just check in there that the user has admin privileges by virtue of being a member of an appropriate group. The so-called "path length" and overhead would be so much less.

Posted: Wed Dec 19, 2007 7:58 am

Hhmmmm... caching a serialized object of a user's permissions... hhhmmmmm.... Wink

Posted: Wed Dec 19, 2007 4:28 pm

OOP's