FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups  Who is OnlineWho is Online   Join! (free) Join! (free)  
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 
  • Welcome
  • Guest

Page 1, 2, 3, 4, 5  Next
 
Post new topic       myfreeforum.org Forum Index -> Forum Blogs
Site Admin
Site Admin



View user's profile Send private message Visit poster's website

A Blog by - myff admin


PostPosted: Sat Jan 30, 2010 9:47 pm

Slower forums part 1


A title that plays on our very successful forum optimisations over the summer of last year, optimisations that were followed by adding a lot more server power to the mix, to the point that I think most people agree that (when advertising is not slowing things) out forums are just about the fastest you will find anywhere

So why the "slower forums" title?

Essentially as whilst speed is obviously important, it is not the only word in the dictionary.

1) Reliability
2) Redundancy
3) Ease of maintenance
4) Cost
5) Scalability
6) Flexibility

Are all good words as well and may come with a speed penalty.

In particular I'm looking at our storage, currently the servers have their own RAID 10 or RAID 5 arrays. These give screamingly fast file performance, but don't tick many of the other boxes  

Adding NAS storage to our servers would tick the other boxes, but what would happen to speed? I think it is 100% certain it would slow things a bit, but I think anyone reasonable would have to agree that at current forums speeds we can afford to lose a notch or three.

I'm therefore currently contemplating adding something like a 1.5tb NAS system, the cost will be over 2000 and add a good 100 a month to running costs, but as I see it, if it flops on the forums front it could still basically replace the current backup and gallery server and so get us back to the same running costs, and we would then be able to let people upload far more and also run even more comprehensive backups.

2 comments have been made on this topic View/Post Comments
PostPosted: Fri Jan 29, 2010 8:24 am

Grouch


DNS servers are on my mind lately as there are a few issues to clear up our end.

So are server moves as they seem to be all I have been doing more months  

Every set of server moves leads to a load of questions as to what is "eddy/marvin/trillian" etc doing in my URL?

Now personally I think the given explanation is quite clear, but this is not a grouch about people understanding that.

It is a grouch about why people see it in the first place? now they have to see it briefly if they are using a forum continually as when the forum moves, its new ip address takes time to propagate. So we set the new IP, but place a redirect at the old IP going to the preset "eddy" (or whatever) addresses that already propagated for the new server.

We also when we are doing server moves, set a 2 hour TTL (time to live) for IP addresses for the domain. Now that's the grouch most of the worlds ISPs seem to ignore that totally. The external nameserver I use here tells me our TTL is in fact 12 days   This false TTL then gets honored by our internal nameserver and we get stuck with the address unless we reset the thing. Meanwhile the external nameserver does seem to actually update, not in two hours as we ask for, but certainly not in 12 days.

We'd all have a lot less problems if ISPs honored basic internet protocols.

http://ask.slashdot.org/article.pl?sid=05/04/18/198259

0 comments have been made on this topic View/Post Comments
PostPosted: Sat Sep 12, 2009 11:24 am

Faster forums part eight (php)


This is not something we can do a lot about until we do go down the new server road, which will allow us to run code exclusively tailored to the forums alone.

But when I was benchmarking forums looking to improve upon database access times, it was quite apparent that when a forum gets displayed with the database doing well, e.g. queries coming from cache etc, then the bulk of the time taken in showing a forum is in the php code.

php really is awfully slow, of course 2 or 3 hundredths of a second spend in php does not seem slow to the user, but in raw computing terms that time should be a small fraction of that.

Even decoding our new cookie system is taking thousandths of a second. If I coded it in 'c' it would take more like a millionth!

It will be interesting to look at:

http://www.roadsend.com/home/index.php?pageID=compiler

to try and deal with this.

2 comments have been made on this topic View/Post Comments
PostPosted: Thu Sep 10, 2009 10:20 am

Google maps portal


Just a thought, something so that you can let your member mark themselves on a map.

It should be easy enough, though I imagine you use it you will have to register with google, as these things need an APIKEY and if we use the same one for all the forums we will blow any limits sky high.

Any thoughts? examples? expertise?

3 comments have been made on this topic View/Post Comments
PostPosted: Wed Sep 09, 2009 3:31 pm

A principle to abandon or not?


This probably won't arise regardless, as I have no intention of deliberate vandalism, but one principle we have struck to is that we keep our phpbb2 forum databases compatible with standard phpbb2. That they if someone wants off the system they can get an export and face no unusual difficulty.

The last week or so has seen (or will see when released) some quite significant changes and bug fixes to phpbb2 as we delve into bugs that must be present on hundreds of thousands of phpbb2 forums around the world   They were never addressed by the phpbb2 team and are in areas where I have been previously reluctant to tread, reasoning that if the core team failed to fully sort the issues in 8(?) years, then I had no business being arrogant enough to dive in.

Having started though, it has been easier than I thought, and that raises the question as to what other areas in phpbb2 might be worth doing better and what to do if that means abandoning compatibility  

2 comments have been made on this topic View/Post Comments
PostPosted: Sun Aug 30, 2009 5:10 pm

Faster forums part seven (memcache)


I think I am going to have a look at:

http://uk.php.net/manual/en/book.memcache.php

It may be ideal for caching portal options. These are currently retrieved in a fashion that can only be call painful, and as the portal system grows something cleaner and quicker is needed.

0 comments have been made on this topic View/Post Comments
PostPosted: Sat Aug 29, 2009 11:37 am

faster forums part six (forum errors)


Each time a forum has an error in it, perhaps in a template or perhaps some code that does not anticipate that it may legitimately get an error response and try and deal with it, we get log entries generated It's hardly a big server hit but it can mount up.
So I have been spending time tweaking peoples templates where I see errors in the logs. This is making the logs a lot smaller, some templates have been generating 3 log entries each time someone views any page on a forum

1 comments have been made on this topic View/Post Comments
PostPosted: Sat Aug 22, 2009 12:13 pm

Faster forums part five


This one requires a little archeology.  We are running non standard code to clear out old search results, and I can't quite fathom why at the minute.

But the principle here is that every search on a phpbb2 forum does the work of clearing down old results. This in entirely unnecessary.  Sure it needs doing, but not on every search.

I have implemented this on downsizer (which runs the standard code) and "View Latest Posts" is noticeably nippier

3 comments have been made on this topic View/Post Comments
PostPosted: Thu Aug 20, 2009 8:05 am

phpbb3 seo code


Let's start with the standard point. A forum with little content or content that is not keyword rich, or a forum which does not have links to it from other sites will NEVER do well with search engines. Nothing we can do will change that.

But we can and should start lending a helping hand by optimising site URLs. Having reviewed the solutions available I am not impressed, they make far too many changes to the code, some times even needing template edits, and all too frequently giving people a shed load of admin panel options.

This is definitely a case where options distract from the fact the core business of SEO is about the content.

What I plan to start doing is to take an approach which is compatible with out phpbb2 SEO, this will have the added advantage that old links known by search engines will continue to work if a phpbb3 upgrade is done.

In my view this approach will quietly get 90% of the possible SEO mileage, and as it will do it without fuss, it will actually be more effective that more complex solutions that will focus people on the tech and not on the real job at hand.

0 comments have been made on this topic View/Post Comments
PostPosted: Mon Aug 17, 2009 12:58 pm

Faster forums part four


The next move on phpbb2 forums is to implement forum search using mysql full text search. This would replace the native phpbb2 search system with the one now available in the database system.

This has the benefit of being far far faster both when searching and when a post is posted/edited. When a post is entered to the database every word has to be laboriously processed into an index system, on large forums that process is bound to be slow.

There are downsides to this, namely you will get a slightly different set of search results when searching a forum. Basically all searches on text are subject to somewhat arcane decisions made by the search designers, rules that are needed both for speed and for keeping database sizes to a sane level. phpbb2 has its own ideas on this, mysql has another set of ideas.

4 comments have been made on this topic View/Post Comments
PostPosted: Sun Aug 09, 2009 8:03 pm

Faster forums part three (slow queries)


Another review on slow queries on the forums shows up a very frequent culprit.

This is the bit that shows who is online. There are two notable issues there, one is that phpbb2 lacks an index on session_time and the other is that this is a query that is being made perhaps many times a minute whilst we don't really need to see the information changing more than once a minute at most.

The lack of an index whilst suspect is not necessarily wrong, as it is the sort of query where mysql may decide to ignore the index anyway. But we can and should cache the results and not repeat the query incessantly.

7 comments have been made on this topic View/Post Comments
PostPosted: Mon Aug 03, 2009 7:16 pm

Wiki forums


Following on from the thread on guides.

It would be rather good to have forums operating as Wikis.

I don't see this as a terribly difficult job.

If we make it so that the Edit permission on a Wiki forum controls who can edit not just their own posts, but the first post in a thread regardless of who posted it, then that deals with Wiki type permissions without too much aggravation.

We let people put a button in the template that would access a wiki editing system.

When an edit it stored we change the first post of the thread to the new version and store a "diff".

There would be little difference in the phpbb2 and phpbb3 code, as all that would differ is saving the posts.

38 comments have been made on this topic View/Post Comments
PostPosted: Fri Jul 31, 2009 7:22 pm

Speeding up large forums


I am planning some database experiments with busy forums, starting of course with the busiest of them all downsizer.net.

I have been logging database queries and seeing some long query times These are not really explainable by lack of server power, but are fairly explainable by the fact that with the MyISAM table types used on all the forums, a write to a table locks the entire table, so if as you might imagine on a busy forum topic views are accumulating at a ferocious rate, then the topics table may be pretty much under constant lock, queuing up loads of requests both to read and write it. Almost no amount of hardware may compensate for this.
There is another table type that can be used  InnoDB tables that only do row level blocking on a write. Of course these types of tables are not without their downsides as well, so it will be interesting to see the results.

2 comments have been made on this topic View/Post Comments
PostPosted: Sat Jul 25, 2009 9:22 am

Faster forums part two


The code does seem just about ready for this, the only question is rolling it out.

Probably it will start Tuesday with the new code to deal with template editing and copying of templates to the template host, combined with a neutered version of the server outage detection system that will not integrate with forums code at all, the office version of the code is seeing the odd few seconds of outage every several hours, and so it trips the system and resets it a minute later. This of course is what it is meant to do, but is it leaping too quickly and how will it behave on a live server?

We need to run for a few days and see what happens.

24 comments have been made on this topic View/Post Comments
PostPosted: Thu Jul 23, 2009 10:44 am

faster forums


I think I have perceived some speed increases on the forums in the last week as we have (on phpbb2 for now) moved smilies/ranks/gallery avatars off on to there own servers. This allows browsers to get more parallel requests in at a time and at the same time decreases load on the main boxes.

What I think is more exciting is the code is also ready now, to do much the same thing with template images.

Nightly a script will look at the templates and if you are not in the process of editing a template and a copy has not already been made, then the template will be copied (creating a backup) to yet another server, the template will then be marked a backed up and that its images should be served from the template server.

If the admin ever enters the template editor then the backup and redirect marks will be deleted. e.g. if you are developing a template then things will be absolutely the same as normal. Once you are finished then your forum will get the boost of an additional server.

Of course we won't quite get a 400% speed increase, but if a forum is now using 4 servers to deliver its content simultaneously, and the load is also less on the main box, then there should be quite a notable boost.

The one main thing left to do, is that whilst we can live with smilies or ranks or avatars being inaccessible with a server issue, we cannot live with template images not working.

If we do rely on 4 servers each with 99.9% uptime, then overall uptime becomes 99.6% to avoid this a polling system needs to be written that will check the 3 image servers every minute or two and stop the forums using them if they are down. Since all images will still be on the main servers this should be quite transparent.

A small but useful side effect of the new system is that templates will be getting nightly backups and can be removed from the main system backup.

This has one drawback in that if a rogue admin messes up a template, then there will be a more limited window for restoration But we cannot run the system around the need to deal with rogue admins.

0 comments have been made on this topic View/Post Comments
Post new topic       myfreeforum.org Forum Index -> Forum Blogs All times are GMT
Page 1, 2, 3, 4, 5  Next
Page 1 of 5
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum

Card File  Gallery  Forum Archive
Powered by phpBB © 2001, 2005 phpBB Group
Create your own free forum | Buy a domain to use with your forum