FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups  Who is OnlineWho is Online   Join! (free) Join! (free)  
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 
  • Welcome
  • Guest

Faster forums part two
Page 1, 2  Next
 
Post new topic   Reply to topic    myfreeforum.org Forum Index -> Forum Blogs
View previous topic :: View next topic  
Author Message
Please Register and Login to this forum to stop seeing this advertising.






Add Karma

rated by members
Add Comment
Show Comments


online/offline
Posted:     Post subject:

Back to top
myff admin
Site Admin
Site Admin
FLAG

Virtual Cash: 92920

Joined: 22 May 2005
Posts: 32206



Add Karma

rated by 281 members
Add Comment
Show Comments


online/offline
PostPosted: Sat Jul 25, 2009 9:22 am    Post subject: Faster forums part two  Reply with quote

The code does seem just about ready for this, the only question is rolling it out.

Probably it will start Tuesday with the new code to deal with template editing and copying of templates to the template host, combined with a neutered version of the server outage detection system that will not integrate with forums code at all, the office version of the code is seeing the odd few seconds of outage every several hours, and so it trips the system and resets it a minute later. This of course is what it is meant to do, but is it leaping too quickly and how will it behave on a live server?

We need to run for a few days and see what happens.
_________________

Family Friendly Shareware | | Web Design/Services | Free Forums
Back to top
View user's profile Send private message Visit poster's website
myff admin
Site Admin
Site Admin
FLAG

Virtual Cash: 92920

Joined: 22 May 2005
Posts: 32206



Add Karma

rated by 281 members
Add Comment
Show Comments


online/offline
PostPosted: Mon Jul 27, 2009 4:48 pm    Post subject: Reply with quote

I have started the template copying process on one server to see how it goes, did some last minute recoding as where forums have moved servers they may be old copies of the template lying around and that could have lead to old copies overwriting new edits on the template server

copies are progressing at about one per minute, so this will not be a fast transition.
_________________

Family Friendly Shareware | | Web Design/Services | Free Forums
Back to top
View user's profile Send private message Visit poster's website
myff admin
Site Admin
Site Admin
FLAG

Virtual Cash: 92920

Joined: 22 May 2005
Posts: 32206



Add Karma

rated by 281 members
Add Comment
Show Comments


online/offline
PostPosted: Mon Jul 27, 2009 9:12 pm    Post subject: Reply with quote

Still under 200 done, and this really does need pacing. Not much point doing all three servers and then finding out there is an issue.
_________________

Family Friendly Shareware | | Web Design/Services | Free Forums
Back to top
View user's profile Send private message Visit poster's website
myff admin
Site Admin
Site Admin
FLAG

Virtual Cash: 92920

Joined: 22 May 2005
Posts: 32206



Add Karma

rated by 281 members
Add Comment
Show Comments


online/offline
PostPosted: Tue Jul 28, 2009 6:44 am    Post subject: Reply with quote

Found a small issue just now when I started the integration, and it will be a few more days before the support forums own template is old enough to go live with the system. Add that to the fact that only 523 template from one server are saved as yet and you end up with the conclusion that we may as well be patient. It will get there.
_________________

Family Friendly Shareware | | Web Design/Services | Free Forums
Back to top
View user's profile Send private message Visit poster's website
myff admin
Site Admin
Site Admin
FLAG

Virtual Cash: 92920

Joined: 22 May 2005
Posts: 32206



Add Karma

rated by 281 members
Add Comment
Show Comments


online/offline
PostPosted: Tue Jul 28, 2009 9:32 am    Post subject: Reply with quote

Trouble is that at this rate it may take more than a week to get there There are more than 10,000 templates on each server and after about 18 hours of processing we have only copied 624 templates.

In the long term that is not an issue, we can as a low priority process clearly save 400 templates a night, which is well in excess of the number likely to need copying.
_________________

Family Friendly Shareware | | Web Design/Services | Free Forums
Back to top
View user's profile Send private message Visit poster's website
myff admin
Site Admin
Site Admin
FLAG

Virtual Cash: 92920

Joined: 22 May 2005
Posts: 32206



Add Karma

rated by 281 members
Add Comment
Show Comments


online/offline
PostPosted: Tue Jul 28, 2009 3:20 pm    Post subject: Reply with quote

1052...

I'm now running two processes one to copy the system templates. One to copy the user templates.

I'll code something up for the other servers to just copy the user templates, whilst marking system templates as already copied.
_________________

Family Friendly Shareware | | Web Design/Services | Free Forums
Back to top
View user's profile Send private message Visit poster's website
myff admin
Site Admin
Site Admin
FLAG

Virtual Cash: 92920

Joined: 22 May 2005
Posts: 32206



Add Karma

rated by 281 members
Add Comment
Show Comments


online/offline
PostPosted: Wed Jul 29, 2009 8:26 am    Post subject: Reply with quote

All system templates were copied.

I am now running the copy user template process on all servers, the code is also there on all servers, so very slowly all forums should be pulled into the system.
_________________

Family Friendly Shareware | | Web Design/Services | Free Forums
Back to top
View user's profile Send private message Visit poster's website
myff admin
Site Admin
Site Admin
FLAG

Virtual Cash: 92920

Joined: 22 May 2005
Posts: 32206



Add Karma

rated by 281 members
Add Comment
Show Comments


online/offline
PostPosted: Wed Jul 29, 2009 10:54 am    Post subject: Reply with quote

2147

I guess templates are copying at about 3 a minute. This may be a glacial pace, but when you think about it long term, are 3 templates a minute reaching a point where no one has edited them for 4 days? More like 3 templates an hour I would guess.
_________________

Family Friendly Shareware | | Web Design/Services | Free Forums
Back to top
View user's profile Send private message Visit poster's website
myff admin
Site Admin
Site Admin
FLAG

Virtual Cash: 92920

Joined: 22 May 2005
Posts: 32206



Add Karma

rated by 281 members
Add Comment
Show Comments


online/offline
PostPosted: Wed Jul 29, 2009 5:42 pm    Post subject: Reply with quote

2850...


The length of this process is certainly going to put a halt to phpbb2 work for a bit, maybe that means it is time to get to grips with a phpbb3 biggy like the profile fields.
_________________

Family Friendly Shareware | | Web Design/Services | Free Forums
Back to top
View user's profile Send private message Visit poster's website
myff admin
Site Admin
Site Admin
FLAG

Virtual Cash: 92920

Joined: 22 May 2005
Posts: 32206



Add Karma

rated by 281 members
Add Comment
Show Comments


online/offline
PostPosted: Thu Jul 30, 2009 9:06 pm    Post subject: Reply with quote

5469...

Not a lot else to say really, but at least it is setting aside time to look at other issues.

The phpbb3 profile fields has got to wait, there are no good solutions out there yet that I can see, and it is not time for our own bespoke solutions.

Instead todays work on optimising phpbb2 databases has been very productive, and hopefully will be able to go forum wide soon.
_________________

Family Friendly Shareware | | Web Design/Services | Free Forums
Back to top
View user's profile Send private message Visit poster's website
Ask Mr. Religion
Craftsman
Craftsman
FLAG

Virtual Cash: 4500

Joined: 01 Aug 2008
Posts: 423



Add Karma

rated by 3 members
Add Comment
Show Comments


online/offline
PostPosted: Fri Jul 31, 2009 8:32 am    Post subject: Reply with quote

Once everything is copied I hope you have some sort of diff copy scheme that will only pick up templates that were 'touched' so you don't have to go through this again. (I think I am understanding what is going on...then again...

AMR
_________________
Ask Mr. Religion (a.k.a. 'Patrick')
Founder, Reformed Theology Institute
Back to top
View user's profile Send private message
myff admin
Site Admin
Site Admin
FLAG

Virtual Cash: 92920

Joined: 22 May 2005
Posts: 32206



Add Karma

rated by 281 members
Add Comment
Show Comments


online/offline
PostPosted: Fri Jul 31, 2009 8:48 am    Post subject: Reply with quote

The system is simple, when anyone even enters the template editor for a template, then the template is marked as needing to be loaded locally and to be copied again. This is a little brutal but how many people seriously edit templates that much?
_________________

Family Friendly Shareware | | Web Design/Services | Free Forums
Back to top
View user's profile Send private message Visit poster's website
myff admin
Site Admin
Site Admin
FLAG

Virtual Cash: 92920

Joined: 22 May 2005
Posts: 32206



Add Karma

rated by 281 members
Add Comment
Show Comments


online/offline
PostPosted: Fri Jul 31, 2009 10:36 am    Post subject: Reply with quote

6612..

As a matter of fact on this issue, I am running it as a manual process on each server from an ssh terminal. So when something glitched this morning and those terminals dropped out, so did the processes

When I kicked the processes off again, they took only a little while to process templates that had been previously deal with, and to deal with ones that were newly needing copying before they have carried on plodding away through the rest of the templates.
_________________

Family Friendly Shareware | | Web Design/Services | Free Forums
Back to top
View user's profile Send private message Visit poster's website
SRSeditor
Craftsman
Craftsman
FLAG

Virtual Cash: 3420

Joined: 14 Apr 2008
Posts: 331


Location: UK
Add Karma

rated by 1 members
Add Comment
Show Comments


online/offline
PostPosted: Fri Jul 31, 2009 11:17 am    Post subject: Reply with quote

admin (no pm's please) wrote:
This is a little brutal but how many people seriously edit templates that much?


I would say I make changes to templates at least once a month on average. Sometimes more often, sometimes less so.
_________________
http://audisrs.myfastforum.org
Back to top
View user's profile Send private message Visit poster's website
myff admin
Site Admin
Site Admin
FLAG

Virtual Cash: 92920

Joined: 22 May 2005
Posts: 32206



Add Karma

rated by 281 members
Add Comment
Show Comments


online/offline
PostPosted: Fri Jul 31, 2009 11:46 am    Post subject: Reply with quote

So with the system your template would be offsited 85% of the time of thereabouts.

In practical terms I doubt people with the templates onsite will see a visible speed penalty, You go from using 4 servers to using 3 and the server you are falling back on is more lightly loaded than it was, and since the offsite templates are served from "shared" hosting it is questionable if they are that fast anyway! The reason they will appear fast is because of the greater parallelism involved.


_________________

Family Friendly Shareware | | Web Design/Services | Free Forums
Back to top
View user's profile Send private message Visit poster's website
Display posts from previous:   
Post new topic   Reply to topic    myfreeforum.org Forum Index -> Forum Blogs All times are GMT
Page 1, 2  Next
Page 1 of 2
 
 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum

Card File  Gallery  Forum Archive
Powered by phpBB © 2001, 2005 phpBB Group
Create your own free forum | Buy a domain to use with your forum