General overhaul of the backup system.

[admin (no pm's please)]

Speed, with a files data striped across multiple drives RAID systems depending on the RAID level can be very fast.

---

_________________

Family Friendly Shareware | | Web Design/Services | Free Forums

[panther_dust]

it is an a way a backup as there is one that mirrors the 1st array. that i class as a backup..

if anything goes wrong with the first array then you have it how it was on the seconed
_________________
PD, Pantherdust, Benjamin, BMDProductions

[admin (no pm's please)]

Let's not play semantics here. Of course it is a copy of data and you can choose to call that a backup if you want. Many people have chosen to view things that way.....





































and regretted it!
_________________

Family Friendly Shareware | | Web Design/Services | Free Forums

[query]

[admin (no pm's please)]

The database backups continue to complete reliably under the new system. In fact the main culprit came in more than 2.5 hours ahead of its worst completion time

Hopefully with more database pruning that can improve further, another two hours quicker and the backup would be completed at 0815 in the British morning taking will well into the quiet hours.
_________________

Family Friendly Shareware | | Web Design/Services | Free Forums

[admin (no pm's please)]

Trimmed another 14% off the slarti database size, taking us down to 9.41am as the completion time.

I can see room for a bit more trimming today, but much more would require a tougher purge strategy.

I have half an idea to place inactive forums into a forum of "purgatory" where they would be simply and safely removed from the database. If someone actually does access the forum the files would be pretty much instantly restored.

Databases tables in such a place would be saved say once a week.
_________________

Family Friendly Shareware | | Web Design/Services | Free Forums

[admin (no pm's please)]

All system weekly backups have completed on the new system. One a mere hour and 20 minutes ahead of last week, one 5 hours ahead and one more than 10 hours ahead of last week

The system where there was only an hour or so gain, is the one with a much smaller backup size and the system where performance is the least issue, so that figure is to be expected.
_________________

Family Friendly Shareware | | Web Design/Services | Free Forums

[admin (no pm's please)]

Bumping this again. The thread here was overtaken by events as we had to get a new backup server and that has required a fair bit of work improving the general backup infrastructure.

But now is the time to get back to deal with the way backups can effect forums performance, and this means getting back onto the idea of removing unvisited forums from the backup systems by moving them out of the database directory structures, but making is so that when someone accesses the forum they are snapped back in seamlessly.
_________________

Family Friendly Shareware | | Web Design/Services | Free Forums

[admin (no pm's please)]

In one sense this is mostly done, the system can move a forum to "purgatory" and if it is accessed it snaps back with no perceivable delay on the office systems

I do need to do some work though on calling forums back from purgatory to be purged and last but not least whilst the system works in the office security is a lot tighter on the actual servers and will need tweaking to allow access to the code to return a forum from purgatory.

It all takes a little thought, for example I was thinking I'd need to notify the system that a forum had return from purgatory, but is it really needed? actually probably it is.
_________________

Family Friendly Shareware | | Web Design/Services | Free Forums

[admin (no pm's please)]

or not.

Trouble with a notification is that we run a client server system. Your forum may be a myfreeforum.org, but might well reside on the myfastforum.org server, so a notification would for a myfreeforum forum would go from the server the forum is on, to the server for the forum domain and then back to the server the forum is on, this both introduces a dependency and does little for the process being seamlessly quick.

Instead the forum now ask themselves, was I flagged as in purgatory? If so then change those bits in my configuration that deal with that to remove the flags and the return from purgatory code.
_________________

Family Friendly Shareware | | Web Design/Services | Free Forums

[admin (no pm's please)]

The new code is now installed, every night we run a script that processes all the forums, tonights should start placing forums that no one has accessed in 100 days into "purgatory", we will see the effect of this in the next nights backup.

The one flaw in the system is that the backup system is currently geared to only keep backups for 3 months, thus forums in purgatory would be ultimately vulnerable. The way round this will be to create another backup process to save the purgatory file system once a month.
_________________

Family Friendly Shareware | | Web Design/Services | Free Forums

[admin (no pm's please)]

The first night sees 25% in purgatory at the start of the day.

I will measure again later and see what the "churn" factor is, even a bot visit will take a forum out of purgatory.

25% is a quite solid number though, and one that we should be able to increase.
_________________

Family Friendly Shareware | | Web Design/Services | Free Forums

[Nascarfan13]

http://snw.myfreeforum.org/index.php


Could this cause my forum to not be able to be accessed? I could get to my forum about 15 minutes ago, now all of a sudden I can't even get to the index page, and then I checked my modem to see if my internet was out and its not, so something is causing my forum to be inaccessible.  And I know its not A rouge admin because right before it went down, there were no members on online at all.

[LFF]

http://lancasteroh.myfreeforum.org/
We are down also. Support is down too. I fought my way in to leave a message that MFF is down. orcikat@aol.com

[namecaps]

 When I try to access my free forum I get this message    

phpBB : Critical Error

Could not connect to the database

my forum is

http://aimclan.myfreeforum.org/

my email is    spaceman@bigpond.net.au
could you please advise me if you know what problem is
thanks

---