IPB

Welcome Guest ( Log In | Register )

 
Reply to this topicStart new topic
> Maintenance down time, Never touch a running system?
schnoog
post Nov 2 2017, 09:37 PM
Post #1


Group Icon Master of Desaster

Group: Management

Joined: 7-March 06
From: Wehr
Member No.: 23716



Hi folks,

6 and a half years ago I was asked to become admin of the Splatterladder.
During the last 6 years we had a total downtime of 51 hours, caused by several defects like broken hard disks (in total 9 were destroyed yet), a broken power supply or a runaway update.
A lot of work was necessary to patch an outdated operating system to keep it in a working and safe state.
But now the time has come to upgrade the operating system to ensure safety and continuity without compiling everything from source.

The SL codebase was written a long time ago, and I'm not able to guess how long it will take to bring all services back to life.
I'll do my very best to minimize the interruption(s).


Greetz
Schnoog







--------------------



Go to the top of the page
 
+Quote Post
macbeth
post Nov 3 2017, 01:31 AM
Post #2


Group Icon Captain

Group: Members

Joined: 12-February 11
From: France
Member No.: 97334



Good luch Schnoogy
we all know u do and will your best for us


--------------------
Go to the top of the page
 
+Quote Post
{WeB}*DoG
post Nov 3 2017, 01:48 AM
Post #3


Group Icon Second Lieutenant

Group: Management

Joined: 18-August 05
From: Wellingborough UK
Member No.: 13132



Superb News Schnoog.... we look forward to the improvements and we will try our best to help in any useful way we can!!


--------------------




LONG LIVE ET ! LONG LIVE SL ! LONG LIVE {WeB}!!!
Go to the top of the page
 
+Quote Post
Wolf Enstein
post Nov 3 2017, 01:50 AM
Post #4


Group Icon General

Group: Management

Joined: 5-August 05
From: Scotland
Member No.: 12398



QUOTE (macbeth @ Nov 3 2017, 02:31 AM) *
Good luch Schnoogy
we all know u do and will your best for us





Thanks for all your efforts, Schnoogmei$ter cheers.gif .


--------------------


Go to the top of the page
 
+Quote Post
$mart
post Nov 3 2017, 09:38 AM
Post #5


Group Icon Major

Group: Members

Joined: 19-September 07
From: South of France xDDDDD
Member No.: 59683



51h during 6years means a SLA of 99,9 % !!!

Impressive !!! cheers.gif


--------------------
Go to the top of the page
 
+Quote Post
schnoog
post Nov 3 2017, 12:53 PM
Post #6


Group Icon Master of Desaster

Group: Management

Joined: 7-March 06
From: Wehr
Member No.: 23716



Thank you all for the flowers :)

But I need to compliment our hoster. Beside the fact that round about 4k Euro flow to them during this time (approx. 1k through donations), the support was always the best.
Never had to wait to exchange any hardware for longer than 4 hours.

If you like some statistics:
SL has moved 3times during the last 6 years. (Hetzner -> Hetzner -> Server4You)
During those 6 years I was connected for 613 hours via SSH to the server, today the 2300th database backup was created (today: 28GB database backup).
Our backup-drive now has a capacity of 3TB which is filled up to 90%




--------------------



Go to the top of the page
 
+Quote Post
Wolf Enstein
post Nov 3 2017, 05:47 PM
Post #7


Group Icon General

Group: Management

Joined: 5-August 05
From: Scotland
Member No.: 12398



QUOTE (schnoog @ Nov 3 2017, 01:53 PM) *
.....today the 2300th database backup was created (today: 28GB database backup).
Our backup-drive now has a capacity of 3TB which is filled up to 90%



Good grief schockiert.gif .


--------------------


Go to the top of the page
 
+Quote Post
SL.Baldi
post Nov 3 2017, 05:59 PM
Post #8


Group Icon Master Sergeant

Group: Management

Joined: 21-April 08
From: Deutschland - Germany
Member No.: 70785



ZITAT(Wolf Enstein @ Nov 3 2017, 05:47 PM) *
ZITAT(schnoog @ Nov 3 2017, 01:53 PM) *
.....today the 2300th database backup was created (today: 28GB database backup).
Our backup-drive now has a capacity of 3TB which is filled up to 90%



Good grief schockiert.gif .




cheers.gif cool grin.gif


--------------------
#########################################################



>>>>>>>>> ADD SL.'Baldi TO YOUR BUDDYLIST<<<<<<<<<<<
Go to the top of the page
 
+Quote Post
schnoog
post Nov 4 2017, 11:36 PM
Post #9


Group Icon Master of Desaster

Group: Management

Joined: 7-March 06
From: Wehr
Member No.: 23716



Hi folks,

.....several hours of frustration and crying
.....one complete rollback
.....another hour of pure fun
.....
...
..
.
but now the main work is done.
The server SL is running on is up to date, the operating system upgraded (tracking out time 1 hour)
Now the real debugging begins:
The new php version misses some functions used at SL.
Since I need to find all calls of those functions, this will be a "longer-term" work.

So if you find any page responding with an Error 500, please drop a note here in this thread



--------------------



Go to the top of the page
 
+Quote Post
Wolf Enstein
post Nov 5 2017, 08:53 PM
Post #10


Group Icon General

Group: Management

Joined: 5-August 05
From: Scotland
Member No.: 12398



QUOTE (schnoog @ Nov 5 2017, 12:36 AM) *
.....several hours of frustration and crying.....but now the main work is done.




respect.gif cheers.gif


--------------------


Go to the top of the page
 
+Quote Post

Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 18th November 2017 - 04:51 PM