Get Paid Forum - Get Paid Discussion > Before anyone panics...

onlooker

Jun 12 2004, 01:33 PM

Just to let everyone know, SAS is down because the entire RackShack.net server network is down at the moment. I know there are quite a few sites here that are hosted with them.

They host THOUSANDS of websites so I am sure they well fix this issue shortly.

Nothing I can do on my end

Rod

steelie

Jun 12 2004, 01:38 PM

Thanks for the update, Rod!! Hope all is well soon!!

manueluis

Jun 12 2004, 01:46 PM

thanks for the news at least now i know what happened

onlooker

Jun 12 2004, 01:49 PM

To prove this point, goto http://rackshack.net

I am trying to get into their site now...grrrrr

Rod

maninbusiness

Jun 12 2004, 02:01 PM

MandyMooCow

Jun 12 2004, 02:42 PM

Its up again - or looks like it to me!

.....or not

arben

Jun 12 2004, 03:54 PM

QUOTE (MandyMooCow @ Jun 12 2004, 04:42 PM)

Its up again - or looks like it to me!

.....or not

Yep it's back up

freeandeasy

Jun 12 2004, 08:15 PM

From Rat Shack:

QUOTE

Earlier today, we experienced a very substantial power spike that damaged one of our UPs units at DC#1. As a result of this, which was actually multiple high voltage spikes, the charging and power unit in one UPS failed. Upon failure, this UPS went into emergency bypass mode, notified our NOC, and left it on core utility power.

Some minutes later, we were affected by approximately 4 brown outs that caused our PDU units on this UPS to shut down and re-cycle their power to prevent damage to the computers and equipment connected. These units worked as designed by preventing damage to actual servers.

Only a very small number of client servers experienced issues upon hard reboot. These servers are being worked as I write this both by our core DC staff as well as off duty staff that has been called in. We are working to provide response asap to each affected client.

Our UPS technicians arrived on site within 35 minutes along with ALL of our core management team including Randy Williams, Patrick Smith, and Jeff Lowenberg. At the same time, we diverted some call center personnel to the DC and called in additional off duty personnel.

We made a decision to transfer from utility power to generator power for that UPS to ensure that additional brown outs would not affect server operations again. This unfortunately, caused an additional power cycle for a small percentage of our customers servers as well as some of our core servers, including the ones that power this forum.

Each of these outages was less than a few seconds in duration.

The affected part has been identified and a replacement has been pulled from our cold spares on hand. Replacement should be complete within the next few hours. When final testing is complete, we will switch back to primary utility power on that UPS which should be transparent to the customer.

In addition, we will be completing testing on all of our other UPS control systems to ensure that the power spike did not cause any damage to them.

Our personnel are working through all tickets as fast as possible. We appreciate your patience as we work as rapidly as possible to respond to all customer requests.

Robert Marsh

Ocram

Jun 12 2004, 08:58 PM

QUOTE (freeandeasy @ Jun 13 2004, 03:15 AM)

From Rat Shack:

QUOTE
Earlier today, we experienced a very substantial power spike that damaged one of our UPs units at DC#1. As a result of this, which was actually multiple high voltage spikes, the charging and power unit in one UPS failed. Upon failure, this UPS went into emergency bypass mode, notified our NOC, and left it on core utility power.

Some minutes later, we were affected by approximately 4 brown outs that caused our PDU units on this UPS to shut down and re-cycle their power to prevent damage to the computers and equipment connected. These units worked as designed by preventing damage to actual servers.

Only a very small number of client servers experienced issues upon hard reboot. These servers are being worked as I write this both by our core DC staff as well as off duty staff that has been called in. We are working to provide response asap to each affected client.

Our UPS technicians arrived on site within 35 minutes along with ALL of our core management team including Randy Williams, Patrick Smith, and Jeff Lowenberg. At the same time, we diverted some call center personnel to the DC and called in additional off duty personnel.

We made a decision to transfer from utility power to generator power for that UPS to ensure that additional brown outs would not affect server operations again. This unfortunately, caused an additional power cycle for a small percentage of our customers servers as well as some of our core servers, including the ones that power this forum.

Each of these outages was less than a few seconds in duration.

The affected part has been identified and a replacement has been pulled from our cold spares on hand. Replacement should be complete within the next few hours. When final testing is complete, we will switch back to primary utility power on that UPS which should be transparent to the customer.

In addition, we will be completing testing on all of our other UPS control systems to ensure that the power spike did not cause any damage to them.

Our personnel are working through all tickets as fast as possible. We appreciate your patience as we work as rapidly as possible to respond to all customer requests.

Robert Marsh

And the scary thing is that it makes complete sense to me

freeandeasy

Jun 12 2004, 10:09 PM

QUOTE (OcramC @ Jun 12 2004, 09:58 PM)

QUOTE (freeandeasy @ Jun 13 2004, 03:15 AM)

From Rat Shack:

QUOTE
Earlier today, we experienced a very substantial power spike that damaged one of our UPs units at DC#1. As a result of this, which was actually multiple high voltage spikes, the charging and power unit in one UPS failed. Upon failure, this UPS went into emergency bypass mode, notified our NOC, and left it on core utility power.

Some minutes later, we were affected by approximately 4 brown outs that caused our PDU units on this UPS to shut down and re-cycle their power to prevent damage to the computers and equipment connected. These units worked as designed by preventing damage to actual servers.

Only a very small number of client servers experienced issues upon hard reboot. These servers are being worked as I write this both by our core DC staff as well as off duty staff that has been called in. We are working to provide response asap to each affected client.

Our UPS technicians arrived on site within 35 minutes along with ALL of our core management team including Randy Williams, Patrick Smith, and Jeff Lowenberg. At the same time, we diverted some call center personnel to the DC and called in additional off duty personnel.

We made a decision to transfer from utility power to generator power for that UPS to ensure that additional brown outs would not affect server operations again. This unfortunately, caused an additional power cycle for a small percentage of our customers servers as well as some of our core servers, including the ones that power this forum.

Each of these outages was less than a few seconds in duration.

The affected part has been identified and a replacement has been pulled from our cold spares on hand. Replacement should be complete within the next few hours. When final testing is complete, we will switch back to primary utility power on that UPS which should be transparent to the customer.

In addition, we will be completing testing on all of our other UPS control systems to ensure that the power spike did not cause any damage to them.

Our personnel are working through all tickets as fast as possible. We appreciate your patience as we work as rapidly as possible to respond to all customer requests.

Robert Marsh

And the scary thing is that it makes complete sense to me