Learn about Centmin Mod LEMP Stack today
Become a Member

Linode Datacenter Outage - Fremont

Discussion in 'VPS Provider Network Status' started by eva2000, May 30, 2015.

  1. eva2000

    eva2000 Administrator Staff Member

    55,155
    12,249
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,825
    Local Time:
    4:32 PM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    May 30, 05:08 UTC
    Update - Our upstream provider has provided a new estimated time to restoration of 11:30PM PDT.

    May 30, 04:37 UTC
    Update - Our upstream provider has provided a new estimated time to restoration of 10:00PM PDT.

    May 30, 03:21 UTC
    Update - At approximately 6:30PM PDT, the Fremont datacenter experienced a power utility outage. One out of eight generators also experienced an electromechanical failure. The estimated time to restoration is currently 9:30PM PDT. Linode has all hands on deck to get Linodes online as soon as power is restored.


    May 30, 02:25 UTC
    Update - We have received word from our colocation provider that there has been a power event in a section of the Fremont datacenter. The affected space is where a critical part of the datacenter network is located. The datacenter's electric provider is working with staff to restore power as soon as possible. Please watch this status page for further updates.

    May 30, 01:47 UTC
    Identified - We are aware of an issue within our Fremont datacenter and are investigating at this time. We will provide additional information as it becomes available.

    Continue reading...
     
  2. eva2000

    eva2000 Administrator Staff Member

    55,155
    12,249
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,825
    Local Time:
    4:32 PM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    very nasty outage that affected this forum :(

    So much so decided to setup a DNS based 503 maintenance failover back up for the forums so in future if such outages occur, it will failover at DNS level to an off host secondary VPS server setup for community.centminmod.com for SSL https which is permanently in 503 status maintenance mode. So from end user's perspective hitting forum domain when Linode datacenter is down will instead of broken page unavailable browser message, it will direct to secondary VPS and show a 503 maintenance message for visitors :)

    The secondary VPS setup 503 maintenance message

    cm_siteunavailable_htmlindex.png

    from visitor perspective just 503 unavailable maintenance message on secondary VPS

    cm_siteunavailable_ga_realtime.png
     
    Last edited: May 30, 2015
  3. rdan

    rdan Well-Known Member

    5,449
    1,410
    113
    May 25, 2014
    Ratings:
    +2,204
    Local Time:
    2:32 PM
    Mainline
    10.2
    Yeah, some of the site I managed badly affected on this :/
    Almost 8 hours of downtime I think.