Linode All Linode Manager passwords have been expired!

rdan · Jan 6, 2016

I just posted this screenshot because I can't browse the blog itself :/.

Does Linode database recently hacked? !!!

ModeltogTossen · Jan 6, 2016

Perhaps they have found out something - and as extra step to make it more secure - they ask us to change password - or maybe they don't know what happens but want us to change?

rdan · Jan 6, 2016

Related Feed: Linode - Security Notification and Linode Manager Password Reset | Centmin Mod Community

ModeltogTossen · Jan 6, 2016

Yeah - I just read it..

eva2000 · Jan 6, 2016

yikes thanks for heads up !

eva2000 · Jan 6, 2016

looks like they took down Linode Manager ?

This webpage is not available
The connection to manager.linode.com was interrupted.
Click to expand...

eva2000 · Jan 6, 2016

And Linode Status - Security Notification and Linode Manager Password Reset

In progress
Security Notification and Linode Manager Password Reset
Posted about 5 hours ago. Jan 05, 2016 - 19:44 UTC

Scheduled
Effective immediately, Linode Manager passwords have been expired. You will be prompted to set a new password on your next login. We regret this inconvenience, however this is a necessary precaution.

A security investigation into the unauthorized login of three accounts has led us to the discovery of two Linode.com user credentials on an external machine. This implies user credentials could have been read from our database, either offline or on, at some point. The user table contains usernames, email addresses, securely hashed passwords and encrypted two-factor seeds. The resetting of your password will invalidate the old credentials.

This may have contributed to the unauthorized access of the three Linode customer accounts mentioned above, which were logged into via manager.linode.com. The affected customers were notified immediately. We have found no other evidence of access to Linode infrastructure, including host machines and virtual machine data.

The entire Linode team has been working around the clock to address both this issue and the ongoing DDoS attacks. We've retained a well-known third-party security firm to aid in our investigation. Multiple Federal law enforcement authorities are also investigating and have cases open for both issues. When the thorough investigation is complete, we will share an update on the findings.

You may be wondering if the same person or group is behind these malicious acts. We are wondering the same thing. At this point we have no information about who is behind either issue. We have not been contacted by anyone taking accountability or making demands. The acts may be related and they may not be.

The security of your data, the functionality of your servers, and your confidence in Linode are extremely important to all of us. While we feel victimized ourselves, we understand it is our responsibility, and our privilege as your host, to provide the best possible security and service. You can help further enhance the security of your account by always using strong passwords, enabling two-factor authentication, and never using the same password at multiple services.

We sincerely apologize for the recent disruptions in your Linode service. Thank you for your patience, understanding and ongoing trust in Linode.
Posted about 5 hours ago. Jan 05, 2016 - 19:44 UTC
Click to expand...

eva2000 · Jan 6, 2016

Discussions also at

Security Notification and Linode Manager Password Reset | Hacker News

I'm glad to see that this information has now been publicly disclosed. In July 2... | Hacker News

2nd link is concerning !

TheSwordsman 4 hours ago | parent

I'm glad to see that this information has now been publicly disclosed. In July 2015, we suffered a compromise at PagerDuty via the Linode Manager. I hope that we can provide a bit more of an official in-depth post-mortem of our compromise, but I'd be happy to disclose some of the details here.
Using the access gained within the Linode Manager, the attacker reset the root password on a few systems, and used Lish to gain root access. We were alerted to this activity and fully revoked the attacker's access within 60 minutes of the first node being compromised. Working with Linode support, we discovered which user account was being used and completely deactivated the user. We also isolated the VMs, and performed forensics on read-only copies of their disk images.

In our situation the attacker knew one of our user's passwords and MFA secret. This allowed them to provide valid authentication credentials for an account in the Linode Manager. It's worth noting that all of our active user accounts had two-factor authentication enabled. An interesting data point was that the user who had their account compromised was no longer in possession of the MFA secret themselves. Their cell phone had been reset (thus deleting all data) 8 months prior. The user could not log in to the Linode Manager if they wanted, so it was our determination that the key could not have been obtained from the user and was more likely on Linode's side.

We also have evidence from access logs provided by Linode that the attackers tried to authenticate as an ex-employee, whose username ONLY existed in the Linode database. It was absolutely unique and was not used elsewhere by the employee making the username an accidental honeypot. This was another piece of data supporting that Linode was the source of our compromise.

We immediately reached out to them not only to inform them of their compromise, but to assist them in investigating it. We were confident that the Linode database had been breached, and that the secret key used to encrypt information in the database had been compromised as well.

In addition to reaching out to Linode, we also worked with a third-party security firm to audit our work done during the incident. Likewise, around the same time we reached out to law enforcement to assist in investigating the attack. I believe our public disclosure includes this information[1]. This was in the middle of July 2015.

We did not get confirmation in July that there was a breach of the Linode Manager or any associated credentials.

In the end, we migrated away from Linode because of this breach (even before it was publicly disclosed) in Aug 2015. We also never were able to confidently disclose that Linode was the vector due to lack of confirmation from their end. While all of us who responded to the incident were confident they were the source, we now thankfully have the data to confirm it.

[1] https://www.pagerduty.com/blog/july-2015-security-announceme...
Click to expand...

Skippy · Jan 6, 2016

Yeah that is concerning, You would think these type of issues would be handled with high care and urgency after that security incident in 2013. But we have to remember its only one side of the story.

eva2000 · Jan 6, 2016

Skippy said: ↑

Yeah that is concerning, You would think these type of issues would be handled with high care and urgency after that security incident in 2013. But we have to remember its only one side of the story.
Click to expand...

Yeah true hard to know the true story. Although key to that one was the hacker knew the user's MFA secret however they managed that ! Those are usually time limited validity wise from each generation !

dorobo · Jan 6, 2016

This doesn't look good

ModeltogTossen · Jan 6, 2016

I think this gets more and more 'weird' - correct that we dont have the full story but I'm sure this is not good if the incident July 2015 is true and Linode have not done anything to it.

As mention elsewhere I was ready to get 2 nodes at them - but for now, thats not going to happens. While this is written I can't even connect to linode.com from Denmark. It does not load correctly..

Edit: At 12.02 local time I can again access linode.com and log in.

Log in / Register

Forums

Want to subscribe to topics you're interested in?

Linode All Linode Manager passwords have been expired!

rdan Well-Known Member

ModeltogTossen I wish I could??

rdan Well-Known Member

ModeltogTossen I wish I could??

eva2000 Administrator Staff Member

eva2000 Administrator Staff Member

eva2000 Administrator Staff Member

eva2000 Administrator Staff Member

Skippy New Member

eva2000 Administrator Staff Member

dorobo Active Member

ModeltogTossen I wish I could??

.

Log in / Register

Useful Searches

Want to subscribe to topics you're interested in?

Linode All Linode Manager passwords have been expired!

rdan Well-Known Member

ModeltogTossen I wish I could??

rdan Well-Known Member

ModeltogTossen I wish I could??

eva2000 Administrator Staff Member

eva2000 Administrator Staff Member

eva2000 Administrator Staff Member

eva2000 Administrator Staff Member

Skippy New Member

eva2000 Administrator Staff Member

dorobo Active Member

ModeltogTossen I wish I could??

.