Amazon EC2 4-day downtime debacle: Keeping your users in the dark is naughty

Amazon Web Services logo

A few days ago, Amazon’s Elastic Compute Cloud (or EC2 to you and I) had a catastrophic failure. The world kept turning, but unfortunately, the third-party collaboration/SVN tool we use was on the affected East USA zone where the issue started on Thursday. It wasn’t until late Sunday night that we regained access to our SVN server.

That’s effectively four days of downtime, where our Alfred development was at standstill at a point where we had huge plans for the next release. The third-party was kept in the dark by Amazon as much as we were, twiddling our thumbs waiting for things to get moving.

In the past year, Twitter’s uptime has improved and it has become even more of an essential tool to many people than before. Increasingly, Twitter is in fact being seen as a source of up-to-the-second information and news, with the newly redesigned homepage further driving the point home.

Twitter homepage

Yet, at the time of writing, Amazon hasn’t used the awscloud account to update customers of the status of the outage or the reasons behind it. There are plenty of theories floating around about hardware failure, Amazon’s Cloud Player becoming too popular too soon, but we’ve not heard an official word.

AWS Health Dashboard - April outage

It isn’t for lack of smart cookies at Amazon either; knowing a few of them, I’m baffled why no one felt it was worth using it as a channel for communication. The AWS Health Dashboard was updated fairly frequently with obscure, meaningless status updates and no background information.

Many organisations dismiss Twitter as a social network made for sharing what you’ve had for breakfast but in times of crisis, it can truly come into its own. As far back as 2007, emergency services have used Twitter to disseminate information and help the population when fires raged across Southern California. The Los Angeles Fire Department as well as news outlets tweeted updates to help people get to safety or stay away from affected areas.

More recently, Japan’s phone networks were overloaded after the earthquakes in March – with NTT DoCoMo restricting up to 80 per cent of voice calls, especially in Tokyo – but Twitter, Facebook, Mixi and Skype were lifelines for those hiding under desks during the seemingly never-ending earthquake.

While the EC2 debacle was nowhere near as life-threatening as an earthquake, it was the perfect opportunity to post short, simple updates on Twitter, letting those directly and indirectly affected know that Amazon wasn’t asleep at the switch.

My confidence in cloud computing has been less dented by the outage itself, and much more by the feeling of helplessness Amazon caused by giving us no clue what was happening! I wonder if we’ll ever find out why they chose to be so uncommunicative, and whether they’ll improve if there’s a “next time”.

Twitter's other spam problem: Username chaos

A few months ago, Twitter published a State of Twitter Spam blog post. It claimed to have reduced spam from fake accounts to little more than 1%, lowering the number of offers for prescription drugs, dodgy online scams and invitations from busty babes. We complain about Twitter more than we praise them, but this deserves a “well done!”

What’s on the increase and quite possibly trickier to control is the noise created by users who don’t understand how Twitter works – usernames in particular.

There seem to be a few trends:

Retweets as replies

Users in Malaysia and Indonesia seem to use retweets as a way to “thread” conversation. As a result, the oldest words get truncated. As Veronica seems to be a common name, I’ll often have more misappropriated foreign tweets than real @replies in my stream.

Failing to understand usernames

The next flavour of Twitter spam is due to users who don’t understand that @Vero or @Jake or @Bob are really someone’s usernames and use them willy nilly.

I often see “@Vero’s house for a party” and hope that some stranger isn’t on the way to my place with a couple of crates of beer for an impromptu gathering.

What’s next?

I’ll be interested to see how Twitter tackles this kind of issue. Having steamed past the 100 millionth user some time ago, the noise level is quickly becoming deafening.

Unlike classic spam where a user publishes the same thing hundreds of times, this can’t be fixed as easily as it’s a user education issue.

How could Twitter handle this?

Torchwood Writer Gets Online Abuse: Where social media stops being fun

A few days ago, I wrote about the Torchwood 5-day mini series which ended on Friday. During that same evening, it’s with great amusement that I also discovered that James Moran, writer for Severance, and episodes of Doctor Who, Torchwood, Primeval, Spooks, and Crusoe, was on Twitter.

In many ways, I enjoy seeing these backstage celebs on Twitter. By backstage celebs, I mean people who aren’t Britney, MC Hammer or Stephen Fry. Those people can be on Twitter all they like, but they’re already in the limelight. Seeing those who usually don’t get the limelight finally interact with the public through more than their scripts, stories or stage direction is more exciting, as we don’t usually get to hear them speak other than through their characters.

However, this evening, I came across a post by James which allowed me to realise just how seriously some people take television. He received many positive and praising messages, but was also highly criticised for a storyline that even upset me. Yes, I did have a tear in my eye when Rhys held the camera for Gwen to record her final words as “the world ended”.

Some have been spewing insults and passive aggressive nonsense. Accusing me of deliberately trying to mislead, lie, and hurt people. Telling me I hate the fans, that I’m laughing at them, that I used them, that I’m slapping people in the face, that I’ve “killed” the show, that I’m a homophobe, that I want to turn the fanbase away and court new, “cooler” viewers, even that I’m hurting depressed people with dark storylines. Asking me to pass on vitriolic, hateful messages to people I love and respect.

Not cool.

As James says, this just isn’t cool.

I love letting a story envelope me and take me away from work, home, the fact that the kitchen’s still not tidy and the stairs need hoovering. I love a story that lets me get a tiny little crush on one of the characters and picture travelling the stars with them. [Hell, I named my cats after Jack Harkness and Rose Tyler!] And yes, sometimes I want to shout at the TV and disagree with their stupid actions. “Don’t go in there alone and DO NOT put your gun down, you idiot!”

But people, all of you people who’ve given James abuse, get. a. fucking. grip.

“Hurting depressed people with dark storylines”? Please, get some real help. And I’m not saying this intending to offend, but with a true concern that if a TV show is enough to make you cross that line, it’s time to look at getting real help.

And if this isn’t your situation, then please go outside and get some perspective. This is television, and for a change, hey, it’s good enough to make people feel strongly about it by choosing a path less travelled. If the writers had taken the usual path, the same people would have clamoured that the ending was cheesy and predictable!

So have some respect for people and their trade. If a writer can’t join Twitter and enjoy it for what it is – a totally open means of communication with the audience – then writers, actors, authors and other backstage celebs will pull back and let their PR agencies do the talking. And that’s not what we want, is it!?

Here’s 50p, go buy yourself an ice cream and some perspective.

Bloggers: Be confident, positive (and humble)

This evening, I came across a post where I couldn’t help but think that I had to share with fellow bloggers.

Darren Rowse, pro-blogger and six-digit-salary man, tells bloggers to get rid of their inferiority complex, and I could not agree more! He gets loads of emails from bloggers asking for tips or advice (why don’t I get more mail from you readers?!) with many self-deprecating comments, claiming they’re “no A-list blogger” and “don’t write as well as they do…”

So this is to tell you, my fellow bloggers, to take pride in what you do. It doesn’t matter if all you write is a weenie little blog to track your child growing up, your BMI going down by preparing for a half-marathon for charity, or a technology rant.

Think positive, be proud of the fact that you’ve braved the wild world of blogging. You may not realise it but you’re boldly going where most of people around you won’t have. So grab that blog by the horns and be a sassy self-promoter. Whether it gets you a job, helps you find like-minded people for a project or just gives you an outlet to blow some steam off, enjoy the fact that you’re still more cutting edge than you might think.

Why Twitter is so unbelievably awesome

Anyone who’s witnessed a typical weekday for me will have noticed my slight addiction to Twitter, a service that simply can’t be explained and has to be experienced.

But in my attempt to justify the thousands of updates I’ve posted on it, I’ll highlight a few amazing ways Twitter has helped me and those around me this week.

  • It helped me discover how other bloggers felt about being accosted by PR agency, resulting in an article for The Blog Medic called “Marketing Ethics: Ten ways to piss off a blogger”.
  • An ad hoc conversation led to a friend getting a job offer, and the entire conversation up to scheduling an interview call happened over Twitter.
  • It allowed me to find a couple of new contracts for Pepsmedia redesigning blog templates & site launches.
  • Since SXSW, I’ve managed to stay in touch with many of the lovely people I met there without going through the usual “ok I’ll reply to that email later”, where later becomes never. By keeping it bite-sized, Twitter makes it easy to stay in touch.
  • I’ve found amazing support for the idea of SocialMediaCamp in London in July through fellow Twitter users who are interested and can provide skills and contacts I wouldn’t have had access to otherwise.
  • A few people offered sound advice with regards to the process to setting up a limited company, again calling on the experience of others.
  • It was the fastest channel through which I heard about Russell’s decision to stop developing Mowser on Monday night.
  • It’s a great way to swap kitty photos with Mel Kirk 🙂

So there you go, it’s a business resource like no other, a great communication tool and an entertaining place to have water cooler conversations with like-minded people.

The Twitter backlash begins: Welcome to a world of pain and spam

A couple of days ago, Hugh quit Twitter to work on writing his book. Now I’m considering quitting Twitter, but nobody’s signed me up for a book.

The reason? Spam, spam, eggs, bacon and spam. Well, without the eggs or the bacon. The sheer volume of new followers I’m getting these days who are blatantly spammers is getting increasingly frustrating. Sure, I can block them one by one, or simply ignore them, but if Twitter could implement a “flag as spam” a la Blogspot, then we could help each other and avoid 10,000 other users getting the same spammy follower message.

Such a pain, Twitter spam takes over my inbox

To add to the frustration, a friend pointed out that spam followers could very well use your RSS feed to create random copy for spam emails or blog comments in the future. I haven’t come across it yet, but I wouldn’t be surprised if it’s already happening.

I guess for now, the less drastic route for me to take will be to create a rule where all notifications of new followers will go straight into a mark-as-read folder. It won’t solve the problem that my feed could end up as spam material for some unscrupulous asshole out there, but it’ll have to do for now.

What this means is that if you start following me and you want to have a conversation, you’ll need to send me a message @vero for me to react and add you as well. Crappy, but it’s the best solution I can think of.

Anyone got a better idea?

Hippie 2.0: Reflecting upon SXSW 2008

South by SouthWest is over. Well, the interactive bit is anyways. Music is clearly still going strong, as I witnessed walking down 6th Street and lucking out on seeing Simian Mobile Disco at La Zona Rosa with a few of the geeks still left in town.

Reflecting upon the past week, it’s comforting to see a clear sense of community amongst the geeks. Topics that kept reoccurring were ones of social capital, change and collaboration. The jaded half of me couldn’t help but snicker. Is this hippie 2.0* or something?

Don’t get me wrong, I find this “Let’s hug, love and help each other” attitude immensely endearing and refreshing, but I can’t help but be tickled by some of the more naive ideas that were exchanged over the course of the week. Not every single one of our ideas will live on past the panels, not every one of our harebrained startup ideas will become the next Facebook and not every suggestion is revolutionary. But it doesn’t matter, it’s motivating to be surrounded by people with faith in their ideas and seemingly endless energy to turn them into reality.

So if it’s up to me, I’ll be attending SXSWi again next year. The panels may not all have been oh-so-fabulous, but regardless, meeting so many new people is an injection of energy, if nothing else.

I now need to somehow make sense of this creative energy and communicate it to my team at work. I’m not sure I can express it in words. Maybe I need a Kumbaya 2.0 to express my feelings?

[* I seemingly didn’t coin the word, as it comes up on the Interweb in a different context, but I think it’s terribly fitting here as well.]