loving the dot com

While trying to book tickets to this year’s Reading Festival, I was once again frustrated by the main website (readingfestival.com) and it’s lack of ability to handle the increase in traffic as tickets are released for purchase and the lineup is announced. This frustration reminded me of a few sites I came across recently, some related research into designing sites for High Scalability, and systems for temporary load management. Here’s some stuff developers might find useful….

The first of these is the site HighScalability.com – as the title suggests, if offers tips and articles on exactly how to build a scalable site, and has some very handy interviews with tech staff from some very high traffic sites, such as YouTube and Flickr.

Looking at YouTube, there’s a great video from the Seattle Conference on Scalability from one of the YouTube staff about how they have successfully managed to grow their system on commodity hardware and open source software, such as MySQLкомпютри втора употреба. Watch the video here.

If you’re interested in working with MySQL, there is a PDF presentation available on Scribd which talks through a lot of the key issues developers face when scaling to heavy traffic loads. This one is well worth a read – see it here.

Lastly is a system which will allow you to quickly scale your application at low cost if you’re in absolute dire need. Enter Amazon’s Elastic Compute Cloud (EC2). There is too much behind the system to explain here, but it basically allows you to manage exactly how and what you want to scale by booting your own reusable images inside the cloud. Fans of Facebook will have no doubt heard of the iLike application, there’s a good explanation of how they used the cloud to scale their application here.

Oh – and don’t forget to get your Reading Festival tickets! No doubt the main site is still crippled under the load, so you can read the line-up and get ticket information from readinglineup.com.

Administrators of Internet Relay Chat (IRC) servers say they have recently become aware of dropping user counts from certain ISPs, especially TimeWarner owned Cox.net.

It is well known that botnets consisting of large numbers of compromised PCs are becoming more of a problem and are harder to find, harder to fight and harder to destroy. Until recently, the only people who seemed to be doing the fighting against this threat were the producers of anti-virus software. It is then up to the computer administrator, or home user to install this software in an attempt to remove any malicious programs from their system. As more and more viruses and trojans are released every day, this is not as simple as it sounds.

According to a number of IRC administrators, the Internet Service Provider Cox.net seems to have taken this fight away from it’s customers and into its own hands.

Many botnets use the IRC protocol for communication and control, so it makes sense that this is where Cox decided to strike. After administrators of the Ablenet IRC network noticed it’s users disappearing they started investigating the cause and found that a DNS server owned by Cox was returning falsified data and were hijacking DNS entries which caused any legitmate IRC connection to be redirected to a server owned by Cox.

After being redirected to Cox’s server a number of commands are sent to the user. To most users this is uncomprehensible data, but the server is infact issuing commands which attempt to uninstall one particular type of malicious trojan.

By hijacking a user’s connection in this manner, IRC networks are receiving a bad reputation because most connecting users are not aware that their connection has been hijacked. Users are left guessing why they cannot chat with their friends like normal, and aim their frustration toward the administrators of the IRC network. For most home users, avoiding this connection hijacking is not a trivial task.

More recently, administrators from EFnet, the oldest and 4th largest IRC network in the world, which currently holds around 60,000 concurrent users, have also suffered from their DNS entries being hijacked by Cox.

At present there are no laws in the United States to stop Cox, or other ISPs, hijacking and falsifying DNS entries, however the borderline on actually cleaning trojans automatically is much more vague. In the United Kingdom it is a definite breach of the Computer Misuse Act for any system or user to perform ‘Unauthorised Modification’ of another computer system. By issuing commands to a user’s PC without their prior consent, this is exactly what Cox appears to be currently doing.

One thing is for certain, with all the current emphasis on Net Neutrality laws in the United States, Cox may have very effectively demonstrated why so many people are campaigning to see it become a reality.

p.s. this isn’t me

Recently I setup another blog at spampress.net. This new blog (as suggeted by the name), will publish nothing but the spam it receives via email.

What’s the point?

Spampress.net has been setup as both a demonstration and experiment. Primarily, it will show exactly how much spam is received to a domain which is completely unpublicised. The second reason is an experiment to demonstrate how spammers are becoming more savvy with including specific keywords in their emails.

Why use keywords in email?

The term ‘keywords’ usually brings to mind SEO and other types of web targeting, but recently spammers have started adding more and more ‘random’ keywords to their email as a way of fooling spam filters. If you view your email in HTML format you might find a lot of spam is made up of one single image, by using this tactic the spammers can overcome a lot of the spam filters because the text isn’t directly readable, and therefore cannot be analysed. By viewing these same emails in text-only format often the email contains nothing but random keyword text. These are used to throw off the spam filter further.

What will this accomplish?

After the blog has been running a while I will be publishing the results collected by Google Analytics and the webserver’s local copy of Awstats.

Stay tuned.

Having been rediculously busy with various things, it has been rather a while since my last post, so here’s one that’s been sitting as a draft for ages, hopefully some of you will find it useful.

Wordpress Plugins – what’s the point?

Wordpress is a great publishing platform from the outset, but Matt also built in a great architecture for coders to expand the system to do all sorts of other neat stuff. I know of some blogs which use tens of plugins to make their site work, but at the moment I only use 4 main ones. here’s a quick rundown of what they do, and why I use them.

Akismet – stopping comment spam

If you haven’t already got some sort of anti-spam plugin installed on your blog, it’s more than likely you are getting lots of comments submitted which seem like utter junk, or just blatant spam. This is where Akismet comes in. This plugin actually comes preinstalled with Wordpress, but you need to signup to Wordpress.com and get an API Key to use it.

Since I started making more regular posts, the amount of comment spam I received has just been rediculous, and Akismet has successfully sorted all that out for me. Make sure you have this turned on, seriously.

Feedburner Feed Replacement

As I mentioned in my redirecting your Wordpress feed to Feedburner post, this plugin provides you an easy way of redirecting all subscribers to your feed through the Feedburner service, allowing you to accurately keep track of reader statistics.

Name Dropper – let your visitors publicise your posts

The name dropper plugin gives bloggers a simple way to allow their visitors to submit their blog posts to a number of social bookmarking sites, including StumbleUpon, Digg, Del.icio.us and many others. It’s well worth installing it – you can see it in use at the bottom of every post here.

Optimal Title – fix your titles for SEO

Optimal Title replaces Wordpress’ wp_title() function and just moves the seperator to after the title, rather than before it. This means the title of your post will be picked up by search engines, BEFORE the name of your blog. That way visitors will more easily find their way to your posts directly.

Google Sitemaps – helping Google spider your site

Having an accurate sitemap of your site can often appear to be more trouble to setup than it’s worth, but thanks to this plugin, the task no longer needs to be a tough one. Just install the plugin and hit Activate, all the rest is done for you. See the sitemap it generates for this site here.

Twitter Tools – simple integration to Twitter

My last post was all about the Twitter service. This plugin allows you to simply integrate with their API to view and submit your latest tweets. Seen here in the sidebar on the right.

Ok, so I couldn’t quite believe it when I read it. At the launch of the PS3 in London’s Virgin Megastore yesterday, Sony decided to give out 100 brand new Bravia HDTVs – worth £2500 each.

I know they’re getting a bit desperate to attract some positive publicity toward the PS3, but surely this is a bit of an extreme measure.

Still, if I’d been queueing for 36 hours straight and had to pay £500 at the end of it, I’d probably need something to cheer me up too. I think I’ll be waiting a while before I buy myself a PS3 – like until they make all the good PS2 games work. If you’re hoping to play Gran Turismo, Grand Theft Auto (Various) or any of the Pro Evolution series, good luck.

Source: [1]

On the whole, randomly searching for ways to cash in on the blogging boom will return you the same set of generic results, but occasionally you come across something out of the ordinary – such as the personal blog of John Chow. John’s blog is a shining example that it is possible to get financial gains purely from blogging, but also keeping your blog entertaining in the process – lets face it, nobody wants just another feed dedicated to technical crap and the increasingly quirky world of SEO.

One of the main reasons for Chow’s success is his attitude to the whole area of SEO and the funny spin he puts on his articles, one thing is for sure though – if you’re wanting to make money online from writing your blog, there aren’t many better examples of people who have achieved that very goal.

Promote something where everybody wins

Campaigns like this one are a fantastic way of getting traffic driven to your site and increasing your search engine ranking at the same time. From John’s point of view, this is certainly win/win. He can demonstrate to potential advertisers how many people read his blog and take note of what he’s written, at the same time he has a sure-fire way of getting others to link back to his blog – affirming his own search engine ranking. Just try googling for “root of all evil”, and you’ll see what a success his last attempt at driving up traffic was.

Always remember – people love competitions, people love campaigns. If there is a queue, people will fall in and join it – the web is no different. If you can convince people they are getting something for nothing, you’ll be a guaranteed success.

Amazon’s latest project, ‘Mechanical Turk‘, provides a web services API for computers to integrate ‘artificial artificial intelligence’ into their software, by asking humans to complete certain tasks rather than relying on automatic processing.

The name itself is a little odd, and refers to a machine built in the 18th century which played chess against real people and frequently beat them – but almost 100 years later it was revealed that there was a man inside the machine playing the games.

Before you think “this sounds like another useless waste of time and TCP/IP packets”, take a moment to think what it can do for you…

The pseudo-code example Amazon use in their FAQ is a particularly good one:

read (photo);
photoContainsHuman = callMechanicalTurk(photo);
if (photoContainsHuman == TRUE) {
	acceptPhoto;
}
else {
	rejectPhoto;
}

What this basically means is instead of writing a program to read in an image and writing rediculously complex algorithms to try and determine whether a particular photo contains a human, you can submit the request to Amazon’s Mechanical Turk. From there someone else can choose to take on the task (for a fee) and quickly (and more reliably) give you an answer to your question.

This service isn’t just restricted to tasks like simply identifying photos. It could be used to do anything, translate and sort RSS Feeds, click on advertising – whatever you can think of.

Remember – if you will pay for it, someone will do it.

Everyone knows that it’s been coming for a while, and it finally has. YouTube has been served with the biggest lawsuit yet in relation to copyright infringement. Viacom, who own MTV, Nickelodeon, VH1, Comedy Central, and others; reckon that YouTube’s constant disregard for copyright law has gone far enough – stating that over 160,000 clips are listed on the site completely unauthorised – and have been viewed a total of more than 1.5 billion times.

Viacom states that YouTube has deliberately not acted to combat copyright infringement in order to drive up traffic statistics and sell more advertising space, however YouTube’s owner, Google disagree with this and is reportedly confident that YouTube has properly satisfied the requirements of copyright law.

Ever since the search giant Google bought the video sharing giant YouTube last year for a whopping $1.65 Billion it has been desperately working with content providers to secure deals which would allow it to legally broadcast content, without fear of such lawsuits coming about. However, a public forum such as this where users can upload their own means copyright infringement is a sure thing.

The sheer size of this case could cause a lot of problems for YouTube – and any other sites currently sharing videos – if it rules in favour of Viacom.

Watch this space.

Sources: [1] [2]

If you’ve ever wondered how cool it would be to be some sort of half-borg-half-human-hybrid, technology might be able to satisfy your curiosity within the next few years….

Neurosky, a company based in Silicon Valley has developed a headset which can literally read your thoughts. Well, ish.

Stanley Yang, Neurosky CEO: “We essentially created a bunch of different algorithms. One for each emotion, for example the attention algorithm.”
…
Interfaced with a video game, you place the cursor on an object. A strong “attention” reading enables you to push or pull objects.
…
A strong “meditation” reading enables you to levitate objects.

I’m not entirely sure what to make of it, because it seems to guess what you’re feeling, as opposed to directly responding to what you’re thinking – just imagine trying to control your attention on a Friday.

Check out the original article and make sure you watch the video.