loving the dot com

The BBC’s weekly technology programme Click has just finished filming a special on the dangers of hackers controlling botnets.

The programme researched and demonstrated these dangers by acquiring access to 22,000 computers in various locations around the world. These computers were then each used to send hundreds of emails to BBC email accounts hosted on Google’s Gmail and Microsoft’s Hotmail services; and also to launch a Distributed Denial of Service (DDoS) attack against one of security firm PrevX’s websites, who also contributed to the programme.

Here is an excerpt from the program, which is to be broadcast on Saturday 14th March 2009:

The BBC correspondant Spencer Kelly states that the programme gained access to “around 20,000 infected computers – If you were to do this with criminal intent, you’d be breaking the law.” However, it is on very dubious legal ground that this claim is made. The UK Computer Misuse Act 1990, Section 1 states that:

Under these terms, having “criminal intent” is not required to be in violation of the act. By securing access to a program running on the computer (ie, the ones used to send the emails or that generated the DDoS traffic) the BBC breaches part 1-a. The access to these systems was also totally unauthorised, breaching part 1-b. The BBC also admits that they were aware that the access to these systems was unauthorised, breaching part 1-c.

Struan Roberrtson, a technology lawyer and editor of OUT-LAW.com seems to have confirmed this:

To add to the BBC’s misuse of remote systems, the end of the report explains how the offending bots were cleaned and a message left on the computer’s desktop background warning the owner that their computer had been compromised. The BBC is therefore also guilty of illegally modifying the contents of a computer without authorisation, this puts them in breach of Section 3 of the Computer Misuse Act:

Although this section of the act does apparently require more criminal intent than section 1 in order to be in violation, the BBC have still hindered access to a program (the bot) by removing it from the system. Regardless of whether the program should have been there in the first place, it is not up to the BBC to decide whether or not it should have been removed; therefore parts 1 and 2 are satisfied. Also, having the intention of performing this action when access was gained to these systems, the BBC appears to be in breach of Section 2 of the act, titled “Unauthorised access with intent to commit or facilitate commission of further offences”.

While it is unlikely that anyone from the BBC will be prosecuted as a result of these offences, it is incredibly irresponsible that the BBC have chosen to demonstrate the dangers of botnets by gaining control of remote computers and clearly breaching the Computer Misuse Act in the process.

This argument has been raised in numerous articles across the Internet over the last day, but one point I have failed to see repeated much is the fact that Google and Microsoft’s email systems were abused by the process of sending spam. The accounts on these systems may have been owned by the BBC, but at no stage do they mention contacting Google or Microsoft in order to warn them of the large quantities of intentionally malicious email they were about to send, affecting the spam filtering capabilities of these systems.

Having spent a number of years running high-traffic Internet Relay Chat (IRC) servers on EFnet, I am no stranger to having to deal with DDoS attacks – frequently reaching sizes into Gigabits of data per second; the bots themselves connecting to and flooding the network, and also the hackers who use IRC as a control mechanism to access their botnets. It is a never ending challenge to make sure these systems are kept away from regular users and not allowed to utilise our network to perform abusive tasks, such as spreading trojans further to other users. All measures we have in place to combat this scourge have to be ‘defensive’ in nature as we deal with users from across the whole world and every country has their own laws which govern computer misuse. This limits us to potentially denying access to services for legitimate users, simply because there is a bot somewhere on their network. We fully recognise that it is not our place to connect to or alter the contents of a remote computer system without authorisation.

It should also be noted that by sending large volumes of email and DDoS traffic, the BBC has not only affected innocent remote user’s systems and the systems on the receiving end; but they have also affected every single data network inbetween those two points. Sending DDoS traffic over the Internet is something that should be taken very seriously. Every packet of data sent costs someone money somewhere, even more so if that data has to travel over trans-continental links. I wonder if the BBC considered this before performing their little demonstration; and how much it actually cost.

Submit this post to:
StumbleUpon | Digg | Del.icio.us | Dzone | Newsvine | Spurl | Furl | Reddit | Yahoo! MyWeb

Since late April 2008 a great many websites (this one included) have started noticing changes in their daily traffic – large sites especially. For some sites this traffic increase is huge, which can not only drastically increase bandwidth costs, but can also massively skew web analytics numbers.

As a result of this, it has become very difficult to report accurate numbers to advertisers of the actual eyeballs hitting the page. Many web-robots (GoogleBot/MSNBot et al) clearly identify themselves by the ‘user agent’ field they send to servers. This allows analytics software to simply ignore them. The problem with AVG’s LinkScanner software is that it does notonline casino netrent a car bulgaria clearly identify itself, instead the the user agent appears as follows:


Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1;1813)


Unlike many other kinds of shady web-spiders, the AVG software does not have any malious intent. Instead, it aims to protect it’s users from clicking on links (which may lead to exploits, virus or trojan infection etc) by visiting other links on a page before you do – so while you’re reading down a search results page trying to decide what to click on, the AVG software has already gone there and seen it.

Unfortunately, this means your PC will be visiting a lot of pages that you probably won’t even visit. Not only does this artifically inflate the traffic for the websites it visits, but it also means that your PC is potentally transferring a lot more data than you want it to. For users on limited-bandwidth contracts, this could be a problem: “What do you mean I’m over my traffic limit? I’ve hardly visited any websites!” – well, you haven’t, but your PC might have.

Back to a webmaster’s point of view – it would totally defeat the point of the software if AVG released ways to identify their bots, so don’t expect them to. The best way to figure out what to look for is to use Google, and search for what other webmasters have noticed. For the time being the software appears to use the user agent shown above, but that’s not to say it won’t sporadically change to more commonly used ones.

Luckily, the bot doesn’t appear to execute JavaScript, which means stats packages like Omniture that rely on events being fired off shouldn’t be affected. However, packages like DoubleClick’s DART which monitor traffic as it is fetched from their remote servers will.

If you’ve been suffering from this problem and want a quick idea of how to possibly fix it, here’s some PHP to drop into the top of a page which should stop AVG traffic from hitting it (for now!):

<?php

if (preg_match('/MSIE 6.0; Windows NT 5.1;1813/',
                     $_SERVER['HTTP_USER_AGENT']))
    return 0;

?>

If anyone else has been affected by this problem or has come up with some alternative solutions to deal with it, please comment and let me know.

Perhaps if enough people kick up a fuss, AVG will be inclined to start a discussion on how to better identify and deal with the traffic they are now generating.

While trying to book tickets to this year’s Reading Festival, I was once again frustrated by the main website (readingfestival.com) and it’s lack of ability to handle the increase in traffic as tickets are released for purchase and the lineup is announced. This frustration reminded me of a few sites I came across recently, some related research into designing sites for High Scalability, and systems for temporary load management. Here’s some stuff developers might find useful….

The first of these is the site HighScalability.com – as the title suggests, if offers tips and articles on exactly how to build a scalable site, and has some very handy interviews with tech staff from some very high traffic sites, such as YouTube and Flickr.

Looking at YouTube, there’s a great video from the Seattle Conference on Scalability from one of the YouTube staff about how they have successfully managed to grow their system on commodity hardware and open source software, such as MySQLкомпютри втора употреба. Watch the video here.

If you’re interested in working with MySQL, there is a PDF presentation available on Scribd which talks through a lot of the key issues developers face when scaling to heavy traffic loads. This one is well worth a read – see it here.

Lastly is a system which will allow you to quickly scale your application at low cost if you’re in absolute dire need. Enter Amazon’s Elastic Compute Cloud (EC2). There is too much behind the system to explain here, but it basically allows you to manage exactly how and what you want to scale by booting your own reusable images inside the cloud. Fans of Facebook will have no doubt heard of the iLike application, there’s a good explanation of how they used the cloud to scale their application here.

Oh – and don’t forget to get your Reading Festival tickets! No doubt the main site is still crippled under the load, so you can read the line-up and get ticket information from readinglineup.com.

Check out this latest facebook app from TV.com. You can rate/review all your favourite tv shows, check out what’s currently hot, see what you should be watching. It’s also got a cool little quiz.

View the application here.

Administrators of Internet Relay Chat (IRC) servers say they have recently become aware of dropping user counts from certain ISPs, especially TimeWarner owned Cox.net.

It is well known that botnets consisting of large numbers of compromised PCs are becoming more of a problem and are harder to find, harder to fight and harder to destroy. Until recently, the only people who seemed to be doing the fighting against this threat were the producers of anti-virus software. It is then up to the computer administrator, or home user to install this software in an attempt to remove any malicious programs from their system. As more and more viruses and trojans are released every day, this is not as simple as it sounds.

According to a number of IRC administrators, the Internet Service Provider Cox.net seems to have taken this fight away from it’s customers and into its own hands.

Many botnets use the IRC protocol for communication and control, so it makes sense that this is where Cox decided to strike. After administrators of the Ablenet IRC network noticed it’s users disappearing they started investigating the cause and found that a DNS server owned by Cox was returning falsified data and were hijacking DNS entries which caused any legitmate IRC connection to be redirected to a server owned by Cox.

After being redirected to Cox’s server a number of commands are sent to the user. To most users this is uncomprehensible data, but the server is infact issuing commands which attempt to uninstall one particular type of malicious trojan.

By hijacking a user’s connection in this manner, IRC networks are receiving a bad reputation because most connecting users are not aware that their connection has been hijacked. Users are left guessing why they cannot chat with their friends like normal, and aim their frustration toward the administrators of the IRC network. For most home users, avoiding this connection hijacking is not a trivial task.

More recently, administrators from EFnet, the oldest and 4th largest IRC network in the world, which currently holds around 60,000 concurrent users, have also suffered from their DNS entries being hijacked by Cox.

At present there are no laws in the United States to stop Cox, or other ISPs, hijacking and falsifying DNS entries, however the borderline on actually cleaning trojans automatically is much more vague. In the United Kingdom it is a definite breach of the Computer Misuse Act for any system or user to perform ‘Unauthorised Modification’ of another computer system. By issuing commands to a user’s PC without their prior consent, this is exactly what Cox appears to be currently doing.

One thing is for certain, with all the current emphasis on Net Neutrality laws in the United States, Cox may have very effectively demonstrated why so many people are campaigning to see it become a reality.

Recently I’ve noticed more and more posts and articles relating to this “new” Web 2.0 service called Twitter. It’s been around for almost a year now (it was born on March 2006 I think), but it mainly took off in the US and hasn’t really appealed to the UK market much yet. What is it anyway?

Put simply – Twitter is a social networking site which allows it’s members to post updates of what they’re currently doing to each other – in 140 characters or less (basically like an SMS). At first glance, most people are going to think “What the hell is the point in a service which just lets me tell my friends what I’m currently doing?.” Well, it’s a good argument. A fair amount of people will disregard the service and not care – there are a few people who are familar with something similar already though. If you’ve got an account on Facebook, chances are you’ve updated your status at least once. This is what Twitter does best, except streamlined and linked to your IM client and mobile phone.

The advantage of the service being linked to your mobile phone instantly makes it easy to post your own little messages, co-ordinate when you’re going out, or even submit micro blog posts, wherever you are – especially for those moments when you don’t have a piece of paper to hand, or just want to quickly rant about something.

As Twitter seems to be steadily growing still, I thought I’d give it a look. If you’ve been reading this page recently you might have noticed the “Twitter feed” column in the right sidebar. This little feed comes directly from my Twitter account and is updated every 30 minutes – thanks to Alex King’s Twitter Tools plugin for Wordpress. As I’ve only been playing with it briefly there isn’t much going on in my feed. Maybe I’ll start updating it more frequently once I get suitably addicted to it.

Of course, being a Web 2.0 service, Twitter have published a handy API enabling people to plug all sorts of applications into their service (like the Twitter Tools plugin for Wordpress). There are some quirky amusing ones, such as Twittervision, which gives you a real-time feed of the latest Twitter “tweets” from the public feed. It’s quite amusing for the first few minutes, but you can get bored of it fairly quickly.

Some of the other people to plug into the service include BBC News, who send headline updates to the site – great if you like to keep track of breaking news. They also have a load of other Twitter feeds you can keep track of.

I have to say though, the most useful thing I’ve found on the service so far is the Twitter Tube Tracker by Tom Morris – it lets me get an SMS update of any delays on the London Underground so I don’t have the joyous pleasure of arriving at the station and finding the line is suspended.

There are a whole load of other ‘Twitterized’ services to play with at the Twitter Fan Wiki. If you’ve got a Twitter account, do check out the fan wiki – you might find something you like. Why not add me to your friends too.

How’s that for timing. Just after I posted yesterday about the one negative point of clickdensity’s service being that you couldn’t track clicks on subdomains through their reporting tool – they’ve gone and added it today. I had a quick play this afternoon and I can confirm that it works really well too.

They’ve slightly changed the look of their reporting pages to accommodate the change (see above), but other than that everything works in just the same way. The other good news is that you won’t have to change the configuration in any of your accounts at all – it just works.

Read about their new support for subdomains.

These days there are a billion and one scripts and companies which can tell you all sorts of statistics about your website traffic, but there’s a new kind of statistic in town – the heat map

Instead of the conventional way of collecting statistics by parsing through log files or logging page hits in real time, heat mapping scripts actually respond to where users click on your pages. By using this data it’s possible to draw some really pretty pictures which can clearly illustrate how users physically move about the site – which elements are more popular than others? How far are people willing to scroll down? Is this Ad actually working? A heat map is the answer.

Commercial Heatmaps

The main names on the commercial side seem to be clickdensity, crazyegg and ClickTale – although ClickTale are still in beta testing and have yet to officially launch their service.

Here’s an example of the offering from clickdensity:

Once you sign up for the service, all you have to do to get their service working is paste a little bit of javascript directly before your closing </body> tag in your page and they immediately start tracking your visitors. As you can see from the picture you also get a lot of filter options on how to narrow down the results shown to you – whether it is in a heat map, or more traditional form. One of the particularly neat features splits the page up into it’s individual elements and gives you individual statistics as you hover over each one.

After trialing clickdensity’s service on a couple of websites at work it was immediately clear that there are a huge number of clicks which just appear to go absolutely nowhere – whether this is because of users clicking on their browser window to bring it into focus, or just randomly clicking it isn’t clear. It is very clear where the large majority of clicks are going though, and I think their map is very nicely rendered.

If you’re looking to test out heat mapping for your website, I would very much recommend you take a serious look at clickdensity as they have more detailed statistics than other providers I have seen. One thing to be wary of though, statistics are (currently) restricted to the exact domain names associated with the accounts – so subdomains count as seperate domains. This means if you have a large site which sprawls over a large number of subdomains, you’ll have to add a tracking account for each one.

Non-Commercial Heatmaps

Wherever there’s a good commercial project, there’s often a non-commercial open-source one too. This one isn’t as good as it’s paid-for counterparts, but then it is being built in people’s spare time and they have no financial motivation to make it as pretty.

The non-commercial offering comes in the form of LabsMedia’s ClickHeat. This system is written in JavaScript (of course) with a PHP/GD backend. Here’s what theirs looks like:

As you can see the interface can’t really compete with that of clickdensity, but most the functionality is there. I have briefly tested this system too and while it does work, I would be more inclined to use a paid-for service at the moment simply because the reporting side isn’t entirely up to scratch.

That said – having had a flick through the code which makes it work, it becomes clear that the system isn’t half as complicated as you might think. All the JavaScript has to do is tell the server where on the page the user has clicked, the majority of the code is dedicated to drawing the heatmap image. This is also fairly trivial in the grand scheme of things.

It’s often said that simple ideas are the best, and I’m surprised that more people haven’t implemented a similar sort of system to this. As it is so simple it does make you wonder how long clickdensity, clickegg and ClickTale can stay ahead of the game.

More and more websites these days are adding what’s called a “Favicon” (or ‘Favourites Icon’) so you can identify them better. For those of you thinking “wtf is a favicon”, it’s the little picture which might be shown next to the URL in the address bar, next to the website title or as part of the tab title – depending on your web browser. For example, if you’re really lucky, and look really carefully you might just see a very small picture of me!

Why are Favicons useful?

The main reason Favicons are useful these days is to promote brand familiarity – it’s the ideal place to put your company logo or some other recognisable little picture there. The problem is actually creating one – or at least, that’s what most people think. It’s actually really easy.

Favicons are stored as a .ico (Icon) file, which is really just a very small BMP file. There are a few programs which let you edit them easily but recently I came across this little gem:

http://www.rw-designer.com/online_icon_maker.php

By using this tool you can easily draw your own little logo, or upload a picture and have it automatically translate it for you. You can then easily save the icon and upload it to your website.

How to get the Favicon to display

Some web browsers support automatic discovery of favicons, that means all you have to do is upload your .ico file as ‘favicon.ico’ to the root directory of your site and it’ll do the rest – BUT – the best way to make sure is to also add the following tag into the header section of your website template, between the <head&ht; and </head> tags:


<link rel="shortcut icon" href="/favicon.ico" />

Adding a tag like this means the browser will be able to find your favicon wherever the user has navigated to (or entered) in your site.

Creating Animated Favicons

Some browsers, such as Firefox, have the ability to display animated favicons – these are just regular animated .gif files so they are even simpler to create. You can see an example of an animated favicon here.

To make your life simpler still, there’s an online generator for animated icons here! For maximum compatibility it’s always best if you’re going to include an animated favicon to also include a static one too because otherwise some browsers won’t show one at all.

To include your animated favicon along side your regular one, put the following code in your page header (as described above):


<link rel="icon" type="image/gif" href="/icon.gif" />

Just replace icon.gif with the filename of whatever you uploaded your animated favicon as, and there you have it.

Job done.

If you’ve ever been sat at a computer and thought “!*@%, I only need to resize (or crop) this picture”, but haven’t had anything to hand except MS paint, Picnik is the solution for you. If you want to quickly import and edit your photos from Flickr, Picnik is the solution for you. If you want to then export your saved photos to Flickr, Photobucket, or Imageshack – Picnik is for you.

Oh yeah – it can take pictures straight from your webcam too.

Picnik is written entirely in Flash so there’s no danger of worrying about Javascript/CSS incompatibilities with your browser. It’s just a slick, quick, and simple way to edit your photos. The main tools it sports are Auto-Fix (available for most filters), Rotate, Crop, Exposure, Colours, Sharpen and Red-Eye removal. Navigating through the list of tools brings up simple easy to use menus for each item.

Having played with Picnik for a little while it’s easy to see that there will certainly be an audience gagging to get their hands on it’s smooth rounded controls and cuddly interface, just because it’s Web 2.0. There is however a much bigger question potential audience available. Obviously Picnik isn’t a replacement for programs like Paint Shop Pro and Adobe Photoshop, but it certainly removes the need to have another image editor installed on your desktop for when you just want to resize something quickly, or change image saturation, colour balance etc. That is Picnik’s biggest selling point.

At the end of the day, if you’re someone who lives in Photoshop for it’s creative abilities – this isn’t going to help you much. Picnik doesn’t allow you to paint with the brush or fill areas etc, but if you only open it to resize your photos before you save them to your blog, visiting the Picnik site will save you about 30 seconds on the loading time and if you make use of their automatic publishing systems it’ll save you a whole lot more too.

Check it out at http://www.picnik.com