Tom's Blog

Back to Tom's Web

Sun, 09 Dec 2012

Wireless and Copper Routing on Windows

Ever used a laptop with a wireless and a copper network interface? Practically every laptop have both, so chances are you have. Have you ever wondered what happens if you plug in with copper, turn wireless on and have both interfaces configured with an IP address. I decided to do this and have have a quick look at the routing table to see what goes on.

TL;DNR windows is sane and uses copper is you configure both.

Here is a sample ipconfig with some slight anonymization:


Windows IP Configuration

Wireless LAN adapter Wireless Network Connection:

   Connection-specific DNS Suffix  . :
   IPv4 Address. . . . . . . . . . . :
   Subnet Mask . . . . . . . . . . . :
   Default Gateway . . . . . . . . . :

Ethernet adapter Local Area Connection:

   Connection-specific DNS Suffix  . :
   IPv4 Address. . . . . . . . . . . :
   Subnet Mask . . . . . . . . . . . :
   Default Gateway . . . . . . . . . :

Tunnel adapter

   Media State . . . . . . . . . . . : Media disconnected
   Connection-specific DNS Suffix  . :

Tunnel adapter Local Area Connection* 11:

   Media State . . . . . . . . . . . : Media disconnected
   Connection-specific DNS Suffix  . :


and here is the routing table:

>netstat -rn
Interface List
 14...10 0b a9 96 fc 2c ......Intel(R) Centrino(R) Advanced-N 6205
 11...00 21 cc ba 7f 38 ......Intel(R) 82579LM Gigabit Network Connection
  1...........................Software Loopback Interface 1
 13...00 00 00 00 00 00 00 e0 Microsoft ISATAP Adapter
 10...00 00 00 00 00 00 00 e0 Teredo Tunneling Pseudo-Interface

IPv4 Route Table
Active Routes:
Network Destination        Netmask          Gateway       Interface  Metric
     30         On-link    306         On-link    306         On-link    306         On-link    266         On-link    266         On-link    266         On-link    286         On-link    286         On-link    286         On-link    306         On-link    266         On-link    286         On-link    306         On-link    266         On-link    286
Persistent Routes:

Persistent Routes:


As you can see there are two default gateways (ie destination and the one with the lowest metric uses the interface, which is the copper one. So it turns out that Windows seems sane and favours the faster copper interface when both are configured and working.

As an aside on Linux using a default network-manager setup, is also seems sane. You get one default gateway but the copper interface is used if it is available. The default gateway changes when the network interface go up and down.

posted at: 20:51 | path: /2012/12/09 | permanent link to this entry

Thu, 11 Oct 2012

3g HTML Abuse

While waiting to be connected to my new ISP I was forced to use a 3G dongle for my main Internet connection. Yikes you say! Well it's not entirely crap, I get about 2M and ssh is mostly usable. It's like having a cable modem in 2000. As a side note giffgaff make this sort of thing really quite easy (and cheap). Anyway while browsing I noticed the usual crap low res images that appear when surfing on mobile phones, so I decided to do some investigation into what was going on. It turns out to just be fairly straight forward, all web traffic is just transparently proxied. So all that happens is a simple JavaScript injection.

Say there is a simple website with the following HTML:

<body><img src="0239-mod.png" /></body>

When I load that file I actually get the following:

<html><script src="" language="javascript"></script>
<head><title>Test</title></head> <body><img src="0239-mod.png" /></body> </html><script language="javascript">

Yikes was my first reaction, bmi.js was nice an un-readable, however if you reformat it a bit it's mostly readable, it's on pastebin if your interested. As an aside is actually a bogon network! Whois says "Address blocks listed with this contact are withheld from general use and are only routed briefly for passive testing." Well at least that are not using genuine allocated addresses. I'd be more worried of the mess it left the HTML in!

After a bit of googeling, it turns out it's being going on for a while, bmi is some bytemobile product. As ever a stackoverflow was most useful. It seems some operators allow you to turn it off.

posted at: 00:19 | path: /2012/10/10 | permanent link to this entry

Wed, 11 Jul 2012

Minimal pppd Setup For GPRS Dongles

So you have a mobile broadband dongle and you want to use it on Linux, one option is to just plug it in and from Gnome 3 is just works and can be set-up instantly with NetworkManager. However I require something more permanent, something that will start at boot, be less interactive and stay up. There is a lot of (mis)information out there about how to do this with various programs and scripts to copy and paste. Lots of wvdial configurations and lots of poking about in /etc/ppp creating and modifying files etc.

Thanks to the wonder that is Debian, it's quite easy to get a nice setup without copying and pasting loads of config and scattering a dozen files all over /etc. I wanted a nice minimal set-up that will cope with Debian upgrades without breaking horrendously. All you need to do it create a new file in /etc/ppp/peers called something sensible like /etc/ppp/peers/giffgaff. Here is my config:

user "giffgaff"
password "password"
connect "/usr/sbin/chat -v -f /etc/chatscripts/gprs -T"



holdoff 10
maxfail 0

You might need to change some of the above. It's got the gprs username and password. It turns out that Debian already has a chatscript for gprs included in the ppp package, so we don't need to mess about with that. We just pass in the APN to that script via the -T parameter. You will have to know these options, however the mobile-broadband-provider-info package contains a file (/usr/share/mobile-broadband-provider-info/serviceproviders.xml) that has all these in, this file is used for all the automatic set-up with NetworkManager. The rest of the options are quite straight forward and they are all listed in the pppd(8) man page.

Next is the automatically starting at boot, for this we edit the /etc/network/interfaces file. Just add the stanza below:

auto ppp0
iface ppp0 inet ppp
        provider giffgaff

Then the ppp0 interface will just start at boot and the usual debian ifup and ifdown commands will work. Something like vnstat will be good enough to capture and log the traffic.

posted at: 13:18 | path: /2012/07/11 | permanent link to this entry

Wed, 23 May 2012


TL;DNR - Buy Super Unleaded petrol! (or don't)

I usually buy my petrol from Tesco for my Celica, I mostly use their super unleaded 99 octane. I have tried both the normal and the super, I've not really noticed a massive difference, however the super feels better, but that could just be me imagining it. So I decided to do a not really scientific experiment to find out which one really is best. Is that extra 5p per litre worth it? I get petrol every week and an half or so and fill my 50 litre tank, so thats about £85 difference.

My Celica does not have mileage readout so I had to calculate it the old fashioned way. Which is: get the tank almost empty, fill it to the brim, take note of the odometer reading, drive until the tank is almost empty, fill it again to the brim and take note of the odometer reader again. That way the petrol sloshing about in the bottom of the tank is not counted. The odometer difference is the total distance and the amount filled the second time is the fuel volume. I did this twice, once with regular and once with super. Between each test I did a complete fill to make sure each test only had the fuel of that test.

On with the results:

Fuel Type Miles Litres Paid Cost per Litre Mileage Pounds per mile
Regular 334.5 mi 52.09 l £ 69.23 £ 1.329 29.19 mpg 4.832 mi / £
Super 333.3 mi 50.01 l £ 68.96 £ 1.379 30.30 mpg 4.833 mi / £

So as you can see the numbers are all very similar, the Super is 1 mpg better but as it costs a little more it actually works out the same! So I'm going to stick with super which feels better and gets be an extra 15 miles per tank.

posted at: 22:32 | path: /2012/05/23 | permanent link to this entry

Wed, 01 Jun 2011

Milk to Coffee Ratio

Whenever I buy coffee I usually get a small or a medium. Up to now I've always seemed to enjoy the small ones more. I've always assumed it was because a medium was too much of a good thing. Which has lead me to hardly ever get a large.

However I was thinking recently that maybe I was wrong and the small ones just taste better. This lead me to wonder why. Anyone that knows me knows that I love milk. I thought that maybe I liked more milky ones better. I decided to conduct an experiment, over a few days I got a small, medium and large coffee to measure how much milk will fit in each one.

I didn't bother measuring the volume of the paper cups I just read the volume from some text on the bottom of the Starbucks cups, assumed each shot was 30ml and I also did not take into account milk foam or dead space at the top of the cup. Anyway here are the results:

Cup Size Cup Volume (ml) Shots Coffee Volume (ml) Total Milk (ml) Milk to Coffee Ratio
Large 605.8 3 3*30=90 605.8-90=515.8 515.8/90=5.73
Medium 473 2 2*30=60 473-60=413 413/60=6.88
Small 355 1 1*30=30ml 355-30=325 325/30=10.83

I can now conclusively state that the milk to coffee ratio is highest for small cups, almost double that of the Medium and Large. Where as Large and Medium are relatively similar. Further investigation will be required to measure the cups and shot sizes more accurately. As well as real mugs rather than paper ones. As a side note getting a medium with one shot will result in a milk to coffee ratio of around 15, which could even be too milk for me! However a Large with two shots comes in around at 9 which is similar to the small ratio.

For now on I will just get smalls and if I want more I'll just buy two cups!

posted at: 01:09 | path: /2011/05/31 | permanent link to this entry

Tue, 29 Mar 2011

vimrc for nagios

I use nagios to monitor servers. It's great. I just thought I would share a quick snippet that I just put in my vimrc. It sorts the servers listed in host_name lines so they are in alphabetical order. Put the following in your vimrc, move your cursor over a host_name line and hit F2 and it will sort the server list.

map #2 V:! sed 's/host_name//' \| sed 's/[ \t]//g' \| tr ',' '\n' \| sort \| uniq \| xargs \| sed 's/ /, /g' \| sed 's/^/\thost_name\t\t\t/'

posted at: 16:57 | path: /2011/03/29 | permanent link to this entry

Mon, 28 Mar 2011

Petrol Price

Petrol Prices are allays on the rise. I decided to do some analysis on some raw data comparing the price of Brend Crude Oil and the price of petrol. The details of the analysis is on my site. I found out that there is a direct correlation between the price of Brent crude oil and the price of petrol at the pump excluding duty and VAT.

I concluded that if the price of Brent crude oil stays at $115 per barrel then the price of petrol will increase to 136.14 +-3.92 pence per litre, it being around 125 at the time of the analysis.

posted at: 21:30 | path: /2011/03/28 | permanent link to this entry

Fri, 04 Mar 2011

Spotify Traffic Analysis

A colleague asked me how much bandwidth Spotify uses. I basically had no idea. I want to run Spotify on my mobile at some point it so it got me thinking. I decided to do some basic analysis. I ran the client behind a http proxy for a day or so and ran a tcpdump at the same time with a filter to capture all the traffic to the proxy. The dump ran from 19/10/2010 11:46 to 20/10/2010 17:19 and produced a 362M capture file. For the most part I was not playing any music, then for the last few hours I played music I had not played before.

I then started trying to analyse the dump. As amazing as Wireshark is it's really not too good at looking as large dumps or large amount of data. I then found Chaosreader which made it very easy. It basically looks at one big pcap file and splits it into the individual tcp steams, analyses them and produces a nice html report. Check out the example report for some examples of what it can do.

The Spotify client tries a few ways of connecting. It first tries to connect to a server on port 4070, which fails because the proxy does not allow CONNECT's to that port.

s: HTTP/1.0 403 Forbidden

Next it tries port 80, which is also blocked.

s: HTTP/1.0 403 Forbidden

And finally it tries 443 and establishes a ssl connection. It is this connection that lasts for the duration that the Spotify client runs.

s: HTTP/1.0 200 Connection established

I'll get back to the main stream of data in a minute. While the client is running it makes various requests to some other sites. I think these only happen when Spotify social is turned on. There are many many short lived connections to Twitter and Facebook.

User-Agent: Spotify-Win32/0.71/40800213
Keep-Alive: 300
Connection: keep-alive
Accept-Encoding: gzip

It made 354 connections and GET requests to the above twitter json api. It also made 89 connections and GET requests to the below Facebook api. This works out at about 12 connections per hour for twitter and 3 connections per hour for Facebook. Which all sounds quite reasonable.

User-Agent: Spotify-Win32/0.71/40800213
Keep-Alive: 300
Connection: keep-alive
Accept-Encoding: gzip

Now back to the main stream. Once the client gets a connection to the Spotify servers and it establishes what looks like an encrypted stream. Once this happens the entirety of the conversation with the server looks like gibberish. This is a bit annoying as I can't really look further into how it works. However I'm still able to see how much data is transfered and thus can calculate average data rates.

I looked at the TCP stream between 0:00 and 6:00. The music was stopped for the whole of the time. As you might expect not much activity happened during this time, 7.5 KiB was sent to the server and 9.7 KiB was received. The average packet size was 77 bytes long. The overall data rate was 5 bytes per second, ie 0.005 KiB/s. Almost no traffic!

I also looked at the half hour window from 12:00 to 12:30 where I played music I had not played before and thus was not cached locally. Spotify relies heavily on local cache, ~/.wine/drive_c/users/thomas/Local Settings/Application Data/Spotify/Storage is currently 1.2G on my work desktop where I don't have any off line play lists. During this 30 minute window, 10.2 KiB was sent from the server and 26.4 MiB was received. The average packet size was 775 bytes long. The Overall data rate was 16.5 KiB/s. This is alot less than I expected.

After looking at the Wikipedia list of mobile telephone bandwidths I conclude that to run Spotify on a phone both GSM and GPRS are too slow, 1.8 KiB/s and 7.2 KiB/s respectfully. Only EDGE and faster is good enough at 48KiB/s. Of course this assumes that the mobile client uses the same bandwidth. The other thing to note is that an 8M internet connection should be able to sustain 62 Spotify users, assuming that they are all listening to new music.

(Note: I did most of this back in October 2010 but only got round to writing the concluding 4 paragraphs today.)

posted at: 16:59 | path: /2011/03/04 | permanent link to this entry

Tue, 04 Jan 2011

New Year's resolutions

This year seems to have flown by so it's resolutions time again. I've done quite well, I manged to stay relatively healthy and read loads of books but I failed completely to keep up to date with LWN.

Including the books from July I've manged to read 18 books, which is ace! Please make recommendations.

As for this years ones, It's almost going to be the same.

posted at: 20:56 | path: /2011/01/04 | permanent link to this entry

Wed, 06 Oct 2010

Major Filesystem Corruption

On Sunday (03/10/2010) my laptop has a bad crash. The screen went blank, I could still see the cursor and move it around but could do nothing else. I could not even login on a virtual console, after I typed the user name it came straight back with a login prompt again. I rebooted, grub loaded, Linux ran, the initramfs loaded and started and then it moaned about not being able to mount the root filesystem and dropped me to a prompt.

After this I decided to pull out my GRML recovery cd. What happened next was what only can be defined as Major Filesystem Corruption. I ran the usual "sudo fsck -C -t ext3 -f -y /dev/mapper/ikaite-root" inside a script session to capture the output. This grew to 42M (a clean fsck is about 150k of script output). After it finally finished I ran it again and it came back clean. After I mounted the filesystem there were no files left and 6000~ entries in lost+found. Looks like I'll be reinstalling it!

I then did some minor analysis on the fsck output. First I wanted to see what the common errors and questions were. A quick use of sed to remove numbers enabled me to cound them up with: "cat fsck.root | sed 's/[0-9][0-9]*//g' | sort | uniq -c | grep -v "^ 1" | sort -n | tail -10" produces:

  32901 Inode  has imagic flag set.  Clear? yes
  34415 Illegal block number passed to extfs_test_block_bitmap # for multiply claimed block map
  36479 Inode , i_size is , should be .  Fix? yes
  40991 Inode  has a extra size () which is invalid
  41422 Inode , i_blocks is , should be .  Fix? yes
  41438 Inode  is in use, but has dtime set.  Fix? yes
  45689 Fix? yes
  61402 Illegal block # () in inode .  CLEARED.
  73733 Clear? yes

Something else I noticed were many "WARNING: PROGRAMMING BUG IN E2FSCK"! messages, 23 of them!

posted at: 10:36 | path: /2010/10/06 | permanent link to this entry

Thu, 16 Sep 2010

ODBC 2 OpsCenter

OpsCenter is a Symantec product that makes reports for various backup products including Symantec NetBackup. I recently decided that I wanted to improve the reporting of our backups so looked into paying with OpsCenter. It turns out that it's not actually that configurable which is a shame as it's mostly quite good.

It does however use a standard RDMS that appears to be an OEMed version of Sybase's SQL Anywhere to store the data. I thought, great now I'll be able use my own tools to query and report on the back. I want to make pretty pie charts, report on total data transfered and as well as actual important thinks like jobs that have failed.

I did a bit of light goggling without much success, however I was convinced that I could configure an ODBC driver on windows to talk to the database so that Crystal Reports could create said pir charts!

I then remembered, we pay quite a substantial amount of money to Symantec for support. I thought, amassing I'll just ask them, they can tell me how to configure the ODBC driver and save me a bunch of time figuring it out! Roll forward one months of emails (they don't seem to like the phone), and I got this conclusion

On 2010-09-13 06:21, wrote:
> Further to our discussion regarding this case, it has been confirmed
> by backline that we can only provide you the ODBC drivers for the
> OpsCenter(As ****** has already sent to you), however unfortunately we
> cannot support or provide the technical know how regarding integrating
> OpsCenter database with any third party frond end reporting software
> like Crystal Reports.

So much for Support! The age old saying, if you want something done, do it your self, thus I have documented ODBC2OpsCenter for any one else out there that needs it! I really wish I could use something more free to do all this stuff.

posted at: 21:50 | path: /2010/09/16 | permanent link to this entry

Fri, 27 Aug 2010

SMS Notifications for Nagios

I use Nagios for monitoring. Up to recently I used a regular modem to send sms text messages to various people when systems are going wrong. The way this works is by using smsclient which dials up to a TAP server. TAP is a fairly archaic way of sending messages. It's been fairly reliable however it has two major drawbacks, sending takes along time and it's limited to 160 characters. As far as I can tell it will not do long text messages. Which are really just multiple short message combined together in a special way.

I wanted a better way! Something that sounded nice was using the mobile network to send messages, that way I could bypass most of the internal infrastructure and take the notifying system as out of band as possible. After a good bit of research I finally settled on a Masetro 100 from RF Solutions. It also turns out that smstools can use it out of the box. I'm so pleased!

posted at: 16:10 | path: /2010/08/27 | permanent link to this entry

Thu, 29 Jul 2010

Weird Traceroute

I was looking at a development web site I am involved with and I was interested in where the site was in the big bad world, so I decided to traceroute to it[1]. What seemed very unusual was that the 5th hop reported an ip address in the private address space. To quote Sam "10.what now?". I'm still amazed that packets with source addresses with private addresses are routed across the internet![2]

$ sudo traceroute -m 10 -N 1 -M default
traceroute to (, 10 hops max, 40 byte packets
 1 (  0.389 ms  0.174 ms  0.154 ms
 2 (  0.366 ms  0.440 ms  0.326 ms
 3 (  6.822 ms  6.707 ms  7.314 ms
 4  * * (  11.099 ms
 5 (  9.041 ms  8.237 ms  7.706 ms
 6  * * *
 7  * * *
 8  * * *
 9  * * *
10  * * *
$ sudo traceroute -m 10 -N 1 -M icmp
traceroute to (, 10 hops max, 40 byte packets
 1 (  0.216 ms  0.134 ms  0.092 ms
 2 (  0.404 ms  0.382 ms  0.451 ms
 3 (  8.107 ms  7.459 ms  7.462 ms
 4 (  7.653 ms  7.440 ms  8.356 ms
 5 (  10.940 ms  20.268 ms  9.236 ms
 6  * * *
 7  * * *
 8  * * *
 9  * * *
10  * * *
$ sudo traceroute -m 10 -N 1 -M tcp
traceroute to (, 10 hops max, 40 byte packets
 1 (  0.323 ms  0.437 ms  0.200 ms
 2 (  0.446 ms  0.333 ms  0.391 ms
 3 (  7.120 ms  6.713 ms  6.800 ms
 4 (  7.925 ms  7.737 ms  7.881 ms
 5 (  7.816 ms  7.500 ms  8.614 ms
 6 (  21.213 ms  21.239 ms  21.242 ms
 7 (  20.770 ms  21.662 ms  22.061 ms
$ sudo traceroute -m 10 -N 1 -M udp
traceroute to (, 10 hops max, 40 byte packets
 1 (  0.276 ms  0.096 ms  0.091 ms
 2 (  0.444 ms  0.409 ms  0.403 ms
 3 (  6.747 ms  6.797 ms  6.844 ms
 4  * (  7.620 ms  7.488 ms
 5 (  8.766 ms  7.501 ms  8.638 ms
 6  * * *
 7  * * *
 8  * * *
 9  * * *
10  * * *
$ sudo traceroute -m 10 -N 1 -M raw
traceroute to (, 10 hops max, 40 byte packets
 1 (  0.314 ms  0.097 ms  0.093 ms
 2 (  0.378 ms  0.412 ms  0.330 ms
 3 (  23.388 ms  8.331 ms  7.084 ms
 4 (  8.294 ms  7.517 ms  8.027 ms
 5 (  8.502 ms  7.589 ms  7.450 ms
 6  * * *
 7  * * *
 8  * * *
 9  * * *
10  * * *
$ tshark -r cap -R "ip.src =="
177   8.113729 -> ICMP Time-to-live exceeded (Time to live exceeded in transit)
183   8.122267 -> ICMP Time-to-live exceeded (Time to live exceeded in transit)
187   8.130944 -> ICMP Time-to-live exceeded (Time to live exceeded in transit)

posted at: 22:56 | path: /2010/07/29 | permanent link to this entry

Thu, 22 Jul 2010

Spinning Phrases

I go to spinning quite a bit and have had I think six teachers over the last year since I started. I think every teacher has one phrase they use most, this not a bad thing, it's just something I have noticed. I present an anonymous list of them here.

posted at: 21:36 | path: /2010/07/22 | permanent link to this entry

Mon, 05 Jul 2010

Resolutions and Reading

Since Christmas I have been up to loads. I went skiing in March and I went to Cuba in June. I have done well with resolutions, I managed to stay super healthy for January and managed to mostly keep to it since. However I utterly failed to read LWN :-(

I did however keep up the reading and by my standards I have read loads! So far I have read:

I'm currently reading The Magicians - Lev Grossman. I'm also almost finished the backlog of books everyone suggested after Christmas, so any recommendations we most welcome!

posted at: 22:06 | path: /2010/07/05 | permanent link to this entry

Mon, 04 Jan 2010

New Year's resolutions

It's time for some new years resolutions! Hopefully if they are cast in stone (well blog) there is more of a chance of success.

posted at: 23:35 | path: /2010/01/04 | permanent link to this entry

Fri, 27 Nov 2009

Intermediate Exam

I passed my amateur radio Intermediate exam, so now I can kick out 50W! My new call sign is 2E0TDS. My thanks goes to all at SADARS that helped with the practical day and two days of lectures and exams. All I need now is some sort of radio!

posted at: 15:22 | path: /2009/11/27 | permanent link to this entry

Fri, 20 Nov 2009

Dublin Trip

Last week I got back from a little holiday in Dublin. I went over for my cousins graduation. I had an amassing time, it was a very relaxed and slow paced get away from everything. We went to all sorts of different cafes, restaurants and live music bars in the evening. It felt so luxurious to stay in a hotel in the centre of town, we were able to roll out of which ever bar we were in, hail a cab and be back home in no time. Coming back to a made bed and fresh towels can't be beaten.

A few places stood out. One night we went for a Mexican in Acapulca which was amazing in it's self but it was topped off by their "Deep Fried Ice Cream" desert. We also went to a restaurant type cafe one day. I can't remember the name of it now, but it served the best hot chocolate I have ever had. Then there was the Queen of Tarts which is a lovely little cafe with amazing carrot cake.

We did not stay in the hotel for the whole time, the last night one of my cousins friends put us up in their amazing house. They took us to a very strange but also very cool place that only served tea. It was called The Tea Garden. It had one of the most relaxing atmospheres I have been in. It was divided up into small booths with raised floors, cushions and small tables. We took our shoes off and sat round the small table and just relaxed! The waitress then came round and was able to recommend which teas to try. It was cool socialising without booze. When the tea came the pots had little night lights to keep them warm. We then just whiled the time away chatting.

It was not all food and drink mind, we did fit in a trip to the book of kells. We finished the trip off on the last day with an late afternoon visit to Bewley's.

posted at: 21:58 | path: /2009/11/20 | permanent link to this entry

Fri, 09 Oct 2009

GPG Bench Cipher

After reading depesz's great speeding-up-dumprestore-process blog post. I started thinking about how to securely transfer a file from one server to another in the fastest possible way. The problem being that scp/sftp is slow for various reasons. ftp, http, nc, cifs, rsync are all plain text so can be quickly discounted. I don't know if ssltunnel suffers from the same window limitations that ssh suffers from. I guess using a dedicated vpn would do the trick. However I liked the idea of using gpg. I was not sure which cipher to use so I decided to run a few benchmarks to see. The results are on my wiki. Once the encrypted file has been created then it can be transfered using any of the available plain text mechanisms. I think nc or ftp have the least overhead.

posted at: 16:53 | path: /2009/10/09 | permanent link to this entry

Tue, 15 Sep 2009

Big Disk

So today I played a bit with some big cheep disks. I have 3 fairly old desktops each with 4*1TB disks, all exported via ATA over Ethernet to a much more modern sort of disk head. Basically it's a "build a large store on half a shoe string" project. I've not quite got the network side of it sorted yet. Currently the disk head's gigabit card is being saturated. On a single disk node each disk can do about 80M read sustained. If all 4 are read at the same time it goes down to about 50M, which is 200M in total. Which seems quite amazing for a Pentium 4 3.0Ghz. Also this seems a bit high as the PCI bus can only do 133M, I'm guessing that the onboard sata ports are somehow separated from the extra pci sata card I added. Interestingly one disk node can sustain 80M read from disk to the disk head. Again this backs down to 30M if all 4 disks are read, thats 120M total, no surprise this is saturating the gigabit link. So the major bottleneck is the disk head. Currently max raid sync speed is 10M, ie 120M total for 12 disks. Ideally 3 nics in the disk head would be best, but then there is no way for the data to get to the disk head.

The /proc/mdstat seems quite impressive (slow rate due to mkfs):

md0 : active raid5 etherd/e3.12[12] etherd/e3.11[10] etherd/e3.10[9] etherd/e3.9[8] etherd/e2.8[7] etherd/e2.7[6] etherd/e2.6[5] etherd/e2.5[4] etherd/e1.4[3] etherd/e1.3[2] etherd/e1.2[1] etherd/e1.1[0]
      10721828480 blocks level 5, 64k chunk, algorithm 2 [12/11] [UUUUUUUUUUU_]
      [==>..................]  recovery = 13.4% (130652288/974711680) finish=158828.3min speed=87K/sec

The mkfs looks like it won't finish for ages, also theres still not definitive information about stride, stripe and other random things.

$ sudo mkfs -t ext3 -E stride=16,stripe-width=176 /dev/soda/store 
mke2fs 1.41.3 (12-Oct-2008)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
670121984 inodes, 2680456192 blocks
134022809 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=0
81802 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks: 
        32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208, 
        4096000, 7962624, 11239424, 20480000, 23887872, 71663616, 78675968, 
        102400000, 214990848, 512000000, 550731776, 644972544, 1934917632, 

Writing inode tables:  3351/81802

posted at: 22:40 | path: /2009/09/15 | permanent link to this entry

Wed, 12 Aug 2009


So my gym is closed for 10 days for a refurbishment. They are going to replace all the cardio equipment with awesomely new kit. I'll be able to check my progress online and I'll be able to plug in a video iPod and watch on the screen. The problem is, while it's closed I have no gym to go to! So I decided to try some classes instead. Tonight I tried a Yoga one and it was great, both relaxing and tiring at the same time, I really enjoyed it. I think I'll be going more often in the future and hopefully it will help improve my skiing.

posted at: 22:07 | path: /2009/08/12 | permanent link to this entry

Mon, 03 Aug 2009

Wedding Meal

I went to Tony and Grainne and Tonys wedding on Friday. It was a lovely day and I wish them all the best in the future.

Anyway, I learnt something while at the hotel that annoyed me. The name of the meal at the wedding reception. Hint: it's not lunch. It's called the Wedding reception, I'm not entirely sure why, but it really does get on my nerves.

posted at: 20:16 | path: /2009/08/03 | permanent link to this entry

Fri, 24 Jul 2009

I'm back from Ireland

I got back on Monday night after cutting it worryingly close to get to the airport, 2 hours in advance try 40 min! Only for Ryanair to delay the flight by an hour and tell no one. Then there's the Post Holiday Blues, aka PTD. It hit bad on Tuesday night but I think I'm used to the grind stone again now.

Also I'm not laughing now:

$ psql -c "select count(*) from rssr.entries where read = false;" rssr
   (1 row)

In other news I might be going to see "STOCKWELL: THE INQUEST" in the Landor Theatre on Monday night. My post holiday resolution is to go to London for more culture.

posted at: 00:18 | path: /2009/07/24 | permanent link to this entry

Thu, 02 Jul 2009

I'm off on my holidays

No Internet for 2 weeks, just as I catch up on my feeds too!
$ psql -c "select count(*) from rssr.entries where read = false;" rssr
      (1 row)

posted at: 23:53 | path: /2009/07/02 | permanent link to this entry

Wed, 24 Jun 2009

Cineworld have an public xml data source

I just found out that Cineworld have a public xml data source. It's over in the syndication subdir of their main site. There are more technical details over on my CineworldScrape page. I feel a bit sad in a way, my scraper has been working for more or less 3 years now in some form or another. The annoying thing is that they don't actually export all the data I use. Being quite visual I have grown used to using the thumbnails that each movie has to help me pick which films to see. Apart from this it's a great step forward. However the problem is there's know knowing if Cineworld will pull this service or even if they want it public. For now I'll continue to use my scraper.

posted at: 21:30 | path: /2009/06/24 | permanent link to this entry

Thu, 18 Jun 2009

Passed Foundation

I passed my foundation exam on Sunday. Big thanks to all the helpfull folk at SADARS. I am now M6TDS.

posted at: 17:05 | path: /2009/06/18 | permanent link to this entry

Sat, 13 Jun 2009

ICMP Redirect

Today I found out where Linux exposes the extra routing information gathered from ICMP redirects. "ip route show cache" will show the entire cached routing table. It's a bit hard to read so "ip route show cache" is better. For example is a network the is connected via a host on my 192.168.0/24 network. My default gateway ( has a static routing entry to the host who gateways for the network ( So When a random host on the network pings a host on the network it first sends to but it sends an ICMP redirect saying that in the future it would be better to just send direct to in the future.
$ ip route show cache
$ ping
PING ( 56(84) bytes of data.
64 bytes from icmp_seq=1 ttl=63 time=2.25 ms
From icmp_seq=2 Redirect Host(New nexthop:
64 bytes from icmp_seq=2 ttl=63 time=2.34 ms
64 bytes from icmp_seq=3 ttl=63 time=1.32 ms
64 bytes from icmp_seq=4 ttl=63 time=1.24 ms
--- ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3012ms
rtt min/avg/max/mdev = 1.241/1.791/2.344/0.511 ms
$ ip route show cache from via dev eth1 
    cache   mtu 1500 advmss 1460 hoplimit 64 via dev eth1  src 
    cache   mtu 1500 advmss 1460 hoplimit 64
The Guide to IP Layer Network Administration with Linux is an excellent guide!

posted at: 15:46 | path: /2009/06/13 | permanent link to this entry

Fri, 12 Jun 2009

Libata Errors

I learned over the years to read the old ide subsystem errors for Linux and genrally am able to get a feel for the sort of hardware error thats coming. However i have yet to really get a feel for the libata errors, I'm not really used to reading the errors. I friend linked me to a page on the lib ata wiki

posted at: 14:48 | path: /2009/06/12 | permanent link to this entry

Sat, 06 Jun 2009

Fedora needs legwork

I played with fedora more today. There are loads of packages in the default repos these days. However I still find my self missing things from Debian. For example, I can't do a "yum search" while doing a "yum upgrade". Why? I also miss the small helper scripts that save on legwork. For example update-grub, in debian when a new kernel gets installed the initrd and grub gets updated and just works. With Fedora I have to manually update the grub config and if say I forget to create the initrd it, it means a trip to the 13 PC's I just upgraded!

posted at: 00:06 | path: /2009/06/05 | permanent link to this entry

Sun, 31 May 2009

CineworldScrape released to public

I have finally documented cineworldscrape and made it available to the general public. So with any luck google will index it!

posted at: 20:51 | path: /2009/05/31 | permanent link to this entry

Wed, 27 May 2009

First Post!

Finaly a I have a blog!

posted at: 19:27 | path: /2009/05/27 | permanent link to this entry

Powered by PyBlosxom