Planet LSU Computer Society

B-roll footage

2009-01-02T02:04:13+00:00

I've been thinking for a long time about B-roll footage. It takes a long time to edit video so that the audio lines up in both sources, then to cut between them, i.e./ One audio source, two video feeds and you choose "the best".

From what I understand, to edit a simple 5 minute piece of footage like this takes about an hour, replaying the same piece over and over until you can find the right splice points - or you're just very experienced and can split it at the right moments, risking a potentially dodgy cut.

I've been learning Python3.0 recently, with the eventual goal of being able to do the following in a nice little programme:

Play a section of footage, roll A. Hit a cue button when something simultaneously happens on both cameras/audio feeds.
Play another section of footage, roll B. Hit a cue button in the same circumstances. You now have the two pieces of film synced.
Create a grid of four spaces: Top Left is Roll A. Top Right is Roll B. Bottom Left is Output, Bottom Right is a summary of useful information (Timecodes, Project Name, Intended Duration, Editors Notes etc)
Play the footage, and have be able to hit the space bar when you'd like a cut - i.e./ you think a cut is needed there.
When reaching the end, hit a button that allows you to easily fine tune the cuts - it takes you to 2 seconds before the first cut, and you can pan through the footage with the mouse wheel to the exact frame you want to splice, and move the cut there. Rinse and Repeat.
Now optionally choose an audio feed you'd like to use, be it seperately recorded audio, or a feed from one of the cameras.
Then, you're handed an XML save file, which has a list of the attributes associated with that footage.

You can then go and edit the next piece of footage. What would happen is that on a second machine, you can drop in the two reels, the XML file and run a programme. You run a piece of software, which reads in the footage as necessary and either re-renders it to an output file, or in the case of formats that supports it, just chops and concatenates.

What you're left with is a piece of footage you can import into a video editor and remove the flubbs etc, without having to worry about desyncing the video stream.

Seems quite simple to implement, just wondered what other people thought - a worthy project? It could quite easily be extended to use more than two streams, or even be turned into a "gallery" style programme dealing with live programming outputing to a save file and a stream out to VLC or something...

Thoughts, questions, comments welcomed.

Drupal Modules

2008-12-12T20:39:00+00:00

I thought I might as well ‘publish’ two Drupal modules that I’ve been working on.

Account Reactivation (Drupal 5)
This was created for LSU CompSoc’s new site, to enable members to request that their account is re-activated at the start of the new year, once they have signed up again.

Download the module here: Drupal Account Reactivation Module

Timetable (Drupal 6)
This is a basic module I created as a test module to get back into Drupal development. It creates a timetable, with the ability to add and remove items via a form, and to set the slots available during the day on the timetable.

Download the module here: Drupal Timetable Module

————————————————————-

Please note that both these modules are provided ‘as is’ and may not be completely stable/include bugs . Do not use on a production site without first testing thoroughly.

However I will provide help/improvements/fixes if time allows for the modules.

Should Arrows be Placed Before Link Text or After?

2008-10-30T11:20:08+00:00

An interesting article at the Usability Post poses the question "Should Arrows be Placed Before Link Text or After?" I only tend to use arrows with previous / next links where I've always felt that the arrows should point away from the links but there are some interesting views from other people here too...

Robots - To index or not to index?

2008-09-25T12:11:20+00:00

I'm having a look around the site at the moment and trying to think what I can do to help search engines spider MilkHub better. After reading about sitemap files I've started creating my sitemap.xml and I'll be fleshing that out a bit over the next week or so and pinging that to the search enginges. What else can I do?

Dear Lazyweb (CentOS)

2008-08-27T14:48:40+00:00

Good evening Lazyweb,

After many hours of research, I'm still none the wiser, and I thought I'd ask around first before making the "CentOS sucks" decree.

A system was running on a VMWare instance using CentOS 5.1 x86_64, which had LVM installed.

B system was installed with CentOS 5.1 x86_64 without LVM.

Both systems were booted from live CDs then the following command was run:

rsync -avze 'ssh' --progress --delete --exclude /dev --exclude /boot --exclude /etc/fstab --exclude /etc/mtab --exclude /sys --exclude /proc --exclude /lib/modules source:/mnt/source/ /mnt/dest/

Upon reboot, the system comes up, and all works fine. We then decide "oooh, old kernel, upgrade time!" and get CentOS to install the new kernel. It doesn't boot. The kernel shows errors related to not being able to mount /dev/root and it's complaining it can't find /dev/VolGroup00/LogVol00 - because it doesn't exist!

Ok, so it appears to have overwritten /etc/fstab. Modify that, modify /boot/grub/menu.list (well, grub.conf which is a symlink) to /dev/cciss/c0d0p1 etc, and still, not booting :S

All I can presume is that the initrd is stale, so I regenerated it with mkinitrd, and the same happened. I guess CentOS works "very very differently" to Debian/Ubuntu systems I've tried in the past.

Any ideas?

cURL Error 26: Failed to open/read local data from file/application

2008-08-26T11:57:59+00:00

Last night I was battling against cURL Error 26: "Failed to open/read local data from file/application" and eventually found a very simple solution...

Dear Lazyweb (Nagios)

2008-08-13T12:15:34+00:00

I'm having an issue with Nagios (or more specifically, Opsview).

I've set-up a system, but it's only monitoring hosts every 5 minutes. As it's monitoring a routing infrastructure, I'd ideally like to monitor every 1 minute - but without using SNMPtraps (which I'm sorting out later).

Any ideas how to configure that in either Nagios or (preferably) Opsview?

Image Replacement + Google

2008-07-05T21:01:36+00:00

Dave Shea asked Google the all important question "what does Google think of CSS image replacement, anyway?" to which they answered "Google's okay with it, you won't be penalized for using image replacement properly."

How-To: Get your Growl back from Logitech

2008-07-05T20:58:36+00:00

For the past few days, since running some updates on my MacBooks, Growl's not been showing any messages. I've since found out that it was Logitech Control Center blocking the messages. The solution is quite simple, delete /Library/InputManagers/LCC Scroll Enhancer Loader then log out and everything should be fine. Thanks Zamboni.

Note (9 June): Growl 1.3.0 has a work around to combat this problem.

New Growl works with Logitech

2008-07-05T20:57:22+00:00

Today Growl released version 1.3.0 which has added a work around for Logitech Control Center 2.4. No need to Get your Growl back from Logitech any more!

Tracking visitors with Google Analytics

2008-07-05T20:56:41+00:00

Sorry to anyone who got a few old posts through from my blog last week. I've been playing with the Google Analytics code a bit and trying out new things, one of which is a way of tracking when people have come to my site after clicking a link in one of my feeds. I found an interesting article about campaign tagging which describes how to add extra query string parameters that Google Analytics will read and use to split up my statistics for me.

If you're adding my feed to an aggregator or a public listing please feel free to add a utm_source to the url so I can tell where people are coming from, as well as which type of feed they're coming from; I will pass through the utm_source to the links in the feed. For example, I've got this in the facebook importer: http://www.milk-hub.net/blog/feed?utm_source=facebook.

Next step is to track when people read the articles.

The Best Way to Distribute Files?

2008-06-19T07:17:19+00:00

I was thinking abou the Firefox 3 distribution method, and whenever anyone wants to distribute large files, the immediate answer is "use BitTorrent" by people who generally don't know what they're talking about. To these people, because they have 100 people talking on a torrent instead of just 1, you get faster download speeds.

The trouble is threefold:

It relies on the path from you to an uploader being free. ie/ if you have to pay for out-of-country net access like the Australians do, you can quickly find yourself with huge bills.
It relies on the network not having a bottleneck. Obviously, with me in Europe, the maximum I can download is goverened by the transatlantic links - which aren't a problem, until you consider everyone else in the world wants a copy of this file (or should...) and the links become saturated quickly if it weren't for an Akamai style HTTP download distribution platform. Put simply - distance is not taken into account.
As connections are normally limited to somewhat lower than the connection can take, your computer doesn't care whether it's downloading from the West Coast US or from next door.

So, I have a few proposed extensions to the BitTorrent protocol:

When setting up the tracker, a copy of the latest BGP table is downloaded. People in the same AS get to know of each other's peers first, and as availability drops, the number of "hops" to the next peer increases until it is globally available.
ISP's could offer a service whereby a BitTorrent cache can exist, hosting chunks of files that popularly pass through the net. Obviously, the ISP will not know what these chunks represent (saving them from a liability point of view) and less traffic goes out over one of their expensive transit connections. Theoretically, this could be at the DSLAM level (for DSL) so that if a movie was being distributed, it wouldn't have to touch ANY of the internet.
Using an independant PKI key, upload and download figures can be tracked and stored on some kind of service, showing which people upload/download the most (ratio wise) and people who upload more get first pick on downloads - ie/ if two people claim the same file, and have the same hop count, the ratio is checked on this third party site. It *has* to be third party, and only uploadable from trackers as some people are bastards and run clients that report false information.

Any thought, questions, comments, additions?

Prevented from re-joining VATSIM

2008-06-04T21:52:52+00:00

I've not had any contact with VATSIM (a virtual air traffic simulator) since Mike Dean stopped having weekend ATCathons in the hotel in Paignton. There's a very clear reason why I don't participate any more, and that's an email I received from the then VP Deployment about my offer to help do some coding on a new cross-platform radar client:

While no developer is paid, the property still remains with VATSIM as it's for its core server facilities, so we cannot have "competing" networks look at the crux and possibly benefit by our work.

VATSIM is not ever going to charge for the services, so the developers are basically donating their time (as am I) for the betterment of our network.

If you're not interested in non-open-source, please let me know.

After an email trying to explain that the code would still be owned by VATSIM, I'm not advocating the GPL to them, just the idea of sharing sources so that others can write applications to talk to their (modified) FSD server, I get the response:

[Since] this is regarding our servers, there's higher importance in ensuring that the "competition" cannot usurp the sensitive parts of it.

Fair enough. I attended Mike Dean's event with a Mac laptop and specially installed Parallels on the laptop so that I could run Windows XP. When the trial expired, I stopped using it. Kees Leune also broadcasted this message to VATSIM members:

2 jan 2005: VATSIM DROPS SUPPORT FOR PCPROXY: By word of Richard Critz, the Vatsim Board of Governors announced today to a large group of developers that Vatsim chooses to concentrate all their development efforts on the development of SB3 and FSInn. In addition, the Board of Governors states that they do not believe that Open Source development is desirable or useful for Vatsim.

As a matter of Policy, Vatsim only allows software which has been formally approved.

Jamie Fox, who also attended these events with me, wrote to me in Feb 2005 saying:

Some time ago I remember Richard Critz saying that the reason for their
secret protocol is that the servers are insecure, and that knowing the
details of the protocol would make the servers vulnerable. In the short
term that made sense, although the long term solution is obviously to
fix the servers, and that was well over a year ago now.

There is a *lot* of anti-open-source rhetoric on VATSIM (or at least there was when I was active) and I didn't like the fact I couldn't improve it myself. Not just that, but because I had no source code, and the controller clients are closed source, you couldn't recompile them for another operating system, leaving my mac laptop and linux desktop out in the cold.

I looked at the situation again today, and there are still no cross-platform clients, although there are some *seriously* nice looking Windows cients. If Cider (I think that's what Wine is called on OSX) reaches an acceptable standard, I may well join in, but I'm not liking the vibe put out.

Which is a real shame, because VATSIM-UK is one of the most pleasant organisations I've ever helped with - VATSIM-UK are the UK administrative division of VATSIM, and run all the UK airspace control, and I spent literally hundreds of hours on East Midlands Approach, Tower, Ground and indeed flying trusty Donair 24 around the local area. The staff were incredibly friendly, the weekends were fantastic, and the support was tremendous. It saddens me that in my self-imposed exile from the project (initially due to final year exams at Uni, now due to no available client) I'm missing out on something truly remarkable.

If it was as easy as "move to IVAO" or "start your own network" I would, but the VATSIM-UK team made the project, and I remember all of them with fond memories. Maybe, when I move house and get my Linux desktop up and running again, I might give Wine a stab at Euroscope/VRC/ASRC again, who knows ;)

Let Google Growl!

2008-06-01T14:38:27+00:00

I've been using the Google Notifier for quite a while now and it works great except for one small gripe - the notifications don't use Growl. I've now found a plug-in called Google+Growl that adds Growl support. It works along side the Google Notifier and allows you to configure how you want your notifications to display.

How-to: Get a # on MacBook with Kubuntu / KDE 4

2008-06-01T14:38:14+00:00

I recently decided to give Kubuntu with KDE 4 a try on my MacBook, the installation was painless and aside from a few patchy bits (to be expected from beta software) it's running great. I was trying to fix a few bugs on the website and couldn't figure out how to access the # sign, normally alt+3 in OS X. Thankfully D_Ed came to my rescue...

Windows error/crash debugging

2008-05-28T16:39:47+00:00

Recently my pc has been crashing a bit, which has caused it to reboot suddenly, and the error messages haven’t shown any useful details of the cause.

After a bit of googling I came across this guide which guides you through using the windows debugging tools to find the cause of the crash.

It will pinpoint the driver at fault etc. so you can update your drivers or look into ways of working round the issue.

BAUD: The First Meeting

2008-05-28T11:14:53+00:00

(copied from baud.org.uk)

The first meeting is planned to be held THIS Saturday (31st May 2008) at 8pm BST.

It will be held on TalkShoe, which requires no third party software. There is a text client, and a dial-in service.

You can listen to the streaming audio and participate on the text client without any third party software by visiting http://www.talkshoe.com/tc/20200 and starting the web client.

For those with VOIP, you can use your favourite client and dial SIP:66.212.134.192 or SIP:123@66.212.134.192 which will get you into the service, then SLOWLY dial 20200# to get into the call. If you have a TalkShoe PIN, type it in now, alternatively, just dial 1# to get right in. If you have free credit to USA numbers, you can also dial +1-724-444-7444 from any phone and get in through that method.

By default, you will be muted which means you can not speak. To request to speak, dial *8 - this will flag you up to the host as wanting to speak. When you hear "You are unmuted" you are on the call, please speak clearly.

The call will be recorded, and available via TalkShoe to listen to afterwards.

The meeting is estimated to last 30 minutes, and will concentrate on initial planning of the structure of BAUD and the opening event, which is expected to be a fairly large LAN party.

How-To: Make Interarchy 9 more like Interarchy 8

2008-05-26T17:34:27+00:00

When Nolobe released Interarchy 9 I screamed at the features that had been removed that I used daily without even thinking about it. Luckily 9.0.1 restores most of these features, although you have to get the settings right.

SFTP in Mac OS

2008-05-26T17:33:57+00:00

When Interarchy updated to version 9 and some of the features I relied on stopped working I went in search of a new SFTP client. I've finally got around to writing up my findings. As you can see, the only client that I liked was still Interarchy so I ended up using the old 8.5.4 build. Luckily, Interarchy have since fixed things and I'm now once again happy with the way it works.

Browser Navigation

2008-05-22T00:00:14+00:00

Why don't more websites take advantage of the rel attribute for their relative links? Perhaps because there's not enough browser support? Opera has supported these attributes for as long as I've been using it and I recently found a couple of plug-ins for Firefox that give the same functionality but I cannot seem to find a solution for Safari...

Lufbra.net saved?

2008-05-19T13:37:37+00:00

Source #1 tells me: "my sources tell me that uniserv is going into administration" - Uniservity is what Lufbra.net is based on, and in my opinion sucks big time.

Source #2 tells me: "The Union are choosing Warwick Union's site to emulate".

Ah well, good while it lasted... Their site isn't bad, but it could be a whole lot better... I'd like to see an entire category for election manifestos etc. We'll see if that works.

UPDATE: A friend at Keele (also Uniservity, but not as bad as ours) tells me that he's thinking of introducing an easy to use, well-designed system. I'd be interested to see what comes out of this.

LSU Reform

2008-05-09T14:05:59+00:00

I know I'm going to get asked this, so I'm putting it here for posterity's sake.

The Union is in financial trouble - yet Leeds get half as much money and run with twice as many students. We get more money per head than anyone else, it seems. So, we must be wasting money somewhere - here are *my* recommendations. Feel free to argue, and yes, I realise I'm being big-headed here.

Incidentally, I've asked for a copy of the latest accounts and/or budget but not received anything back from VP Finance yet. We'll see whether my figures add up then.

The Exec just got a pay-rise from £220 pw to £250 pw (supposedly backdated to the beginning of the year!) giving them (over 52 weeks) a grand total of £13000 pa. If they only get paid during term time, that's £7500 pa. Or nearly twice as much as a student loan. I would recommend £150-200 pw, with a possible exception for those who work after-hours, who could get minimum wage in addition. I've been told this is "bare minimum funding" yet I believe £13k is about what you expect when you're on Industrial Placement, and they don't get the benefit of having student housing...
The Exec to be cut down drastically - do we need so many full-time officers? I can understand having President, Finance, Democracy, Education - and Media does bring in additional funds which could require a full time employee, but do we really need a full time Action chair? Environmental and Ethical officer (surely People and Planet would do this for free)? POSTGRAD officer (surely this is a maximum of one-day-a-week!) Community Students Officer, Department Committee Officer? Get rid of those 5 people, and you've paid for the Student Advice centre already. I understand some are not full-time employees, but I presume they still get paid...
Union rents - if a juice bar can afford to set up within the Union, then obviously the rents are too low. I'd be intrigued to see how much they're paying for prime commercial estate...
Support staff - As far as I can remember in my experiences setting up CompSoc, the Clubs and Societies staff never seemed to do a lot to help, and they were a large hindrance in some aspects. While I'm sure they do valuable work - do we really need the amount of staff we have so far? There are other departments with staff I don't quite understand either...
Media. While I agree Loughborough benefits from it's (mostly unheard) radio and TV stations, was it *really* worth spending the money on that? I hear over £1,000,000 was spent on those facilities, and they aren't that great - I wouldn't even say they're taken full advantage of either...
The website sucks - big time. If it was easier to use, a LOT easier on the eye, and actually designed rather than having a mish-mash of different fonts, colours and poor spelling/grammar then maybe some funding could be drawn up through adverts on the site, rather than the measly 4 square inches at the top that no-one sees, let alone clicks on. Get rid of Uniservity, pay a local company like Cuttlefish to do you a site, or give CompSoc £1000 and they'll do one in Drupal that kicks butt. Do *not* just go with a cheap company, get professional advice from someone like... Leeds, Warwick, even Cambridge!
All nights bar Hey Ewe, FND and EOTP are free. You'll get more people in, and more will be spent on the bar. While Queue Jump may bring in additional revenue, I think this goes against the spirit of the Union.
Sponsorship - While I'm dead set against having sponsors on stage telling you about their latest and greatest properties to rent (and I was there when they actually did this) I wouldn't mind banners around the place with sponsors names on. No leafleting though.
Label - How much does the magazine cost to produce? If it's making money, keep it, if it's breaking even, cut the circulation (they're all left in Dining Halls anyway) if it's losing money, cut it to internet publication only.
Publish your full accounts, and ask the University to do a budget review as part of the Business School's next year syllabus or something - so much cruft must be able to be cut, it's untrue.

And that's about it. Everything else is small-fry and I believe it will get cut anyway once students realise that actual services are under threat.

Comments are more than welcome - if you don't know what I'm on about, visit www.lufbra.net and see how poor we present ourselves on a national stage, compared to (bringing back the old example) Leeds Union at http://www.luuonline.com/index who don't do an excellent job, but they're clear, easy to read, and do it on a budget far less than LSU.

How-To: Customise your SSH Config

2008-05-08T16:08:57+00:00

I am someone who feels more comfortable using the command-line than many GUI set-ups and who is often SSH-ing between servers; many of these servers have lengthy host names and require various different usernames; luckily the SSH developers thought ahead and came up with a way of pre-setting most options per-host...

How-To: Stop iPhoto From Opening Automatically

2008-05-08T09:38:50+00:00

I've stopped using iPhoto recently in favour of flickr but every time I plug in a camera or something that looks like it could be a camera iPhoto opens. A quick hunt around the internet and I found the solution...

Life Membership at Loughborough

2008-05-08T09:32:25+00:00

The Union are instituting a policy which would drastically change the Life Membership scheme, which is unchanged (except for formatting) since it's inception in 1988.

The main points of the changes are:

The £15 fee will now be an annual subscription
Members get 5 free entries to Stupid Tuesday/Hey Ewe/FND for their own sole use
Members get queue jump for themselves + 2 guests
Exec get unlimited free entries for themselves + 1 guest, plus queue jump
Exec get invites to all Fresher/Grad balls
Honorary membership will only be awarded for 1 year

It's really quite annoying, and the LSU Computer Society (amongst others) are taking this on personally.

As I'm a Life Member, and unable to attend the meeting on Monday at 5pm (Board Room, LSU, hint hint) to discuss this, I've decided to draft a letter to the Exec responsible. Please feel free to leave comments, and I'll submit it to them on Monday if I feel it would be beneficial. For the trendies amongst you, there has been a Facebook group created by the LSU Secretary, Kirsty Smith, at http://www.facebook.com/group.php?gid=26661743800&ref=ts

Sorry about the crazy formatting, it's copied from a PDF as I left the OpenOffice document at home before I came to the Office.

*****

Dear Sirs,

Document UC-082-paper-07 (Life Memberships Policy) has been brought to my attention. As a
respected member of the LSU Computer Society, I have a number of concerns which I wish to raise
as this document may well affect myself in the future, in addition to the numerous friends and
colleagues I have accrued over the past 5 years at Loughborough University.

The current policy, as you so describe, has been in effect for 20 years. The document does not
address at any stage a suitable reason for the review, and the adage “if it ain't broke, don't fix it”
comes to mind.

In particular, you raise a valid point:

“The working group agreed that one of the reasons for becoming a Life Member is continue to
support Loughborough Students’ Union and the work it undertakes. These are normally students
who have affinity to LSU that will continue into their working life’s and beyond. Life members
play a valuable role by supporting LSU. Life Members by giving support means that LSU can
maintain the range of services and activities that it undertakes to ‘Better Student Life’ and the
Loughborough Experience for it’s membership.”

Those who choose life membership fall into one of two categories:

Those who are residing in Loughborough, and have friends around that they wish to continue meeting through clubs and societies, in addition to attending occasional events such as FND etc.
Those who have invested a lot of valuable time, effort and funds into a club, society or group that they would like to remain involved with. This is by far the largest proportion of life members in my opinion.

I fall into the latter category (graduated July 2007), having founded and subsequently followed the
Computer Society since it's inception. I was considered for the member of the year award in this
academic year, which goes to show how valuable life members are to clubs and societies –
especially those with strong communities.

I am therefore astonished to read that you are intending to launch a “Life Membership Plus” scheme
that throws away “five free entries to certain nights at the union”. Supposedly, those that are not at
LSU are presumably employed and therefore have more funds available than your average student.

You also appear to have instated a “Queue Jump” scheme, allowing “privileged” guests to jump
past all the queues, and extended this to include the Life Membership Plus scheme. Students should
be given priority over life members and guests to all events, and they should not have to pay
additional fees to skip the queues as this means the student who is tight on cash and can't afford to
buy such a ticket is disadvantaged.

Similarly, the annual report – during my four years at Loughborough, I never even realised that an
Annual Report existed! As Life Members do not have a vote at any event – Union, Club, Society or
otherwise – I see little point in paying for the distribution of these reports. At most, an e-mail would
be of interest.

The £15 annual fee is also a little worrying – every society benefits from the involvement of past
members. These past members are less likely to contribute if they're paying £15 every year, and not
using any of the so-called “benefits” that you propose to include. The idea of Direct Debit also
scares me – the thought of the Union having direct access to withdraw funds from my account
worries me, thinking that should you impose one of these “fines” that seem to have arisen in the
past year, I have no right of challenge before you dip your hand in and withdraw funds! A standing
order would suffice, an e-mail reminder to pay dues would be best – which also means that you'll
receive less complaints as a result of withdrawing funds without prior notice when someone moves/
loses contact with the club/society they have been involved with.

If anything, every committee member of a club or society should receive automatic life membership
to their club (and therefore, the Union) as they generally are the people who are going to help the
most – in my experience, a lot have far greater influence, respect and use to members than many of
the Exec from previous years!

To be told that “The current LSU Clubs and Societies Honorary Membership Policy be amended so
that the free honorary life membership be only granted for one year” smacks of money grabbing and
no real interest in the pursuits of a better community. You are supposed to be honouring the
members that the society has greatest benefited from, yet you once again seem to be demoting them
to a sub-class of membership.

All in all, despite the numerous (and exhaustive) spelling and grammatical errors indicative of
Keystage 3 students, this policy is ill-thought-through and lacks any motivation other than an
attempt to open a new (minor) revenue stream, and would seek to undermine the efforts of past
students who wish to stay in contact with, aid and assist with old friends, clubs and societies.

Since 2003, I have seen the Union deteriorate to the state that many societies are now having to
spend much of their time with needless Union paperwork, rules and regulations that back then were
minimal, unimportant, and for the most part, ignored. It saddens me to say, if I were a finalist now, I
would not consider life membership for an instant, and instead make contact with friends still at
University through alternate means.

Regards,

Matthew Walster BSc(Hons) MBCS ON

Loughborough Students Union - Watch This Space

2008-05-07T10:56:28+00:00

All I'm willing to divulge at the moment is that LSU seem to be acting for themselves, rather than for the students they represent.

They appear to be seeking new revenue streams that will devalue the Union, and hinder rather than help clubs and societies.

At their present rate of development, I'll be recommending an invasion of the next Union General Meeting to voice concerns over the allegations of worrying neglect towards clubs, societies and those that seek to improve student life.

No doubt the Union will respond by increasing funding to Media, the AU or the Exec.

If you're a geek at LSU (or indeed in a similar situation at another Students Union), watch this space - something bigger, better, more democratic and less secretive is coming your way in the next month.

Plus.Net's Dodgy Figures - And How They're Not So Dodgy

2008-05-03T23:15:12+00:00

Plus.net responded to a twitter post of mine calling them expensive. I've used their own figures (and updated them for their latest prices) to show something:

Option 3 (their highest package) allows you 30GB/month at £19.99/month.

We can ignore their "value added features" such as overnight free bandwidth, because that's using the net as it's supposed to be in my opinion - allowing congestion to determine usage. Other features such as Usenet etc - also ignore because the on-the-spot cost to them of these must be <£1/customer/year unless they're doing it wrong.

Take out the "IPStream Home 256kbps/512kbps/1Mbps/2Mbps - £7.63" costs they have to pay (+ VAT) and you're left with £11.02. You can probably add in another £1.50 if I've read the rebate thing correctly. [1]

1Mbit is about 300GB in a month. (28 days is 302.4GB) 1Mbit allegedly costs them £126.86 [1] before transit and routing costs.

I've been informed you pay about £15/Mbit for transit these days from a London supplier, less if you're a big organisation.

So, all in, it's about £140/month for 300GB. Or £14/month for 30GB.

Essentially, if everyone used their entire 30GB quota every month, Plus.net make a loss - but that's where overages and overselling come in.

Not every uses 30GB/month - some may only use 1GB. That's essentially free bandwidth in the bag for them. If you go over your limit, you pay a whopping "75p per GB if you add it upfront or £1 per GB if you just pay for any extra as you use it." [2] which rivals 3G for expensiveness.

Unless you use 30GB/month, Plus.net makes a profit. That's how the business works, and Plus.net is doing nothing wrong by charging these figures - if people don't like it, they'll move.

But wait - there's a big glaring problem here - and I'm not talking about using basic statistical theory to prove in a normal distribution fah lah lah - no. They're paying BT *£126.86* PER MEGABIT to get the traffic from the exchange to their nearest POP (place where Plus.net can distribute the bandwidth to "the internet"). That's INSANE - this is either BT charging a metric fucktonne for their services, or British telecomms infrastructure is decidedly more expensive to operate than most other European countries.

The final word on this is - if you can avoid telephone lines for broadband, do so. Virgin Media offer 20Mbit, and you get it - none of this DSL rate-adaptive crap. There are no limits, and they have fair throttling during peak times for the most part (if you're a crazy downloader, they limit you to about about 20-25% of your supposed speed for a few hours - still faster than my mum gets on DSL!) which keeps their network healthy. If you can't get cable, Plus.net have a fantastic community element - but BeThere are cheaper and SamKnows says [3] that my Mum's exchange only really has Sky, Bulldog and BeThere LLU enabled. And my new place I'm moving into says "BT Wholesale has set an enablement date of 18/02/2009 for their 21CN Wholesale Broadband Connect product at the Sherwood exchange" [4] or "9 months after I move in" and the tracker estimates an ADSL2+ speed of less than 2Mbit. Sorry! I'm sticking with Cable! Shame on you, BT!

[1] http://community.plus.net/blog/2008/02/28/how-uk-isps-are-charged-for-broadband-the-cost-of-ipstream/
[2] http://www.plus.net/support/broadband/quality_broadband/traffic_prioritisation.shtml
[3] http://www.samknows.com/broadband/exchange/EMHUCKN
[4] http://www.samknows.com/broadband/exchange/EMSHRWO

Website Access Restrictions

2008-04-28T18:09:24+00:00

I manage/admin a fair few sites covering an oil exploration equipment company to a church website. Unfortunately over time I’ve had a some issues with spammers posting random unwanted guestbook messages and so on.

Therefore I decided to go ahead and implement some measures to start blocking them. The main site this occurs on is the church site I run, which has an option to blacklist IP addresses. However as the spam comes from a range of IP addresses each time the site is ‘hit’ this would involve a tedious process of adding each address.

Therefore I decided to use the .htaccess allow/deny restrictions. While these are simple to use, getting the information required isn’t always as easy.

The method I took used three steps:

Determining the IP Range involved
Calculating the network prefix of the range
Banning them

For the first step I noticed the IP addresses each time I was hit were similar, so I looked them up on http://software77.net/cgi-bin/ip-country/geo-ip.pl to see if they were in a single range assigned to a particular organisation or country. In this case they were.

Then I converted the range from a start and end IP address to a single address with a prefix. In my case it was 123.112.0.0/12. (you can check your conversion using http://www.csgnetwork.com/ipinfocalc.html) [The look-up link above now gives you the CIDR range]

Finally I added the following lines to my .htaccess file, including a comment for future reference

Order Allow,Deny
##Chinese IP range banned for guestbook spam 28/4/08
Deny from 123.112.0.0/12
Allow from all

As a side note banned users will get a HTTP 403 error page. I also created a custom page to be loaded for banned users, defined by the following line in my .htaccess file:

ErrorDocument 403 /errordocs/403.html

You also need to test that they have access to view this page, so I added another .htaccess file in the errordocs folder with the following lines:

order deny,allow
allow from all

Latest LUGRadio episode comments

2008-03-03T10:16:19+00:00

Last episode, the crew took the shoe and beat Chris with it royally. Much fun was had! However, Chris was obviously drawing a blank on some points, and I thought I'd fill others in on the bits he missed :)

OpenSolaris is fantastic - if you don't have any expectations. With Linux, you just boot it and it works, with OpenSolaris, there's a LOT of configuration to be done first! However, if you use the excellent Nexenta distribution, which is based on Debian, it's a case of install and forget. All the drivers are in there, the default shell is bash (and works correctly) and apt works as expected. In fact, Nexenta has a killer feature called apt-clone - when you do an apt-get dist-upgrade to get the latest core stuff, it warns you that this could be a potentially damaging operation, and stops itself, telling you to run apt-clone dist-upgrade instead.

When you run this apt-clone, it takes a snapshot of the filesystem in it's present condition, and installs all the new kernels/drivers etc into the new data area. This uses ZFS, which I'll get to later - but suffice to say, it's a lot easier than LVM, especially since it tries to get you to run a ZFS-mirror for your root filesystem (think RAID-1 root/boot).

Once you add in the CSW (Blastwave) repo into apt, you're looking at a huge range of software at your fingertips - even if you don't use the server for anything like apache (which is very stable) it makes a fantastic storage server.

What's more is that Nexenta seems rock-solid. There's no way to really describe it - when you've installed Nexenta and boot a Linux system, it feels like it's a rickety system - patched up on top of patches on top of more patches and it's just barely working, whereas Nexenta and OpenSolaris auto-detects whether you're 32-bit/64-bit and runs silently, so it just seems like it's a tiny well-refined kernel and user-environment that just "feels better". There's no foundation for this - it just seems so. Incidentally, it wears off - when you actually look at the research, I still prefer Linux, but Nexenta really does blow your socks off.

ZFS is the next thing I wanted to look at - Matthew Garrett has some bad things to say about it, but then again, he sees the flaws in everything, and we appreciate (in the long run) that he does see these flaws - so they can be fixed. He recommended another filesystem to try out (bvgfs or something similar) and when that becomes stable, I look forward to trying it out.

EDIT: http://oss.oracle.com/projects/btrfs/ - sounds exciting, nowhere near production ready.

Why is ZFS so great?

For a start, ZFS is rewritten from the ground up - none of this "extended 4 fs" rubbish. I expect to be berated for this, don't worry. It's got something about it - that when you're dealing with quotas in ext3 it seems like another patch that doesn't quite live up to expectations. ZFS has it all built in - it gives you a guaranteed minimum reservation of space if you so choose, and a soft/hard limit to hit if you set it.

Plus, it's relatively easy to create a new filesystem - it's a case of doing "zfs create tank/home/dotwaffle" and since tank/home is already mounted on /home, it creates /home/dotwaffle and sets up this new filesystem.

"Big deal" - right? Well, this new filesystem can be customised as you wish, you can have it compressed using gzip, you can have up to three copies of each block so that if you're not using mirror (RAID-1) or parity (RAID-Z) you have a chance of recovering your data. Recovery is made easier too because every piece of data is checksummed, so that ZFS knows on read whether or not the data is corrupted - something someone on ext3 will not necessarily have known until they md5sum'd the data. Trust me, this happens more than you think. The data in a mirror/parity pool is protected so it automatically fetches the pristine copy, and mails you a human-readable explanation of what went on. Nice!

If you use ZFS on an entire disk (that's the entire disk, not just making a partition the size of the disk and telling it to use that) it'll do some jigger-pokery to maximise the use of the write-cache on the disk too. Which is nice. I heard it might then monitor SMART data too, but that's just a rumour.

This filesystem can also be shared - zfs share tank/home/dotwaffle - and bang, it's available over NFSv4, CIFS, even iSCSI if you choose. There are plenty of configurable items, just have a read and I guarantee it'll be easy for you to pick it up.

ZFS also has a feature which you probably didn't even realise recent Linux kernels do - suppose you have a zfs pool with 12 disks in it. You want to migrate it to a new system, so you export it. Put the drives in a pile, and crap, was drive #1 the bottom disk or the top? Doesn't matter, put them into the system in any order, and ZFS will automatically reorder them into the right configuration. Of course, mdadm has done this for a while now, but I think ZFS did this first...

The final feature that makes me excited is "slog". Your RAID-5 array may write at what - 30MB/s? ZFS will write faster than this, but what if your controller is the bottleneck, and all the drives must be on the same controller (say, you've only got one)? In steps the log device - I saw on the net someone add one of these Gigabyte iRAM drives, 4GB of DDR-RAM on a PCI card that communicated over SATA-I. He added 2 of these, on different channels, so 8GB. He created a zfs pool, and added in the log devices. Writes saturated the SATA buses - writes at 300MB/s and above. Why? The data was being written to the log device first, then streamed off that log device slowly onto the real storage media. Think of it as a buffer, it's taking your randomly sparse, randomly mega-busy I/O and making it into a stream to be handled by the disks. If the power goes down, the iRAM preserves it's contents for up to 12 hours. If it's lost, sure, you've lost the data, but if you call a sync(), I'm assured the iRAM writes all it's data to the disk straight away. Mega-fast read, mega-fast write. Mega.

ZFS is nice, if you're dealing with storage, it eases the workload a LOT, and with the final killer feature - on-line disk checking, you're laughing.

That sums up my rebuttal, hopefully that's what was in Chris Proctor's head at the time, and he couldn't vocalise it. I just wrote this without checking any facts (no internet down here at my mum's house in West Sussex) so there are bound to be errors and spelling errors, and I just intend to copy/paste this into LiveJournal, so don't take it as canon :)

I would, however, advise you to download "NexentaStor" Evaluation version from Nexenta.com, which is their enterprise-level Nexenta product (there's free stuff on nexenta.org which does the same kind of thing without the web interface) and run it in a VMWare machine or similar. Be sure to make a few files on the VM and use them as a ZFS pool to get the desired effect =)

I will still run Ubuntu-server, and I'll still prefer the GPLv2, but Nexenta has one hell of a product for storage, and it's all thanks to Sun's ZFS... Nice!

Finding Suid security holes

2008-02-21T10:34:15+00:00

What is “Suid”
Suid is a special flag which can be set on a file in a Unix based system, called set-user ID. When set on a file it will cause the program to run as whoever owns that file rather than whoever ran the program.

Why would you want to that?
It is commonly used to allow programs that need root access, but we want to run them without users having to type passwords all over the place, the “mount” program is a good example. This needs to be called from programs running at just a user level, but because they create a lot of high level access need root priveledges.

Isn’t that dangerous?
Not really, only the root user can set something to be root suid. So you need to have root access to give the access to people.

The danger comes if we have a file set suid, for performing an operation and a user is able to change it. If a malicious user (or ‘virus’) with access on your system was able to write to the file, they could swap the file out with some different code (i.e rm * -Rf) and when they run it, it will run as the root user and pwn your system.

This python program scans your entire filesystem for security holes, and notes when it finds a program set suid. If the user also has write access it will print this out with the word “panic”

Download Here and run “./suidfinder” to test your system.

British Computer Society

2008-02-20T23:17:02+00:00

On 12th February, 2008, I was awarded a professional membership of the BCS, and got (more) letters after my name!

The certificate only just arrived, hence me posting about it - wanted it to be confirmed first!

So, I guess that means my name is now:

Matthew James Anthony Walster BSc(Hons) MBCS ON

Bit of a mouthful, you can call me waffle =)

Computer Science Timetable -> Vcalendar convertor

2008-02-10T16:21:51+00:00

I was about to start the task of adding all my lectures to my calendaring program, when I thought. Why spend 10 minutes on this, when instead I could be writing some mammoth regular expressions to do this for me.

As such I have written me an awesome python script to convert it for me. For any the niche of other computer science students who keep a calendar this is the script for you!

Download the script
Fetch your timetable from https://co-teach.lboro.ac.uk/corgs/timetbl/tt.php?Personal&text=1
Select all the text and save it as to text file somewhere
This program reads from STDIN so we need to run ./parse.py saved_text_file > calendar_out

My First LSUCS Planet Post

2008-02-09T00:38:08+00:00

A big hi to all those reading this from the Planet LSUCS.

First Post! w00.

LSUCS talks

2008-02-05T20:54:18+00:00

I've been thinking of doing a few talks for LSUCS, maybe a regular lecture series where I get guest speakers etc, take a bit of an active role, meaning I'd take Wednesday afternoons off work for a while.

The talks would be technical of nature, and would run for an hour, with an hour's workshop afterwards if needs be.

Talks I'm thinking of doing:

LSUCS LANs

Managing the Nortel 450-24T
Managing the Extreme Networks 48 (15000)
Copper vs. Fibre
Spanning Tree Protocol
Load Balancing and Trunking
Network Services - HTTP/DHCP/DC

Networking - Routing

RIPv1
Subneting
OSPF

Typesetting

The beginners guide to LaTeX and why you should use LyX instead for your Final Year Project.

Networking - Monitoring

NMAP
Nessus
Nagios
SNMP

Software Configuration Management

Why the hell you care

svn vs bzr vs git vs hg
SourceForge vs Belios.de vs Launchpad

This is all I can think of off the top of my head - any more suggestions? I think people would become quite bored with this after a while, but it might inspire others to do better talks.

That, or convince people to come to FOSDEM with me in three weekends time =)

PHP Server Monitoring Script

2008-01-25T21:32:31+00:00

We’ve been having some issues with the availability of the LSUCS server recently, so I decided to try and put some monitoring in place. Unfortunately we’re not in the position to deploy a full monitoring suite with heartbeat monitoring etc.

So I developed a PHP script to check the status of the server remotely. This is done by checking the IP address, and connections on specified ports. The script can be configured to sends status emails, and to send error emails.

The script also accesses data from a remote PHP script on the monitored server to access disk usage data if available.

The script can be downloaded from here.

~~The script will probably be changed over time to support multiple servers and extra functionality.~~

26 Jan 2008: I’ve added support for multiple servers and extra features (see code for full details). The link above now points to the latest version.

PHP: Late Static Bindings

2008-01-23T18:03:25+00:00

New to PHP 5.3.0 will be late static bindings. Currently you can use the self keyword in PHP to access static methods of the current class, the new static keyword allows you to access static methods through the inheritance tree.

Hello Planet LSUCS

2008-01-14T20:12:55+00:00

As of "not long after this is posted", this should show up on Planet LSUCS.

Woo!

Goodbye Newshutch

2007-11-01T18:47:10+00:00

I went to read my feeds, as I do every day, and was greeted with a sad message... Newshutch will be closing down on November 10th!

Goodbye Newshutch

2007-11-01T18:38:00+00:00

I went to read my feeds, as I do every day, and was greeted with a sad message... Newshutch will be closing down on November 10th!

File Renamer

2007-05-14T17:25:00+00:00

ETW pointed out to me a cool perl script File Renamer which tidies up directories of TV episodes, and it will fetch the episode titles from EpGuides.com or TV.com.