Pirate Bay Is No More?

It's old news by now but The Pirate Bay has been bought by Global Gaming Factory. So what will be next for the Pirate Bay crew? Should be interesting.

Low Sound On 2.6.30

I upgraded the kernel on my desktop to 2.6.30 today but later found the sound was broken. Actually, sound worked but the volume was so low that even with the amplifier volume turned way up it was barely audible. When i discovered the problem, I just rebooted back to 2.6.29 since I had other things to do.

Fortunately, the problem wasn't too difficult to solve. In fact I found the solution on the first hit on Google, which leads to the this page on the ALSA web site. In a nutshell, just crank up all the VIA DXS channels to 100%.

I wish all problems could be solved by cranking it up! ;)

Linux LVM

I decided to abandon /home as an NFS mount. It worked OK but the performance affected Firefox which writes a lot of files to its cache. I also had edge cases like this one. But before moving /home back to a local drive, I needed to rearrange the space. I have enough disk space but it is not divided up efficiently. These days I use the Linux Logical Volume Manager (LVM) which makes shuffling disk space more convenient than direct partitioning of the drive. However, the risks are the same so backups are essential.

I've only shrunk/extended LVs once before so it was the first test of my notes. When doing this type of work, you are either in single user mode or running from a rescue CD, so your access to the Internet is not always dependable. With this in mind I checked the notes against known good references before starting. The LV resizing looks like this:

$ umount /var
$ e2fsck -f /dev/mapper/vg0-lv1
$ resize2fs /dev/mapper/vg0-lv1 1048576K
$ lvreduce -L 1048576K /dev/mapper/vg0-lv1
$ e2fsck -f /dev/mapper/vg0-lv1
$ mount /var
$ lvm pvdisplay
...
Total PE 1024
...
$ lvextend -l +1024 /dev/mapper/vg0-lv2
$ resize2fs /dev/mapper/vg0-lv2
$ e2fsck -f /dev/mapper/vg0-lv2
$ mount /dev/mapper/vg0-lv2 /home

I omitted some details here which were specific to the system. For example Debian starts a few daemons in single user mode, which I had to stop those before I could unmount /var. A rescue CD wouldn't have that problem.

I made one mistake which fortunately I caught in time. I left out the "K" on the end of this command:

$ resize2fs /dev/mapper/vg0-lv1 1048576K

That resulted in the file system being 4GB (1048576 4KB blocks) instead of the 1GB I had intended. Fortunately, I caught the error before reducing the LV which then would have truncated the file system. As I said before, the risks remain the same.

07-Ghost #8

07-Ghost is a sci-fi anime series with a gothic atmosphere. The story revolves around Teito Klein and his mysterious past. Teito is introduced as a former slave who was adopted by a high ranked military official, which allowed him to enter the military academy. Just before graduation, Teito suddenly remembers a small fragment of his past. He finds himself pitted against the powerful military elites and is forced to flee the academy. He escapes with the help of his only friend, Mikage, whom Teito has to leave behind, which weighs very heavily on him. In fact Teito has an excessively brooding nature which might be his only serious character flaw. As Teito remembers more of his past he also begins draw on a source power he never knew he had before.

Teito takes refuge in a church which provides much of the gothic atmosphere that is completed with an organ music sound track. The priests of the church are a little quirky and provide much of the comedic elements to balance the heavy dramatic side of the story. The priests' oddball behaviour notwithstanding, they all have hidden abilities and are extremely powerful. The church is protected by seven ghosts, from which the title of the series is derived, but they haven't actually made an appearance yet. But I have a strong suspicion who might be using the ghosts' powers.

Ah, I love a story with so many mysteries!

Microblogging

Still haven't developed much feeling for Twitter. I often forget to post anything for days at a time although I didn't do too badly this week. Usually when I'm doing something, the last thing on my mind is, "Got to tell some about this!" Maybe this will change over time.

I found out Twitter has an XMPP gateway. That's handy, I thought. My IM client is always running so it would be more convenient than posting from my Twitter home page. Unfortunately, XMPP isn't a priority for the Twitter developers so the gateway doesn't work too well when it worked at all. Others such as tweet.im and twitterspy have step in to fill the need. Other microblogging sites like Identi.ca have an XMPP gateways and connect to Twitter. I decided to try Identi.ca first since you are posting to two sites at once. Hey, its hard to resist a two-for-one. ;)

So far my impression of Identi.ca is mixed. It has a much nice web UI than Twitter but the service became very unreliable just after I signed up. Twitter is can be slow but I haven't seen it become inaccessible yet. Identi.ca's XMPP gateway works and the posts are forwarded to Twitter, but the XMPP gateway is annoying. It constantly logs in and out for some reason. One post I made tonight showed up on Twitter a couple of hours later. Well, at least it showed up.

The two-for-one maybe wasn't a deal after all so I'll have to try the other options.

Pylons Progress

I was hoping to plough through the The Definitive Guide to Pylons ("The Pylons Book"), but I only made it to chapter seven since last Saturday. Most of that was achieved in the last two days, so it was mostly down my not keeping up the pace. Chapters five and six covered Mako and FormEncode respectively. I have not used FormEncode at all and Mako only slightly more than that, which slowed me down too. Chapter seven was faster because I have a little more experience with SQLAlchemy, but I still learned a few new things. The book has introduced enough material to start writing a simple application so naturally chapter eight does just that.

I understand why I suffered information overload while trying to learn Pylons and develop an application at the same time, even though that is my usual learning style. Pylons is a loosely couple framework that ties unrelated modules together. Mako, FormEncode, and SQLAlchemy are completely separate projects, each with its own documentation and tutorials. The shorter Pylons tutorials naturally assume some familiarity with the modules, which meant that I spent much time digging through their documentation, without a clear understanding of what was important from a Pylons perspective. This is where The Pylons Book is proving really helpful.

Pylons appeals to me because it doesn't reinvent modules unnecessarily but that also made it harder to learn in my usual way.

F1 Peace At Last

It looks like FIA blinked first. The 2010 season will use the 2009 rules and the budgets will reduced to the 1990s levels over two years. Fota will not start their own championship series.

What was really surprising is that Mosley will not run for re-election. The official story is that he decided to stand down months ago but it was kept secret. If that's true, why take such hardline position and create a huge mess for your successor to clean up?

The teams can finally get on with designing their 2010 cars which is already behind schedule because of all the uncertainty over the rules.

Tears To Tiara #7

Tears to Tiara is another fantasy anime from the Spring season. The first episode uses the classic summon-a-demon-to-rule-the-world plot device but then the second episode throws a nice twist. The summoning resurrects the main character, Arawn, who is anything but evil. In fact he saves the sacrifice and kills the summoner! The first rule when summoning anything, is be absolutely sure you know what you are calling forth.

The story weaves several myths and legends together. One of the characters is named Arthur and the OP has a scene with a sword in a stone. Arthur is the lead warrior of a tribe called the Gaels. When the Gaels' island is attacked, they travel to another island called Albion. The enemy is the Divine Empire and their city on Albion is called Londinium. Unsurprisingly, the Empire's troops look like Romans soldiers. Oh yes, and there's Elves and magic too. I'm sure other fantasy staples will appear as the story progresses.

With all that thrown together, if the series tried to be a serious drama it would never work, but the story is definitely tongue-in-cheek which creates the right balance for me. The first few episodes were mainly character development and plot set up, so I expect the story will pick up the pace now.

F1 Great Britain

Redbull have really turned it up notch. Vettel got his second poll in a row and Webber came really close to making it an all Redbull front row. Brawn GP were definitely off the pace but the real surprise was that Barrichello out qualified Button for the first time this season. I have a suspicion that Barrichello may not have shared all of his practise data with Button this time. I have no evidence to support my theory; it is just a hunch.

In the race, Vettel kept it on the track for the first lap this time and pretty much cruised to the victory, with Weber passing Barrichello in the pits to make it a Redbull one-two finish. Thankfully, there was plenty of good racing in the rest of the field. Who could have imagined Hamilton and Alonso fighting over fifteenth place, but it was fun to watch them throwing their poorly handling cars around the track. Unfortunately this is the only proof that McLaren were actually at the race. Ferrari showed better performance in the race than in qualifying but were completely unable to explain why. Williams is also improving. The rest of the season is going to be interesting.

Finally, Silverstone is an historic F1 circuit. The fact that the F1 management are willing drop it from the calender is, I think, indicative of what has gone wrong with the sport. The modern circuits might be better in some technical ways but they feel sterile compare to places like Silverstone.

Lazy Bum

Wow, what an unproductive day. I got up late and, except for one "must do" chore, just goofed off all day. I watched F1 qualifying (which I'll comment on tomorrow after I watch the race) and some anime. What can I say, sometimes I'm just a lazy bum!

Back To Pylons

Last year I tried to learn the Pylons web framework while developing a personal project. That was a bad idea. The framework is just a little too complex to learn that way. I'm still interested in Pylons but this time I'm using The Definitive Guide to Pylons to learn the framework in a more systematic way. I'm only on Chapter 5 now which is still in the beginner section. Since I already know some of this material, I won't comment on how useful the book is until I get to the advanced sections.

F1 Self Destructing

The Formula One Teams Association (Fota) has followed through with their threat to start their own series next year. The FIA has already announced that they will start legal action against those teams. There will be lots of blood on the ground before we see any resolution to this situation.

Could this be the end of F1? That might not be such a bad thing. If the management cannot place the actual racing ahead of its own business agenda, there is no future for the sport.

The Slippery Slope

Here we go again. A new Bill introduced today will reduce our freedom for the sake of catching a few "gangsters, sexual predators and terrorists" (Ottawa Citizen). I fear excessive police powers far more than I will ever fear any terrorist or other criminal. I will never trade my freedom for any illusion of increased safety, end of story. The cops already have the powers to snoop on anybody as long they can convince a judge the snooping is justified, i.e., the system has checks and balances. It seems that's not good enough for the cops.

Read Micheal Geist's analysis here.

PyGTA On Wednesday?

This month's PyGTA meeting was moved to Wednesday to accommodate a speaker's schedule but he canceled at the last minute, so Mike Fletcher had to improvise a little. Toronto PyCamp 2009 was mentioned. The course is an affordable (only $375!) week long introduction to Python. Mike talked about some ideas he he has for a game network library for Python. As is typical for Python, there are a plethora libraries already but Mike has yet to find one that is easy to use and scales well.

WRC Greece

After a rare but costly mistake by Loeb on day 2, the rally was Hirvonen's to lose. Given the reputation of this rally being a real car breaker, it was never a sure thing, but for once Hirvonen's luck held and he achieved his first win of the season. With Loeb not scoring any points, Hirvonen is now only only seven points behind in the driver's championship. Suddenly, Hirvonen has a fighting chance again but the real question is, does Hirvonen have the skills to match Loeb?

Accidents in any motor sport are always scary but WRC always seems much worse because there are no sand traps and safety barriers to scrub off speed and absorb energy. Loeb's mistake sent him off the road at such high speed the that the car easily tumbled at least another 200m before coming to a stop. The car barrel rolled 5 times and yet both Loeb and Elana were complete unscathed. WRC cars are strong!

Petter Solberg's bad luck has definitely returned. The right front suspension failed on day 1 but he managed to continue even though the crew did not have the correct bolts at the temporary service. On day 2, after Loeb crashed, Solberg found himself in second place. In keeping with his style, Solberg decided to push for the win and broke the left front suspension. Despite a fantastic effort to temporarily repair the damage and make it back to normal service, he had to retire because he didn't have the spare parts. This is the first hint of the budget difficulties Solberg must be facing if he is missing critical spares like that.

Learning New Things

I received two books as a gift from my sister in England. The titles are A Little Book of The Green Man and A Little Book of Misericords. The books are about certain types of sculptures found in medieval churches and monasteries.

The Green Man is a head or face that is surrounded by foliage which is sometimes growing from the head or spewing from the mouth. Apparently variations of this image is known in many different parts of the world so it is not specific to medieval Europe or any one culture. The best part is no one seems to know exactly what the image means so it is quite mysterious. The book does not delve too deeply into the meaning but focuses on the visual enjoyment of the imagery. It is not difficult to find Green Man images on the web.

The word misericord is derived from the Latin for "pity" and "heart" and is a small ledge on which a monk could rest during a long church service. Rather appropriately named! The misericord is actually on the underside of the normal seat and is only visible when the seat is upright. For something that was completely hidden part of the time, the detail carved into the misericord is quite extraordinary. Even more curious is that the carvings rarely depicted religious scenes as one might expect, but scenes from everyday life. Here is a gallery dedicated to misericord images.

Although this subject is not one of my usual interests, it is never-the-less quite fascinating.

Catching Up With Family

I dropped in on my sister Pat for few minutes today. She and my brother-in-law John returned from England a week ago after a two week visit with our sister Marie. They has a great time on the trip although it sounds like it was a little hectic, but I joked that as retirees, both Pat and John could relax after the trip.

Marie is doing well. She has more management responsibilities at the stable where she works, but she was able to take two weeks off to spend with the visitor so it doesn't sound like there's too much pressure. My two nieces are also OK so it looks like the English contingent of the family are in good shape.

Asura Cryin' #6

I'm really enjoying Asura Cryin'. The story is good, there's lots of great mecha action, and there are some really kooky characters, but after six episodes, the questions are still piling up. The story so far has hinted the mecha are a product of both magic and science, and the power source is the spirit of a living being, but I get the feeling that there is more to it than that. There is a sense people are dabbling with apocalyptic forces and some of the characters seem a little fearful of the power that they are tapping into. I have absolutely no idea how this story is going to end, which is probably why I'm enjoying it so much.

Clannad After Story

The anime series Clannad and the sequel Clannad After Story, are very down-to-earth compared to the sci-fi and fantasy I usually watch, but I thoroughly enjoyed both series. The stories are just wonderful. Both these series would be at top of the list of anime recommendations to someone who doesn't like sci-fi or fantasy. Clannad After Story is actually better but is much more serious than Clannad which is more comedic.

Pandemic

So the World Health Organisation (WHO) has officially declared the swine flu to be a pandemic. I found this confusing at first until I understood that the WHO is referring to how much the swine flu has spread around the world. Pandemic refers to geographic distribution, not severity.

Then it occurred to me that this out break is indistinguishable from normal seasonal flu to my untrained eyes. The distribution is the same as normal seasonal flu, perhaps even lower since no one closes schools for seasonal flu for example. The fatality rate of swine flu is not that high, 141 in 30000 confirmed cases so far. That's only 0.4% compared to seasonal flu's fatality rate of 10-16 percent using the Wikipedia infection rates.

So, why doesn't the WHO declare pandemics for seasonal flu every year? It meets all the criterion, doesn't it? I'm confused again. What am I missing?

GIMP

Last night's TLUG meeting featured Giles Orr demonstrating GIMP, the free image processing application. He showed some of the common tasks he performs before posting his photographs to his web site. As only an occasional GIMP user myself, I found the talk very instructive and practical. Giles has been using GIMP for about 9 years but during the question period, even he had to admit there were corners of GIMP that he has not fully explored. I don't feel so bad now. ;-)

Telecommunications In Canada

Here is a transcript of Micheal Geist's statement and follow up questions at the Senate Standing Committee on Transport and Communications. It is a long read but it gives a great summary of the sorry state of telecommunications in Canada. The follow up question period is the best part where Geist doesn't pull any punches on the subject. The problem is, will anybody act on the committee's report? Unfortunately, knowing how these things work, the most likely answer is, No.

F1 Turkey

When Vettel grabbed poll position, I was really looking forward to a good race between the Brawns and Redbulls. Unfortunately, Vettel threw it away before the first lap was over and Barrichello's bad luck continued with a horrible start and the first retirement for Brawn GP this season. This race proved that passing remains a problem in F1 when Vettel closed up on Button but then could do nothing more.

The problem with the season so far is that nobody else has consistent pace which is practically handing both championships to the Button/Brawn GP combination. Even if the other teams somehow get their act together, Brawn GP would have to not score any points to loose their lead in the constructors championship. But more importantly, the season is at risk of becoming totally boring if the other teams don't get it together soon.

Pirate Party

Looks like Sweden's Pirate Party will win a seat in the EU Parliament, with a strong chance of a second but that won't be confirmed for another week. It really is immensely satisfying to see grass roots democracy still works somewhere in the world. Wish that were possible in Canada, but not with our "first past the post" election system. Sigh...

We Have Ignition

OK, it hasn't quite achieved that yet, but here are a few pictures of the National Ignition Facility at Livermore. While those are interesting, they are not nearly as impressive as the pictures of CERN's Large Hadron Collider. You know, those old sci-fi movies start to look kind of realistic when compared to this stuff. ;-)

Motivation

I pretty good at keeping myself motivated... well, most of the time :-). Occasionally, like today for example, I'll get have absolutely no desire to get anything useful accomplished. I'll feel guilty about it too which of course doesn't help much. Thankfully, my laziness usually doesn't last too long, but I wish I knew how to avoid it.

SQLAlchemy

SQLAlchemy is the most powerful and flexible database access library for Python. I've made several attempts in the past to get to grips with the module, but I'm finally making progress. SQLAlchemy is probably overkill for my Tom's TV Utility project, which requires a very simple database, but actually using the library for a practical project is definitely helping me to learn it. I decided to avoid SQLAlchemy's Object Relational Mapper (ORM) this time, and stick with the SQL Expression Language which seems to be a lot easier for me to understand. I suspect that is because I already know SQL so the ORM seems very clumsy when I can write the query in my head faster. The Expression Language is like writing SQL in Python so it is easier to convert the query.

The real benefit of SQLAlchemy is database independence. For example, all the unit tests use SQLite but I will use PostgreSQL when I finally get Tom's TV Utility into a usable state (which isn't too far off actually). SQLAlchemy will allow me to make that switch without any fuss. Why use such a powerful database engine for a trivial project? Why not? I already have a PostreSQL server running for SQLedger and several Trac projects so adding more databases is trivial and I get to learn how more about PostreSQL.

The Phillips Machine

I saw this story about a hydraulic computer from the 1950s called the MONIAC. The computer tries to model the economy using water to simulate the flow as funds. It is interesting as a mechanical system but I wonder: would it have predicted the current economic problems?

There is also Wikipedia article which is more detailed than the videos linked in the above story.

ScriptTest

Most of the programs that I write for myself have command line interfaces. My functional tests for command line programs were always a little clunky and I never had a satisfactory solution for testing file system modifications that occur when the program creates or updates files. Functional test frameworks for web and GUI apps are quite common, but not so much for command line programs.

A few months ago I discovered ScriptTest which solves the problem quite nicely. I've been using it in my Tom's TV Utility project and am very pleased with the results. ScriptTest is quite easy to use but the online documentation is a little out of date so I recommend reading the __doc__ strings. Oh yes, ScriptTest is written in Python.

The online documentation describes a simple test so I'll show a more complex one from my project.

import os
from scripttest import TestFileEnvironment

class TestTomsTV:
    def setup(self):
        self.env = TestFileEnvironment(
                base_path=os.path.abspath('tests/test-output'),
                cwd=os.getcwd(),
                script_path=[os.getcwd()],
                environ={
                    'PATH': '/usr/local/bin:/usr/bin:/bin',
                    'HOME': os.path.abspath('tests/test-output'),
                   },
                template_path=os.path.abspath('tests/data'),
                ignore_hidden=False)

OK, maybe that's a little too complex ;-) but it shows ScriptTest's flexibility.

• base_path sets the directory where the utility will write files. The default is ./test-output.
• cwd sets the working directory for the program under test. The default is base_path.
• script_path limits where the framework will search for the program under test. If you do not give the full path of the program, it will be found the normal $PATH which is not desirable in development.
• environ passes a custom environment to the program under test. My utility depends on $HOME so I ensure that the variable points to the base_path directory.
• template_path sets the directory where preset files will be found. This is handy feature which allows tests to initialize files in the base_path area.
• ignore_hidden toggles how the framework reports hidden file and directories. My program creates a hidden directory so I require the framework to record that detail.

Here's a sample test.

    def test_list(self):
        'Test: tomstv list'
        self.env.writefile('.tv/db.sqlite', frompath='file1.sqlite')
        cmd = 'tomstv list'
        result = self.env.run(cmd)
        assert result.returncode == 0,\
                'Expect returncode=0, got %r' % result.returncode
        assert result.stderr == '',\
                'Expect stderr=%r, got %r' % result.stderr
        expect = data.file1_list
        print result.stdout.split('\n')
        for r, e in zip(result.stdout.split('\n'), expect.split('\n')):
            assert r == e, '\nExpect %r\ngot %r' % (e, r)

The writefile method copies a test database from the template_path mentioned before and the run method executes the program. When it completes, you can verify output is correct and check files that where changed, created, or deleted, but this test doesn't do that.

If you write functional test for command line programs, you should definitely add ScriptTest to your toolbox.

RTorrent Remote Control

My preferred bittorrent client is rtorrent. I like it because the text user interface makes it convenient for running on server without X installed. rtorrent has a XML-RPC interface which offers some interesting possibilities for remote control from another application. XML-RPC is a moderately complex protocol and parsing XML is much easier in a high level language. Fortunately someone wrote an XML-RPC utility in Python specifically for rtorrent which makes tinkering a whole easier.

Enabling the XML-RPC interface only requires one line in the .rtorrent.rc config file:

scgi_local = /home/tlow/.rtorrent/socket

You can also enable the interface on a TCP socket but the rtorrent wiki advises against this because of the security implications.

The Python utility can be used on the command line. The following returns the hashes of all the .torrent files currently loaded into into rtorrent:

$ ./xmlrpc2scgi.py scgi:///home/tlow/.rtorrent/socket download_list ''
<?xml version="1.0" encoding="UTF-8"?>
<methodResponse>
<params>
<param><value><array><data>
<value><string>D19075DB7A5BF5B0B9B89B30EE39CC132BF64F4A</string></value>
<value><string>B00F710FFC9FC09F019FEA2A4718CCFF794475F6</string></value>
<value><string>D7521742B5037E691EFB9CA33C24A29B73F5C68C</string></value>
<value><string>1B74111F0848AA7C056867AEB1A3FBAA744DB356</string></value>
</data></array></value></param>
</params>
</methodResponse>

Obviously you still need to pipe the output to another utility for further processing but the output sort of readable.

The utility can also be used a library module which is much more interesting:

$ ipython
Python 2.5.4 (r254:67916, Feb 17 2009, 20:16:45)
Type "copyright", "credits" or "license" for more information.

IPython 0.9.1 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object'. ?object also works, ?? prints more.

In [1]: import xmlrpc2scgi as xs

In [2]: rtc=xs.RTorrentXMLRPCClient('scgi:///home/tlow/.rtorrent/socket')

In [3]: rtc.download_list('')
Out[3]: 
['D19075DB7A5BF5B0B9B89B30EE39CC132BF64F4A',
 'B00F710FFC9FC09F019FEA2A4718CCFF794475F6',
 'D7521742B5037E691EFB9CA33C24A29B73F5C68C',
 '1B74111F0848AA7C056867AEB1A3FBAA744DB356']

Which is the same as the XML output above but as a Python list.

The main problem with rtorrent's XML-RPC interface is that very few of the commands are documented, so it requires some more digging to understand exactly what is possible. I already have an idea of how I want to use the interface, but I'll leave that for another time.