ASCIIMath creating images

Tuesday, January 18, 2011

Further adventures with Mercurial: Converting a repository from Subversion

I have been using Mercurial for a little while now, and have become accustomed enough to try converting my rather large Subversion repository I have sitting on my home server.  This has proven to be a bit challenging.

Mercurial tracks files only, not directories - it does track the location of files, but it is not possible to have an empty directory in a Mercurial repository (I have taken to creating empty hidden files as a workaround).  Apparently this causes problems for some of the conversion/interaction tools, such hgsvn.

After several tries of various things, I've had success with hgsubversion python extension.  However, installing in under Ubuntu (10.10, or Maverick Meerkat) is not completely trivial. Here is what's needed to make it work:

  1. To build a version of subvertpy newer than what's in the repository, install libsvn-dev and python-dev: sudo apt-get install libsvn-dev python-dev
  2. Download subvertpy from http://samba.org/~jelmer/subvertpy/, then build and install it locally. You need at least 0.7.4 (Ubuntu's repositories have 0.7.3).
  3. Finally, follow the instructions on http://mercurial.selenic.com/wiki/HgSubversion to install hgsubversion.  I moved it into /usr/local/lib.
Then, a new Mercurial repository can be cloned with (for example) hg clone svn+ssh://myhost.org/myrepo.  Although the HgSubversion site warns about memory leaks, I found this procedure quite well.

Thursday, January 13, 2011

Using MATLAB with simple single-threaded scripts on a cluster

McGill University is part of CLUMEQ, and it is relatively easy to gain access to the Krylov computer, a small(ish) system of 300 cores. While finishing up my thesis, I made some use of this cluster to run simulations in MATLAB.  Here are a few notes on efficiently using the system.

Initially, one would assume the best way to utilize the system is to use the Parallel Toolbox, and surely some applications can fit very well to this mold.  In my case, the parallelism within my code was very data-oriented and thus I couldn't get a great speedup.  Instead, I simply used the cluster as a set of easily queued single-thread computers.  A simple script turns a file of MATLAB commands into a series of queue submissions:

cat matcommands | while read line; do
        echo "#!/bin/bash" > temp.sh
        echo "#PBS -l nodes=1:ppn=1,walltime=1:00:00:00" >> temp.sh
        echo "#PBS -V" >> temp.sh
        echo "#PBS -N Bmatlab_s" >> temp.sh
        echo "cd WorkingDirectory" >> temp.sh

        echo "matlab -nojvm -nodisplay -nodesktop -nosplash -r \"" $line '"' >> temp.sh

        msub temp.sh
done

The "matcommands" file is simply a series of single lines of
script(arguments); exit();
script(arguments); exit();
and so on; these will be executed in parallel independently of each other. Very handy if testing a script with varying data.  The script must be written such that the output is saved to a file whose name depends on the arguments.  I then would download all the result files to my desktop and analyze everything there.

This method of using MATLAB on the cluster also works around the restriction that Krylov only has a 16-cpu license for the Parallel toolbox.  Instead, 30 jobs would typically be processed at the same time (which I assume is simply a local policy restriction).

Wednesday, January 5, 2011

Netbook!

A new addition to my menagerie of computers at home, I have gotten myself a netbook.  I haven't had a laptop since I sold my Mac Powerbook G4, opting instead for a Mac Mini: I had been using the PB effectively as a desktop anyways, on a pedestal with an external keyboard and mouse.

On our last trip to the in-laws in Kitchener, this created a problem, of course: I was still working on my thesis, so I used Madeline's Macbook, much to her annoyance; she has been pushing me to get a portable ever since.  I maintain that the primary purpose of a portable is to be portable - if I need power, I use a desktop, or connect remotely to one!  Thus a netbook is the best choice for me.

I just bought a Asus EEE 1000HE, used but effectively new - let someone else pay the initial depreciation!  This was chosen in part for its Linux compatibility, but the EEE's also have a pretty good reputation as far as netbooks are concerned (without being super expensive).  I named it Earendil (it should have an umlaut on the a).

There is very little that needs to be done to make it all work: I found I had to disable a few modules from being loaded to make sure the wireless resumes after waking from suspend. A fix was posted here: http://www.twentyways.com/2010/11/ (basically a few lines to add to /etc/modprobe.d/blacklist.conf)  It's a bit noisy, since the fan effectively runs all the time, but not excessively so.

I did manage to mess the install up once already though, losing the ability to suspend.  Trying to get the CPU to truly idle (such that the fan can turn off), I installed eee-applet, powertop, and some other packages - and something messed things up.  Even apt-get purge in reverse order didn't fix it - I ended up reinstalling.  I might try again with more care (and backups!)