Better decision tree graphics for rpart via party and partykit

I’ve been using Graphviz to create better decision tree graphics “by hand” for rpart objects created in R (final tree). I stumbled on this post that shows how one could convert an rpart object to a party project via the as.party function in partykit to utilize the plot functions in party. It looks quite nice.

I might have to do additional hacking as I like to display the node size and percentage of success in every node. For example, in rpart, I do something like

## rpartObj created from rpart
textRpartCustom <- 
{
    nclass <- (ncol(yval) - 1L)/2
    group <- yval[, 1L]
    counts <- yval[, 1L + (1L:nclass)]
    if (!is.null(ylevel)) 
        group <- ylevel[group]
    temp1 <- rpart:::formatg(counts, digits)
    if (nclass > 1) {
        ## temp1 <- apply(matrix(temp1, ncol = nclass), 1, paste, 
        ##     collapse = "/")
      temp1 <- matrix(as.numeric(temp1), ncol=nclass)
      ##temp1 <- paste("p=", round(temp1[, 2] / apply(temp1, 1, sum)*100, 1), "%", "; N=", apply(temp1, 1, sum), sep="")
      temp1 <- paste("", round(temp1[, 2] / apply(temp1, 1, sum)*100, 1), "%", "; ", apply(temp1, 1, sum), sep="") 
    }
    if (use.n) {
        out <- paste(format(group, justify = "left"), "\n", temp1, 
            sep = "")
    }
    else {
        out <- format(group, justify = "left")
    }
    return(out)
}

rpartObj$functions$text <- textRpartCustom
plot(rpartObj)
text(rpartObj)

to get these information to display for a classification fit.

Enable root account in Ubuntu?

After my recent experience with broken su and sudo commands in a failed system upgrade, I realized that although disabling the root account has many advantages, one of the disadvantage is that I can’t login as root in the terminal when I’m physically in front of the system. This is a major issue if su, sudo, and passwd binaries are broken somehow. Luckily, chroot was there to the rescue for me. Now, I contemplate whether I should enable the root account on my systems…

Flipping the classroom: creating screencast lectures in Linux

I’m debating the idea (hype) of flipping the classroom for one of my classes next Fall where students watch lecture videos at home (or elsewhere) so I could spend class time doing more hands-on activities like discussing the art of data analysis and how to solve problems with statistics. I think Khan Academy, Udacity, and Coursera are doing a great service for humanity by offering high quality courses taught by excellent teachers online that are accessible to anyone with an internet connection.

I don’t claim to be a great teacher, but I think my own students might benefit from this pedagogical method. My main concern with this approach is that not all students will watch the lectures, just as how not all students read the assigned readings (guilty as a student). I guess I can give students short quizzes during lecture to push them to watch the videos. Also, I’ll give my usual challenging homework so that only students that study the material well could excel. By flipping the classroom, more material could be covered, students have access to the recordings in addition to my slides, and I could make sure everything I want to be said are recorded (as opposed to a live session where I could forget a few points). Lecture times can then be more interactive as opposed to me lecturing them for an hour.

I think most of the online education sites use Camtasia with a Wacom Cintiq to produce their videos. I use Linux and cannot afford such an expensive device. I plan on using a screencast software like recordMyDesktop or Istanbul to record the desktop screen and audio. For recordMyDesktop, I had issues with the encode on the fly option, which means recording very long videos could be an issue (1 minute of raw video takes up about 210MB, and 1 minute encoded video takes up about 8MB). Istanbul records on the fly without problem (I think). I haven’t tried recording for an hour and 20 minutes yet.

My plan is to create my lecture slides with LaTeX Beamer and use Xournal to annotate the slides as I’m lecturing; hopefully my Asus T101MT netbook is strong enough to do the recording as I utilize it’s touchscreen capabilities. I can just switch over to Emacs to illustrate data analysis in R when needed. My main concern now is where I could host these (large) videos…

Update 4/27/2012: Screencast with ffmpeg

After some testing, I think the best screencast software on Linux would have to be ffmpeg. First, remove ffmpeg and compile it from source based on the latest version per this post. Then, create screencast.sh:

#! /bin/bash
DATE=`date +%Y%m%d`
TIME=`date +%Hh%M`
ffmpeg -y -f alsa -ac 2 -i pulse -f x11grab -r 24 -s $(xwininfo -root | grep 'geometry' | awk '{print $2;}') -i :0.0 -c:v libx264 -preset veryfast -crf 22 -c:a libmp3lame -ar 44100 -ab 24k -threads 0 /tmp/screencast_$DATE-$TIME.mp4

For more libx264 options, see this page.

Record streaming radio with streamripper

Yes, many radio shows are available as podcasts. However, some are not. If a radio show is also broadcasted via a live stream online, then we could record it with streamripper. I did so as follow:

sudo apt-get install streamripper ## install
## record for 60 seconds
streamripper http://kpcclive1.publicradio.org:80/ -s -l 60 -a "Prarie - %d"

To have the show be recorded automatically, first create record.sh:

#! /bin/bash

## set in crontab:
## 59 17 * * 6 /path/to/record.sh
cd /path/to/save/; streamripper http://kpcclive1.publicradio.org:80/ -s -l 3720 -a "Prarie - %d" &> /dev/null

Then add the following cron entry via crontab -e:

59 17 * * 6 /path/to/record.sh

Test ram with Memtest86+ and ignore bad parts with badram in grub

Recently, my computer kept freezing whenever I started conkeror (with 100+ buffers loading from a previous session). Folks over at #conkeror on freenode suggested that the problem might be due to faulty ram. They suggested testing my ram with Memtest86+. It is installed by default on Ubuntu.

If you have multiple sticks of ram, test one stick at a time. It’s best to test one stick per night as the test can take hours. To test the ram, restart your computer and go to the grub menu (hold shift if your grub menu doesn’t display automatically). Then, select the “Memtest86+” boot option. Press “c”, “4″, and “3″ to display the error locations according to the BadRAM syntax (converting the default faulty memory addresses is not obvious to me and others). If you don’t do this, you will end up wasting time fixing your boot options (details later).

If you know which ram sticks are bad, replace them if they are under warranty. If they are not under warranty and you can’t afford new ram, you can make use of BadRAM, incorporated by default in grub2, per these documentations. That is, edit /etc/default/grub and specify the faulty ram addresses with the GRUB_BADRAM option.

More information on running Linux with broken memory can be found here.

When I tried this out, I did not use the proper memory address syntax so my computer failed to boot. What made things even worse was that my hard drive was encrypted. Luckily, I can still access grub, and after many trials and tribulations, I fixed the problem by booting the computer with an Ubuntu live disk (usb), mounting the first, unencrypted partition (/dev/sda1) of the hard drive that stored /boot, and removing the badram option in /boot/grub/grub.cfg (replace “boot” with the mount path). Before figuring out the solution, I was trying to mount /dev/sda5, the encrypted partition, according to this and this as I thought that was where /boot resided. I also thought I had to generate a new initrd image. Luckily I didn’t have to (and didn’t succeed in trying) as that would have further complicate my boot options as I have experienced in the past.

After removing the bad ram, conkeror still crashed for me. Either something is wrong with other pieces of my hardware or something is going on with the xulrunner sucking up my system resources. I was able to stop the crashes by placing this in my conkeror rc file.