Nick's posterous http://nicktj.posterous.com Most recent posts at Nick's posterous posterous.com Mon, 06 Feb 2012 12:30:00 -0800 Share a single X window with someone over VNC http://nicktj.posterous.com/share-a-single-x-window-with-someone-over-vnc http://nicktj.posterous.com/share-a-single-x-window-with-someone-over-vnc

I recently found a need to share an NX session open on my desktop with someone else remotely in Linux.  Sure, I could have setup a VNC sharing my entire X desktop,but I didn't see the need to send 3840x1080 over the WAN for a smaller section of interesting data.

This slick little trick comes compliments of a StackExchange based website, Super User:

 

x11vnc supports sharing a window based on it's id.  Basic steps:

  1. Run xwininfo from a console. It will change your cursor. Click on the window you want to share. xwininfo will print out the window id.
  2. Run x11vnc -id "id from xwininfo" replacing "id from xwininfo" with the appropriate id.

 

Permalink | Leave a comment  »

]]>
http://posterous.com/images/profile/missing-user-75.png http://posterous.com/users/heOPIhURBYqT0 Nick Jones nicktj Nick Jones
Thu, 12 Jan 2012 12:41:00 -0800 Quickly tidy XML in vim on Linux http://nicktj.posterous.com/quickly-tidy-xml-in-vim-on-linux http://nicktj.posterous.com/quickly-tidy-xml-in-vim-on-linux

If you find yourself staring at an ugly representation of an XML doc, use the following command to tidy it up using xmllint:

$!xmllint --format --recover - 2>/dev/null

Permalink | Leave a comment  »

]]>
http://posterous.com/images/profile/missing-user-75.png http://posterous.com/users/heOPIhURBYqT0 Nick Jones nicktj Nick Jones
Wed, 14 Sep 2011 06:54:00 -0700 Hadoop on LVM2 http://nicktj.posterous.com/hadoop-on-lvm2 http://nicktj.posterous.com/hadoop-on-lvm2

I had built up a cluster from the remains of past projects and other odds and ends of hardware around the labs so inevitably the nodes were configured with mostly odd sized disks. Most of the time everything runs smoothly; however, the amount of stored data on the cluster has been growing quite a bit. We had plenty of capacity for it but several nodes with smaller disks would fill up while executing, get blacklisted by jobs, and become less useful to the cluster.

Amidst upgrading (i.e. adding additional disks), I decided to give LVM2 a shot.  One large expandable volume freed us from dealing with unique Hadoop config files. LVM2 (using defaults) will get you a single volume to use but the performance of multiple disks is lost.  Nevertheless, LVM2 can be configured to stride across disks (much like a RAID 0) but without the requirement that disks are of the same size.  I recommend at least trying it; it does make system management a whole lot easier.

These are my notes on how I setup the volume; feel free to adjust as you see fit.

  1. fdisk both physical drives with a partition type of 8e (Linux LVM).
  2. Create a new LVM volume group with the physical disk partitions
    # vgcreate hadoopdisks /dev/sda1 /dev/sdc1
    Volume group "hadoopdisks" successfully created
  3. The disks I used were one terabyte in size, but not all of that space is actually usable. This was a pretty easy way to get LVM to tell you the number of extents available.
    # lvcreate -L2T -i2 -nhadoop hadoopdisks /dev/sda1 /dev/sdc1
    Using default stripesize 64.00KiB
    Insufficient free extents (476932) in volume group hadoopdisks: 524288 required
    
    # lvcreate -l 476932 -i2 -nhadoop hadoopdisks /dev/sda1 /dev/sdc1
    Using default stripesize 64.00KiB
    Logical volume "hadoop" created
  4. One last change tells LVM2 it's free to allocate anywhere it can.
    # lvchange --alloc anywhere /dev/hadoopdisks/hadoop

Permalink | Leave a comment  »

]]>
http://posterous.com/images/profile/missing-user-75.png http://posterous.com/users/heOPIhURBYqT0 Nick Jones nicktj Nick Jones
Mon, 22 Aug 2011 08:01:16 -0700 Hadoop and bnx2 http://nicktj.posterous.com/hadoop-and-bnx2 http://nicktj.posterous.com/hadoop-and-bnx2

Awhile back, I setup a hadoop cluster here at the office using some discarded old boards.  They're actually a decent platform to use as Hadoop nodes as I came to find out; however, I kept having random nodes die every so often with the following kernel message:

bnx2: Chip not in correct endian mode

The solution?

...disable the IRQ balancer daemon.

It turns out that when the IRQ balancer reallocates the device's interrupt to another CPU's local APIC, it must first disable it before moving it to the new APIC ID.  However, if the device had reset itself during that time, you'll end up with this endian mode problem.  Running the IRQ balancer on a TaskTracker node after they've been balanced once doesn't really have a benefit anyway since you're likely scaling to the entire machine's size.

Permalink | Leave a comment  »

]]>
http://posterous.com/images/profile/missing-user-75.png http://posterous.com/users/heOPIhURBYqT0 Nick Jones nicktj Nick Jones