Nick's posterous http://nicktj.posterous.com Most recent posts at Nick's posterous posterous.com Mon, 06 Feb 2012 12:30:00 -0800 Share a single X window with someone over VNC http://nicktj.posterous.com/share-a-single-x-window-with-someone-over-vnc http://nicktj.posterous.com/share-a-single-x-window-with-someone-over-vnc

I recently found a need to share an NX session open on my desktop with someone else remotely in Linux.  Sure, I could have setup a VNC sharing my entire X desktop,but I didn't see the need to send 3840x1080 over the WAN for a smaller section of interesting data.

This slick little trick comes compliments of a StackExchange based website, Super User:

 

x11vnc supports sharing a window based on it's id.  Basic steps:

  1. Run xwininfo from a console. It will change your cursor. Click on the window you want to share. xwininfo will print out the window id.
  2. Run x11vnc -id "id from xwininfo" replacing "id from xwininfo" with the appropriate id.

 

Permalink | Leave a comment  »

]]>
http://posterous.com/images/profile/missing-user-75.png http://posterous.com/users/heOPIhURBYqT0 Nick Jones nicktj Nick Jones
Sat, 04 Feb 2012 16:57:45 -0800 Beer of the night: Convict Hill, Oatmeal Stout http://nicktj.posterous.com/beer-of-the-night-convict-hill-oatmeal-stout http://nicktj.posterous.com/beer-of-the-night-convict-hill-oatmeal-stout

-1382861174

Slightly hoppy flavor with a smooth and clean finish. Willing to drink but only a maybe on a repurchase.

Permalink | Leave a comment  »

]]>
http://posterous.com/images/profile/missing-user-75.png http://posterous.com/users/heOPIhURBYqT0 Nick Jones nicktj Nick Jones
Fri, 13 Jan 2012 09:36:43 -0800 Publish syntax colorized code from html http://nicktj.posterous.com/publish-syntax-colorized-code-from-html http://nicktj.posterous.com/publish-syntax-colorized-code-from-html

I came across this pretty slick way of getting reasonable syntax highlighting for code copied into PowerPoint, email, etc. by exporting the buffer as HTML directly from Vim.

Now you have no excuses for ugly formatting and lack of syntax highlighting in your presentations!

Permalink | Leave a comment  »

]]>
http://posterous.com/images/profile/missing-user-75.png http://posterous.com/users/heOPIhURBYqT0 Nick Jones nicktj Nick Jones
Thu, 12 Jan 2012 12:41:00 -0800 Quickly tidy XML in vim on Linux http://nicktj.posterous.com/quickly-tidy-xml-in-vim-on-linux http://nicktj.posterous.com/quickly-tidy-xml-in-vim-on-linux

If you find yourself staring at an ugly representation of an XML doc, use the following command to tidy it up using xmllint:

$!xmllint --format --recover - 2>/dev/null

Permalink | Leave a comment  »

]]>
http://posterous.com/images/profile/missing-user-75.png http://posterous.com/users/heOPIhURBYqT0 Nick Jones nicktj Nick Jones
Tue, 27 Dec 2011 09:34:03 -0800 Web vs product design for web apps http://nicktj.posterous.com/web-vs-product-design-for-web-apps http://nicktj.posterous.com/web-vs-product-design-for-web-apps Great article on working at mutiple levels in the stack and focusing on strengths within a team.

http://jeffcroft.com/blog/2011/dec/25/2012-web-design-vs-product-design/

Permalink | Leave a comment  »

]]>
http://posterous.com/images/profile/missing-user-75.png http://posterous.com/users/heOPIhURBYqT0 Nick Jones nicktj Nick Jones
Fri, 14 Oct 2011 07:11:00 -0700 APT Repository Keys behind a HTTP Proxy http://nicktj.posterous.com/apt-repository-keys-behind-a-http-proxy http://nicktj.posterous.com/apt-repository-keys-behind-a-http-proxy

We have an HTTP proxy at work that doesn't like hkp style requests; however, there's this little gem to make it work perfectly.

sudo gpg --keyserver yourkeyserver.somewhere.org --keyserver-options http-proxy=http://myworkproxy:port --recv-keys ABCDEF01

 

Permalink | Leave a comment  »

]]>
http://posterous.com/images/profile/missing-user-75.png http://posterous.com/users/heOPIhURBYqT0 Nick Jones nicktj Nick Jones
Fri, 16 Sep 2011 20:30:59 -0700 Acts As Statistic Rails Gem http://nicktj.posterous.com/acts-as-statistic-rails-gem http://nicktj.posterous.com/acts-as-statistic-rails-gem I finally got around to putting time into building out a gem for unified statistics gathering. The goal is to build up enough support to mimic something like the Benchmark class but log to an ActiveRecord model in a normalized format. However, unlike Benchmark, it will also support stats of counts and rates.

It was almost a year to the day before I put some effort into it beyond script/generate plugin. I do hope to keep up the development at a significantly faster rate. :-)

https://github.com/nickjones/acts_as_statistic

Permalink | Leave a comment  »

]]>
http://posterous.com/images/profile/missing-user-75.png http://posterous.com/users/heOPIhURBYqT0 Nick Jones nicktj Nick Jones
Wed, 14 Sep 2011 06:54:00 -0700 Hadoop on LVM2 http://nicktj.posterous.com/hadoop-on-lvm2 http://nicktj.posterous.com/hadoop-on-lvm2

I had built up a cluster from the remains of past projects and other odds and ends of hardware around the labs so inevitably the nodes were configured with mostly odd sized disks. Most of the time everything runs smoothly; however, the amount of stored data on the cluster has been growing quite a bit. We had plenty of capacity for it but several nodes with smaller disks would fill up while executing, get blacklisted by jobs, and become less useful to the cluster.

Amidst upgrading (i.e. adding additional disks), I decided to give LVM2 a shot.  One large expandable volume freed us from dealing with unique Hadoop config files. LVM2 (using defaults) will get you a single volume to use but the performance of multiple disks is lost.  Nevertheless, LVM2 can be configured to stride across disks (much like a RAID 0) but without the requirement that disks are of the same size.  I recommend at least trying it; it does make system management a whole lot easier.

These are my notes on how I setup the volume; feel free to adjust as you see fit.

  1. fdisk both physical drives with a partition type of 8e (Linux LVM).
  2. Create a new LVM volume group with the physical disk partitions
    # vgcreate hadoopdisks /dev/sda1 /dev/sdc1
    Volume group "hadoopdisks" successfully created
  3. The disks I used were one terabyte in size, but not all of that space is actually usable. This was a pretty easy way to get LVM to tell you the number of extents available.
    # lvcreate -L2T -i2 -nhadoop hadoopdisks /dev/sda1 /dev/sdc1
    Using default stripesize 64.00KiB
    Insufficient free extents (476932) in volume group hadoopdisks: 524288 required
    
    # lvcreate -l 476932 -i2 -nhadoop hadoopdisks /dev/sda1 /dev/sdc1
    Using default stripesize 64.00KiB
    Logical volume "hadoop" created
  4. One last change tells LVM2 it's free to allocate anywhere it can.
    # lvchange --alloc anywhere /dev/hadoopdisks/hadoop

Permalink | Leave a comment  »

]]>
http://posterous.com/images/profile/missing-user-75.png http://posterous.com/users/heOPIhURBYqT0 Nick Jones nicktj Nick Jones
Mon, 22 Aug 2011 08:01:16 -0700 Hadoop and bnx2 http://nicktj.posterous.com/hadoop-and-bnx2 http://nicktj.posterous.com/hadoop-and-bnx2

Awhile back, I setup a hadoop cluster here at the office using some discarded old boards.  They're actually a decent platform to use as Hadoop nodes as I came to find out; however, I kept having random nodes die every so often with the following kernel message:

bnx2: Chip not in correct endian mode

The solution?

...disable the IRQ balancer daemon.

It turns out that when the IRQ balancer reallocates the device's interrupt to another CPU's local APIC, it must first disable it before moving it to the new APIC ID.  However, if the device had reset itself during that time, you'll end up with this endian mode problem.  Running the IRQ balancer on a TaskTracker node after they've been balanced once doesn't really have a benefit anyway since you're likely scaling to the entire machine's size.

Permalink | Leave a comment  »

]]>
http://posterous.com/images/profile/missing-user-75.png http://posterous.com/users/heOPIhURBYqT0 Nick Jones nicktj Nick Jones
Sat, 13 Aug 2011 15:00:00 -0700 Lone Star Ruby Conference 2011: Day 2 Notes http://nicktj.posterous.com/lone-star-ruby-conference-2011-day-2-notes http://nicktj.posterous.com/lone-star-ruby-conference-2011-day-2-notes
These are some of my (terse) notes from day two of the LSRC.  It would probably be helpful to read my colleague's notes as well.

Blow up Your Views

  • Instance variables in views suck
    • Real interface between C/V
      (github: voxdolo/decent_exposure)
    • Added "exposes(:foo)" in the controller and the view starts using just "foo" instead of "@foo"
  • Kill helpers
    • Example of timestamps
      • Should be writing helpers to format timestamps
      • Usually helpers are for specific objects (e.g. formats the timestamp on a particular kind of object)
    • We write helpers like print_name(user) instead of user.print_name but we want the later.
  • View Objects Implemented
    • Create an /app/decorators folder
    • Implement an attr_accessible approach to the view: Limit what it is allowed to call.
    • Github: jcasimir/draper
  • Should be splitting up responsibilities
    • Always render the same thing from the controller (define an API)
    • The view should deal with drawing the display for the client
    • Send them a JS template, assets, etc.
  • Render Engines
    • Plain HTML with jQuery Replacement
    • Mustache
    • Handlebars (SproutCore)
    • ICanHazJS

ZeroMQ

  • ZeroMQ is a messaging bus
  • Zero stands for 'zero brokers'
  • Scalability
    • Serve requests faster
    • Deals with complexity scalability
  • Uses an actor model
    • stateless
    • self-contained pieces
  • What ZeroMQ isn't
    • ZeroMQ isn't designed to write an HTTP server
      • It doesn't care what you send in messages but it cares how it sends it
      • ZeroMQ sockets only talk to other ZeroMQ sockets
    • Could write a layer that interprets HTTP and sends it into ZeroMQ (Mongrel2)
  • How to use it
    • Ruby: gem install zmq
    • Based in C and supported in every language
    • Threadsafe but can't share a socket between threads
  • Example: Request/Reply Sockets
    • Can start a client and it starts trying to communicate but will block until the server responds
    • If the server crashes in the middle and restarts but it wont automatically work.
  • Publish/Subscribe model
    • Uses option "SUBSCRIBE"
    • If the publisher goes down, the subscriber is durable
  • Push/Pull sockets
    • Multiple pull sockets can bind to a single push socket
    • Spawn more workers if the queue is backing up
  • Durable Sockets
    • Set a socket identity: give it a name
    • Setup spooling to disk for unreceived messages

JavaScript for people who didn't learn JavaScript (@jwo)

  • Prototype Inheritance
    • Only one prototype per function (but can be switched out when needed)
    • Like Ruby modules but you only get one.
    • Calling super
      • JavaScript doesn't provide a method to call super
      • var Note = Backbone.Model.extend({
          set: function(...) { } <-- override behavior and call Backbone.Model.set.call(...)
  • Using 'new'
    • Don't just use new because you think you have to
    • Use 'new' if you need the prototype of the object
    • Don't use it if your just creating a new Array, etc.
    • Object.create is useful but some browser dependencies
    • "Don't write new Array()"
      • Arrays in JS don't have limits
    • "Don't write new Function()" - just use the funciton() literal
  • Invocation
    • Functions get declared-parameters, this, and arguments
    • Creating a new object with Object.create(HelloWorld) and calling hi.speak() you get the prototype
    • Using just HelloWorld.speak doesn't provide scope so 'this' isn't bound to the object
    • JavaScript allows you to pass in the object that you want 'this' to reference (so function() in the method can actually operate on whatever object you provide)
  • JavaScript puts semicolons in where you "forgot them"
    • return
      {
        javascript: "foo"
      }; // This will fail because return gets a magic ';' on the end.

Testing JavaScript with Jasimine

  • Jasmine has rspec like syntax
  • Jasmine with jQuery
    • jasmine-jquery adds more matchers
    • Lettering.js
      • Allows you to throw spans in between chars in a particular HTML DOM element
    • SpecRunner.html
      • Added jquery and lettering js files
  • Jasmine with Rails
    • Add 'jasmine' to Gemfile
    • rake jasmine / rake jasmine:ci
  • Easier way with Evergreen "gem install evergreen"
    • Create public folder (with js in it)
    • Create spec folder with specs in it
    • evergreen run
    • Evergreen follows the name_spec.js file naming convention
    • Install coffee-script and works with Jasmine (make .coffee file)
  • Headless running
    • Install capybara-webkit
    • Switched from selenium to webkit and runs faster
  • Rosie gives you factory_girl for JS
    • github: bkeepers/rosie
  • jasminerice - jasmine + coffee script + rails 3.1

The Ballad of Goliath, EventMachine, and MongoDB

These notes are bit short since a lot of the talk was random aspects of MongoDB towards the end.
  • Goliath
    • Event Driven (Built upon EventMachine)
    • Web and app server
    • Uses fibers in Ruby 1.9
    • Matches a rackup app somewhat
      • Inherits from Goliath::API
    • Get an array of command line options
      • -u for actually specifying the user
    • By default it doesn't log at all
  • EventMachine: Watch peepcode screencode
  • MongoDB
    • Just doing Mongo::ReplSetConnection.new(host), only R/W to primary
    • Use Mongo::ReplSetConnection.new(host, :read_secondary => true) to get better read performance

Polygot Parallelism

This talk started off with a brief intro to Erlang and continued a discussion into how Rackspace was able to successfully solve a massively parallel data gathering problem.
  • Erlang
    • Immutable Data Structures
    • Single assignment => everything is immutable
  • Webmachine
    • Similar to Sinatra or Goliath
    • Basically developing a custom web server for your application's behavior.
    • http://webmachine.basho.com
    • Leaves sensible defaults for the whole HTTP life cycle and can override at any step.

Permalink | Leave a comment  »

]]>
http://posterous.com/images/profile/missing-user-75.png http://posterous.com/users/heOPIhURBYqT0 Nick Jones nicktj Nick Jones
Sun, 03 Jul 2011 11:48:00 -0700 Capistrano and Bundler http://nicktj.posterous.com/capistrano-and-bundler http://nicktj.posterous.com/capistrano-and-bundler

Sounds great doesn't it? Carefree deployments and gem management for your app. However, I ran into a few pitfalls that made the first experience less than carefree.

bundle:install step

After integrating bundler into capistrano with

require 'bundler/capistrano'

a bundle:install step gets kicked off to install your gems locally.  Pretty cool, but beware of defaults if you're on a multiserver deployment setup.  The default step flags are --deployment --quiet which will download all of the gems from the net.  Maybe not so bad for some folks but on a beefy rails app, this took forever across four servers and 38 gems.  Nevertheless, capistrano is easily customizable and can make this easier with the following line:

set :bundle_flags, "--deployment --quiet --local"

In order for this to work properly though, make sure and package up the gems with your application!

Custom rake tasks

The app I work on has a few custom steps for compiling a custom Ruby library, building extra documentation, etc. so there are a few more after "deploy:update_code" steps to address upon deployment.  Everything was pretty simple and easy to update for bundler; the only trick is knowing you have to do this update.

Each custom step was built into a rake task for lazyness in development, so each resembled something like this:

namespace :xapian do
   desc "Rebuild Xapian index."
   task :rebuild, :roles => :xapian_index do
     run "cd #{release_path}; rake xapian:rebuild_index models=\"MyModelName\"", :env => {'RAILS_ENV' => stage}
   end
end

We were calling rake directly, but this will lead you to step failures pretty quickly if the version of rake differs (or doesn't exist) in the deployment environment.  Simply replace rake with #{rake}.  The step will automatically be executed as #{bundle_cmd} rake instead (e.g. bundle exec rake by default).

 

Permalink | Leave a comment  »

]]>
http://posterous.com/images/profile/missing-user-75.png http://posterous.com/users/heOPIhURBYqT0 Nick Jones nicktj Nick Jones
Wed, 18 May 2011 18:56:00 -0700 Versioning ActionWebService on Rails http://nicktj.posterous.com/versioning-actionwebservice-on-rails http://nicktj.posterous.com/versioning-actionwebservice-on-rails

I started a Rails project a few years ago that has grown up.  These days, there is a main consumer of a SOAP API provided through ActionWebService (with more on the way).  Most generally try and head down the RESTful path, but with server-to-server communication, it just made it easier to take a structured path.  (Custom XML/JSON output, yuck.)

Since the client system is actually the server for a desktop client, it became more necessary to independently update the two servers.  This did pose a bit of a problem with SOAP API bindings in .NET.  We wanted to establish a backwards compatible base API (to maintain the status quo) and also start working on a few API breaking updates.  This would normally be pretty easy in rails with namespaces but it unfortunately doesn't work out so well for ActionWebService.

ActionWebService actually supports a few different models of dispatching actions: direct, delegated, and layered.  Direct just maps a controller's actions to the entire described API.  Delegated enables exposing single methods as SOAP actions from another controller.  Lastly, layered is a lot like direct and delegated mixed; separate ports are created for each layered implementation, and everything is described in the same wsdl.

So...everything should be great, right? Well, it turned out that all of the session code broke.  Since the 'versioned' implementations were actually instantiated from the original controller, they're not provided the same environment as the top level. This little trick from the ruby-forum re-instated the functionality (with a minor edit):

class ServicesController < ApplicationController
    web_service_dispatching_mode :layered
    web_service(:v1) { ServicesV1Controller.new(self) }
  end

 

class ServicesV1Controller < ApplicationController
    def initialize(self)
      @controller = self
      @session = @controller.session
    end
  end

We're still working through some of the kinks in making the API fully backwards compatible, but this got us a long ways to supporting multiple API versions with SOAP.

 

Helpful post on API versioning: http://stackoverflow.com/questions/389169/best-practices-for-api-versioning

Permalink | Leave a comment  »

]]>
http://posterous.com/images/profile/missing-user-75.png http://posterous.com/users/heOPIhURBYqT0 Nick Jones nicktj Nick Jones
Tue, 10 May 2011 14:09:00 -0700 Appcelerator: Titanium http://nicktj.posterous.com/appcelerator-titanium http://nicktj.posterous.com/appcelerator-titanium

I've been intrigued by Titanium's cross-platform support and use of HTML/CSS to code up cool apps instead of in Adobe AIR.  However, I've been curious if I could avoid using Appcelerator's cloud building services for work projects.  Trying this forum post out: http://developer.appcelerator.com/question/73271/how-to-package-your-desktop-app-yourself-ie-not-use-the-cloud  Hopefully it works...

Permalink | Leave a comment  »

]]>
http://posterous.com/images/profile/missing-user-75.png http://posterous.com/users/heOPIhURBYqT0 Nick Jones nicktj Nick Jones
Sun, 08 May 2011 10:11:58 -0700 Photoshop with large files http://nicktj.posterous.com/photoshop-with-large-files http://nicktj.posterous.com/photoshop-with-large-files

My wife and I had been hunting down a wedding photo of my parents and finally got one yesterday (thanks Grandma!).  I quickly scanned it at her house at 2400dpi into TIFF to make sure I had everything possible for correcting it and enlarging the 5x5" print to 11x24".  Luckily, I had my laptop to transfer the 300MB file.

As one would expect, the photo is riddled with color decay, red spots across a large section, and dust and scratches all over it.  I've been editing all morning with nine layers adjusting colors and such.  Suddenly, Photoshop gives me a warning about being unable to save past the 2GB boundary.  Never fear though, switch to Adobe's "Large Document Format" (aka PSB) and keep working.  Unfortunately, PDB is likely to be less compatible with non-Adobe products though.

Hope this helps...

Permalink | Leave a comment  »

]]>
http://posterous.com/images/profile/missing-user-75.png http://posterous.com/users/heOPIhURBYqT0 Nick Jones nicktj Nick Jones
Tue, 26 Apr 2011 20:50:07 -0700 Family Computer Help Request http://nicktj.posterous.com/family-computer-help-request http://nicktj.posterous.com/family-computer-help-request How's the fam. How's the Job. Hope all is well. Blah Blah Blah. I have a computer problem.

Permalink | Leave a comment  »

]]>
http://posterous.com/images/profile/missing-user-75.png http://posterous.com/users/heOPIhURBYqT0 Nick Jones nicktj Nick Jones
Wed, 20 Apr 2011 06:27:00 -0700 Wicked fast mallocs (tcmalloc) http://nicktj.posterous.com/wicked-fast-mallocs-tcmalloc http://nicktj.posterous.com/wicked-fast-mallocs-tcmalloc

"TCMalloc is faster than the glibc 2.3 malloc (available as a separate library called ptmalloc2) and other mallocs that I have tested. ptmalloc2 takes approximately 300 nanoseconds to execute a malloc/free pair on a 2.8 GHz P4 (for small objects). The TCMalloc implementation takes approximately 50 nanoseconds for the same operation pair. Speed is important for a malloc implementation because if malloc is not fast enough, application writers are inclined to write their own custom free lists on top of malloc. This can lead to extra complexity, and more memory usage unless the application writer is very careful to appropriately size the free lists and scavenge idle objects out of the free list"

http://goog-perftools.sourceforge.net/doc/tcmalloc.html

Permalink | Leave a comment  »

]]>
http://posterous.com/images/profile/missing-user-75.png http://posterous.com/users/heOPIhURBYqT0 Nick Jones nicktj Nick Jones
Tue, 19 Apr 2011 20:33:00 -0700 Hello (Posterious) World http://nicktj.posterous.com/hello-posterious-world http://nicktj.posterous.com/hello-posterious-world

Giving posterious a whirl, so..... hello world. :)

Permalink | Leave a comment  »

]]>
http://posterous.com/images/profile/missing-user-75.png http://posterous.com/users/heOPIhURBYqT0 Nick Jones nicktj Nick Jones