March 2006 Archives

We've been pushing hard on ZenTest and I'm proud to announce that we just released ZenTest 3.1.0! In this release we've added multiruby and massively enhanced autotest. I have to admit, I'm falling for autotest in much the same way Eric fell for unit_diff. Unfortunately for both of us, marrying software isn't legal in Washington state.

Install Now, Read Later

sudo gem install ZenTest or visit the rubyforge project.

I'm going to write up some real stuff on using multiruby and autotest, but for now here is the standard marketing blurbies:

ZenTest provides 4 different tools: zentest, unit_diff, autotest, and multiruby.

ZenTest scans your target and unit-test code and writes your missing code based on simple naming rules, enabling XP at a much quicker pace. ZenTest only works with Ruby and Test::Unit.

unit_diff is a command-line filter to diff expected results from actual results and allow you to quickly see exactly what is wrong.

autotest is a continous testing facility meant to be used during development. As soon as you save a file, autotest will run the corresponding dependent tests.

multiruby runs anything you want on multiple versions of ruby. Great for compatibility checking!

Changes:

2 major enhancements

+ Added multiruby! YAY! + Massive improvements to autotest: speed, reliability, reporting, etc.

10 minor enhancements

+ multiruby builds in a centralized location. YAY! + multiruby now allows reinstalls quickly and easily (can even skip config). + multiruby exits with total sum of exit codes. + autotest file search is muuuuch faster. + autotest automatically detects rails mode. + autotest deals with rails dependencies much better. + autotest reruns a full suite after you go green to ensure full coverage. + autotest always runs with unit_diff -u. + autotest can now run cvs/svn/p4 up periodically to be a mini-tinderbox. + autotest now has real help.

4 bug fixes

- ZenTest is now zentest. Yay for consistency! (do a rake uninstall to clean) - ZenTest excludes pretty_print methods. - Fixed unary operator issues (they were backwards... oops!) for ZenTest. - unit_diff now runs diff.exe on Windoze. dunno if that will work.

I haven't done nearly enough to thank Pat Eyler for all his great work writing about ZenTest. He's done an awesome job of getting the word out there about ZenTest (and pretty much everything else we do). Here is some of the stuff he's written:

Welcome to The Gemcutter's Workshop covers ZenTest, unit_diff, and autotest along with a plethora of other great testing and development tools.

How to Use ZenTest with Ruby - included with ZenTest - covers refactoring and unit testing in the more traditional (non-TDD sense).

Test-first Ruby programming (requires login) - shows how to use ZenTest with in a test-first environment.

Again... thanks Pat!

Tired of your rails functional test failures being completely unreadable? I'm not terribly fond of rails' assert_tag but it is better than nothing. However, I never like to read something like the following:

expected tag, but no tag found matching {:attributes=>{:action=>"/admin/themes/update/1"}, :tag=>"form"} in:
"<!DOCTYPE [...3k worth of crap cut...]</html>".

Ugh! It just does nothing to help you and since it isn't expressed as a diff, unit_diff is no help in this arena. I have however figured out how to make it much more manageable with the following snippets:

class ApplicationController < ActionController::Base
  def initialize(testing=false)
    super()
    self.class.layout(nil) if testing
  end
  [...]
end

and then in your functional tests:

def setup
  @controller = MyController.new(true)
  [...]
end

Now your failures contain just the content from the page:

expected tag, but no tag found matching {:attributes=>{:action=>"/admin/themes/update/1"}, :tag=>"form"} in:
"\n<ul>\n <li>Name: Blue</li>\n <li>Folder: blue</li>\n <li>Description: A Plain blue theme</li>\n <li>Masthead: Blue</li>\n <li>Editor: n/a</li>\n</ul>\n\n<a href=\"/admin/themes/show/1\">Cancel Editing</a>\n".

Tonight's Seattle.rb weekly hackfest featured Francis Hwang who is in Washington visiting his family this week. Good food and much fun was had. Eric Hodel and I did some bug squashing and feature implementing on ZenTest. We actually closed out every bug on ZenTest so I'll be releasing tomorrow!

I also wrote a small emacs library called toggle.el that lets you teach it a bunch of patterns so it can easily toggle between two related files. A simple example of that is "blah.rb and "test_blah.rb", but I've provided it with patterns to work with rubygems-style "lib/blah.rb" and "test/test_blah.rb" and rails-style "app/model/blah.rb" and "test/unit/blah_test.rb" as well. It is pretty clean and you can use it for any languages you want. I've been wanting this in emacs for a while and am surprised that I haven't found something like it yet (the reason I got on #emacs was "most people probably roll their own"). I'll be releasing that tomorrow as well.

Awesome hacking session tonight. Our ruby obfuscator is now actually on track to be generally usable by the end of the month. (Although we still need documentation--badly)

We have an awesome architecture to write these types of tools in, but there are several things that get complicated and having an overly clean architecture actually gets a bit hard to deal with. The biggies are that C and Ruby disagree on what is an expression vs statement... as such, we often have to rearrange the order of things, and the way our architecture works, sometimes that is easy, sometimes not.

Today we got things like map working. More specific iteration has worked for some time, but now we can do more general iterations with blocks. There are restrictions currently, like we don't even remotely honor block closure semantics (and, I'm not sure we ever will), but we're actually doing a fair amount now and it is going pretty smoothly.

Next up will be extending our splat arg support to convert to C var-args so they actually work and generalizing our block support (where possible).

P.S. I should mention, the work on the obfuscator is also a nice forcing function for solidifying ruby2c and ParseTree.

Testing across multiple versions of ruby has flushed a bunch of potentially lethal bugs in ParseTree. Nothing huge in the actual logic of ParseTree, but logical testing errors that could have hidden nasty problems (or shown false negatives that were near impossible to debug). Absolutely lovely. And the ability to simply drop in a new tarball of ruby and have it automatically configured, built, installed, and used on your next run is even more lovely.

ZenTest 3.1.0 will release with multiruby within the next week, along with improvements to autotest.

<zenspider> singletons are almost always a sign of bad design.
<suryam> but i need it in my design... [...]
<zenspider> "need" is almost always a sign of bad thought.

Lets say you maintain a library or tool that could be sensitive to particular changes in ruby. In my case, ParseTree is very dependent on the internals of ruby staying the same. Maybe in your case you are using some methods that changed their argument semantics (they way public_methods changed in 1.8). Making sure your code worked from version to version is a bit of a pain.

Until now.

% ./bin/multiruby -I../../RubyInline/dev ../../RubyInline/dev/test_inline.rb 

VERSION = 1.8.2

Loaded suite ../../RubyInline/dev/test_inline
Started
...................................................
Finished in 3.385808 seconds.

51 tests, 78 assertions, 0 failures, 0 errors

RESULT = 0

VERSION = 1.8.3

Loaded suite ../../RubyInline/dev/test_inline
Started
...................................................
Finished in 3.21357 seconds.

51 tests, 78 assertions, 0 failures, 0 errors

RESULT = 0

VERSION = 1.8.4

Loaded suite ../../RubyInline/dev/test_inline
Started
...................................................
Finished in 3.642159 seconds.

51 tests, 78 assertions, 0 failures, 0 errors

RESULT = 0

This will be released with ZenTest very very soon.

Here is my very simple rule for optimizing:

Never ever code for speed until you need to code for speed!

Consider it an addendum to Knuth's "root of all evil" law. There are two corollaries to this law:

Always measure.

and

Only rewrite the slow part in C, not the whole damn thing.

Some of this came up recently because of a dialog last week on ruby-talk between Sascha Ebach and JEG2 about making FasterCSV even faster. James wants to keep the module as pure ruby (totally laudable goal) but Sascha thought it'd be neat to take advantage of RubyInline if it were available. I'm all for keeping it pure ruby, but I wanted to see how you'd do the dual implementation version in a way that still read clean. Here is my first scratch at it:

begin; require 'inline'; rescue LoadError; end

class MaybeFast
  def blah
    puts "slow version"
  end unless respond_to? :inline

  inline do |builder|
    builder.c 'void blah() { puts("fast version"); }'
  end if respond_to? :inline
end

MaybeFast.new.blah

I'm proud to say that we've finally released ZenTest 3.0!

ZenTest has had a nice overhaul, is much better on working with rails, and has better support for naming conventions. unit_diff has been solidified into a scary monster I can't live without. New to the game is autotest, a continuous testing tool (originally for rails) that gives you near-immediate feedback as you develop and refactor. Oh, and it is now available as a gem and uses rake (so it should be a better citizen on Windows).

Please, check it out and let us know what you think!