Rails: April 2007 Archives

ZenTest provides 4 different tools and 1 library: zentest, unit_diff, autotest, multiruby, and Test::Rails.

ZenTest scans your target and unit-test code and writes your missing code based on simple naming rules, enabling XP at a much quicker pace. ZenTest only works with Ruby and Test::Unit.

unit_diff is a command-line filter to diff expected results from actual results and allow you to quickly see exactly what is wrong.

autotest is a continous testing facility meant to be used during development. As soon as you save a file, autotest will run the corresponding dependent tests.

multiruby runs anything you want on multiple versions of ruby. Great for compatibility checking!

Test::Rails helps you build industrial-strength Rails code.

3.5.2

• 4 bug fixes:
  • Patch up Rails fixture defaults for Test::Rails::TestCase.
  • Session now properly hooked up to controllers.
  • ruby 1.8.6 has a bug on 'raise Interrupt' with no args. Fixed on both sides.
  • Fixed redgreen to work with new getc/putc-based output. (from Finn Smith)

Damn good writeup on all but the newest autotest plugins:

PH7 - Getting started with Autotest - Continuous Testing

Thanks to Jean-Michel Garnier for pointing this out to me!

ImageScience is a clean and happy Ruby library that generates thumbnails -- and kicks the living crap out of RMagick. Oh, and it doesn't leak memory like a sieve. :) For more information including build steps, see http://seattlerb.rubyforge.org/

Changes:

1.1.2 / 2007-04-18

• 2 bug fixes:
  • reports bad height/width values for resize
  • explicitly removes ICC color profiles from PNGs (bug in freeimage).

ZenTest provides 4 different tools and 1 library: zentest, unit_diff, autotest, multiruby, and Test::Rails.

ZenTest scans your target and unit-test code and writes your missing code based on simple naming rules, enabling XP at a much quicker pace. ZenTest only works with Ruby and Test::Unit.

unit_diff is a command-line filter to diff expected results from actual results and allow you to quickly see exactly what is wrong.

autotest is a continous testing facility meant to be used during development. As soon as you save a file, autotest will run the corresponding dependent tests.

multiruby runs anything you want on multiple versions of ruby. Great for compatibility checking!

Test::Rails helps you build industrial-strength Rails code.

Changes:

* 3.5.1 / 2007-04-17

• 4 bug fixes:
  • Fixed gem name to be camel-case again. Fixed on rubyforge too.
  • Fixed rdoc for hooks.
  • Fixed redgreen, results changed to an array.
  • Patch up Rails fixture defaults, since they're not inheriting properly.

Probably the best write-up on autotest I've seen out there:

DRY up testing in Rails with Autotest: "Autotest runs your tests automatically whenever your files change. As Rails developers, we've been trained hard to test early and test often. We are also acutely aware of the DRY principle (Don't Repeat Yourself). However, these ideas don't quite agree in Rails because in our test-code-test cycle, we're constantly typing rake every time we need to run our tests.

Autotest will DRY up your testing by running your tests automatically whenever your files change."

(Via "Maintainable Software")

ZenTest 3.5 shipped with rspec support built in. It works if you install rspec as a gem, but if you installed it into your rails app via a plugin, then Josh has some tips:

http://joshknowles.com/2007/4/13/zentest-3-5-rspec-0-9-autospec

ZenTest provides 4 different tools and 1 library: zentest, unit_diff, autotest, multiruby, and Test::Rails.

ZenTest scans your target and unit-test code and writes your missing code based on simple naming rules, enabling XP at a much quicker pace. ZenTest only works with Ruby and Test::Unit.

unit_diff is a command-line filter to diff expected results from actual results and allow you to quickly see exactly what is wrong.

autotest is a continous testing facility meant to be used during development. As soon as you save a file, autotest will run the corresponding dependent tests.

multiruby runs anything you want on multiple versions of ruby. Great for compatibility checking!

Test::Rails helps you build industrial-strength Rails code.

Changes:

* 3.5.0 / 2007-04-12

• 5 major enhancements:
  • Now requires RubyGems 0.9.1 or newer.
  • Autotest and unit_diff are both unbuffered. Results are more live.
  • Refactored and redesigned how files map to test from Sean Carley. See fixtures plugin as an example.
  • Generalize how autotest handler is instantiated and invoked, allowing for many more autotest types, including combos.
  • functional_test_matrix.rb added!
• 23 minor enhancements:
  • Added all_good hook if initial run was all_good as well.
  • Added assert_in_epsilon to ZentestAssertions.
  • Added autotest plugin to auto-update source ala tinderbox/cruisecontrol.
  • Added autotest plugin to update ichat/adium IM status with code stats.
  • Added autotest plugin to update the GNU screen statusbar from Yuichi Tateno.
  • Added autotest syntax error handling from Ryan Platte.
  • Added autotest/emacs emacs integration plugin!!! YAY!
  • Added autotest/migrate.rb.
  • Added camping support from Geoffrey Grossenbach.
  • Added changed file reporting to autotest via -v flag.
  • Added informative summary and filtering via ENV['VERSIONS'] to multiruby.
  • Added libnotify support from Kazuo Saito.
  • Added lots of rdoc patches from Hugh Sasse.
  • Added rjs files to view_test_case.
  • Added rspec_autotest "stolen" from caldersphere.net
  • Added run_command hook to trigger the start of a test run.
  • Added tmp to rails' exceptions list.
  • Added unit_diff command variable to autotest for customizing flags and such.
  • Added zentest_mapping.rb and test.
  • Allow session to work in Test::Rails view tests.
  • Improved autotest/growl.rb output from imajes.
  • Improved autotest/timestamp output from Josh Susser.
  • Test::Rails works with Rails 1.2.
• 2 bug fixes:
  • Accelerated Test::Rails unit tests via sensible defaults.
  • Better assertion messages for ZentestAssertions.

[UPDATE: the final version of this code was added to ZenTest 3.5.0] As continuation from my previous post, I offer a visualization to help clarify things:

Before	After

Click the images to see the code in more detail.

The most important thing to understand is that every matrix line from the "after" version like:

matrix :edit, :edit, :edit, :edit, :edit, :e_RO, :e_RO, :e_RO, :e_RO

is an entire page of the "before" version.

I've made some really good progress with the Functional Test Matrix idea. The code compression ratio is phenomenal. I'm writing up a comparison of the two, but without finesse the post would be huge and boring. I'll summarize here as best as I can.

I think most important idea is that high-level reviews or even edits of the functional tests require reading 1 line per action instead of N average lines per action * M edge cases. Very rough estimates show that my traditional functional test writing style weighs in at 179 lines of test for two actions against the following matrix:

setups         :u_w,   :u_r,  :g_w,  :g_r,  :o_w,  :o_W,  :o_r,  :o_R
matrix :edit, :edit,  :edit, :edit, :edit, :e_RO, :e_RO, :e_RO, :e_RO
matrix :view, :view,  :view, :view, :view, :view, :view, :view, :e_NF

Compare that to the 3 lines above and an additional 61 lines of "legos" used to put the matrix together (and, if you want, another 17 lines for the matrix framework itself). The liberal estimate puts that at 2.8:1, and the conservative estimate at 2.2:1. I suspect these numbers to be a bit high and for it to probably move down closer to 2.0:1.

The kicker for me tho, is that once the testing matures, I think we actually settle on the ideal of only really working with the matrix itself, pushing us up to nearly a 60:1 ratio! Like I said, this is rough, so YMMV. Look above. When my client comes to me and says "I think it'd be more secure all read-only errors were not-found errors", I wind up modifying only 4 result codes. If we change our permissions model and add unreadable and unwritable to group, I add 4 result codes and get 4 more tests for free.

A side benefit (from my perspective) is that the functional test matrix itself is high-level enough that it can be read and understood by the business owner or domain analyst. Allowing them to glance at it and immediately point out either missing edge cases or incorrect results.

I just released toggle.el, version 1.2 and autotest.el 1.0 beta 2.

Get them via: http://www.emacswiki.org/cgi-bin/wiki/RyanDavis

toggle.el

toggle.el allows you to quickly open corresponding files via a dynamic mapping. In this release I added some bug fixes and a mapping for rspec.

There are now 4 different mapping styles in this version: zentest, rails, rspec, and ruby. Feel free to submit more and I'll incorporate them.

Example Mapping (rspec style):

app/models/blah.rb      <-> spec/models.blah_spec.rb
app/controllers/blah.rb <-> spec/controllers/blah_spec.rb
app/views/blah.rb       <-> spec/views/blah_spec.rb
app/helpers/blah.rb     <-> spec/helpers/blah_spec.rb

autotest.el

autotest.el is a godsend for me. It runs all my tests inside emacs itself, providing hyperlinks in backtraces and the ability to quickly switch between the autotest output and the last buffer I was working in. The combination of autotest.el and autotest speeds me up tremendously.

This is an idea I've been tossing around for a while. Akin to Ward's FIT, how do you make testing complex specifications with many edge cases clearer?

I have something similar to the idea below that I'm slowly evolving towards this idea. I'm not entirely sold on it, but it does seem to cut down on the amount of code I have to write and how I have to think about it. What do you think? Suggestions?

The Idea:

This is supposed to get us thinking about the various dimensions our testing should address. If there are states orthogonal to each other (eg. readable vs unreadable, logged in vs not logged in) each of those states should comprise a dimension in the matrix. By addressing it this way, we should be able to minimize the amount of setup/teardown code and get full coverage across our actions for all these edge cases and as a result have extremely clear tests.

Example Test Matrix Specification:

setups          :edge1, edge2, :edge3, ...
matrix :action1,  :new,  :err,   :err, ...
matrix :action2,  :del,  :err,    :na, ...
matrix ...

setups:

I envision the setups being a code that combines the different dimensions of edge case state.

Something for a CMS might look like: [df][uga][rRwW] where:

• [df] for dir/file
• and the rest is in the style of symbolic args to chmod.
• lowercase X == Xable, uppercase X == unXable, where X is read/write

matrix:

:new/:err/:del are just examples, they should have semantic info attached to them.

Edge cases specific to an action that fall outside the matrix are regular tests.

How it Fits Together:

Setups stores off the edge cases. Matrix creates a test method for every applicable (read: a non :na result) in the (rough) form of:

def test_#{action}_#{setup}
  matrix_setup_configuration #{setup}.split(//) # global setup
  matrix_setup_#{action} #{setup}, #{expected}  # action setup + execution
  matrix_test_#{expected}, #{setup}             # expected verification
end

I'm not really good on the sales and marketing side, but a number of people have been bugging me to go public about my availability, so here it is. I'll be posting this in a more permanent location (linked on the sidebar) once I get it polished up and finalized:

I'm available for consulting on a part-time basis, 10-20 hours a week, at $100 per hour. I'm usually brought in for short-term gigs as a troubleshooter/problem-solver. Probably not worth mentioning, but this page should be proof that I'm better working on backend issues than UI. Email me at: ryand-ruby@zenspider.com.

What I do:

• Bootstrap new projects.
• Troubleshoot deep/confusing problems so the main developers can focus.
• Identify and fix problems to accelerate the development process.
• Design and implement solid, well-tested, efficient backend code at a rapid development pace.
• Retrofit or shore up tests.
• Refactor or reimplement brittle code.
• Profile and fix bottlenecks.
• Develop developer tools to improve the development process.
• Mentor.

What I Prefer to do it with:

• Ruby (yes, and rails)

What I also do:

• Java
• Python (but please no more twisted python!! blech!)

What I used to do and might be bribed into doing again:

• Smalltalk
• Perl
• C/C++/ObjC

Past Clients (ones I can mention):

• Limewire
• brightside broadcast
• Technology Group NW
• PNN

My Projects:

• BFTS
• metaruby
• ParseTree
• ruby2c
• RubyInline
• RubyTests
• Seattle.rb (png, hoe, ruby2ruby, heckle, image_science, box_layout)
• ZenHacks
• ZenTest
• ZenWeb
• zero2rails

I also Contribute to:

• ruby
• rubygems
• codeforpeople (rubyforge)

Coming soon:

autotest_emacs_integration.mov (2212 Kb)
(direct link in case servers are borked)