Recently in MetaRuby Category

miniunit is a completely minimial drop-in replacement for ruby's test/unit. This is meant to be clean and easy to use both as a regular test writer and for language implementors that need a minimal set of methods to bootstrap a working unit test suite.

Changes:

1.1.0 / 2007-11-08

• 4 major enhancements:
  • Finished writing all missing assertions.
  • Output matches original test/unit.
  • Documented every method needed by language implementor.
  • Fully switched over to self-testing setup.

• 2 minor enhancements:

  • Added deny (assert ! test), our favorite extension to test/unit.
  • Added .autotest and fairly complete unit tests. (thanks Chad for help here)

• http://rubyforge.org/projects/bfts

I love testing. I just do. I do it all the time. I'm a big TDD dork. I push for it because it makes my life and my code easier. That said, there are times when I really dislike test/unit. It is a lot bigger and more complex than it needs to be.

Working on metaruby it is our very first hurdle for every class we try to reimplement (nuke all of the methods in Array and see how far YOU get :P) We have to do a phase where we figure out what essential core methods are needed for test/unit to run the rest of our tests.

Sometimes, "essential" is subjective and it gets in our way.

Regardless, test/unit is 3300 lines of complex code for a system that as originally described is not that complex at all. To be fair, test/unit has a lot of bells and whistles, but I'm guessing most people never ever touch most of them. Me? I use the basics + regular expression filtering. That's it. No GUI, nothing extra. And it does great (*).

So, while working on BFTS (Big "Formal" Test Suite--our fork/port of the rubicon test suite), I wrote a drop-in compatible replacement for test/unit. This will let us have a much smaller set of essential core methods for our metaruby work. I've got the newly ported tests for Array, Hash, String, and Time running with it.

So... the clencher? What really makes me love this code to the point that I smile every time I read it? It is a beautifully clean 71 LOC long!!! (92 with whitespace and comments!) The bells and whistles in test/unit might be nice, but I highly doubt you get 36x (92 vs 3300) the functionality from it.

*) My only complaint about test/unit, besides its complexity, is the error when you have a TestCase subclass that doesn't have a test method in it. That is just wrong in my opinion. I don't think you get that much out of that type of error, and I don't have a single project that doesn't refactor test code into an abstract superclass.

This was our old milestone list for metaruby/ruby2c. It does not reflect current status, but status then (approx 2004-10).

1: (done) parse_tree.rb
2: (done) 1st compilable unit
3: (done) massive cleanup and rearchitecture (node class and SexpProcessor
composite class)
4: basic sexp interpreter (numerics), use that to drive the doco for
the ruby subset we support.
5: advanced sexp interpreter (objects)

basic units/goals (in no order)

memory allocation & deallocation
sexp interpreter + documented ruby subset
array
object
string
inlining

huge units/goals:

parser
runtime

• Plethora of passing unit-tests: check
• Simple reproducible system: check
• Ability to override core classes: check
• Binary with our core classes: check
• Segfaults: check

% cd MR
% make libmetaruby.a
...
% cd RUBY
% patch -p0 < metaruby.patch
...
% ./configure --program-prefix=meta
...
% cp ~/Links/MR/libmetaruby.a .
% make metaruby
...
% ./metaruby bin/irb 
irb(main):001:0> 1+1
=> 2
irb(main):002:0> false
/usr/local/lib/ruby/1.8/irb.rb:298: [BUG] Segmentation fault
ruby 1.8.3 (2005-09-10) [powerpc-darwin8.2.0]

It is time to split the tail of ruby2c and start making a translator for ruby internals. So far we've only done generic C.

We are very close... very very close...

My rubyconf 2005 presentation proposal was accepted!

I plan on talking about all the technologies and toys that have spawned out of the metaruby project. The whole fam.

I plan on illustrating the family's usefulness and power through examples including: an example of RubyInline, a language extension for RubyInline, aparse tree complexity metric grapher, a simple example of ruby2c translation, amaintainable and fast profiler, a Ruby obfuscator, and a dynamic Ruby optimizer that requires zero effort to use.

For fun, here is something I scratched up tonight with Eric to play with the idea of lisp-like ruby posted on redhanded recently. Check it, real s-expressions in ruby:

require 'ruby2ruby'

class Module
  def _(sexp)
    self.module_eval RubyToRuby.new.process(sexp)
  end
end

class Foo
  _ [:defn, :example, [:args], [:call, [:lit, 1], :+, [:array, [:lit, 1]]]]
end

p Foo.new.example
=> 2

Eric did something very very cool the other day. He wrote a new tail to the ruby2c pipeline that outputs Ruby C instead of generic C. As a result, he is able to automatically translate the following:

$ cat demo/factorial.rb
class F
  def factorial(n)
    f = 1
    n.downto(2) { |x| f *= x }
    return f
  end

  def main # horrid but funny hack
    return factorial(5)
  end
end

into:

// BEGIN METARUBY PREAMBLE
#include <ruby.h>
#define case_equal_long(x, y) (rb_funcall((x), rb_intern("==="), 1, (y)))
// END METARUBY PREAMBLE
// class F < Object

static VALUE
rrc_cF_factorial(VALUE __self, VALUE n) {
VALUE f;
VALUE x;
f = LONG2FIX(1);
x = n;
while (RTEST(rb_funcall(x, rb_intern(">="), 1, LONG2FIX(2)))) {
f = rb_funcall(f, rb_intern("*"), 1, x);
x = rb_funcall(x, rb_intern("-"), 1, LONG2FIX(1));
};
return f;
}

void
Init_F() {
VALUE rrc_cF = rb_define_class("F", rb_path2class("Object"));
rb_define_method(rrc_cF, "factorial", rrc_cF_factorial, 1);
}

As a result, pretty much any ruby code can be automatically translated to C with no real effort on our part. We can even bypass the type inference engine (for now at least). I think this is the first real step towards a ruby obfuscator.

(click for a prettier version)

This is the world I'm living in right now... Let me introduce you to the fam'.

The core of the fam here is the ruby2c tool chain. That includes: ParseTree, an independent package that mines ruby's internals for method syntax trees; Rewriter and R2CRewriter, layers that make language translation easier; TypeChecker, a type inference engine for ruby; and finally RubyToC, which translates a subset of ruby into static C code. We released a beta of ruby2c in February and are still working on it in parallel with our work on metaruby.

Also related to ParseTree are ABC, Dep, Graph and ruby2ruby. They are much more in the "toy" category, mainly in the amount of polish more than utility. ABC, Dep, and Graph help you analyze code and visualize code complexity and relationships. ABC generates whitespace independent complexity metrics. Dep shows very basic dependencies between classes/methods. Graph translates parse tree output into a DOT file for the GraphViz tools.

Ruby2Ruby translates the output of ParseTree back into ruby. At first glance, it doesn't sound very useful. After all, you started with ruby, right? Why translate it back to ruby?. But imagine how many things you can build on this foundation: syntax highlighters, indentation tools, code annotation tools, enhanced debugging output, and truly awesome dynamic optimizers/rewriters/inliners. There are a lot of possibilities sitting on top of this 150 lines of code.

ZenTest is my test auditing and generation tool. It analyzes your implementation and unit tests and generates missing methods (both test and implementation) based on naming convention (Blah#blah should be tested by TestBlah#test_blah). It is my hope to tie ZenTest, ABC, and Graph together so you can visualize your test and implementation interdependencies and focus on the writing tests where they are most needed, based on complexity.

The blue nodes are things we haven't written yet, but hope to in the near future. ZenHack is going to be a repository of useful tools for hacking on ruby, prototyping, and generally playing with the language. It will tie into a lot of the tools above by using and abusing them. Parser will eventually be a full fledged pure ruby parser that we can use to feed ruby2c and can also feed to ruby2c. We also have planned two new tails to the ruby2c tool pipeline, one to generate extension code for regular ruby extensions and another to generate code for metaruby itself. The current ruby2c translator was a throwaway proof of concept.

So, that is the fam! Ruby community, the fam. Fam, the ruby community. I hope to see you mingling soon and enjoying each other as much as I have.

Things have slowed down of late. I think that means that we have reached a new plateau, and that can certainly be considered a good thing. Eric and I have started moving on the metaruby side as a forcing function to get ruby2c even more complete. We've also done a better job of illustrating what ruby2c really is in relation to itself, extension writers, metaruby, and the rest of the parse tree family. I hope to have this blogged in the near future.

We've been wanting to do something called ping-pong pairing, to force us to switch more and get us even more robust code. It is an interesting idea but we haven't been able to sit and code for a while. I've been doing it with myself though, using ruby2c and metaruby and DAMN, it works well. Granted, I have easy pickings right now as we are at the front of the project, not the back, but still. I have this mental image of me wearing my metaruby hat finding something that doesn't work (equivalent to a failed unit test). Then I switch to my ruby2c hat and trudge over to that emacs buffer and fix it. This usually causes a new breakage over in metaruby, so I get to trudge back... Clean separation, and a nice forcing function for me to get both more robust.

Now... if only I weren't on the bus while doing this...

"ParseTree is a little brown stinky ferret that digs down a hole and violently rips the AST away from the warm bosom of ruby."