Debugging and Tuning

Friday last week I had a major break-through: I saw the shell again after more than four weeks. Debugging was exactly as much fun as everybody says it is. It's particular frustrating if you spend five hours from not having any clue what part of the system might be causing a crash over narrowing it down to a small number of hypotheses to finding a really stupid mistake that you can't believe you are responsible for. Gdb is a close friend now.

For testing the interpreter I was basically using the startup procedure. When the interpreter starts up it reads in and parses a bunch of source files with definitions. So there surely are a couple of more bugs that this "test" did not expose. Before I try running other programs to tease out more bugs, though, I will spend a little more time on performance tuning. Having to wait a minute or two until the interpreter got to a point where it broke while parsing the standard files was an annoyance while debugging last week (and besides, that performance is unacceptable).

Three items are on my performance tuning to-do list for now:

• Do some profiling and try to speed up the hot spots in the MM code
  
• Implement asynchronous collect (so far there's only synchronous)
• Implement a smarter collection trigger policy

On Designing Interfaces

I talked a little bit about ATs in an earlier post. To summarize, ATs serve three different roles in the original Lush interpreter, namely

1. As generic, unique identifiers for Lisp objects
   
2. As carriers of runtime type information
3. As carriers of information for memory management (reference counts)
   

In the branch that I am currently working on, the third point is no longer true and ATs serve only the first two purposes. In this posting I want to talk about the first purpose.

There are many type-generic functions in the interpreter that may work with many or all kinds of objects (creating a string representation, writing an object to a file, computing a hash code, etc.). Many other functions work with objects of particulars type only (open a file given a name (string), slicing an array, arithmetic functions, etc.). And a third kind of function are found in the class structure of Lisp objects. They are type specific versions of functions that should work with all objects (dispose, creating a string representation, listeval, etc.).

The natural signature of type-generic functions include an AT argument pointing to the object in question. They may be implemented as dispatchers to type-specific functions of the third kind. Function signatures for the second kind may or may not include an AT.

When you look at the original Lush interpreter implementation you will find that functions of the second and third kind do have generic signatures as well. The ones of the second kind typically do a runtime type-check, the ones of the third do not. I consider this bad design. If a function works with arguments of a particular type only, then the signature should say so and the implementation should not waste (run)time doing type checks. This is one kind of changes that I have been making since I started hacking Lush, and there's still a lot of work to be done on overhauling internal interfaces.

Yesterday I was churning through the dispose functions, which are functions of the third kind. Objects with non-trivial dispose functions need a finalizer to invoke the dispose function. Lush also has a function delete to explicitly destroy a Lisp object, no matter if it is still referenced or not. Delete calls an object's dispose function and then modifies the AT to make it look like a NIL. So a dispose needs to be idempotent since it may be called twice (once by delete and a second time by the finalizer).

Since I was going to look at all dispose functions I decided to change their signatures from generic to type specific along the way. When I got to dispose for OO objects, I hit a snag: This dispose invokes a user-defined destructor (if one has been defined), and at that point, I need to know the object's AT because the destructor is a Lisp function and evaluate has a generic signature. After making dispose type-specific, however, I don't have the object's AT anymore.

That led me to wonder if maybe I was wrong and there is some value in having all signatures generic. Anyway, I fixed it by adding a "back pointer" to the OO object structure (a pointer that points back to the AT referencing the object), which is not nice. Perhaps the dispose interface still isn't quite right?