Mining CVS Data

I've been reading a book on Data Mining and was wondering how I might actually apply what I've read. Fixing some bugs today got me thinking about mining CVS data. If your cvs comments can be linked to defects (i.e. the comment contains a defect number) you might be able to make some useful observations.

Code Complexity
If a "chunk" of code has lots of defects assigned to it, that might indicate a high level of complexity. A "chunk" could be lots of things: a function, a class, a module/package, anything. My intuition is that the more bugs, the more difficult the code is to understand, maintain and test.

Probabalistic Reasoning
Given a "commit" into cvs, what is the probability it will contain a bug? What factors play into the decision?

• Number of lines changed.
• Number of previous bugs in this chunk?
• Did the current author make the previous, change, or is she working on this code for the first time?
• Number of dependancies introduces (i.e. a new import statment in a java class).

javolution.org

I recently ran across a nice project that has some support for memory pools in java: javaolution.org. Since it works with the current JVM you need to do a bit of work. Instances of a pool context are created and bound to the current thread. Then, any objects created using a special factory are actually allocated in the pool. When the thread leaves the pool context the objects are recycled.

Java Memory Pools

Having developed primarily in garbage collected languages (perl, java, php), I read this article on memory allocation techniques mostly out of curiosity. The most interesting section was on using memory pools instead of the standard calls to malloc() and free(). Apache allocates a memory pool for each request. Any memory needed only for the request is put into this pool. At the end of the request the whole pool is freed.

I've been working on a garbage intensive java application that does lots of EDI and XML translation. Reading XML and mapping objects creates lots of garbage quite quickly and the JVM spends a fair amount of time clearing up garbage. It seems that some sort of pool would help in this case. I would think this could be useful for lots of transactional and web based systems.

How it would work is another matter. Perhaps a "pool context" could be associated with a thread.

 MemoryPool pool = Runtime.createMemoryPool();

 Thread.currentThread().setMemoryPool();



 Foo foo = new Foo();   // allocated in pool

 Bar bar  = new Bar();   // allocated in pool

The garbage collector would not see foo or bar. It would only see and collect from pool.It's more complicated than that, though, since you could allocate
something in the pool and then assign it to a variable (for example,a Map) outside of the pool.

  HashMap map = new HashMap();



  MemoryPool pool = Runtime.createMemoryPool();

  Thread.currentThread().setMemoryPool();



  Foo foo = new Foo();   // allocated in pool

  map.put("Foo", foo);

In this case the garbage collector would see foo.

The compexity probably outweighs any real benefits, but I find the idea interesting either way.