stty -sane

Versioning Database Data

2010-11-29T08:29:00.000-08:00

Some links to articles about tracking historical data in your database:

Products:

http://download.oracle.com/docs/cd/B28359_01/appdev.111/b28424/adfns_flashback.htm

JBoss WebsphereMQ Failover

2008-01-08T20:21:00.000-08:00

IBM's MQSeries Message Broker doesn't have any built-in failover support. The reccomended solution is to use IP level takeover where a standby machine monitors the main machine via a heartbeat. If the heartbeat goes away, then the failover machine does IP takeover. Your clients will need to re-connect, but that should be all.

We didn't want to do that so instead I needed to have our MDBs and other JMS client code running in JBoss to failover to a different MQ server.

The solution I chose was to have an MBean that monitors the "main" server by listening on a "monitor" queue (just a normal queue created for monitoring purposes). When a failure is detected (by a JMS exception listener) it re-populates the MQ JMS objects in JNDI.

The MDBs will automatically try and re-connect when they detect a failure. Eventually, after the JNDI is updated to the new MQ server it will reconnect properly.

Other client code (MDBs that send messages, servlets that send/receive) need to know to re-connect when there is a failure. I created a notification system so that they can re-get any objects from JNDI after they are re-populated. In addition, any temporary queues that were created will need to be re-created since they now refer to the bad MQ server.

Hopefully, I can detail this out better over the next couple of weeks.

Predicting bugs in code changes using SCM information

2006-12-14T20:46:00.001-08:00

This is to remid me that I might want to watch the whole video.

Google TechTalks
March 8, 2006

Jim whitehead
Jim Whitehead is an Assistant Professor of Computer Science at the University of California, Santa Cruz. He has recently been developing a new degree program on computer gaming, the BS in Computer Game Engineering. Jim received his PhD in Information and Computer Science from UC Irvine, in 2000

Abstract:
Almost all software contains undiscovered bugs, ones that have not yet been exposed by testing or by users. Wouldn't it be nice if there was a way to know the location of these bugs? This talk presents two approaches for predicting the location of bugs. The bug cache contains 10% of the files in a software project.

e^pi*i = -1 ; my favorite web comic

2006-11-03T06:48:00.001-08:00

http://xkcd.com/c179.html

Filtering Blog posts

2006-10-18T13:19:00.000-07:00

I use bloglines to read weblogs. I recently added a site that posts different deals on the web every day because I was going to buy a new monitor. Anyway, I still like to see what is going on, but there are a lot of deals I'm not interested in. I'd really like to be able to filter the results.

Bloglines doesn't have any filtering capability, but I could create a proxy server to do the filtering for me. So for example: Instead of giving bloglins http://feed.com/atom.xml I would give it http://myproxy.com/blogfilter/feed.com/atom.xml. My 'blogfilter' application would make the request to feed.com, get the data, filter stuff out and return the results.

How should I filter? The simplest would be by keywords. Remove entries with these keywords in the title or body. You could have per-blog filters, too. A next step might be to use Bayesian filtering. But then you'd need an easy way to "train" the system. The proxy could add a small snippet of javascript at the end of each post. The javascript would be loaded from my site and would create a little panel with options to somehow rate the blog. It would then use this data to train the bayesian filter.

Update:
Somes services that do this already:

Articles on filtering feeds:

http://www.readwriteweb.com/archives/rss_filter_prod.php

Super Artificial Intelligence

2006-08-10T20:59:00.000-07:00

Like most programmers, I've always been interested in AI, or Artificial Intelligence. Countless books and movies have been based on machines that gain the intelligence of humans. Usually, the plot involves the machines turning against their creators.

But right now, AI is being put to other uses: machine vision, language translation, even managing restaurants.

The above tasks are accomplished quite well by humans, but we do error on occasion. To err is human, we all accept that (to some degree). But what happens when machines start to make the same mistakes. Will we accept it? I'm not sure that we will. I think that we will demand machines that don't make the same mistakes we do. What we really want is Super Artificial Intelligence; machines that are superior to us.

That doesn't bode well for the day its Humanity vs. The Machine.

Evolution of Microformats

2005-08-20T23:03:00.000-07:00

For the last year or so I've been keeping an eye out for interesting articles on the Semantic Web, NLP (Natural Language Processing) and other goodies. Most of the work around the Semantic Web seems to be defining ways to annotate your content to contain semantic information. Other work centers around using NLP to extract semantic information from content that isn't marked up.

Microformats (www.microformats.org) falls into the first camp. Instead of being accademic, however, they offer very practical ways to add semantic information to your content (i.e., this represents a person, event, etc ...). As simple as the formats are, nobody is going to add this to content without the help of tools .

My questions are how will new Microformats (or whatever ends up being used by our tools) be derived? Will there be a voting procedure? Some sort of standards group? I can image lots and lots of different formats. And how will those be integrated into new tools?

One final thought is: will advances in NLP make semantic markup obsolete?

Mining CVS Data

2005-03-28T20:51:00.000-08:00

I've been reading a book on Data Mining and was wondering how I might actually apply what I've read. Fixing some bugs today got me thinking about mining CVS data. If your cvs comments can be linked to defects (i.e. the comment contains a defect number) you might be able to make some useful observations.

Code Complexity
If a "chunk" of code has lots of defects assigned to it, that might indicate a high level of complexity. A "chunk" could be lots of things: a function, a class, a module/package, anything. My intuition is that the more bugs, the more difficult the code is to understand, maintain and test.

Probabalistic Reasoning
Given a "commit" into cvs, what is the probability it will contain a bug? What factors play into the decision?

Number of lines changed.
Number of previous bugs in this chunk?
Did the current author make the previous, change, or is she working on this code for the first time?
Number of dependancies introduces (i.e. a new import statment in a java class).

javolution.org

2005-03-03T08:04:00.000-08:00

I recently ran across a nice project that has some support for memory pools in java: javaolution.org. Since it works with the current JVM you need to do a bit of work. Instances of a pool context are created and bound to the current thread. Then, any objects created using a special factory are actually allocated in the pool. When the thread leaves the pool context the objects are recycled.

Java Memory Pools

2004-12-03T21:42:00.000-08:00

Having developed primarily in garbage collected languages (perl, java, php), I read this article on memory allocation techniques mostly out of curiosity. The most interesting section was on using memory pools instead of the standard calls to malloc() and free(). Apache allocates a memory pool for each request. Any memory needed only for the request is put into this pool. At the end of the request the whole pool is freed.

I've been working on a garbage intensive java application that does lots of EDI and XML translation. Reading XML and mapping objects creates lots of garbage quite quickly and the JVM spends a fair amount of time clearing up garbage. It seems that some sort of pool would help in this case. I would think this could be useful for lots of transactional and web based systems.

How it would work is another matter. Perhaps a "pool context" could be associated with a thread.



 MemoryPool pool = Runtime.createMemoryPool();

 Thread.currentThread().setMemoryPool();



 Foo foo = new Foo();   // allocated in pool

 Bar bar  = new Bar();   // allocated in pool

The garbage collector would not see foo or bar. It would only see and collect from pool.It's more complicated than that, though, since you could allocate
something in the pool and then assign it to a variable (for example,a Map) outside of the pool.




  HashMap map = new HashMap();



  MemoryPool pool = Runtime.createMemoryPool();

  Thread.currentThread().setMemoryPool();



  Foo foo = new Foo();   // allocated in pool

  map.put("Foo", foo);

In this case the garbage collector would see foo.

The compexity probably outweighs any real benefits, but I find the idea interesting either way.