Anaphylaxis

Not sure if I mentioned it already, but I don't really like the taste of honey.  I keep bees because they are fascinating creatures with highly complex behavior, both individually and as a colony.  The mixture of practical and theoretical skills required to keep honeybees well appeals to me.

Of course, when you say "bees" people immediately think "stings", probably before they even think of "honey".  I have all the protective gear, but inevitably as a beekeeper you will get stung now and again.  I find that autumn is the worst time of year; understandably perhaps as the bees are protecting their stores built up over summer with little prospect of replacing them before going into winter.

Most stings are harmless enough.  Full coverage clothing means that the bee's stinger does go in far, and if it is far enough for me to notice I can take it out easily.  However, recently I had two different episodes where the reaction was a bit more dramatic.

In one case I was carrying a hive full of bees to move it to another apiary.  The entrance was sealed-up, but a couple of bees were hanging underneath looking for the way in, and took exception to my hand being there nearly squashing them.  I got a full dose of venom in my finger, which was un-gloved as the bees were sealed in, right?  Rather than drop the hive with 50,000 more bees inside I just had to grin and bear it.

The normal response to a bee sting is short intense pain followed by localized swelling, redness and swelling, and a wheal around the sting site lasting about a week.  The reaction is localized to within ~10cm of the sting site.

My reaction was a bit different.  Within ten minutes I had a severe pounding sensation in my head, dizzyness, itchy rash (ironically called "hives") over my whole body, swelling in my armpits and groin, and a feeling of pressure on my chest.  Bugger!  I had to lie down, took a triple dose of anti-histamines, and waited about six uncomfortable hours for it to pass.

The second time I was mowing the lawn quite close to the hives, and I guess one bee took exception, and without warning stung me on my temple.  Bees usually will give you a warning by "head-butting" you a few times, and irately buzzing round you -- this one just went for it!  The result was a similar whole-body reaction.

I'm not big on medications, but the thought has crossed my mind about what would happen if I was careless enough to get multiple stings simultaneously.  So I've now got an epinephrine autoinjector next to the other bits and pieces in my bee kit.

The best description of the recognition and treatment of anaphylaxis I have found is published by the Resuscitation Council (UK).

Provided you watch and listen to what the bees are telling you, then with proper handling and the right protective equipment, getting stung should be a rare occurrence for a beekeeper.  I'm still learning.

IBM and OpenJDK

IBM and Oracle are going to bring their combined resources together to collaborate in OpenJDK. The natural question arises about what this means for the Apache Harmony project.

Apache Harmony has always been clear about the goal of innovating on a compliant and compatible implementation of Java SE. It's also common knowledge that Apache have been requesting a compatibility test kit license for a number of years, and that a suitable license has not been forthcoming. There's little prospect of that situation changing.

So what's best for the Java ecosystem? I believe that compatibility is vital, and rather than risk divergence the right thing is to bring the key platform development groups together on a common codebase. Lessons learned on Project Harmony will be of value to OpenJDK, and I know there is immense mutual respect between the IBM and Oracle engineers.

Java trap not yet disarmed

There is already plenty of opinion written about the Oracle vs. Google action. As a member of the Apache Harmony project I'm an interested bystander in this story, since Android's runtime core libraries come from Apache Harmony. To date, Apache Harmony have not been notified of any involvement in the lawsuit.

Carlo concludes in his blog entry "the open source credibility of Oracle, already damaged by the OpenSolaris affair, is now destroyed" -- and that is the real shame here. Oracle's use of patents in this manner is not good news for anyone in the Java ecosystem who is promoting or using Java as a free and open runtime platform.

Having multiple competing implementations of a specification, that are ultimately held to account for claiming compatibility to the specification by a rigorous test suite, is a good thing.

As is widely known, Apache Harmony has been denied access to the Java SE compatibility test suite in a long running dispute. That situation did not change with Oracle's acquisition of "the most important software [Oracle has] ever acquired". This new action, coming less than six months since the acquisition completed gives me concern for the future of OpenJDK as being the place where Oracle openly and freely advances the most important managed runtime commons.

Contributors to the OpenJDK project have already assigned joint ownership rights to Oracle via the Sun Contributor Agreement. Oracle alone has ownership of the entire OpenJDK codebase.

Apache projects are structured differently. There is no such grant of joint ownership to a single entity, and therefore it would be impossible for a single entity to take control. Apache Harmony is a patchwork of contributions, owned by the community and available to consumers under a liberal open source license. Furthermore, the Apache license has terms that explicitly handle Patent rights (see Section 3), and so it is clear to consumers that they have a license to use any Patented contributions.

I hope Mark is wrong when he says that Oracle are on a "quest to destroy the free Java world" because the only conceivable Java world is one where you can get Java wherever you want it under the freedoms of an open source license. However, it would appear that the trap is not yet fully disarmed, and at least one implementation may be starting to move in the wrong direction.

Oracle += Sun

Well, it took its time coming, but reports are that the acquisition has been approved by the EU today.

I expect a number of Sun employees will be opening their mailer with trepidation when they get into work this morning. Its the end of an era for a big name in the industry, but a great opportunity for a new era in the Java ecosystem.

Congratulations and good luck to all those involved.

Sunset photo by Paul Moody

Dear Santa ...

Thanks for the socks and all that, but my G1 is nearly 18 months old now and I was thinking about getting a Nexus One.

The first problem was that trying to go through the order process using FireFox doesn't seem to work.

Every path after the 'Continue' button results in:I had assumed that the system was busy, but the problem lasted a long time, so I tried with IE, and hey presto it works.

The next 'problem' is that the phone is sold via the US, and as Google are right to point out, that means the sale incurs an import duty when it comes into the UK.

Rather than stumping up the $529 (~=£330) our American friends pay for the phone, delivery included, I was now looking at...

$529 for the phone
+ $19.99 for a UK charger
+ $29.65 shipping

with UK Sales Tax on the total at 17.5% and a 6.5% import duty on all that, which makes it ~$725 (i.e. ~=£453). Ouch.

For the difference I can almost fly to the US and pick it up myself!

Lucky New Year

I went to the Google home page today, and clicked on the "I'm Feeling Lucky" button without anything in the search box.

The result was a page with countdown in seconds,


a quick arithmetic shows it's counting down to the New Year. So somebody is expecting 2010 to be lucky!

UPDATE:

Now the New Year has arrived, the same empty Lucky Button press gives some 'fireworks'!

Programming sockets using temporary ports

In an earlier posting, I showed where there was code in Apache Harmony that had some unsafe parameter checking logic, and I gave a pattern for how to do it right.

Another "anti-pattern" that I see recurring in the Harmony test cases is around client-server socket programming. The typical scenario is that the tester wants to exercise some socket code, so they spin up a new Thread to act as the server, and run the tests on the main thread. The tester either picks a port that they assume will be free, or use some horrible code that tries to find a free port:

   /*
    * Returns a different port number every 6 seconds or so. The port number
    * should be about += 100 at each 6 second interval
    */
  private static int somewhatRandomPort() {
      Calendar c = Calendar.getInstance(TimeZone.getTimeZone("UTC"));
      int minutes = c.get(Calendar.MINUTE);
      int seconds = c.get(Calendar.SECOND);

      return 6000 + (1000 * minutes) + ((seconds / 6) * 100);
  }

Guessing port numbers like this is simply awful and hopeless. Once the port has been established, the poorly written test case opens a server socket to accept connections in the server Thread, while to get the timing right the client goes into a sleep for a while to give the server time to start up. Apart from the fact that the code is going to fail intermittently, it means the test case always takes a minimum length of time to execute as it spends time sleeping.

There is no need for guessing ports or separate threads. In Java, just like other programming languages that expose the underlying platform's TCP/IP stack behaviour, binding to port zero instructs the stack to allocate an ephemeral port.

Furthermore, server sockets have a listen backlog queue capable of remembering 50 outstanding connect requests by default. Here is a example of simple client-server code using an ephemeral port and a single thread to exchange a simple message over TCP/IP sockets.

      // Set-up
      ServerSocket server = new ServerSocket(0);

      Socket client = new Socket();
      client.connect(server.getLocalSocketAddress());

      Socket worker = server.accept();

      // Do some stuff
      client.getOutputStream().write("Hello world!".getBytes("UTF-8"));
      byte[] buffer = new byte[1024];
      int length = worker.getInputStream().read(buffer);

      // Tidy-up
      client.close();
      worker.close();
      server.close();

      System.out.println(new String(buffer, 0, length, "UTF-8"));

Hopefully the code is simple enough to understand without further comment. I'll just point out that the server socket is created with an argument of "0" to mean the listening socket should be opened on any network adapter, with a stack allocated port number, and supporting up to fifty pending connections. Then the client connects to the actual interface and port that was used using getLocalSocketAddress().

The same simple example can also be written using the NIO APIs, which requires passing null to the bind() method, like this:

      // Set-up
      ServerSocketChannel server = ServerSocketChannel.open();
      server.socket().bind(null);

      SocketChannel client = SocketChannel.open();
      client.connect(server.socket().getLocalSocketAddress());

      SocketChannel worker = server.accept();

      // Do some stuff
      client.write(ByteBuffer.wrap("Hello world!".getBytes("UTF-8")));

      ByteBuffer readBuffer = ByteBuffer.allocate(1024);
      worker.read(readBuffer);
      readBuffer.flip();

      // Tidy-up
      worker.close();
      client.close();
      server.close();

      System.out.println(Charset.forName("UTF-8").decode(readBuffer));

Of course, the simple examples still ignore a few return codes and exceptions that should be considered to make the code safer, but the purpose here is to show the structure for tests and applications that need to use sockets to exchange information.

Shoddy reporting from El Reg :-(

I stop by The Register website quite frequently to keep up on the gossip, and quite enjoy their laid back reporting style, but the September 4th article titled "Oracle should relax Sun's Java Community control grip" was a very poor piece of reporting.

It hit my radar with the reference to Apache Harmony, where Clarke writes:

For all its evangelism - and its initial decision to open source Java - Sun has refused to open the TCKs, infuriating and frustrating the open-source community.

This has led to accusations that Sun is hindering - not helping - open-source Java projects such as Harmony from the Apache Software Foundation (ASF), backed strongly by IBM.

While Apache's has been able to build an implementation of Java Standard Edition under Project Harmony thanks to the opening of Java, Harmony cannot be certified because the TCKs contain proprietary code the open-source code cannot touch. Harmony, therefore, remains stuck in a limbo of having been built but being uncertified.

Unfortunately this just serves to illustrate that Clarke doesn't understand the situation, or history, around open source Java SE.

Apache have never asked, or expected, Sun to open source their TCKs. While that would be a fine outcome in itself, there is no reason why Sun cannot maintain their tests as proprietary code and make them available under a variety of license terms as they do with other Java specification test suites. Apache have asked for license terms that allow the code we have written to be released under an open source license, that is to say Apache will not entertain restrictions placed on our code that passes the TCK.

http://www.apache.org/jcp/sunopenletter.html

Furthermore, Apache Harmony and GNU Classpath existed well before OpenJDK, so it can hardly be said that OpenJDK was a prerequisite to alternative implementations being created!

The story attempts to cover a great deal of ground, the possible effects of the acquisition, the Apache dispute, JCP reform, and so on -- each of these is a major story in itself and this shoddy attempt does each a disservice.

Improving parameter checking in Apache Harmony

I'm going through some Apache Harmony code at the moment, and spotting a few places where there is a common pattern of code that needs fixing.

Consider this method:

 public synchronized int read(byte[] target, int offset, int length) throws IOException {
   if (length + offset > target.length || length < 0 || offset < 0) {
       throw new ArrayIndexOutOfBoundsException();
   }
   if (0 == length) {
       return 0;
   }
   ByteBuffer buffer = ByteBuffer.wrap(target, offset, length);
   return channel.read(buffer);
 }

Can you see the problem?
Hint: the parameter checking in the if() statement is wrong.

The original author rightly wants to ensure that the offset and length represent plausible values for the data to be put into the buffer; that is, the method should not attempt to place data outside the bounds of the 'target' byte array.

The problem is that in Java, like a number of other languages, (int) values overflow without any exception being thrown. In the code above we are checking for illegal values by computing (length + offset) and throwing an exception if it is greater than the buffer's length. This ensures we don't "write off the end" of the target buffer. However, if the sum overflows an int representation then it may indeed pass this safety check, and permit a value for offset or length that is too large.

As a practical illustration, consider that the 'offset' actual parameter value is Integer.MAX_VALUE = (int)2147483647, and the 'length' value is, say 42. In this case the sum of these positive values is a negative value '(int)(2147483647 + 42) = (int)-2147483607' so the less-than test passes and the method is broken!

The fix is to avoid the sum, and convert it into a subtraction, e.g.

if (length < 0 || offset < 0 || length > target.length - offset) {
    throw new ArrayIndexOutOfBoundsException();
}

Now we can be assured that there will be no int overflow. The target.length cannot be less than zero, so the subtraction can only produce values in the range (0 - Integer.MAX_VALUE = Integer.MIN_VALUE + 1) to (Integer.MAX_VALUE - 0 = Integer.MAX_VALUE).

I know this is not news for Java developers. Josh Bloch and others have been highlighting the problem for a number of years, yet I've found about four places in the code so far that exhibit this potential problem. They'll all be fixed in time for the next milestone build.

Queen introduction

I got two new queen bees in the post a short time ago. They come in normal padded envelopes via Royal Mail. The postman was wondering why the envelope was buzzing!

Within the envelope each queen and her attendants is protected in a 'puzzle cage'. The cage has some candy to to keep the queen alive, and since the queen never feeds herself there are worker bee attendants to feed her and keep things calm during travel. If you look closely you can see the queen bee on the right, she has a green dot on her back showing she is the latest 2009 model.

To introduce a queen you can either use the puzzle cage they came in which has knock-out tabs enabling the bees to walk out one the candy is consumed, but I used a Butler cage. The Butler cage is a simple wire cage permanently blocked at one end, and which you seal at the other end using newspaper and elastic bands.

The knack is to take the queen out (let the attendants fly away) and put her into a separate cage. I did it on a sheet near a window since the queen may well fly off too, and at £20 each you don't want to loose her! The attendants will sting, but the queen doesn't sting you so you can pick her up by her thorax and pop her in.

After carefully putting the queen into the Butler cage and sealing it up it is ready to go into the hive. Opinions vary, but I take out the old queen about two days before introducing the new queen. The period of queenlessness makes the workers more inclined to accept the new smell. I kept the old queen in a jar with some attendants and candy in the case the introduction failed and I needed to back out of the procedure.

As you can see from the photo I have put a short nail in the cage's block so that I can push it into the comb. It is put in position so that the newspaper end of the cage is centred on the middle frame, right in a patch of brood where the workers would expect to find the queen.

The receiving colony's workers can feed and lick the queen through the cage, but if they try to attack they cannot ball her in there, and she can retreat into the newspaper end if they try to sting her. Over time the new queen pheromone permeates the colony and the workers will be ready to accept her.

The queen is eventually released by workers eating through the newspaper, and I gave two pin pricks on the end for them to get hold of with their mandibles.

Once the cage is in place another recommendation of Professor Ratnieks is to heavily smoke the colony. Not sure if that is a distraction technique or simply another way to mask the new incoming smell. I put in loads of smoke so that the bees were fanning furiously.

Then leave the colony alone for at least two days. After that time I checked to ensure the queen had been released, and removed the cage. The new queen was wandering over the comb without a care in the world, probably quite oblivious of her travels through the postal system a few days earlier.

I left them alone for another week, then looked again and there were eggs a plenty, so she was working away happily. Some people report as low as 50% success rate at introducing new queens. I only did these two, one into a colony that had been queenless for a while and the other where I removed an existing queen. Both worked for me so I'm happy with that.

Organised volunteerism

As part of a longer discussion on a private mailing list[1] Sam Ruby wrote a succinct description of the Apache volunteerism ethos. He wrote:

"The prevailing attitude within Apache is that releases will be done when they are ready, and that such releases will contain only the functions for which there exists volunteers who have an interest in doing the work. At times, there are people who would prefer a more predictable schedule and specific function. There are organizations which provide such assurances. This isn't one of them."

This resonated with me as a participant in the Apache Harmony project which is undertaking a full implementation of the Java SE specification. There are some modules that attract lots of attention in completeness and performance (such as the core LUNI, beans, security, and so on) and others that don't attract so much effort (such as Swing, print, and RMI).

I'm totally fine with that situation since it represents the technological equivalent of the adaptive market hypothesis of financial markets. In our world, people will tend and care for code that is important to them, and the other code by definition is not so important to people.

I've been a contributor to numerous open source foundations and working groups with different styles of working, and there is no "one true way" -- having a variety of organizations with different styles of working ensures that there is going to be a place for a wide variety of people to be comfortable innovating in the advancement of Java technology.

I'm comfortable with those who want to make some code open, and keep other code to themselves. I object to those who claim there is only one way to behave, and try to enforce that on others through restrictive licensing or organizational rules.

[1] Sam kindly gave me permission to take that quote and make it public.

WebSphere Application Server for Developers

IBM's WebSphere Application Server (WAS) is the company's flagship Java EE 5 product used in production environments worldwide.

As you would expect for a product representing over 10 years investment from Big Blue, it comes with a license fee to match. However, if you are willing to forgo formal support, and can live with the developer license offered, you can now download the full application server at no cost!

The "WebSphere Application Server for Developers" package for Windows or Linux is available as a download directly from IBM. Bear in mind that WAS weighs in at 788 MB, so downloading over a 1.5Mbps connection is likely to take about one and a half hours.

Go and grab it, and try your software out on a runtime that matches the production environment. In place of formal support there are WAS developers answering questions on the open newsgroup.

JavaOne 2009

The attendance figures put around for JavaOne 2009 state that there were "around 15,000" people at the Moscone Center -- hmm, that seems generous and it certainly felt a bit quieter than previous years. Not altogether unexpected with the economy impacting on travel, maybe people avoiding large congregations over concern of flu, and so on.

The main hall was arranged slightly differently, with the stage set up at the 'side' of the hall, with large back drapes hung across each end. Ushers did a good job of corralling people to the front, and there was a noticeably large representation from Sun themselves.

Like last year there was much talk of Sun's proprietary technology called "JavaFX", with demos of JavaFX running on a disparate variety of hardware (set top box, TV, phone, desktop). There was discussion of Java performance, as realized by the contributions made from new machine advances from Intel. Of course, it wasn't called out that the Nahelem numbers were published on IBM's Java implementation! There was a demo of the new Java market place where Sun, like others, are aiming to reproduce the success of an iTunes style store front to end users.

There was not much talk about Java SE 7 (Jonathan Schwartz did mistakenly announce it's availability, but was corrected by his Chief Engineer on stage the next day), and not much talk about OpenJDK or the JCP which were prominent topics of calls for community participation in previous years. NetBeans was also notably absent from the opening session.

The opening session was concluded by a moving 'good-bye' speech from Scott McNealy and a symbolic hand-over to Larry Ellison. During the hand-over Larry gave a challenge to the OpenOffice team to rewrite in JavaFX. He also said that Sun could be expected to produce notebooks running a Java-based OS in competition with Android.

Later in the week I gave a demo of the Apache Harmony runtime during the IBM general session. I showed Harmony running Eclipse and developing the NIO module using the Eclipse PDE, then went on to show Harmony running Geronimo and the Roller blogging webapp. Craig Hayman had slides of Harmony running the WebSphere eXtreme scale server too. With the existing modular architecture of Apache Harmony, it is trivial to build right-sized runtimes for each of these.

Many people have speculated that this may be the last JavaOne conference, but I don't think so. It is a well organized shop window for Sun/Oracle, and while it remains important to associate Java the ecosystem with a particular vendor I believe that the marketing event will survive.

Not so fast Dagastine

Earlier this week David Dagastine was quick off the mark reporting HotSpot as:

"the fastest, most reliable, and most widely deployed, open source Java Runtime in the world"

running on an Intel Nehalem. Unfortunately a bit too quick, since he didn't wait for the results to come in from IBM's J9 VM which posted a faster result as reported by Derek (who is, allegedy 8-10% better looking too)!

The HotSpot results were partially attributed to improvements in the class library, which David (when challenged by Mark Wielaard) tells us "need additional work to integrate into OpenJDK, but that work is in progress now" -- I assume that they are in the same queue as the TreeMap performance changes that are apparently in the process of being contributed back to Apache Harmony.

So today "Sun Java powered by the HotSpot Server VM" is not the fastest and not open source. Looks like some catching-up to do again.

For best results ...

This sticker was on the back of a Jeremy Clarkson DVD in Sainsbury's!

Remember to remove all outer packaging first.

Sun welcome at EclipseCon

I'm delighted to see that Sun are Gold sponsors of the EclipseCon event next week in Santa Clara. It's sure to be a great event, and encouraging to see Sun able to demo their Eclipse support for Solaris, Glassfish and JavaFX.

I wonder if the Netbeans girls will be back again too?

The "wrong type" of nature

Here in the South of England spring seems to have arrived, and that means getting back out to the apiary to see how things have faired over winter.

I was pleased to see the colonies in both my hives are still alive and well (sadly not the case for many beekeepers' colonies this year).

You can see from the photo on the left that I left my hives with some winter protection. If you look closely you can see a metal "mouse guard" to keep the hibernating critters out, a super of honey for the bees' winter food, a solid roof which you can't see but has insulation inside, and the whole lot is surrounded with chicken wire -- which I had lying around in the garage.

The chicken wire is there to stop woodpeckers. In winter, when food is scarce, the woodpeckers figure out that there are bees in those big "lunch boxes", and they cling on the side to peck a hole and help themselves. Apparently, they will even tap around a bit and figure out that the handles are the thinnest part of the hive, and will attack there.

Well that was a good move, because the birds had helped themselves to some friends' bees that were placed close by:
Apart from the annoyance of having the woodwork damaged, and having bees eaten and wax damaged - having an extra entrance means the hive is harder to defend for the remaining bees and the reduced colony may well be "robbed" of any remaining food by stronger colonies. Once the woodpeckers learn there are free bees, they also tell their friends and you get them all round for a feast.

Luckily the damage was spotted quite quickly, and repairs made. Besides using chicken wire you can also tack plastic sheeting on the sides to stop the birds getting a hold.

With both my colonies growing again it will be time to restart the regular inspections and get back into the intriguing world of the honeybee.

The Magic Android powered phone

The BBC are reporting a new Android-based phone announcement at Mobile World Congress. The HTC "Magic" will go on sale in various European countries through Vodafone.

Judging by the job advertisements, you can also get a reasonable salary with Android programming skills compared to Java ME experience. It's still a tiny percentage of the demand but maybe a sign of some traction in larger companies. Watch out for the onset of non-phone devices powered by Android.

Bops, tree hugging, and safe wagers

Back in July I wrote about how Apache Harmony's TreeMap implementation had been used (with modifications) to deliver improvements in Sun's SPECjbb2005 score by "a solid 3-5% depending on the platform" according to Dave who also promised that they were "making the necessary steps to give back [the] code changes to Apache Harmony"; even though they are not obliged to do so according to the Apache License.

Did you expect us to sit and wait?!

IBM has just published its latest SPECjbb2005 scores with, amongst other things, an enhanced version of TreeMap from Harmony that gives them a 10% boost, and thereby move ahead in many reporting categories. Derek has all the details, and I encourage you to read them.

It would be interesting to look back and compare the changes Sun and IBM made to Harmony's TreeMap, and merge them for the common good.

While I'm sure Dave's intentions are pure, I doubt the suits will let him contribute his changes back to Apache and all the talk of "requiring innovation sharing in the commons" only applies in one direction. In fact, I'm so sure, that if Sun do successfully contribute their TreeMap changes back to Apache Harmony I promise to give £500 of my own hard-earned money to The Woodland Trust to benefit some real trees.

Debugging mixed-language code in Apache Harmony + Eclipse

I've been hacking in managed runtimes for more years than I'd care to mention, and plenty of that time has been spent in debuggers of one sort or another!

...and that's the point. Implementing a runtime requires that you have a suite of debuggers that cover the different target platform native code and the dynamic late-bound language du jour. You have to switch from Eclipse to WinDbg/gdb and back again frequently.

Intel's Software group have (for some unknown reason) been quite low key about their innovation in this space, with the development of an integrated debugger for Java and JNI environments. This advance has been enabled by the combination of support in the Eclipse Java Debug Tools and Apache Harmony runtime.

Now, as you step into a native method from Java, the Eclipse debugger shows a logical representation of the mixed Java thread / C thread stack frames and allows you to inspect variables etc. in either language. They also support the JNI methods that call back into Java so you get full roundtripping. Cool.

For most people who need to debug a system comprising Java and native code this is exactly what you want to do, and its a real time saver to be able to manage the breakpoints, variable watches and flow of control from a single IDE.

Kudos to Chris, Vasily, Gregory, Ilya, et al who made this work. Clearly people who live in the real world!

Go watch the screencast demo, download it and try it out!