Thursday, December 31, 2009

Lucky New Year

I went to the Google home page today, and clicked on the "I'm Feeling Lucky" button without anything in the search box.

The result was a page with countdown in seconds,


a quick arithmetic shows it's counting down to the New Year. So somebody is expecting 2010 to be lucky!

UPDATE:

Now the New Year has arrived, the same empty Lucky Button press gives some 'fireworks'!

Wednesday, September 16, 2009

Programming sockets using temporary ports

In an earlier posting, I showed where there was code in Apache Harmony that had some unsafe parameter checking logic, and I gave a pattern for how to do it right.

Another "anti-pattern" that I see recurring in the Harmony test cases is around client-server socket programming. The typical scenario is that the tester wants to exercise some socket code, so they spin up a new Thread to act as the server, and run the tests on the main thread. The tester either picks a port that they assume will be free, or use some horrible code that tries to find a free port:


/*
* Returns a different port number every 6 seconds or so. The port number
* should be about += 100 at each 6 second interval
*/
private static int somewhatRandomPort() {
Calendar c = Calendar.getInstance(TimeZone.getTimeZone("UTC"));
int minutes = c.get(Calendar.MINUTE);
int seconds = c.get(Calendar.SECOND);

return 6000 + (1000 * minutes) + ((seconds / 6) * 100);
}

Guessing port numbers like this is simply awful and hopeless. Once the port has been established, the poorly written test case opens a server socket to accept connections in the server Thread, while to get the timing right the client goes into a sleep for a while to give the server time to start up. Apart from the fact that the code is going to fail intermittently, it means the test case always takes a minimum length of time to execute as it spends time sleeping.

There is no need for guessing ports or separate threads. In Java, just like other programming languages that expose the underlying platform's TCP/IP stack behaviour, binding to port zero instructs the stack to allocate an ephemeral port.

Furthermore, server sockets have a listen backlog queue capable of remembering 50 outstanding connect requests by default. Here is a example of simple client-server code using an ephemeral port and a single thread to exchange a simple message over TCP/IP sockets.

// Set-up
ServerSocket server = new ServerSocket(0);

Socket client = new Socket();
client.connect(server.getLocalSocketAddress());

Socket worker = server.accept();

// Do some stuff
client.getOutputStream().write("Hello world!".getBytes("UTF-8"));
byte[] buffer = new byte[1024];
int length = worker.getInputStream().read(buffer);

// Tidy-up
client.close();
worker.close();
server.close();

System.out.println(new String(buffer, 0, length, "UTF-8"));

Hopefully the code is simple enough to understand without further comment. I'll just point out that the server socket is created with an argument of "0" to mean the listening socket should be opened on any network adapter, with a stack allocated port number, and supporting up to fifty pending connections. Then the client connects to the actual interface and port that was used using getLocalSocketAddress().

The same simple example can also be written using the NIO APIs, which requires passing null to the bind() method, like this:

// Set-up
ServerSocketChannel server = ServerSocketChannel.open();
server.socket().bind(null);

SocketChannel client = SocketChannel.open();
client.connect(server.socket().getLocalSocketAddress());

SocketChannel worker = server.accept();

// Do some stuff
client.write(ByteBuffer.wrap("Hello world!".getBytes("UTF-8")));

ByteBuffer readBuffer = ByteBuffer.allocate(1024);
worker.read(readBuffer);
readBuffer.flip();

// Tidy-up
worker.close();
client.close();
server.close();

System.out.println(Charset.forName("UTF-8").decode(readBuffer));

Of course, the simple examples still ignore a few return codes and exceptions that should be considered to make the code safer, but the purpose here is to show the structure for tests and applications that need to use sockets to exchange information.

Monday, September 07, 2009

Shoddy reporting from El Reg :-(

I stop by The Register website quite frequently to keep up on the gossip, and quite enjoy their laid back reporting style, but the September 4th article titled "Oracle should relax Sun's Java Community control grip" was a very poor piece of reporting.

It hit my radar with the reference to Apache Harmony, where Clarke writes:

For all its evangelism - and its initial decision to open source Java - Sun has refused to open the TCKs, infuriating and frustrating the open-source community.

This has led to accusations that Sun is hindering - not helping - open-source Java projects such as Harmony from the Apache Software Foundation (ASF), backed strongly by IBM.

While Apache's has been able to build an implementation of Java Standard Edition under Project Harmony thanks to the opening of Java, Harmony cannot be certified because the TCKs contain proprietary code the open-source code cannot touch. Harmony, therefore, remains stuck in a limbo of having been built but being uncertified.
Unfortunately this just serves to illustrate that Clarke doesn't understand the situation, or history, around open source Java SE.

Apache have never asked, or expected, Sun to open source their TCKs. While that would be a fine outcome in itself, there is no reason why Sun cannot maintain their tests as proprietary code and make them available under a variety of license terms as they do with other Java specification test suites. Apache have asked for license terms that allow the code we have written to be released under an open source license, that is to say Apache will not entertain restrictions placed on our code that passes the TCK.

http://www.apache.org/jcp/sunopenletter.html

Furthermore, Apache Harmony and GNU Classpath existed well before OpenJDK, so it can hardly be said that OpenJDK was a prerequisite to alternative implementations being created!

The story attempts to cover a great deal of ground, the possible effects of the acquisition, the Apache dispute, JCP reform, and so on -- each of these is a major story in itself and this shoddy attempt does each a disservice.

Improving parameter checking in Apache Harmony

I'm going through some Apache Harmony code at the moment, and spotting a few places where there is a common pattern of code that needs fixing.

Consider this method:


public synchronized int read(byte[] target, int offset, int length) throws IOException {
if (length + offset > target.length || length < 0 || offset < 0) {
throw new ArrayIndexOutOfBoundsException();
}
if (0 == length) {
return 0;
}
ByteBuffer buffer = ByteBuffer.wrap(target, offset, length);
return channel.read(buffer);
}

Can you see the problem?
Hint: the parameter checking in the if() statement is wrong.

The original author rightly wants to ensure that the offset and length represent plausible values for the data to be put into the buffer; that is, the method should not attempt to place data outside the bounds of the 'target' byte array.

The problem is that in Java, like a number of other languages, (int) values overflow without any exception being thrown. In the code above we are checking for illegal values by computing (length + offset) and throwing an exception if it is greater than the buffer's length. This ensures we don't "write off the end" of the target buffer. However, if the sum overflows an int representation then it may indeed pass this safety check, and permit a value for offset or length that is too large.

As a practical illustration, consider that the 'offset' actual parameter value is Integer.MAX_VALUE = (int)2147483647, and the 'length' value is, say 42. In this case the sum of these positive values is a negative value '(int)(2147483647 + 42) = (int)-2147483607' so the less-than test passes and the method is broken!

The fix is to avoid the sum, and convert it into a subtraction, e.g.
if (length < 0 || offset < 0 || length > target.length - offset) {
throw new ArrayIndexOutOfBoundsException();
}
Now we can be assured that there will be no int overflow. The target.length cannot be less than zero, so the subtraction can only produce values in the range (0 - Integer.MAX_VALUE = Integer.MIN_VALUE + 1) to (Integer.MAX_VALUE - 0 = Integer.MAX_VALUE).

I know this is not news for Java developers. Josh Bloch and others have been highlighting the problem for a number of years, yet I've found about four places in the code so far that exhibit this potential problem. They'll all be fixed in time for the next milestone build.

Wednesday, July 08, 2009

Queen introduction

I got two new queen bees in the post a short time ago. They come in normal padded envelopes via Royal Mail. The postman was wondering why the envelope was buzzing!

Within the envelope each queen and her attendants is protected in a 'puzzle cage'. The cage has some candy to to keep the queen alive, and since the queen never feeds herself there are worker bee attendants to feed her and keep things calm during travel. If you look closely you can see the queen bee on the right, she has a green dot on her back showing she is the latest 2009 model.

To introduce a queen you can either use the puzzle cage they came in which has knock-out tabs enabling the bees to walk out one the candy is consumed, but I used a Butler cage. The Butler cage is a simple wire cage permanently blocked at one end, and which you seal at the other end using newspaper and elastic bands.

The knack is to take the queen out (let the attendants fly away) and put her into a separate cage. I did it on a sheet near a window since the queen may well fly off too, and at £20 each you don't want to loose her! The attendants will sting, but the queen doesn't sting you so you can pick her up by her thorax and pop her in.

After carefully putting the queen into the Butler cage and sealing it up it is ready to go into the hive. Opinions vary, but I take out the old queen about two days before introducing the new queen. The period of queenlessness makes the workers more inclined to accept the new smell. I kept the old queen in a jar with some attendants and candy in the case the introduction failed and I needed to back out of the procedure.

As you can see from the photo I have put a short nail in the cage's block so that I can push it into the comb. It is put in position so that the newspaper end of the cage is centred on the middle frame, right in a patch of brood where the workers would expect to find the queen.

The receiving colony's workers can feed and lick the queen through the cage, but if they try to attack they cannot ball her in there, and she can retreat into the newspaper end if they try to sting her. Over time the new queen pheromone permeates the colony and the workers will be ready to accept her.

The queen is eventually released by workers eating through the newspaper, and I gave two pin pricks on the end for them to get hold of with their mandibles.

Once the cage is in place another recommendation of Professor Ratnieks is to heavily smoke the colony. Not sure if that is a distraction technique or simply another way to mask the new incoming smell. I put in loads of smoke so that the bees were fanning furiously.

Then leave the colony alone for at least two days. After that time I checked to ensure the queen had been released, and removed the cage. The new queen was wandering over the comb without a care in the world, probably quite oblivious of her travels through the postal system a few days earlier.

I left them alone for another week, then looked again and there were eggs a plenty, so she was working away happily. Some people report as low as 50% success rate at introducing new queens. I only did these two, one into a colony that had been queenless for a while and the other where I removed an existing queen. Both worked for me so I'm happy with that.

Organised volunteerism

As part of a longer discussion on a private mailing list[1] Sam Ruby wrote a succinct description of the Apache volunteerism ethos. He wrote:

"The prevailing attitude within Apache is that releases will be done when they are ready, and that such releases will contain only the functions for which there exists volunteers who have an interest in doing the work. At times, there are people who would prefer a more predictable schedule and specific function. There are organizations which provide such assurances. This isn't one of them."
This resonated with me as a participant in the Apache Harmony project which is undertaking a full implementation of the Java SE specification. There are some modules that attract lots of attention in completeness and performance (such as the core LUNI, beans, security, and so on) and others that don't attract so much effort (such as Swing, print, and RMI).

I'm totally fine with that situation since it represents the technological equivalent of the adaptive market hypothesis of financial markets. In our world, people will tend and care for code that is important to them, and the other code by definition is not so important to people.

I've been a contributor to numerous open source foundations and working groups with different styles of working, and there is no "one true way" -- having a variety of organizations with different styles of working ensures that there is going to be a place for a wide variety of people to be comfortable innovating in the advancement of Java technology.

I'm comfortable with those who want to make some code open, and keep other code to themselves. I object to those who claim there is only one way to behave, and try to enforce that on others through restrictive licensing or organizational rules.

[1] Sam kindly gave me permission to take that quote and make it public.

Monday, June 22, 2009

WebSphere Application Server for Developers

IBM's WebSphere Application Server (WAS) is the company's flagship Java EE 5 product used in production environments worldwide.

As you would expect for a product representing over 10 years investment from Big Blue, it comes with a license fee to match. However, if you are willing to forgo formal support, and can live with the developer license offered, you can now download the full application server at no cost!

The "WebSphere Application Server for Developers" package for Windows or Linux is available as a download directly from IBM. Bear in mind that WAS weighs in at 788 MB, so downloading over a 1.5Mbps connection is likely to take about one and a half hours.

Go and grab it, and try your software out on a runtime that matches the production environment. In place of formal support there are WAS developers answering questions on the open newsgroup.

Monday, June 15, 2009

JavaOne 2009


The attendance figures put around for JavaOne 2009 state that there were "around 15,000" people at the Moscone Center -- hmm, that seems generous and it certainly felt a bit quieter than previous years. Not altogether unexpected with the economy impacting on travel, maybe people avoiding large congregations over concern of flu, and so on.

The main hall was arranged slightly differently, with the stage set up at the 'side' of the hall, with large back drapes hung across each end. Ushers did a good job of corralling people to the front, and there was a noticeably large representation from Sun themselves.

Like last year there was much talk of Sun's proprietary technology called "JavaFX", with demos of JavaFX running on a disparate variety of hardware (set top box, TV, phone, desktop). There was discussion of Java performance, as realized by the contributions made from new machine advances from Intel. Of course, it wasn't called out that the Nahelem numbers were published on IBM's Java implementation! There was a demo of the new Java market place where Sun, like others, are aiming to reproduce the success of an iTunes style store front to end users.

There was not much talk about Java SE 7 (Jonathan Schwartz did mistakenly announce it's availability, but was corrected by his Chief Engineer on stage the next day), and not much talk about OpenJDK or the JCP which were prominent topics of calls for community participation in previous years. NetBeans was also notably absent from the opening session.

The opening session was concluded by a moving 'good-bye' speech from Scott McNealy and a symbolic hand-over to Larry Ellison. During the hand-over Larry gave a challenge to the OpenOffice team to rewrite in JavaFX. He also said that Sun could be expected to produce notebooks running a Java-based OS in competition with Android.

Later in the week I gave a demo of the Apache Harmony runtime during the IBM general session. I showed Harmony running Eclipse and developing the NIO module using the Eclipse PDE, then went on to show Harmony running Geronimo and the Roller blogging webapp. Craig Hayman had slides of Harmony running the WebSphere eXtreme scale server too. With the existing modular architecture of Apache Harmony, it is trivial to build right-sized runtimes for each of these.

Many people have speculated that this may be the last JavaOne conference, but I don't think so. It is a well organized shop window for Sun/Oracle, and while it remains important to associate Java the ecosystem with a particular vendor I believe that the marketing event will survive.

Thursday, April 02, 2009

Not so fast Dagastine

Earlier this week David Dagastine was quick off the mark reporting HotSpot as:

"the fastest, most reliable, and most widely deployed, open source Java Runtime in the world"
running on an Intel Nehalem. Unfortunately a bit too quick, since he didn't wait for the results to come in from IBM's J9 VM which posted a faster result as reported by Derek (who is, allegedy 8-10% better looking too)!

The HotSpot results were partially attributed to improvements in the class library, which David (when challenged by Mark Wielaard) tells us "need additional work to integrate into OpenJDK, but that work is in progress now" -- I assume that they are in the same queue as the TreeMap performance changes that are apparently in the process of being contributed back to Apache Harmony.

So today "Sun Java powered by the HotSpot Server VM" is not the fastest and not open source. Looks like some catching-up to do again.

Saturday, March 21, 2009

For best results ...

This sticker was on the back of a Jeremy Clarkson DVD in Sainsbury's!

Remember to remove all outer packaging first.

Friday, March 20, 2009

Sun welcome at EclipseCon

I'm delighted to see that Sun are Gold sponsors of the EclipseCon event next week in Santa Clara. It's sure to be a great event, and encouraging to see Sun able to demo their Eclipse support for Solaris, Glassfish and JavaFX.

I wonder if the Netbeans girls will be back again too?

Thursday, March 19, 2009

The "wrong type" of nature

Here in the South of England spring seems to have arrived, and that means getting back out to the apiary to see how things have faired over winter.

I was pleased to see the colonies in both my hives are still alive and well (sadly not the case for many beekeepers' colonies this year).

You can see from the photo on the left that I left my hives with some winter protection. If you look closely you can see a metal "mouse guard" to keep the hibernating critters out, a super of honey for the bees' winter food, a solid roof which you can't see but has insulation inside, and the whole lot is surrounded with chicken wire -- which I had lying around in the garage.

The chicken wire is there to stop woodpeckers. In winter, when food is scarce, the woodpeckers figure out that there are bees in those big "lunch boxes", and they cling on the side to peck a hole and help themselves. Apparently, they will even tap around a bit and figure out that the handles are the thinnest part of the hive, and will attack there.

Well that was a good move, because the birds had helped themselves to some friends' bees that were placed close by:
Apart from the annoyance of having the woodwork damaged, and having bees eaten and wax damaged - having an extra entrance means the hive is harder to defend for the remaining bees and the reduced colony may well be "robbed" of any remaining food by stronger colonies. Once the woodpeckers learn there are free bees, they also tell their friends and you get them all round for a feast.

Luckily the damage was spotted quite quickly, and repairs made. Besides using chicken wire you can also tack plastic sheeting on the sides to stop the birds getting a hold.

With both my colonies growing again it will be time to restart the regular inspections and get back into the intriguing world of the honeybee.

Tuesday, February 17, 2009

The Magic Android powered phone

The BBC are reporting a new Android-based phone announcement at Mobile World Congress. The HTC "Magic" will go on sale in various European countries through Vodafone.

Judging by the job advertisements, you can also get a reasonable salary with Android programming skills compared to Java ME experience. It's still a tiny percentage of the demand but maybe a sign of some traction in larger companies. Watch out for the onset of non-phone devices powered by Android.