Thursday, July 26, 2007

Here be (multiple) dragons

Are you thinking about the impact of multi-core processors on the code you write? Are you just thinking about it or actually writing code that will exploit multiple cores?

"The computing industry changed course in 2005 when Intel followed the lead of IBM’s Power 4 and Sun Microsystems’ Niagara processor in announcing that its high performance microprocessors would henceforth rely on multiple processors or cores." [The Landscape of Parallel Computing Research: A View from Berkeley]
Programming for multiple processors is hard. I find myself having to make an explicit mental switch from a 'functional view' of what the program is trying to accomplish, to a 'multi-threaded view' of how the flow of control can help / hinder the program effectiveness in the light of multiple threads running on multiple cores. Then mentally switch back and forth until both worlds are reconciled. I'm sure smarter people can do this in a single pass, but I can't.

There are multiple dragons to consider too, that live in the regular world of parallel algorithms (deadlock, stalling, synchronization, ...), in the nuances of the system's memory model (visibility guarantees, NUMA, data races, ...), and in the language implementation choices (particular compiler's instruction reordering, effects of Java final, pthreads version, ...).

In Java there is a defined memory model (JSR133) and set of concurrency utilities that significantly help with writing correct, portable programs. Intel have released a set of libraries and utilities for C++ called 'Thread Building Blocks' that should improve the correctness and portability of those programs too.

The Intel suite (which appears to be descended from the OpenMP code) runs on Windows/P4 or Xeon, Linux/P4 or Xeon or Itanium2, and MacOSX/Core Duo and has been released under a modified-GPLv2 license, with the option to commercially license at a measly 300 dollars a platform.

That commercial license option means this is another 'parallel open source project'. If you want to contribute to Thread Building Blocks then you must assign copyright to Intel, and grant them a broad patent license. In return they license your code back to you to use as you choose. The Governance model is 'Intel decide' too. I have to say that I'm no fan of this style of project, but back to the technology...

Abstracting the 'multithreaded view' into a set of consumable APIs and pragmas helps the application writer focus on the functional correctness, and allows the compiler writers to optimize for known patterns without the disruption of language changes.

It will be interesting to see if these building blocks are adopted by the other major CPU/compiler vendors, and whether they are extended into languages beyond C++. Will they do it under the Intel terms for this project?

The technology is sound. Anything that helps us voyagers to navigate the uncharted waters and avoid the dragons will be most welcome.

Tuesday, July 24, 2007

Grand Poobah of the Harmony lodge

I'm delighted to have been appointed to the Chair of the Apache Harmony project, taking over from fellow harmonizer Geir Magnusson, who has been chair since the project's inception in 2005. Geir moves on the Apache Board, and remains Apache's JCP SE/EE Executive Comittee member.

In truth, the position of Grand Poobah doesn't grant any exceptional powers -- I don't even get a big hat or fancy medal or anything :-). Of course, the Harmony project is the community of developers who show up on the dev' list every day, and it is managed day to day by the committers on the PMC.

Since Apache is a 501(c)3 not-for-profit corporation, the Board requires a corporate officer acting on their behalf within the project, and in turn all project members provide oversight and guidance to the project. This delegation model provides shared responsibility with the umbrella protection of the Foundation as a whole.

It's an honour to be appointed V.P., Harmony. Now if you'll excuse me, I'm off over to eBay to look at the hats on offer.

[For the avoidance of doubt, "no" that is not me in the photo (or Ken Coar), it is His Holiness Pope Shenouda III photographed by Michael Sleman]

Thursday, July 12, 2007

OSSummit call for speakers is closing

If you were thinking of submitting a talk proposal for the joint Eclipse / Apache sponsored OSSummit conference being held in Hong Kong at the end of November this year -- then you don't have much time left to act! The Call for Participation (CFP) closes on July 13th.

While the CFP promotes the meeting's interest in Apache and Eclipse projects, I'd personally be particularly interested to see a significant number speakers from local free software and open source efforts in the P.R.China.

The organizers are also providing a generous allowance for flights and accommodation to all conference speakers, so it should be a great opportunity for everyone to meet and exchange experiences and advice.

[Disclosure: I'm on the organizing committee for OSSummit]

Monday, July 02, 2007

G-WMAA cleared for take off

I live in Hampshire which, until yesterday, was the only remaining county in the UK that didn't have an air ambulance service. The service will be particularly useful to residents of the nearby Isle of Wight if they need emergency care only available on the mainland, as well as medical emergencies along the area's coast or in the county's broad rural areas.

Through the hard work of the staff and volunteers of the associated charity the Bolkow 105 aircraft has been bought (call sign G-WMAA) and has become operative. The charity needs £2,000 per day to keep the air ambulance aloft, and all of it is raised through community donations since the charity receives no money from the government.

This project has been particularly interesting to me. As a bit of an aircraft nut, and such a clear benefit to the community, I have been happy to contribute (in a very small way) to the challenge over the last couple of years. It's great to see the service finally get off the ground, literally.

I hope never to need the service, of course, and wouldn't wish the need on anyone; but accidents happen, and when they do we should provide the best service we can to help the unfortunate victims.

The beginning of the beginning of the end for the JCP

There is trouble brewing in the world of Java. The evidence is not so apparent at the moment, but it has the potential to rock the Java ecosystem to it's foundations if the issues are mishandled.

The principal issue is around the definition of 'standard' Java. By mutual agreement, companies and individuals contribute to the definition of new Java standards through the Java Community Process (JCP). The 'mutual agreement' of these contributors is written down as the Java Specification Participation Agreement (JSPA), which is the legal document describing the terms under which the contributions are made and can be used. Like all legal documents, it doesn't describe every last nuance of the community interaction, nor should it attempt to do so, but in general the JCP has worked because there has not been an unreasonable agent at play -- everyone has been participating for the general improvement of Java, and everyone benefits from the general improvement of Java.

So what happens when somebody tries to tip the balance of community participation in their favour? Would you expect other community members to cry foul? try to right the wrong? withdraw their collaboration? Well, yes!

It is well documented how Apache have written an open letter to Sun asking for a license to the the Java SE 5 JCK. So far, Sun have refused to grant such a license under terms that community members consider reasonable. The details of the unacceptable terms are not so important here, but if the majority of the community feels that the work they have contributed via the JCP is being misappropriated by an unreasonable agent, what do you think they are likely to do?

I suggest that unless Sun engage in a round of diplomacy and negotiation with the 'heavy-lifters' on the JCP Java SE/EE Executive Committee, such 'heavy-lifters' will decide pretty quickly that contributing to Java via the JCP is not a safe, smart thing to be doing, and the centre of gravity will move elsewhere.

In the current situation, Apache is the fall guy and the JCP members are watching carefully to see how the issue will be resolved. Whereas licensees normally negotiate with dollars behind closed doors, Apache are holding up (open source) principals and (now) negotiating in public. JCP members know that next time they could be the target

We can only speculate as to why Sun are taking the current position. Maybe it is a control thing -- their marketing certainly leads people to believe that Java == Sun, rather than Java is a result of contributions from many people that are brought together by Sun in the JCP.

Maybe it is the fear that the JCK is inadequate to ensure Java's value proposition of 'write once run anywhere' (WORA). There is a huge gap between being compliant (passing the JCK) and being compatible (behaving like the reference implementation). For WORA to work, Java developers have to code only to the specification, or every implementation has to behave the same way down to very small details. For so long, there has been only one game in town that developers have coded to the reference implementation, not the specification. So when Apache deliver a compliant implementation, and Sun have to admit that it is real Java, but it doesn't run half of the Java applications on the planet... what damage will that do to the Java brand?

So I further expect that Sun will try to migrate the work currently conducted "through JSR's in the JCP" to work being conducted "in projects in OpenJDK" -- that is, try to blur the distinction of the specification and the implementation. The goal being to ensure that there is only a single implementation. Of course, with only a single implementation the specification becomes moot -- the implementation does whatever it does.

That's how I get to my conclusion, that this may be the beginning of the end for the JCP. By granting a suitable license to Apache, Sun would demonstrate to the community that the JCP is more than a mechanism by which they attract contributions to a single implementation they control. They may be persuaded to do so, even if they would prefer not to.

[2007-07-09 Edited to remove duplication junk, maybe caused by using a dollar sign?]

Sunday, July 01, 2007

Bank of Tim International opens today!

Being a Dad often feels like being a bank to the family, and the most popular transactions tend to be withdrawals! But this weekend I upped my game and became an International Commercial Banker.

I've made a loan to a Samoan sewing business to be repaid over a term of 18 months. Maybe. I've never met Freida Lisa Makesi (pictured), the businesswoman who took out the loan, which has been granted with no security or guarantee, and for which there is no interest payable (back to me).

Ok, I'll come clean. I signed up with a microloan site called Kiva. The money at stake is only USD$200, and I'm partnering with 15 other people to provide the full value of the $850 loan that Freida's business needs to buy a sewing machine.

Kiva is an interesting way to link those of us for which a couple of hundred bucks is not a life-changing amount, with those for whom it most certainly is. The website details a number of needy businesses that have applied for microloans under conditions that a 'real' commercial bank would not touch. You use your own criteria for selecting the borrower, and lend them any amount from $25 up to the full loan amount.

As the loan is repaid you get to see the payments being returned as credit to your Kiva account. From there you can withdraw the money for a new gadget you don't really need, or lend it out again to help another business move one small step up the ladder. Of course, you may never see the money again for any number of reasons, not least is that the mortality rate in the countries supported is way higher than where YOU probably live, or the borrower simply disappears with the money; but the field workers who administer the loan on your behalf select the borrowers based on their experience, and have an remarkably low defaulting rate on the loans they arrange.

I have a structured approach to my charitable giving, but this is the first time I've tried lending money to a worthy cause. I'll let you know how it goes, but Freida Lisa Makesi has to be a better risk than my daughter for making good use of the money, and maybe paying it back!