After graduating from college I immediately started work at a very large, multi-national communications technology company. At the time, I liked the idea of working for a large company. I thought that it would provide room to grow, job security, and the ability to switch roles easily without leaving the company. Over the past 10 years, I have moved from a company with about 100,000 employees, to a company with about 2,000 employees, and then to a company with 4 employees. Every time I moved to a company with substantially less employees, I noticed that my job satisfaction increased.

I have never been more happy with a job that I am at Signal. And, there are several reasons for this.

The ability to wear all hats

When I started at Signal there were only 2 other engineers (3 if you include the technical co-founder who was looking to focus less on technology and more on growing the business). With only three people on the engineering team, you need to be able to work on any part of the system, at any time. Application design and development, user interface design and development, database administration, system administration, IT, etc. In an environment like this there are many opportunities to learn something new. Database administration and user interface design were not strong areas of mine, but working at a startup gave me the opportunity to develop those skills. Much more so than at company where teams are segmented based on the functional areas of the application. And, even after a year and a half, I’m still learning new stuff like crazy.

This isn’t to say that some engineers aren’t stronger than others in certain areas, and you can’t play to those strengths. You’d be foolish not too. But at this size, it’s simply not practical to have one engineer do all the UI work, one do all of the database work, and one do all of the application development.

The chance to make a really big impact

I was hired as Signal’s fifth overall employee. When there are only five people in the entire company, everybody has the chance to make a really big impact. The harder you work, the bigger contribution you get to make. And contributions can take many forms, like a newly implemented feature, providing awesome support to a customer, a new idea for a product, or a new way of resolving a problem that has been facing the company. There is no product group or customer service team at a very small startup, but the needs traditionally serviced by those roles still exist. In addition, you are able to see the impact of your contribution immediately, be it positive or negative.

You also have the opportunity to play a bigger role in how your product or service develops. If you see an area that needs improvement, a bug that needs fixing, or a gaping hole in the product that could be addressed by a new feature, it’s up to you to address it. There is no committee or change board that needs to approve the change, and no need to line up resources from the database and UI teams to help with the work. You are expected to drive the change. This, for me, is a major source of inspiration and motivation.

The ability to choose your tools

One of the things a software engineer likes to complain about the most are the tools they are forced to use at their company. And with good reason. Using tools that you are comfortable with can make you more productive, and quite frankly, happier. Little is more frustrating than being forced to solve a problem one way when you know you can solve it quicker, and better, another way. This is generally not a problem at startups. While you rarely have the ability to choose all of your tools (programming language for example), you will generally have the ability to choose your work machine, operating system, editor/IDE, web browser, mail client, etc. And, since you are your own IT department, there is no question that you will have root access to your work machine. These may sound like minor details to most people, but any geek will assure you how important this actually is.

The ability to choose the right technology for the job

Many companies standardize on technologies. There are many valid reasons for this. However, sometimes the standardized option makes little sense for the task at hand, and you are forced trying to fit a square peg through a round hole. This generally doesn’t happen at startups. One of the best things about my job is that the engineering team gets to decide which technologies to use after evaluating the technical details of the problem it is trying to address. One might think that a stance like this would lead to an explosion of tools and technologies in our stack, but this is hardly the case. Being responsible and professional developers, we understand that we will have to live with and support the decisions we make. So, we carefully evaluate any significant new technology before bringing it into the stack.

A voice in choosing your teammates

When working on a small team, everybody needs to pull their weight in order for the company to be successful. Bringing new people onto the team is a big deal. A single bad team member can be a cancer to an otherwise strong and healthy team. Therefore, people interviewing for the company talk with everybody. This may be a bit overwhelming for candidates, but I think most appreciate how closely we protect our team dynamic. Now, this isn’t to say that everybody “interviews” each candidate. Our VP of marketing isn’t going to ask a candidate for an engineering position to solve a coding problem on the whiteboard. Instead, a few people will determine if the candidate is qualified for the position, and the rest will determine if the candidate would be a good fit, personality wise, on the team. As an engineer, I think this is great. When hiring a new engineer, we all get to ensure that the candidate is smart, loves technology, knows how to solve problems, and will get along with the rest of the team. And when hiring for a non engineering position, we get to ensure that the person we’re bringing onto the team isn’t a jerk.

A great work culture

Startup culture is great. I’m not talking about $1,000 desk chairs or having a company masseuse. Hell, two other engineers and I were coding around what the CEO called a “nerd campfire” the other day (can’t say that I disagree with that terminology) because we currently have no desks. I’m talking about the camaraderie; eating lunch together, getting to meet each others spouses and children, chilling out and having a few drinks with each other. I think this is very hard to find outside of small company. It seems to me that most startups have a casual, relaxed, and fun work environment. Flexible hours. No dress code to speak of. Signal even has a kegerator in the office which is always fully loaded with great beer. They do pretty much anything they can to make their employees feel comfortable so they be productive.

Flexible work options

Flexible work hours are very popular now-a-days. Even at the larger companies where I have worked, it wasn’t uncommon for some people to come in at 11am and stay until 7pm or 8pm. Having the ability to fit work around your personal schedule is a big plus, and I feel it makes for a much happier employee.

Even more important to me is the ability to work from home. I live about 30 miles from the office, which takes about 90 minutes to travel during rush hour by car or train. I’ve tried not too complain too much about my commute, because nobody forced me to live where I do. It was my choice, and I’d make the same choice again. But, when I was given the opportunity to work from home, I jumped on it. Working from home saves me 3 hours a day. That is huge for me, as it always feels like there are not enough hours in the day. Working from home 4 days a week has given me the ability to see my wife and kids more often, start running, and to spend a little more time on some side projects to improve and expand my skills as a developer. Seeing my kids more often is by far the biggest benefit of working from home. Now that my oldest is in school, I barely see him when I go into the office. Our schedules only overlap for about 30 minutes in the morning, and 30 minutes at night. This is not the case when I work from home.

No bullshit politics

When there are only 13 people in the entire company, there are no politics. There is no jockeying for position. There is no posturing to give the impression that you are an all star employee when you’re really just a leech. There is no room to hide. If you’re not contributing, everybody knows it. If you are trying to exploit others for personal gain, everybody sees it.

Your opinion matters

It’s hard to make a decision that impacts the company in complete isolation at a startup. The founders at Signal always consult the team when it comes to making a big decision. Make no mistake, they have the final say. But they value the opinions of the team and constantly request feedback. This gives the employees a sense of ownership, which is great for morale and can be very motivational.

Conclusion

I think many of these points are the product of a few major differences between a big company and a startup. First, productivity is paramount, and anything that can be done to increase productivity is done. If employees aren’t producing at their highest level, it greatly increases the risk of the startup folding (delaying time to market or being surpassed by a competitor). Second, startups run on the bare minimum. There is no extra weight to carry around. There are no employees that need to justify their existence by adding unnecessary overhead. Everybody has more than enough “real work” to do in order to move the company forward.

Working for a startup does have some drawbacks as well, but the positives far outweigh the negatives. The comparison is not even close as far as I’m concerned. As I mentioned at the beginning of the post, each company I have worked at has been substantially smaller than the previous one, and my job satisfaction has increased with each move. While some large companies can have some really interesting problems to solve (especially when it comes to problems of scale), I’m not sure I could ever go back to working for a large company.

Yo Ho, it’s a startup’s life for me.

First and foremost, this was an incredibly well organized and wonderfully run conference. Ray Hightower and the WindyCityDB team did an amazing job. The space at IIT was perfect. The food and drink was ample and tasty. And the talks were interesting and thought provoking. To top it off, all proceeds from the conference, approximately $1,000, was donated to the Greater Chicago Food Depository.

One theme that took me by surprise was the broad support for relational databases. Almost all of the speakers stated that the relational database still has a place in this new world, along side all of the NoSQL technologies that we had gathered to discuss. This is something I completely agree with, but being a NoSQL conference, I had imagined that there would be quite a bit of RDBMS bashing going on. Nothing could have been further from the truth. Everybody seemed to agree that the RDBMS is a very functional and reliable piece of software that is still very much needed. Everybody also seemed to agree that there are some things the RDBMS doesn’t do too well based on how it is designed, and that the different NoSQL technologies aim to fill this gap. Nobody was advocating their particular NoSQL database as a replacement to the RDBMS. Instead, all were pitched as alternatives, created to solve different problems.

Another theme that sort of caught me off guard was the wide acceptance of Polyglot Persistence as “the future” of application persistence methods. I say “sort of” because I was there to speak about Polyglot Persistence, which of course means that I think it can be a great tool if used properly. However, I thought it was a bit more controversial of a topic. As it turns out, there are many already taking a polyglot approach to solving their persistence needs. This was great to hear, as made me feel a little less insane for wanting to use multiple databases within a single application. Well, I could still be insane, but at least now I know I’m in good company :) Several speakers mentioned Polyglot Persistence in their talks, and I spoke with several individuals at the conference, and after the conference, who are actively using a polyglot approach to address their persistence requirements.

I also found John Nunemaker’s prediction that hosted database solutions are going to be the future of application data persistence to be very interesting. If you think about it, this is exactly what happened with web hosting. Most companies, especially small companies, now use a 3rd party to host their production environments. In fact, I’d bet that the thought of hosting your own environment doesn’t even cross the mind of the vast majority of small companies now-a-days. I can easily see how hosted database solutions, such as MongoHQ or Cloudant, could follow a similar path. You would no longer need to worry about hardware issues, scaling issues, backups, maintenance, etc. It would allow you to simply focus on your data.

Even though I didn’t get a chance to play with it, the WindyCityDB lab was a nice touch, especially for those who had never tinkered with some of the NoSQL databases on display. The lab consisted of several iMacs, each loaded with a particular NoSQL database. All of the databases were preloaded with the Stack Overflow data set. Sharing a data set was a great idea, as it gave you a good way to compare and contrast how the different databases handled the same set of data.

I really enjoyed myself at this conference, and am already looking forward to next year’s. At the rate the NoSQL movement is moving, I can only imagine what the landscape will look like a year from now.

As it stands right now, Apple is king with consumers. Though not the first ones to market with an internet connected mobile device (the Blackberry has been around for a long time, and still holds the majority of the smartphone market share), Apple appears to be the first to really understand what the average, non-corporate consumer wants. A true mini computer. An open device that can play music, run applications, take photos, and provide a pleasant web browsing experience. And they did it in true Apple fashion, making the device extremely easy to use. As a result, the iPhone has become extremely popular with consumers, and is widely considered “the device to have”.

With the majority of people still without a smartphone, much of the market remains up for grabs. Apple’s competitors are scrambling to catch up, trying to ensure that they they get a piece of the pie. But, one very important question lingers. What can Apple’s competitors offer that would give the average consumer a reason to buy their device instead of buying an iPhone? To me, the reasons are few, and becoming fewer.

(I’d love to hear your reasons in the comments. So please, chime in.)

A comparable feature set

This pretty much goes without saying. Any challenger to the iPhone crown must offer similar features to that of the iPhone. It is very unlikely that a competing device will lure anybody away from the iPhone if it is missing a feature that is now expected to be there. The device must be capable of running apps, taking photos, playing music, etc, for it even to be considered.

A better network

AT&T’s network leaves much to be desired. Having never been an AT&T customer, I can only relay the opinions of my friends and family who are AT&T customers. However, their opinions are one in the same. I’ve not heard a single word of praise when it comes to AT&T’s network. All of my friends and family with iPhones have expressed frustration that the device they love is frequently crippled by a network that is spotty and congested.

It’s no secret that Apple has an exclusive agreement with AT&T, and that agreement has an expiration date. Rumors have been circulating about a jump to another carrier, possibly Verizon, sometime next year. The more wireless carriers offering the iPhone, the less valid of a reason this will become for not purchasing one.

A comparable application ecosystem

Competing devices will need to have an application ecosystem that is at least comparable to the iPhone’s. This is no small task. There are over 100,000 applications in the App Store. Sure, several offer the same functionality, and many are of very poor quality. However, nobody can argue with Apple’s tag line of “There’s a app for that”. There really is an application, in most cases many, for everything you could possibly want to do with your iPhone.

Given their head start, beating Apple at this game will not be easy. Google’s Android OS currently stands the best chance of challenging Apple on this front, with over 10,000 applications already available. The Android OS is open, and capable of running on hardware from any manufacturer. In addition, applications written for Android are capable of running on any device that runs the OS (for the most part). Next year is going to be a big one for Android, with several new devices coming to market from many different manufacturers. Some analysts are even predicting that the number of Android devices in the hands of consumers will surpass the number of iPhones by 2012. This will no doubt attract more application developers to the platform.

However, Android has its own set of challenges awaiting. The fact that manufacturers are free to run Android on devices with very different hardware specifications (screen size, input controls, etc) poses a major challenge for application developers. Perhaps the risk of rendering thousands of existing Android applications useless by releasing a device with dramatically different hardware specs will be enough to convince manufacturers not to do it. Perhaps Google will provide a set of Android APIs that can help application developers deal with this issue. Perhaps a set of best practices will emerge as a guide for developers looking to tackle this issue. Perhaps we’ll see something similar to the PC application market in the mid-late 90′s (and the Blackberry application market today), where only certain devices will be capable of running certain applications. Only time will tell if these issues will prevent the development of the Android application ecosystem.

A killer feature

One wild card that is always in play is the killer feature. Apple’s competitors are only one, innovative, killer feature away from stealing the spotlight for themselves. By “killer feature”, I mean a feature so awesome that when you see it in action, you say to yourself, “I need one of those!”.

Version 2.0 of the Android OS took a stab at this with the introduction of Google Maps Navigation. A fantastic feature, Google Maps Navigation morphs your mobile device into a fully functional GPS unit, complete with a synthesized voice telling you where to go, real time traffic information, and several map overlays showing you the location of everything from ATMs to gas stations. But, is this a killer feature? Frankly, I’m not sure. But, its announcement was enough to cause a significant drop in the stock price of traditional GPS manufacturers, and it certainly has potential.

An incredibly easy to use device

Making devices that are intuitive and easy to use has always been one of Apple’s strengths. Look no further than the iPod for an example of this. Competing devices will need to be as easy to use as the iPhone is to appeal to the average consumer.

How do I get my music onto the device? How to I get the photos I take off? These operations should be simple and intuitive. Motorola’s new Android 2.0 device, the DROID, is seriously lacking in this area. Several steps are required to store data on or pull data off of the device:

• Attach the device to your computer
• Use the device’s menu system to instruct it to mount itself as an external drive
• Locate the files on your hard drive that you would like to store on the device
• Copy and paste the files from your hard drive onto the device
• Unmount the device

For the iPhone, the list of steps is much smaller.

• Attach the device to your computer, and let iTunes do the rest

Are the steps required to store data on the DROID too much to handle for an experienced computer user. No, of course not. But, there is still a large percentage of people out there who would struggle with completing those tasks. Believe me, I know. Many are family and friends of mine who I help complete “simple” tasks on their computers all of the time. These people make up a significant portion of the market. If you want them to buy your device, then you have to make it stupid simple to use.

Summary

Apple has set the bar high with the iPhone, very high. While I can think of several reasons why developers and techies would prefer a different device, I can’t think of many reasons why the average consumer would. And, there are a lot more average consumers than there are geeks.

But make no mistake, Apple’s competitors have the iPhone in their sights. The tide can shift very quickly in this market, especially since most people get a new phone every couple of years. Will the iPhone challengers be able make a dent in the iPhone’s market share? Or, will the iPhone be the de-facto standard for smart phones? Only time will tell.

In my opinion, the best feature provided by Parleys.com is the ability to watch the presentations, with all of the UI features, offline. This is HUGE for me, as I spend almost two hours every day sitting on a train. Parley’s provides an Adobe AIR application that makes all of this possible. Adobe AIR is a runtime environment that “lets developers use proven web technologies to build rich Internet applications that run outside the browser on multiple operating systems”. The Linux version of AIR, still in Beta, works wonderfully. The AIR application operates just like the site, and provides some additional functionality to download the presentations for later viewing.

I’ve been spending quite a bit of time lately looking for a site like this. With the current economic conditions, I’d imagine that most companies are cutting back on the number of conferences they send their developers to. According to the site, they plan on continuing to add content. If they do, I see this as an invaluable resource for keeping up to date on what is going on in the Java community.

What was really making me consider picking up a Macbook was iLife, and specifically, iMovie. I’ve heard great things about iMovie. Supposedly, it just works, and it’s really easy to get it to do what you want it to do. Linux has no all-in-one video editing package. Just a bunch of stand alone applications that you can piece together to get the whole picture, but that’s too much work for yours truly. And, the Windows ones I’ve tried I usually end up fighting with to get them to do what I want. So, I was really leaning towards the Mac.

But, this is a deal breaker. My miniDV camcorder will not transfer the higher resolution video over USB. If you want the good stuff, you have to use the firewire port on the camcorder. And, I would like the videos of my kids in high resolution thank you very much. It creates a much better end result to blackmail them with when they’re older and don’t like me anymore.

Sure the Macbook Pros have a firewire port. But you know what else they have? A price tag that’s $700 more than the Macbook.

Shortly after Apple’s announcement today, I was already able to find disgruntled folks like myself on the web via Google. I can only hope that Apple fixes this mistake and puts a firewire port back on the Macbook.. Until then, no Macbook for me.

You need a Sun Developer Network account (free) in order to access the presentations.

• Complext Event Processing at Orbitz
• Design Patterns Reconsidered
• Let’s Resync: What’s New for Concurrency on the Java Platform, Standard
• Top 10 Patterns for Scaling Out Java Technology-Based Applications
• Introducing eBay’s Scalable Innovation Platform
• Programming with Functional Objects in Scala
• Groovy, the Red Pill: Metaprogramming–How to Blow the Mind of Developers on the Java Platform
• JRuby on Rails Deployment: What They Didn’t Tell You
• Asynchronous Ajax for Revolutionary Web Applications

There were a number of sessions that I was not able to attend at the conference that I look forward to watching now. I’d highly recommend taking a troll through http://developers.sun.com/learning/javaoneonline/ to see what else might peak your interest.

But, after every interview I give, I always come out with a feeling that I could have done better. My questions could have been clearer, my topics should have been more diverse, I could have represented Orbitz a little better, etc. After a recent interview, I proposed something to a couple of my teammates…interview practice.

I proposed that those who interview candidates practice their interviewing skills on other Orbitz engineers. I think this could be valuable for many reasons.

It gives you great feedback on your questions. If an engineer you work with and respect isn’t giving you the answer you are looking for on a particular question, perhaps the question is unclear, or too difficult for most of the interviews we are giving. Sure, it could also mean that my colleague may not have detailed knowledge of that topic. But, if I think my colleague is a solid engineer, and that I’d like to hire more people like this person, then I want to make sure that I don’t unintentionally weed-out a candidate of this person’s caliber. This doesn’t mean that you can’t ask this question, but instead maybe weigh the question a little bit lighter, or ask it as a “harder” question while pushing the candidate a bit.

Practice interviews help you hone your “interview agenda”. After a few practice sessions, you should have a fairly good idea how long to spend on each topic you want to cover. And, you should develop more than enough questions to challenge the candidates who easily plow through your warm up questions.

Practice interviews give you a great forum to try out and tune new questions. If you think a coding exercise may be a little too complex for a 5 minute trip to the white board, then ask it and find out. If the mock candidate asks you questions to clarify the exercise that you cannot answer, then your question needs a bit more work. Whatever it may be, the practice interview is a much better place to try out new questions and exercises than a live interview.

If you really get into the role, and act as you would during a real interview, hopefully your trusted friend and colleague will have no trouble telling you if you are acting like a ass. In my opinion, one of the worst things you can do is talk down to, degrade, or make a candidate feel stupid if they are unable to answer a question. It’s unprofessional, mean, and will discourage somebody who may have turned out to be a great employee from joining the company.

These are just a few of the benefits that I see right away. I have a feeling that if we start holding these mock interviews, say once a week for about an hour, that we will quickly find other benefits as well. Should be an interesting experiment. If we kick it off, I’ll let you know how it works out.

JRuby and Groovy were all over the place. There was also a session on Scala. These other languages bring new ways of solving problems to the table. In addition to their expressiveness, JRuby and Groovy bring the power of meta programming to the Java world. Scala also brings a more expressive syntax, and the power and flexibility of two programming paradigms: object oriented and functional.

These were the only non-Java programming languages that had sessions associated with them. At the CommunityOne session I attended which was presented by Charlie Nutter, Charlie said that virtually every language out there is capable of running on the JVM, usually via a sub-project. I think this is a very profound statement. It shows that the language implementers or those strongly associated with the language realize the benefits of running on the JVM, and the ability to integrate with the billions of lines of existing Java code out there. Granted that each one of these projects varies in the level it can interact with existing Java code. However, Sun appears to be making a strong effort to work with the implementers of these languages to make integration with Java easier.

I think this is great. I’m sure that Sun realizes how powerful and mature the Java platform has become. And, as a Java programmer, I love the idea of being able to pick a language that best fits the problem I am trying to solve, while still maintaining interoperability with my existing Java code.

The Java language itself appears to be stalling. No widely adopted programming language lives forever. In order for a language to be successful, it must maintain some sort of backward compatibility. Maintaining backward compatibility, while necessary, slows the evolution of the language, and prevents the language from adopting vastly different programming models that may be a better fit for new problems that developers may be facing. Newer languages, or less widely adopted languages, do not face this dilemma and can change more rapidly. I’m also unconvinced of the effectiveness of the whole JCP process. I some situations I feel it is best to have a small group of intelligent individuals at the helm of a project, making all of the decisions. Usually, the more people involved, the longer it takes to get things done.

I think that making the JVM a more attractive environment for non-Java programming languages will only benefit the platform. I think it will also prevent some developers from jumping platforms to use a different programming language that better suits the problem they are trying to solve. I think this is a win for the Java community.

The last general session of JavaOne 2008 consisted of James Gosling inviting several people on stage to showcase what they have been using Java to create. There were several presentations, but I’m going to only talk about a few that interested me.

First up was Visual VM. Visual VM is a free JVM monitoring tool that can look under the covers of applications running Java 1.4.6 (I think) or greater. The stats gathered include memory usage, thread usage, CPU usage, and more. It also has a series of nice features, like getting a thread dump on your application by simply clicking a button. Best of all is that it does this with almost no overhead on the application. The tool looked very nice, and worth checking out in further detail.

Second was by far THE coolest thing that I have seen here this week; a pen. Yep, a pen…but a VERY smart pen. This pen can record your voice (or somebody else’s) as you write, and when you tap on an item that you wrote with the tip of the pen, it will play back just the portion of that audio that you record when you wrote that particular item. The pen also has several tools, like a translator that will take a word written in English and translate it (verbally) to a number of different languages, by simply tapping the word you wrote. It also stores images of everything you write. You can basically toss the paper you wrote on in the trash, because you can easily transfer the images to your computer through the USB port on the pen. The text in the notes is searchable via the pen software, and audio captured can also be accessed via that same software.

JMars is an open source application that contains loads of detailed images of Mars, collected via the several NASA missions to that planet. It is fully interactive, lets you see several types of maps of a terrain (and combine particular maps), and basically navigate the planet as you wish.

Complex Event Processing at Orbitz

Matt O’Keefe and Doug Barth did a great job presenting our event processing framework here at Orbitz. They took the audience step by step through our API and library that collect the data (ERMA), the commercial third-party tool that we use to aggregate and route that data (Streambase), and our graphing tool that visually represents that data (Graphite). The event processing framework has very high throughput, has little overhead on the running application, and requires very little code in the application.

At the end of the presentation, Doug announced that we would be open sourcing the two pieces of the framework that we own, ERMA and Graphite, and we were looking into bundling an open source data aggregation tool (since we can’t open source Streambase) to provide a complete event processing solution. The audience applauded this announcement, which took Doug a bit by surprise based on the look on his face :)

Good job guys!

Improving the Engineering Process Through Automation by Hudson

Hudson is a continuous integration (CI) tool that can be used to automate building, testing, and deploying your code. PCs are cheap and getting cheaper by the day. Developers on the other hand are not. Hudson is advertised as a cheap “team member” that can take on some of the easier to automate activities.

One of Hudson’s obvious goals is ease of use. And, this is a goal that they have reached in all areas. Even something as complex as distributed builds is a snap in Hudson (more on that later). It installs in a snap, and can run either by itself, or within another web container. It was designed to be extensible, allowing the community to continue development of the tool through plug-ins. These plugins provide integration with several popular version control systems and bug tracking systems.

Several best practices were suggested:

• Componentize builds to reduce the time needed to get feedback. Replace that one, monolithic build with several smaller builds. And, only build what changed.
• Run tests in parallel, or run groups of related tests in parallel.
• Hudson can build, test, promote, do some QA, deploy, integrate, and more. Take advantage of its power and flexibility to automate whatever can be automated. Set it up to do as much as possible.

Hudson also makes it very easy to do distributed builds. The master machine serves HTTP build requests to the slave machines, and stores all of the information collected from the builds. The slave machines are the ones that do the build. The nice thing about distributed builds in Hudson is that slave boxes can come and go as they please. At the beginning of a build, the master checks to see how many slaves are available, and delegates the build to a slave machine. And, slave configuration is easy. A client needs to run on the slave, and some basic configuration is needed on the server for each slave. After that, Hudson takes care of the rest. This feature is great for building and testing on multiple environments and operating systems.

Oh, did I mention that Hudson was free? We use it on the Transaction Services team at Orbitz, even though the company has standardized on Quickbuild. Putting up with maintaining two ci tools tells you 1) how much we like Hudson and 2) how easy it must be to get running, and keep running.

Automated Heap Dump Analysis for Developers, Testers, and Support Employees

This session focused on the use of an open source tool, Memory Analyzer, to help track down memory leaks in a Java application. Finding memory leaks in Java has always been difficult. Who in their right mind wants to wade through a heap dump? Memory Analyzer is a sweet little tool that analyzes the heap dump for you, and provides you with easy to decipher diagnostic information that it pulled from the dump.

Since Memory Analyzer doesn’t understand your application, it can’t really tell you where a leak is. However, it can tell you which classes are holding the majority of the heap, how many instances for a given class have been instantiated and how much memory those instances occupy, and more. This is usually enough to point you in the right direction. Memory Analyzer can also give you the stack trace to a specific memory allocation, helping you further track down the leak.

The reports generated by Memory Analyzer are very comprehensive, and provide tons of useful information about your application’s memory usage. These reports can be used by all areas. They can help support employees track down production issues. They can help developers fix and test memory related issues. And, they can help testers very that the memory usage for a given application doesn’t dramatically vary from release to release. The reports have several useful features to track down leaks, like the ability to group the results by classloader, in an effort to further isolate the problem.

Memory Analyzer also does a static analysis of you code to look for common memory related anti-patterns. This can help find a bug before it is introduced into the codebase. This tool has a lot of promise. I hope I never have to use it, but it’s comforting to know that it’s out there to use, just in case.

Top 10 Patterns for Scaling Out Java Technology-Based Applications

Scalability seems to be one of the industry’s biggest buzz words these days. And, I don’t think that many people really know what it means. It was pointed out to me during a Q and A session after a talk how many people were asking scalability questions regarding topics that had nothing to do with scalability. “Does JAXB scale?” was one of these. Scalability != Performance.

Scalability is the ability to handle an ever increasing amount of requests gracefully. This could include adding servers to your server farm, or upgrading some key components. If you can add capacity to your system or tweak your system to handle increasing numbers of requests, you can scale. If there is a bottle neck in your system that maxes out your capacity, and you can’t easily fix that bottleneck, you can’t scale. Linear scalability, the ability to handle increased traffic with increased hardware, keeping the latency at its normal rate, is the goal. Any sort of up-trend in latency indicates that there will come a point in time where your latency will hit unacceptable rates, and cause a scalability bottleneck.

Scalability is not limited by a technology, a programming language, or an operating system. People have built scalable systems on every possible combination of these. The design and architecture of your system is what will determine if your system will scale or not.

Availability and reliability must be baked into the design of your system. It cannot be an afterthought. Refactoring your code to deal with scalability issues after launch is very difficult, as major design changes are often necessary.

The speaker then went on to discuss some things to consider when thinking about availability.

Latency is not always predictable. Network IO and other tasks performed outside of the application can be unpredictable. Reduce or eliminate these where possible. Remote messaging brings its own set of challenges. If order of the messages is important, how do you control it? How do you make sure the message will get there? How can you make sure you get a response quickly? How can you make sure that subsequent executions (a retry) won’t cause repercussions? Managing these complexities can be difficult, and if done improperly, can limit scalability.

Durability, the ability to survive a failure, is also a major challenge. Writing data to disk or to a DB takes time, and the coordination of writing and reading data necessary to perform a failover can be difficult to manage.

The speaker went on to identify some key areas to focus on when trying to build a system that can scale.

• Routing – Reliable routing is essential for scalability. You must be able to reliably send requests to components that can process those requests in timely manner.
• Partitioning – Spreading out the responsibilities of your system into different components enables scalability. If you notice that one area of the system is becoming a bottleneck, you can always add more capacity to that area, without touching the other areas.
• Replication – The replication of data is necessary for surviving a failure in the middle of a transaction. The routing of a system must also be able to recognize failure, and route the request to another component who can handle the request.

The presentation also covered how to handle load on a system. Some common ways to deal with load:

• Load balancing – Send the requests to a component that has the capacity to process them.
• Partitioning – Partition your system so that you can add capacity to stressed areas.
• Queue – Queue requests for processing when the system has available capacity.
• Parallelization – Execute requests, or parts or requests in parallel where possible.

There are also strategies for when your system is overloaded:

• Turn away requests
• Queue requests to be processed when the system has capacity
• Add capacity to the system (usually in the form of hardware)
• Relax any kind of read/write consistency that you are enforcing
• Increase the size of any batch jobs that you run

The speaker also spent some time talking about failure recovery. You should plan for failures at the component level (a piece of the system) and the systematic level (the entire system). Build some redundancy into your system so you have the ability to failover to a redundant component if one component fails. If you can’t failover to another component, then you need to build recoverability into your component, so that it can handle problems by itself. Critical data should be replicated so one component can pick up where another left off in the event of a failure. This however adds overhead to the system. So, only the critical data should be replicated.

At the end of the talk, the speaker left us with the “secret” to scalability: simplification. The simpler your system is, the easier it will be to scale. If you can’t get it to work on the whiteboard, then there is no way it will work in production.

Spring Framework 2.5: New and Notable

I’ve been working with Spring for quite a while now, and we are currently using 2.0. This presentation gave a 10,000 foot overview of what is coming in version 2.5 of the Spring framework.

• As always, 2.5 will be backwards compatible with previous 2.x releases.
• Greater annotation support.
• Enhanced support of the testing framework provided by Spring. Using the test framework, you can easily test your configuration, your database connections, and even the database transactions you plan on executing.
• Support for Java 6, Java EE 5, and OSGi.
• This release of Spring will be the last release to support Java 1.4.
• Support for OSGi allows for greater modularization.

Why did I sign up for ANOTHER Groovy and Grails talk? That’s a great question. I have no idea. Nothing new was presented here that wasn’t already presented elsewhere.

JRuby on Rails Deployment: What They Didn’t Tell You

This was a pretty comprehensive presentation on what it takes to deploy a Rails application via JRuby. The talk started out with a brief history of how Rails apps have been deployed on the traditional Ruby implementation. CGI/FastCGI was always slow and kind of buggy. The Mongrel Ruby web server is widely used, but large clusters need to be deployed for high traffic sites, due to Rails’ non-multithreading nature. These large clusters have proven to be difficult to manager. There are tools that help with Mongrel cluster management, but it doesn’t change the fact that your cluster size is huge.

Deploying Rails apps on JRuby allows you to take advantage of JVM threading, shrinking the number of real processes you need to have running out there to handle requests. It is possible to configure JRuby on Rails to use a fixed number of JVM runtimes to process requests, so that you can scale it out to fit the needs of your web application.

There are tools available to help with the deployment of a Rails app on a Java application server. Warbler can be used to package the Rails application into a war file that can be easily deployed to the application server. Warbler is pretty easy to setup and use, and only requires you to list the Ruby gems that your application depends on (which can be done in a configuration file).

However, Rails apps on JRuby still require a bit of fiddling to get up and running. You can use JNDI for database connections, but Rails does not close the connections at the end of each request (by design…this is just how rails works). So, workarounds are needed to get the application to behave a little more like a Java app, which usually opens a connection at the beginning of the transaction, and closes the connection at the end. Several configuration modifications are needed to get the Rails app to run optimally in a production Java environment. And, there are several areas where the Rails and the application server don’t mesh too well together.

The author has just released a library, called JRuby Rack, which attempts to bridge the gap between your Rails application and the application server. Among other things, it allows Rails to use JSPs to render the view, makes the Java servlet context available to the Rails app, and passes servlet request attributes from Java to the Rails application.

Programming with Functional Objects in Scala

Martin Odersky, the creator of the Scala language, presented a good overview Scala. Scala is another programming language that runs on the JVM, and boasts full Java interoperability. It was designed to be scalable across all areas of development, which is where it got its name. It can do everything from power the smallest scripts to the largest applications.

Martin considers Scala to be the Java of the future. It can do everything that Java can, and more. Unlike another fully Java interoperable language on the JVM that is getting a lot of attention, Groovy, Scala is fast. Martin says it benchmarks just as fast as Java. It also is much more expressive than Java, letting you write less code. Martin stated that the average Scala project contains 2x (or more) less code than an equivalent Java project. This is due to the syntax of the language, and to the better abstraction that is provided by Scala.

Scala was designed to be extensible. Users can easily add to the language to fit their needs. This is nice, as it keeps the language itself trim. There is no need for the language itself to solve everybody’s issues. You can take it upon yourself to solve your own problems :)

I’m not going to list out all of the features of the language that were identified during the presentation. Check out http://www.scala-lang.org if you are interested. I will however mention some of the features that peeked my interest.

First, Scala supports mixins. You can think of a mixin as a Java interface with a default implementation. You can “mixin” functionality to your class, without clouding the object hierarchy by allowing multiple inheritance. After working with mixins in Ruby, I’ve really started to miss them in Java. It’s nice to see them here.

Second is the concept of actors. Actors add Erlang like concurrency to Scala. Erlang is well known for the way it handles concurrency, using many lightweight processes that interact with each other via simple asynchronous messages. This is great to see in a language with full Java interoperability.

Another thing I like about Scala is the fact that it is an object oriented language AND a functional language. You can code using the paradigm that best suits the problem.

Scala is quickly growing in popularity. Though not as easy to adopt as Groovy, due to the radically different syntax, it is without a doubt a very strong language.

Defective Java Code: Turning WTF Code into a Learning Experience

I was a little disappointed with this talk. I was expecting a lecture on ways to best learn from existing bugs. There was a little bit of that, but the majority of the session simply showcased some not so obvious bugs that the presenter has run into in the past. There was however some good advice that I took away from this session.

First, never assume that a bug is so stupid, or so unique that it doesn’t exist elsewhere in your code or in somebody else’s code. The speaker showed several bugs fitting this description that he was able to find in several different code bases using findbugs. And, these weren’t obscure projects he found these bugs in either. He found several of these bugs in the Java codebase, the Eclipse codebase, and the codebases of several other large, well respected projects.

Make use of every available tool you have to flush out bugs as early in the development cycle as possible. This includes things like the @Override annotation. The speaker demonstrated a bug where a subclass was supposedly overriding the method of a parent class, but used a class in the method signature with the same name as the one in the original method…but from a different package. So, simply looking at the method, it looked like a perfectly valid override. You had to compare the import statements of the parent class and the child class in order to figure out what was happening. Using the @Override annotation would have caught this bug at compile time.

Design Patterns Reconsidered

Design patterns have been getting a lot of negative press lately. This is due to a number of factors:

• They result in people copying/pasting code to implement the patterns without actually knowing what is really going on.
• They are accused of being workarounds for shortcomings in the language.
• They are overused.

Each of these criticisms has its valid points. However, design patterns still play an important role in software development today.

• They form a vocabulary that developers can use to talk about problems, and solutions.
• They help expose real design issues.
• They help compare alternative design choices.

The speaker took us through three design patterns documented in the original Gang of Four Design Patterns book, and how they might be applied differently today.

First up was Singleton. The speaker acknowledged that there is sometimes a valid need to ensure that there is only one instance of a specific object. However, the Singleton design pattern causes the following problems:

• It makes testing difficult by carrying state from one test run to the next. This could affect how the test behaves, and can make test results unpredictable.
• It creates “hidden” dependencies. It is hard to tell, without digging through the code, that an object may have a hidden dependency on a Singleton. This also makes the Singleton hard to mock for testing.
• There is not really just one instance per the JVM. There could be one per classloader.
• Could create memory leaks, since Singletons never fall out of scope.
• Subclassing is difficult, and ugly.

What do we do about this? Simple; don’t use Singletons. Define an interface, and use a dependency injecting framework to handle the creation of the object, and to inject it where it needs to go. This way you can have your one instance of the class, and avoid all of the problems mentioned above.

Next up was the Template Method design pattern, which has the following problems:

• Algorithms in the base class(es) can become very complex, and hard to follow.
• This design pattern usually leads to an explosion of base classes.
• Inheritance cuts of many of your design options for the sub classes.
• No way to combine functionality in sibling sub classes. You may have a FastCar, and a CheapCar, which are both subclasses of Car…but there’s no easy way to have a FastCheapCar without an additional subclass.
• It is hard, without exhaustingly comprehensive documentation, to document the intent of your framework to your user.

How do we address these issues? Use composition over inheritance. Move the logic that would exist in the sub classes into a series of command objects, which you can then inject into the framework. Inheritance is a very strong form of coupling, and should be avoid unless it makes absolute sense.

Lastly, the speaker took a look at the Visitor design pattern, which has the following issues:

• You can easily expand the list of visiting methods, but you can’t easily expand the data structure you visit.
• It is difficult to return a value while visiting.
• It is difficult to throw an exception while visiting.

The speaker offered the following suggestions for the Visitor pattern:

• Put the data structure navigation code in its own visitor, which would take the logic visitor along for the ride, applying it to each node it visits.
• Hold onto return values and exceptions in the navigation visitor.
• Closuers, when they are available in Java, may be able to improve upon this pattern further.

The speaker finished with a list of general design principles:

• Code to interfaces, not implementations, and use dependency injection to reduce coupling.
• Favor composition over inheritance.
• Don’t rely on object identity.
• Separate parts of the design that change at different rates.
• When in Java, take advantage of the strong, static typing system. It is one thing Java does very well. So, use it to your advantage.

Developing Service-Oriented Architecture Applications with OSGi

This was a Q/A session with some OSGi experts. The experts discussed how they have used OSGi on past projects, what they liked about, and what they felt it could do better.

What is OSGI?

• OSGi is a framework that manages the services in a service oriented architecture.
• It has the ability to add services at runtime, and makes it very easy to do so.
• It provides total control over what services you expose, and what services you consume.
• Service “bundles” have a life cycle that is independent of the JVM, meaning you can start and stop services without starting and stopping the JVM.
• OSGi bundles make the dependencies of a service very clear, making sure there are no “surprises”.
• It allows you to deploy separate components of a SOA independent of one another. So, if a service has a bug, you can simply upgrade that service without touching the rest.
• There is no need to restart the application server to deploy a new version of a service.
• Service bundles are platform independent. Your service can run in a web container, a cell phone, a zerox machine…pretty much anything that supports OSGi.

I’ve been hearing a lot of buzz about OSGi the past few weeks leading up to JavaOne, which is why I attend this talk. I have to say, I like what I heard. I think that Orbitz may be able to benefit from such a framework, as the majority of our architecture is service oriented. This could possibly help in the deployment of some of our services. I will definitely be reading some more on this topic, and thinking about how we might be able to use this powerful new technology.

Testing in Groovy

The last session I attended for the day was one on testing using Groovy. Groovy provides many features that make it an ideal language to use to write tests for your Java code:

• Groovy is much more expressive than Java, and has the ability to dramatically reduce the amount of test code you have to write.
• Groovy is fully interoperable with Java, giving you access to all of your Java code that you want to test.
• You can re-use existing Java test frameworks.
• Groovy’s metaprogramming capabilities make mocking objects and stubbing methods a breeze.
• Groovy has several features that can be used for testing, including a GroovyTestCase class, and built in mocking and stubbing capabilities.

I was pretty convinced going into JavaOne that I wanted to start writing tests in Groovy. Everything I have seen here at JavaOne regarding Groovy has re-enforced this decision.

Here is my summary this session:

Buzzword, buzzword, buzzword. Look at this fancy GUI tool you can use to create applications without writing code. Buzzword, buzzword, buzzword. Look how well this integrates with Excel. Buzzword, buzzword, buzzword. Give us lots of money to make your life as a developer as mundane as possible. Bleh.

Developers like writing code. We WANT to know what is going on behind the scenes, and we WANT the ability to completely control that behavior. I think the presenters forgot where it was that they were actually presenting.

Groovy, the Red Pill: Metaprogramming–How to Blow the Mind of Developers on the Java Platform

Sorry Charlie (Nutter), but you just lost the “Best of JavaOne” title to Scott Davis (although, I still REALLY liked yours). This presentation was fantastic. Not only is Scott a very engaging, enthusiastic, and funny presenter, but the material was great. In the matter of an hour, he was able to shoot through many of Groovy’s strongest features.

For those new to Groovy, it is a dynamic programming language that runs on the JVM which has FULL interoperability with Java. Groovy IS Java in the sense that the Groovy is converted Java bytecode before it is run. Groovy can use Java libraries, and Java can use compiled Groovy libraries. Groovy is also able to run valid, syntactically correct Java code. This enables you to start using Groovy with your existing Java coding style. Then, gradually, you can start to use some of the features that make Groovy such a remarkable language. This ability to “ease into” the language is one of Groovy’s greatest strengths.

What are some of these “cool features” you ask? Method pointers, closures, dynamic addition of getters and setters, operator overloading, the ability to dynamically add methods at runtime (even to Java standard library classes), the ability to handle calls to methods that don’t exist, and much, much more.

These capabilities allow you to create domain specific languages that let you code much closer to the problem domain, increasing the readability and maintainability of your code. Scott took us through an IPod example, where an IPod object lets you load songs onto it. Say we have an ArrayList of songs, named songs. “songs.add(new Song())” doesn’t really express what we want to do. We want to “load” a song onto the Ipod, not add a song to the list of songs on the IPod object. So, we can create a new method pointer, named load, and point it to the add method:

load = songs.&add

This lets us code much closer to the domain, loading songs onto the IPod by calling songs.load(new Song()). Very cool.

But, the true power of Groovy, as demonstrated in Grails, is its metaprogramming capabilities. Want to add a new method, shout, to java.lang.String? No problem:

String.metaClass.shout = { ->
delegate.toUpperClass()
}

Methods can be re-defined in a similar way. Re-defining methods gives you a way to easily stub out methods for testing.

Groovy also provides hooks into methods that either allow you to do something before a method in a class is invoked, via invokeMethod(), or when a call is made to a non-existant method on a class, via methodMissing(). These capabilities allow you to intercept method invocations, add dynamic behavior based on a methods name and/or parameters, and more.

Groovy also adds TONS of syntactical sugar on top of Java. Easy iteration of collections, overloaded operators that just make sense, additional operations to the Java Standard Library classes, etc. All of this, and more, is what makes Groovy such an attractive programming language. I’m looking forward to learning more.

Introduction to Web Beans

What is a web bean you ask? That’s a good question. Even after attending this presentation, I’m not quire sure that I know the answer. Web beans appear to be a way to make services available to a client in a way that is non-invasive, and in a way that prevents tight coupling. Service implementations can be injected into a client for use in any layer of the stack, without the need to go through layers upon layers of middle-ware. I’m sorry to report that this is all I got out of this presentation. I will likely review the slides from the presentation and read up on the subject on the web after JavaOne to see if I can learn more about this, as I am disappointed that I sat through this hour long presentation and still do not know that a web bean is.

It’s All About the SOA: RESTful Service-Oriented Architecture at Overstock.com

The folks at Overstock.com gave a good presentation about their use of REST in their service oriented architecture. They moved to an SOA from a monolithic, self contained application (written in a single C file!) to eliminate business logic duplication and the drifting of that logic into other areas. SOA also helped with the isolation of sensitive data, reduced coupling (which enabled easier cross team development via interfaces), and provided a service level domain focus.

Before choosing REST, Overstock.com evaluated some competing SOA technologies, including SOAP/WS-*, JINI, and a few others. REST won out for a couple of reasons. First, they liked that it provided a uniform interface (which is basically HTTP). Also, they wanted to take advantage of their experience as web application developers who already had detailed knowledge of the HTTP protocol. Since they were already familiar with the technology (HTTP), it enabled them to get them up and running quickly, which was a requirement due to an approaching holiday shopping season.

They developed a web framework, built upon the open source Restlet REST framework for Java, to take care of some routine operations that they did not want to burden their developers with. For example, since they didn’t want to muck with the XML directly, their web framework took care of marshalling/unmarshalling the XML to/from their object model via JAXB. It also took care of some logging, and other common concerns. In addition to developing their own framework on top of Restlet, they also contributed code to the Restlet project, and to the JAXB project.

One of their goals was to keep things as simple as possible. Their solution was developed in J2SE, and is deployed via Tomcat web containers, with standard HTTP load balancing. They are able to perform rolling deployments because they always maintain backward compatibility with at least the last version of the application.

Dealing with Asynchronicity in Java Technology-Based Web Services

This presentation covered a technique that was developed to deal with asynchronicity in web services. I must admit, one of the reasons I signed up for this session was to find out what “asynchronicity” meant :) Using asynchronous web services adds complexity, but also provides some benefits. For one, you don’t have to sit there and wait for long running services to complete. You can call the service, and immediately return so that you can do something else. Second, using asynchronous web services gives you the ability to cancel requests at any time.

There are two ways to implement asynchronous web services. The first involves polling. The client will call a web service, which will queue the request for processing, and immediately return to the client. With polling, the client will periodically check back with the server, via another web service call, to see if their request has finished processing and to retrieve the result. This has many issues. It is chatty, especially when you’re dealing with multiple clients issuing multiple requests. It also adds unnecessary latency, as there may be a period of time that passes after the request finishes processing, but before you make the call to retrieve the results.

The second technique, the one covered in detail during the presentation, involves a callback from the server. This technique involves the server exposing the web service, as normal, AND the client exposing a callback web service. When the client calls the server’s web service, the request is thrown into a queue for processing, and the server immediately returns to the client. In the request is a unique ID for the request, and information about the callback service. When the request is finished being processed by the worker thread, the server calls the callback web service on the client with the unique ID passed in the original request and the response, so the client knows which request was processed.

The speakers then proceeded to show how the callback technique would be implemented, step by step. However, I think the main value in this presentation was the description of the technique. The implementation was pretty straight forward and easy to understand, especially if you have experience dealing with web services.

One item the speakers did caution us about was the necessity to persist the queue of requests (either to disk or database). Keeping the queue in memory is not sufficient, because all of the requests will be lost if the application crashes or is suddenly terminated.

JRuby at ThoughtWorks

This was a very interesting talk about JRuby, and how ThoughtWorks uses JRuby for both internal projects, and client projects. Thoughtworks is a consulting company that is well known for its agile methods.

Those who have worked with Ruby know that it is great for agile development because it is very readable and expressive, it leads the industry when it comes to testing innovation, and it allows for easy, quick prototyping.

But, the standard Ruby implementation has many issues. Ruby does not use native threads, which presents scheduling issues for multithread applications. It has weak Unicode support. It has poor performance; even the new version, 1.9, leaves much to be desired. The garbage collector doesn’t work that well, stopping all processing while it collects garbage. The C language extensions to the library have caused many issues, especially in the area of cross platform compatibility. It is also a challenge, politically, to get a Ruby application deployed to production environments in some companies.

JRuby addresses all of these issues. Java threading is used, which addresses all of Ruby’s threading issues. Java has full Unicode support. JRuby performs better than even the new version of Ruby (in long haul testing). The Java garbage collector is used, providing quick, reliable, and dependable garbage collection. All language extensions are written in Java instead of C, solving the portability issue. And, companies who already have a Java production environment in place usually don’t have an issue with deploying JRuby applications, since they can be packaged and deployed just like Java applications. In fact, some may not even know that they are deploying a Ruby application.

Another major strength of JRuby is the ability to fall back on Java in the case were Ruby is not a good fit, like a performance intensive part of the application, or maybe connecting to a database that does not have a Ruby adaptor.

The speaker then described some of the projects that have been going on at Thoughtworks, and demonstrated one of the web applications they user internally to manage projects.

The opening session to JavaOne 2008 was pretty much a showcase of what’s new and upcoming in the world of Java development. There was a preview of several Java enabled devices, including a couple of mobile phones, a mobile book reader from Amazon (with web and Amazon.com integration), and a Blue Ray disc demo presented by Neil Young, who is just about the last person I would have expected to see at JavaOne. All of the products demoed were developed with Java as a core component.

The first part of the keynote talked about how the RFIDs embedded in our JavaOne IDs are used to validate us for the sessions that we have registered for. In addition to that basic usage, the RFIDs are also used to track people as they enter/leave the sessions. This information can be displayed on a graph to see how well the speakers are keeping their audience throughout their presentation.

The speakers also demonstrated how Java can interact with industrial tools that are used to track energy efficiency in buildings. The data collected via these tools can be fed to any number of applications to process or display the data. During the demo, they demonstrated a JavaFX application that visually represented the data, and a graphing tool that created a series of graphs based on the data. Sun has hooked up this monitoring to their data centers, and release information to the public regarding how much energy is being used by their data centers. The speakers stressed the goal to use technology to improve society.

There was a pretty cool JavaFX demo that brought up a portlet like component on a web page. They demonstrated how a user can then drag that component from the web page, out of the browser, and drop it on the desktop. The user can then choose to “save” that component, essentially installing a local copy of that app on their PC. They also demonstrated how that same application was easily accessible via a mobile device. This was the SAME copy of the app. It was not written differently for these different environments.

Overview of the JavaFX Script Programming Language

This presentation, given by the creator of JavaFX, demonstrated the features of the JavaFX language. JavaFX is an object oriented scripting language, built on top of Java, which allows for easier GUI development. It supports closures, first class functions, multiple inheritance, and other syntactical sugar currently not available in the Java programming language. Although JavaFX was crated with GUI development in mind, it is a complete language on its own, and can be used for other tasks.

One of the strengths of JavaFX appears to be in the way that properties can be specified for GUI components. An example (this is going off memory, may not be syntactically correct):


Rectangle {
x : 10
Y : 10
width : 50
height : 50
color : Color.RED
}


The above example draws a 50px x 50px red square, at position 10,10. Instead of hard coding the values for the properties, you can bind them to variables. Binding the properties to variables allows JavaFX to dynamically re-draw your component when the values of those properties change. For example, if you set the width property to “w”, and the height property to “h”, then in an event handler that responded to say, a mouse drag event, you can dynamically resize the component based on how the user dragged the mouse.

I don’t do any GUI development, but I’ve heard a lot about JavaFX lately, and wanted to see what it had to offer. The demos during this presentation were simple, but we have also seen some pretty cool animated, 3D demos using JavaFX here at JavaOne. Some of those demos were very impressive.

Enterprise-Level Testing Strategies

This presentation, given by some fellow Chicagoans at Navteq, described how the presenters went about solving some performance and maintainability problems they were having with their enormous unit test suite.

Some general pointers they offered:

• Know, in detail, what it is that you need to test. Code coverage doesn’t mean squat if your tests don’t exercise your real use cases.
• Build your test framework in such a way that it can grow with your team.
• Poorly written unit tests are viral, and spread quickly. Don’t tolerate poor unit tests. Treat test code like production code, and enforce best practices.
• Grow your test framework with your software. As you refactor your software, refactor your tests and test framework as well.

More specifically, they mentioned a few specific items that you can do to increase test performance and reduce test maintenance (or at least make it easier to do):

• Standardize your test data, and standardize access to that test data. Share standard test data among all of your tests. If you do this, and standardize access to that data, you can cache that data to avoid re-creating it, or re-fetching it from the data store.
• Use long-lived objects to avoid the repeated instantiation of expensive objects.
• Analyze your dependencies and componentize your builds. Break your builds up into smaller parts, so that you only need to run the tests on the component that changes.
• Use a continuous integration tool. Using a CI tool helps you catch problems with your code much earlier in the development cycle, making it easier and less costly to fix.
• Have a configurable and deployable database schema.
• Apply “best practices” to test code, just as you would to production code. Bad tests are always copied more, and propagate faster, than good tests.

With regard to caching test data, the speakers said it is necessary to hand each test a COPY of that test data. This way, the test can modify that data as it needs to without affecting the tests that run after it. Copying the test data for each test is much simpler than implementing a rollback mechanism to be executed after each test. Since this copy would need to be a deep copy, the speakers suggested using your objects’ copy constructors, if they have them, or to serialize the test data when it is created, and de-serialize it into a new copy at the beginning of each test.

Implementing these changes cut Navteq’s test time in half when running the entire suite (from 3 hours to 1.5 hours). However, they pointed out that they rarely need to run the entire suite anymore, since they have componentized their build. Now, they only run the tests for the component that was modified, which takes 20 minutes for their largest component.

Introducing eBay’s Scalable Innovation Platform

This was by far the most interesting presentation of the day. eBay took us on a whirlwind tour of their application platform.

First up was the UI layer. With eBay, everything is in Java. And, everything means everything. They have a home grown DOM implementation to build the web pages. The have home grown, re-usable, self contained UI components which have their own CSS, JavaScript, and content. In addition, the UI components contain dependencies to other UI components, and have a sample data model to aid in testing. One very nice thing about their UI components is their level of documentation. eBay keeps a Component Catalog that fully documents each of the components, how they can be configured, how they can be used, and more. This fosters re-use among the developers.

The platform is aware of all uses of web resources (images, links, external CSS, external JavaScript, etc). eBay has tools that will verify the resources referenced in code at build time. If a resource is missing, or not specified correctly in the code (typo), the build will break. This prevents broken links and images from making it onto the site. They have also developed a set of Eclipse plug-ins to aid in resource management.

All CSS is also defined exclusively in Java. Eclipse plug-ins have been developed to help with this area as well. Code completion is available for all CSS properties for the specific element that they are styling. Build tools have been created to verify that only valid properties are used, and valid values are used for those properties where possible. This helps prevent many common CSS errors.

eBay has crated a set of content creation tools that allow in-browser, in-page content editing. While testing, if you see a content error on the site, you can edit that content right in the browser, and immediately start the patch process to get that content change on the site. To ease internationalization, tools highlight content that does not come from their content management system. This makes it very easy to spot content that will not be translated when viewed in international locales.

As I mentioned before, EBay’s UI is composed of a series of components. A particular key-stroke in your browser will enable the highlighting of all of the individual components on that page. Another custom key-stroke displays the code for that particular component in Eclipse. This enables you to quickly find and easily fix bugs in components.

EBay’s platform was built with scalability in mind. Monitoring and operations was not an afterthought. What to monitor, and how to monitor it was carefully considered when designing the platform. Because of this, they have great visibility into all areas of the site, which enables them to easily find bottlenecks so they can be addressed.

The data access layer was built with performance in mind. It consists of 500+ databases, which store different sets of data. The Data Access Layer (DAL) does caching, partial object persistence (hydration), and routing to a specific database based on the data that is being accessed or stored. eBay has a template-ized SQL framework that will dynamically insert column names, table names, and inserts an SQL JOIN into the query if necessary. Like the UI layer, the data access layer also provides a set of Eclipse plugins to aid in the development of DAL components.

As far as a connection pool, eBay was not happy with any of the open source options, so they rolled their own. Among other features, their connection pool provides easy configuration, complete transparency, supports throttling, and has a storm suppression system that throttles the creation of new DB connections in times of exceptionally high traffic…to keep the database from falling over.

The platform supports a fail fast architecture that determines when to “cut the cord” to another component to prevent that problem from propagating through the platform and affecting other areas.

According to EBay, the Java everywhere approach works great. It enables them to quickly develop and deploy new features to the site (they do a build every week), which was a major goal of the architecture. It does however require that everybody, even sight web designers, know a little bit of Java. But, this does not appear to have caused a problem.

Real-Time Specification for Java (JSR 1): The Revolution Continues

This session attracted my interest, as I was not aware of any effort to provide real-time support in Java. I had always considered real time systems to be based in a language that was sitting a little closer to the operating system (preferably a real time operating system), and not running in any type of virtual machine. But, this effort has been going on for quite some time. After all, it is in fact JSR numero uno.

Currently in Java, there is no way to specify the real priority of a thread. You can “suggest” to the JVM the priority to run the thread as, but there is no guarantee that the thread will run at that priority. Also, there is no real way to specify an exact “wake up time” for a thread that needs to wake up every x seconds to perform some task.

JSR-1 aims to add some of these capabilities. It isolates the application from pauses caused by garbage collection…by essential disabling garbage collection. It also works around other features in the JVM that may delay processing, like JIT compiling.

However, the fact that it disables garbage collection brings up other issues. The app is now responsible for maintaining a tighter grip on its memory usage. There are two ways to do this. Uses “Immortal Memory”, which is just fancy talk for letting your objects live forever, or used a scoped based memory approach provided by the language as part of the JSR.

JSR-282 expands upon JSR-1 by allowing you to specify a processor for your real time processing to use, to specify data to be included along with the time-based events that are fired, and by providing an alternative to the memory management model suggested by JSR-1. JSR-282 is moving slowly (though apparently not as slowly as JSR-1), and the development of this JSR is not funded. So, don’t count on seeing it implemented anytime soon.

JSR-50 contains the specification for distributed real time support. This includes the ability to provide end-to-end time constraints across processors.

JSR-302 is not specifically a real time JSR, but it is related. JSR-302 is the specification for safety critical java applications. Safety critical applications, like the applications that run the flight instruments of a airplane, are required to go through a rigorous certification process. Every line of code that makes up that application, including the JVM that runs it, must be thoroughly inspected. Because of this, these applications are often stripped to the bare necessities. Developing these apps require a different mindset than what most web developers are used to. Instead of a “failures will happen…learn to handle them” mindset, one needs a adopt a “all failures must be prevented at all cost…but handle them gracefully when they happen” mindset.

This talk didn’t quite live up to what I was expecting, but it did provide some insight into an area of Java development that I have little knowledge of.

Let’s Resync: What’s New for Concurrency on the Java Platform, Standard!

This presentation covered the new concurrently features of the upcoming Java 7 release. CPU clock speeds have been holding steady for the past several years. Chip manufactures are providing faster chips by adding cores to the chip instead of by increasing clock speed. This provides an interesting challenge to developers, since taking advantage of those multiple cores is not an option if your application is single threaded. Course grained threading (per user request threading models) works well with a small number of cores. However, once that number of cores starts to dramatically increase, as it is predicted to do, this threading model will be unable to keep all of those cores busy.

Java 7 will be providing a fork/join threading framework to offer a finer grained threading solution, which aims to take advantage of these newer multi-core systems. Fork/join involves taking a data set, chopping it up into chunks, having a series of threads process those chunks of data, and combining the results of that processing at the end. Fork/join frameworks are a good way of solving common divide and conquer problems, like sorting or searching. However, some playing around with the number of threads in a fork/join pool and the size of the data set that a thread operates on is required to get the best performance of this model.

A common problem with the fork/join model is that sometimes threads work through their data sets quicker than other threads. When this happens, the threads that finish early just sit there waiting for the others to finish. This is not a good use of their time. Enter “work stealing”. Work stealing is an idea that allows threads that have completed their work early to “steal” items from the tail of another thread’s work queue. Since the worker threads only pay attention to the heads of their queue, this provides little chance of concurrent access. One common strategy for implementing work stealing is to put the more costly work at the end of your queue. That way, if another thread steals an item off of your queue, at least they’ll grab something that will keep them busy for a while.

Java 7 will come with a ParallelArray class that abstracts away much of the detail around the fork/join model. It lets you filter and map the data set it operates on to reduce the data set before parallelizing the work. Parallelizing with ParallelArray makes it very clear as to what each thread is doing, as opposed to parallelizing work with a series of arbitrary thread objects that simply join to one another. Anonymous inner classes, or Java 7 closures, can provide filtering and mapping logic for the ParallelArray to use when working on the data set.

Although the advantage of processing data like this is clear, I still struggle to find examples of data processing at my job that can benefit from this type of multi-threaded processing. Maybe this requires a different way of thinking about our current set of problems.

CommunityOne opened with a keynote from Ian Murdock, VP of Developer and Community Marketing at Sun Microsystems. Ian feels that the software industry is undergoing a fundamental change, specifically in the area of open source. According to Ian, the software development industry is currently moving from a more proprietary based world, where innovation only happens in specific places (large companies), to more of an open, community engaged world, where innovation happens everywhere. He sees the industry starting to adopt a model that could be compared to a Linux distribution. Smaller, loosely joined software components (web services, packages, etc) developed by a community combined into a larger, functional unit that is greater than the sum of its parts. Ian stressed the importance of community involvement if this model is to succeed.

After Ian’s presentation, a moderated Q and A session was held with a group of individuals with senior roles at some of the larger companies that profit from open source products (MySQL, RedHat, etc), as well as groups that do not profit from their projects (Samba, Python, etc). The questions all revolved around how those people engage their communities, encouraged contributions, and grow their community as a whole. For the people who were associated with a company that profits from open source, questions were also asked regarding how to encourage contribution from their communities and give back to their communities, while still trying to turn a profit and satisfy their investors.

The theme of community was obvious. In the keynote, and in all of the talks that would follow at CommunityOne.

The Road to Babel: Bringing JVM Machine Implementers Together

Probably one of the best talks of the day, in my opinion. Charlie Nutter did a great job running down some of the popular and not so popular languages that can run on the JVM. He said that virtually every language that he knows of can run on the JVM one way or another. It is clear that Sun is working hard to reach out to these language implementers to beef up support for their languages on the JVM. There is also an effort among the community of JVM language implementations to better organize their efforts. The implementers have realized that they can share libraries to avoid duplicating effort. As an example Jython and JRuby now share a common POSIX library. In addition to the push from Sun to better support other programming languages on the JVM, the community is getting organized and doing some pushing of their own. There is now a JVM Language Implementers list on Google, which is a great way to stay on top with what is happening in this arena. Also, there is a JVM Language Runtime environment currently in development that is aiming to bring many of the features used by other programming languages to the JVM, which would hopefully improve support for some of the languages that are dramatically different than Java. This environment is even aiming to address issues regarding how these different programming languages, all living on the JVM, can interact with each other (similar to how the .NET languages can play with each other). With all of this going on, the future is looking bright for the JVM as an overall application platform.

Insight into the Ubuntu Ditro & Community

I started out in the Multicore Processors & Microparallelism talk, but that ended up being way too low level for my liking. So, being an Ubuntu user, I hopped next door to check out this talk. The talk didn’t cover anything revolutionary, just a good overview of how the Ubuntu project is organized, and how it encourages and engages its community.

Quote of the day: “Made a sick in my mouth” (aka Vurp).

Grails: Productive Web Dev. w/Groovy, Spring, and Hibernate

This was another great talk, showcasing some of the amazing features of the Grails Platform. Grails is more than just a web framework. It is a build system, an ORM layer, and more. Written in Groovy, it allows you to write code that is dynamic, flexible, and expressive. And, it doesn’t re-invent any wheels. Grails is backed by Spring, Hibernate, and several other open source Java projects that do their distinct jobs very well.

If you have any experience with Ruby on Rails, think of Grails as the Java Rails. It supports much of the features that made Rails so popular, including:

• Dynamic loading of changes without having to restart your application
• Dynamic generation of convenience methods, like dynamic finder methods on domain model classes. For example, Person.findByAgeLessThan(15) will in fact find a Person object in your database with a age less than 15.
• Scaffolding (dynamic or generated) for CRUD operations.
• Easy configuration of routes.
• Convention over configuration.
• And much more

It has a powerful plug-in system with a wealth of community developed plug-ins. These plug-ins help keep the core small, and give you the flexibility to only install what you need.

The speaker said that Grails was the next step in the continuous simplification that has been taking place in the Java world. J2EE was too complex, so Spring and Hibernate simplified it. Grails takes that a step further by simplifying Spring and Hibernate to the point where you wouldn’t even know that they are doing the work behind the scenes.

Having worked with Rails in the past, one Grails feature did stand out as a big “nice to have”: the ability to do development work with an in memory database. With Rails, you need to create a database before you can do any development work, you need to migrate that database when you update the schema, and you need to maintain test data in that database. It gets to be a pain sometimes. With Grails, you can simply use an in memory database that is crated when the application starts up. This will get you a fresh copy of the database every time you start the application. There is even a hook in the framework that will let you add test data to the database at init, so you can start every time with a fresh set of test data. Very slick.

Grails also has some nifty view technologies. You can either embed Groovy code right in the view (JUST DON’T ADD BUSINESS LOGIC) or use tag libraries. Tags are easy to add, and easy to toss into a namespace to prevent collisions.

Riding on Java technology also gives Grails some advantages, including IDE support in all 3 of the major IDEs (including debugging support), and since deploying a Grails app is just like deploying any other Java web app, we know that the technology scales.

Given our large investment in Java at Orbitz, and the wealth of Java code we have already written, I think Grails would make a great choice for any web applications that would need to interact with our existing infrastructure.

Building Ajax Applications

This talk showcased some of the new features added to the Netbeans IDE to support the development of Ajax applications, and other rich clients.

First up was a whirlwind tour of some of the new support for JavaScript that has been added to Netbeans. Some of the new features include syntax highlighting, code completion, quick access to documentation, refactoring support, and more. The Netbeans team has also added some tools to aid in JavaScript development, including some static code analysis tools, and a full featured debugger.

Also demonstrated were some of the capabilities that Netbeans provides to easily add rich content to your web application. There is integration with the JMaki Ajax framework that enables you to drag and drop some DHTML and Ajax capabilities right into your project. Netbeans also supports an easy drag and drop interface to add pretty graphs to your application.

Netbeans has been making great strides over the last year. Their Ruby editor is very polished in my opinion, they have great Rails support, and I know that they are targeting other languages as well. It’s sort of ironic, but I really haven’t heard much regarding the Java support in the IDE. However, given how well they are doing everything else, it might be worth checking out.

Asynchronous Ajax for Revolutionary Web Applications

This presentation covered a new spin on hot technology. Typically, Ajax involves making requests from the web browser to the server asynchronously, to avoid locking up the web browser. However, some kind of human interaction usually triggers the request, like a button click or some other mouse event. This talk described a new model for Ajax, a push model. This talk focused on some of the newer web servers and Ajax frameworks that allow you to do “event driven” Ajax. This model provides for very speedy updates, with reduced load on the server. They are speedy because the update is sent as soon as the server processes some event. And, the load on the server is lightened because clients no longer have to constantly poll for updates. This concept is known as “Ajax Push”, “Comet”, and “Reverse Ajax”.

An example of this that was demonstrated was a slideshow application. Clients would connect to the web application, and would be shown the current slide in the slideshow. When the meeting organizer advanced the slide, updates would immediately be sent to all of the registered clients with the new slide. This sort of technology could dramatically change how the web works, from request/response model to more of a dynamic, event driven model, or more likely, a combination of both.

The speakers talked about several web containers, frameworks, and other technologies that allow this to happen. Their demo used the ICEFaces framework, and the Glassfish J2EE server.

This approach brings up many questions about resource usage on the server. After all, in order to send updates to clients, the clients must maintain a connection to the server at all times. This consumes threads and sockets. The speakers said that while a server could be configured to allow many concurrent connections, it is not a wise use of resources, and you could eventually max out your resources. As an alternative, they suggested using non-blocking IO while communicating between the client and the server. Doing so lets you service many clients with just a few threads.

This way of doing Ajax is still very new, and not very well supported by all of the different web containers out there. However, many realize its significance, and a standard is on the way.

• Overview of the JavaFXâ„¢ Script Programming Language
• Enterprise-Level Testing Strategies
• Introducing eBay’s Scalable Innovation Platform
• Real-Time Specification for Javaâ„¢ (JSR 1): The Revolution Continues
• Let’s Resync: What’s New for Concurrency on the Javaâ„¢ Platform, Standard
• Event-Driven Service-Oriented Architecture on the Javaâ„¢ Platform with OSGi and Spring
• Jersey: RESTful Web Services Made Easy
• Groovy, the Red Pill: Metaprogramming–How to Blow the Mind of Developers on the Javaâ„¢ Platform
• Introduction to Web Beans
• Dealing with Asynchronicity in Javaâ„¢ Technology-Based Web Services
• The JavaScriptâ„¢ Programming Language for Enterprise Application Scripting: Five Years of Experience
• How to Build RESTful Clients with the JavaScriptâ„¢, Ruby, and JavaFXâ„¢ Programming Languages
• JRuby at ThoughtWorks
• What’s Hot in Javaâ„¢ Programming Language Debugging
• Groovy and Grails: Changing the Landscape of Javaâ„¢ Platform, Enterprise Edition (Java EE Platform) Patterns
• JRuby on Rails Deployment: What They Didn’t Tell You
• Programming with Functional Objects in Scala
• Defective Javaâ„¢ Code: Turning WTF Code into a Learning Experience
• Design Patterns Reconsidered
• Designing Manageable Javaâ„¢ Platform, Enterprise Edition (Java EE Platform) Applications in a Clustered Environment
• Untangling the Asynchronous Web
• Complex Event Processing at Orbitz
• Improving the Engineering Process Through Automation by Hudson
• Automated Heap Dump Analysis for Developers, Testers, and Support Employees
• Top 10 Patterns for Scaling Out Javaâ„¢ Technology-Based Applications
• Spring Framework 2.5: New and Notable
• It’s All About the SOA: RESTful Service-Oriented Architecture at Overstock.com
• Developing Service-Oriented Architecture Applications with OSGi

Go Doug, Jill, and crew!