Tame The Code Beast

Think about where your business was a year ago and where it is now. Whether better or worse, it's likely that your business environment has changed, and chances are those changes have altered how you think about software. Tasks that you haven't automated now have to be done so frequently that they're soaking up too much precious time. And that elegant abstraction you designed? It now takes too long to tailor for each new product.

This pretty much describes SystemsForge, a company that develops e-commerce, content management, and custom Web applications--building more than 200 apps over the last four years alone, in fact. Originally written in ColdFusion, the first version of SystemsForge's software used a component-based architecture built around a content management system, with "Pages" within a "Site." When building a new Site, developers ran a wizard to generate the code for the project, then used the content management system to add Pages with features such as product catalogs, checkout, e-mail newsletter systems, and event calendars.

This approach worked well for the first few clients. When it started, the product catalog feature had 30 options and was implemented in a manageable 2,500 lines of code. But by the time SystemsForge had 40 clients, the feature catalog had more than 450 configuration settings, nearly 13,000 lines of code. That was unwieldy, didn't scale, and was littered with configuration variables, conditionals, and loops that obfuscated the intent of the code. That's when the company opted for a software product line approach.

A software product line (SPL) is an innovative approach to software development that lets organizations develop, deliver, and evolve an entire line of similar products with much higher degrees of efficiency. The key characteristic of an SPL that distinguishes it from other development efforts: the extensive reuse of core assets. Those assets--referred to as "essential practices" by the Carnegie Mellon Software Engineering Institute--include requirements, architectures, de- signs, source code, and test cases. SPLs even include people, who share their expertise around the assets and procedures associated with a product line.

An SPL is most often used to manage a software portfolio, but it can be extended to companies managing a portfolio of end products, like manufactured goods, where the reuse of practices as well as parts is key to lower costs. Companies such as Cummins, General Motors, Hewlett-Packard, Lockheed Martin, and Nokia have invested in using SPLs, looking for better productivity, quality, time to market, and product line scalability.

SystemsForge, like virtually all SPL implementations, identified and built its reusable assets using an iterative process. Iteration reduces the risk of producing the wrong assets and quickly builds a reuse infrastructure, rather than waiting until a product development effort is finished and reacting to that. The core assets are released to the product building teams much like products are released from developers to customers. Both the Rational Unified Process and the agile approach have been effective for guiding product development in an SPL.

Managing Complexity

Before moving to an SPL approach, the biggest problem SystemsForge had was managing the complexity of product features: There were just too many configuration options to implement them all at runtime. To tackle that complexity, SystemsForge created a software product line that used a concatenation-based code generator--meaning the code was linked together in a chain.

This approach worked all right to create multiple applications for clients with similar needs, but it wasn't flexible enough to scale to multiple clients with differing requirements. Consequently, SystemsForge decided to move its SPL to a template-based generator that promised to be more concise, more flexible, and more powerful than the concatenation-based one.

There are a number of product options to support an SPL. The SystemsForge team looked at various ones, including MetaEdit+ for building modeling environments, Microsoft's domain-specific language tools, the openArchitectureWare project (now part of Eclipse Modeling Tools), various UML-based code generators, and products such as CodeFluent, a model-driven tool for creating .Net applications that's described as a "software factory."

They also examined products such as pure::variants and BigLever, which specialize in SPL management market. In the end, SystemsForge opted to write its own code-generation framework, in ColdFusion, in less than 2,500 lines of code. It decided the benefits from control outweighed the robustness of third-party software. While it lacked the sophistication of commercial alternatives, it was a useful starting point.

Over time, the SystemsForge team raised the runtime framework's level of abstraction to replace a lot of the repetitive generated code with a runtime interpretation of XML models. It also created a standard interface to the metadata so metadata storage could be decoupled from the implementation of the generator. And finally, SystemsForge reimplemented a front-end wizard that made it possible to build applications such as e-commerce systems faster than before.

Working With Custom Code

For most SPLs, developers need to support some degree of truly custom, project-specific code. When the company built its first version of SystemsForge, before using SPL, it used a large core framework with project-specific directories for assets (images, CSS, etc.) and custom code. Generally developers did ad hoc customization of features rather than having a well-structured approach to application extension.

When moving to the SPL template-based generator, SystemsForge developed a smaller framework with numerous extension points. Events/listeners let it add custom functionality such as login, saveDomainEntity, and before/after controller actions. Subclassing generated code let SystemsForge overload specific generated methods. Mixins, partial classes, and includes let it mix generated and handwritten code into one runtime script--while still saving it as two separate files. That allowed active regeneration without losing hand-coded changes.

Generated custom tags let SystemsForge reuse complex generated logic such as list pagination and complex administrative user interface widgets in view code. Protected regions let the company customize specific parts of a view template while still allowing for active regeneration of the code as configuration options were changed.

These changes extended the viability of the product line. By having a well-structured, template-based, code-generation step with well-defined extension points, SystemsForge was able to create more than 100 applications with a code base of about 85,000 lines of code, including framework and generator code but excluding project-specific custom code. That's far less than the 140,000 lines with the concatenation-based generator. If they started with an approved HTML/CSS design, configuration would typically take one to two hours; project communications would add another four to six hours; and the custom code would take another week or two to write and test, depending on the complexity.

The changes SystemsForge implemented using a software product line approach were driven by client needs. But they were supported by extensive research into best practices in domain-specific modeling and software product lines. Change is the one thing any company can count on, and being prepared to manage change is a prime benefit of the SPL approach.

Read all about software development at Dr. Dobb's: drdobbs.com