In the telecommunications industry there's a special phrase for that bit of technology that carries data from the last pole or relay box into the customer's home. It's called "the last mile" and it's often seen as one of the biggest challenges because this last step in the technology chain can be a considerable physical undertaking. In the IT industry we also have our "last mile": putting the right application in the hands of the end user. Composite applications address this "last mile", combining a rich user interface with SOA-driven application integration technology.
Composite applications are nothing new. Analysts have been talking about composite applications since the birth of the Internet and in more urgent tones during the Enterprise Application Integration (EAI) generation of integration technology. Today there are plenty of consumer-focused composite applications, colloquially referred to as mashups. Popular examples, fueled by the mapping-oriented Web Services provided by Google and others, have been growing at an exponential pace. These applications typically take data from one source of information and plot that data on interactive maps. Literally dozens of these are introduced every week.
One recent example plots Associated Press news articles by the location of the news story (http://asap.ap.org/fronts/newsmap.s - ASAP); another displays airports with their associated air traffic control delays (www.usaflightinsurance.com/gmap.htm - FAA Flight Delay Information); a third displays movies, theaters and show times by location (www.mashmap.com/ - Mash Map). Composite applications are useful for the consumer. This article will discuss the next logical generation of these applications - the business use of composite applications or, as one market analyst put it, "enterprise mashups."
The Basics of Composite Applications
Since the term "composite application" isn't subject to formal definition by any standards-setting body, there's considerable overuse and misuse of the term in software marketing. The "composite application" label has been applied by consumer and business-oriented software vendors alike in such diverse functional areas as Web portals, business process management, document management, knowledge management, and collaboration. A "true" composite application is probably not any one of these things but rather an application that combines services such as intranet-based search engines, applications and databases exposed as Web Services, messaging systems, business intelligence engines and data integration solutions with extranet services such as data sources and application utilities. Simply defined, a composite application combines functionality from multiple applications to support business tasks that span those existing applications.
Like two streams that join to make a large river, the composite application "buzz" is due to the convergence of two trends: Internet-based user interface technology that's rich (i.e., one that looks and feels like a desktop application but is delivered via a Web browser) and evolving open integration technologies and standards like XML and Web Services.
Read enough of the vendor case studies and you find that the current sweet spot for composite application implementations includes two or more of the following source applications: sales force automation, customer support, contracts/billing, content management, and the ever-present custom legacy application. Composite applications are still new enough that very few qualified and quantified ROI studies exist, but anecdotal evidence indicates that the benefits of composite applications can be great. For IT, composite applications are developed faster than previous integration approaches and can be easier to maintain and upgrade, while also improving the return-on-assets of the existing systems they are built on. For business users, they address issues as far ranging as data quality and data latency, throughput, and end-to-end cycle time, and qualitative areas such as customer service and enhanced ease-of-use.
Now let's take a close look at the underlying architectural components of composite applications.
The 'Application' Side of Composite Application
Internet application development tools have matured greatly in the last decade and they have made great strides towards the goals of true portability, ease of development, and most importantly, true user utility. Internet-based user interface technology has also improved, allowing a "smarter" client that has sophisticated client-side capabilities such as statefulness, fat client-like logic, local data validation, and client-side data control such as local caching and programmatic server interaction, without requiring large downloads or the installation and maintenance of native client-side code.
The most interactive of applications that fit this description are often referred to as "Rich Internet Applications" or RIAs. SOA analyst firm ZapThink estimates that the RIA tools market currently accounts for less than 10% of all application presentation tools but by 2010 will account for over 50% of that market and represent a $1.1 billion industry. There are a number of practical toolsets available to bring a RIA to the user. The three most popular approaches today include Java, Macromedia Flash, and a combination of native browser functionality referred to as AJAX (a combination of Asynchronous Javascript, XML and some supporting technologies). Table 1 compares some the more notable pros and cons of each technology.
In this article, we'll focus on the newest of these approaches, AJAX.
AJAX, or Asynchronous JavaScript and XML, is a Web development technique for creating interactive Web applications. According to the Wikipedia, "The goal of AJAX is to make Web pages feel more responsive by exchanging small amounts of data with the server behind the scenes, so that the entire Web page does not have to be reloaded each time the user makes a change. This is meant to increase the Web page's interactivity, speed, and usability." It's certainly a worthy goal.
Technically, AJAX consists of a combination of:
- Presentation using XHTML and CSS;
- Dynamic display and interaction using the Document Object Model (DOM);
- Data interchange and manipulation using XML and XSLT;
- Asynchronous data retrieval using XMLHttpRequest; and
- JavaScript binding everything together.
AJAX designs are commonly deployed to the user's browser in two parts a pre-built generalized AJAX runtime component, or "client," and your application-specific design specifications. The latter is typically in some XML format. With an AJAX runtime client, the initial download of "code" to the browser will be relatively larger (due to the dual download of AJAX runtime client and your application-specific design specifications) but will make the overall user experience better.
Once the AJAX runtime client is initialized in the browser, it loads the screen layout specification, generates dynamic HTML to render the screen accordingly, and starts to handle all the document object model (DOM) events generated by the user's interaction with the browser. The AJAX runtime client typically handles many user-generated interface events locally, such as field navigation and any associated simple data validation. When communication with the server is necessary (for example, to fetch additional information or to post a transaction), it mediates the network interaction with the server and may, for example, cache results in hidden data tables in the browser to optimize future data-centric activities and manage server interactions better. The AJAX runtime typically communicates with the server in an asynchronous fashion, without stalling a user's interaction with the application or requiring a refresh of the user's HTML interface.
The kinds of tasks an AJAX runtime can handle locally, without server interaction, are rapidly evolving from simple data "checkbox-type" validation constraints such as checking for the presence of mandatory data items or date formats to more complex business rules (perhaps originally represented as BPEL fragments) that are executed in the AJAX runtime. This is some of the magic of AJAX, as it lets you move functions from server to client as necessary. A good generalized architecture of AJAX, depicted in contrast with the more traditional Web application architecture, is shown in Figure 1.
Much like the early days of HTML, you can code AJAX-based applications by hand in Notepad or vi. But considering the many technologies in AJAX (DOM, CSS, HTML, XML, JavaScript, etc.), maintenance and debugging would be at best difficult. You certainly want to use an AJAX integrated development environment (IDE). A good IDE can generate some or most of this fairly complex code, both the server-resident code and the browser-specific code that's ultimately deployed with and executed by the AJAX runtime client.
AJAX IDEs are still maturing. Here are some important functions to look for when acquiring your AJAX toolset:
- Pre-built control library: Look for a large library of pre-built visual interface controls like those you would see in a typical client/server development tool. Ask whether the IDE lets you extend the toolset to meet your own unique needs. Business applications often require custom controls that are unique to your organizational needs and composite applications are no exception.
- Logic/Data binding: The tool should enable the automatic mapping of GUI controls directly to simple services and the generation of code stubs to bind controls to more complex services. In the latter case, this allows the separation of application interface development from business logic/data binding, which can be done in parallel.
- Code development support: Closely related to logic/data binding, code development support allows for that last bit of hand-coding you need to do in Java or .NET to bind a button to a custom application or API. Does the tool include some type of plug-in (or is entirely resident in) for a code development environment like Eclipse, Visual Studio, and/or Dreamweaver?
- Standards support: In the next section we'll outline a number of standards important to an AJAX-based composite application architecture. Choose an AJAX IDE that supports the ones that are important to you project. Get a feel for the product's roadmap for future standards and technologies by asking if the vendor participates in standards bodies like OASIS and W3C. And if the tool doesn't support WS-such-and-such today, when will it?
- Development repository: A repository can be the "lumber yard" from which your composite application engineers pick the raw materials they need to build their composite applications. A repository can store many things: application interfaces exposed as Web Services; logical software services like database tables, stored procedures, and reusable composite application "widgets" such as custom controls; or specialized application binding code. The real trick is to find a repository that makes the storage, searching, and retrieval of these objects easy through simple interaction standards like UDDI 3.0 and WebDAV, and perhaps even an API that allows for advanced programmatic interaction such as the XML Query for Java (XQJ) and the Java API for XML Repositories (JAXR).
- Odds and ends: Some final items to look for in your AJAX toolset include dynamic charts, multi-language support, style templates, and thorough installation and "getting started" documentation. Most but not all AJAX tools support these areas, so trust but verify.
Finally, you want to remember your deployment target when choosing a composite application toolset. Today's estimates are that around 75% of browser installations support AJAX. While this may be less of an issue inside corporations that have constant maintenance/upgrade programs, it's important to know this in advance. While this issue isn't exactly unique to AJAX, it can become a real issue when you want to deploy your new composite application to 10,000 business users.
The 'Composite' Side of 'Composite Application'
Perhaps not coincidentally, Service Oriented Architectures (SOAs) fit nicely into the AJAX-based composite application design goal. SOAs, in case you've missed the buzz so far, bring value to an organization by enabling it to expose important business information and logic in a readily accessible way. Technically SOAs are collections of standardized interfaces, or "services" to and from business applications and data sources allowing loosely coupled access to this data and logic through standardized mechanisms such as XML-based messaging over HTTP. (In fact, if you're reading this journal you likely already understand the ubiquity and utility of an SOA.) There are many parts to an SOA, so we'll restrict ourselves to the SOA principles and technologies that are most relevant to composite applications.
The behind-the-curtain component of composite applications loosely corresponds to the higher-order functions of the Web Services protocol stack. Described top-down from the highest to the lowest these are functions that include:
- Orchestration is functionality that lets the composite application define and execute a "flow" between services Today, this kind of functionality is usually found in Enterprise Service Buses and some BPM products. Popular relevant standards include BPEL (Business Process Execution Language) and BPML (Business Process Modeling Language). BPEL, in particular, is increasingly accepted as an interoperable standard for orchestrated process flows. It emphasizes a loosely coupled orchestration solution that reuses lower-level services (instead of hand-coding an orchestration that is tightly coupled to lower-level APIs). BPEL also includes native functions to persist long-running orchestrations.
- Choreography, or the interaction between independent processes, can be more complex, involving business rules rather than simple conditional logic, and is often laden with elaborate rules for handling exceptions. Because of these factors, it's a tougher nut to crack than orchestration. Relevant standards include the well-supported BPEL, the less popular WSCI, the newer WSCDL, and even human-centric standards like XPDL and the proposed "BPEL for People." There may not be a perfect fit within the standards world for choreography, , as evidenced by recent attempts to "abstract" BPEL for this type of purpose.
- Composition creates coarse-grained services from fine-grained services. The purpose of composition is to eliminate direct application-to-application connections and instead create "composed" services that are reusable many times. For example, fine-gained data services "Get Name," "Get Balance," and "Get Credit Score" can be accessed by every AJAX composite application that needs customer information. But it's smarter to compose a corresponding coarse service, "Get Customer Info," accessible by all applications, particularly if this new service has extra logic like the resolution of data semantics or data rules. BPEL can be used appropriately for composition assuming that your "services" are Web Services only; if your environment includes other "services" such as a SQL interface or JMS API, you may want leave the composition functionality to a tool other than your AJAX tool, such as an Enterprise Service Bus.
- Finally, transaction control manages a reliable relationship between two or more services. Although you could explicitly connect to the source systems yourself and perform your own transaction management, this is hard enough even when connecting to two instances of the same type of system, like two Microsoft SQL Server databases. However, in real-world situations, transactions are just as likely to span multiple, heterogeneous systems over multiple communication channels and APIs such as JDBC, SOAP over HTTP, JMS, and so on. For these reasons, you'll probably want to rely on the more sophisticated, dedicated transaction management capabilities of an Enterprise Service Bus or TP monitor. > Looking forward, a standard you might employ is WS-CAF, a draft OASIS standard "for supporting coordinated and transactional compositions of multiple Web Services applications," that is composed of three existing OASIS specifications: WS-Context, WS-Coordination Framework, and WS-Transaction Management. When complete and approved, WS-CAF could eliminate much of the custom coding inherent in today's approaches to Transaction Control.
In addition to the above, you want to remember a few fundamental areas of technology:
- Security: You should design your entire composite application stack to allow for security to be passed from the highest to lowest level service. This can be complicated by simple issues like different IDs, passwords, and rights. The solution is the management of identity through a "policy" that can be used in all parts of the composite application instance lifecycle. This complex area makes WS-Security and SAML must-have standards to include in your composite application project.
- Management: Once you've deployed your composite application, you'll want to know that it's running and available. Your AJAX runtime environment should include some kind of management interaction through SNMP or JMX.
It's important to realize from this long list of functions that there's unlikely to be any one AJAX IDE tool that addresses all of these functions unless you're willing to include a lot of homegrown code as part of the answer. For example, you might already own an orchestration tool (sometimes called an Enterprise Service Bus or ESB). But using more than one tool to complete your composite application project need not be a deal killer. Many vendors "pre-assemble" some or all of the tools you need, including an AJAX IDE, an ESB, a development repository, and other supporting software. And you may find that your tools interoperate through standardized protocols.
Obviously, all this talk of services means that the composite application development team has to be aware and involved in the service enablement of the source systems. Getting your legacy service enablement wrong will result in longer development times and less value for the ultimate user. Issues that a legacy service enablement vendor can help you address might include:
1. Level of service enablement: Do you enable the legacy database, the legacy application, or the legacy user interface? These are complementary technologies that tackle legacy systems in different ways and with different tradeoffs.
2. Type of service enablement: Service interfaces can be through pre-SOA standards like SQL, the exchange of XML-formatted documents, a more formal WSDL contract, or some sort of messaging system.
3. Granularity of function: It's rare to find a match between the level-of-functional granularity natively offered by a packaged application and what your composite application would "prefer" to consume, so some kind of orchestration, choreography, or composition is often necessary to aggregate a set of lower-level functions exposed by an application into higher-level business services that can be used directly by other consumers.
Finally, keep in mind that not all of the technologies and standards may be relevant to your composite application. Equally important, we couldn't cover them all in this article, so we focused on the most popular and most formal of them. If you embark on a composite application project, you should try to keep up with the relevant technologies and standards.
Finishing the Last Mile
Composite applications benefit the businessperson and the technologist. Business users get a richer user experience. IT has a smarter way to create composite Web applications. Make sure you have a plan to walk this last mile. It's worth the trip.
Composite Applications and Portals
Portals were arguably the first generation of composite applications. Today composite applications can offer a much richer application development platform through service-based integration (instead of the kind of integration in the application server that runs the portal) and the highly interactive user environment offered by AJAX-enabled rich Internet applications. With this trend, some analysts have predicted that composite applications signal the beginning of the end for portal-based applications. In reality, the primary functions of portals and composite applications complement each other well. A portal can "wrap" and pass through users seamlessly to a composite application, containing it wholly. Conversely, a composite application can wrap a portal (or portlet more likely), as part of the application, perhaps even invoking it with parameters that tie it into the rest of the composite application. But they have their differences too.
Different tools for different problems -
Portals began by aggregating functions that typically didn't require coordination between them or transactions across them. For example, Web portals allow UI designers to assemble useful utilities for their user communities but the Company News portlet in Frame 1 doesn't connect to the Customer Reference Database in Frame 2. They don't focus on what one composite applications vendor described as "flow" and "wiring."
While portal technology has greatly matured in recent years, the "Functionality of Composite applications" section in the main article outlines a number of functions and standards that you won't likely see in everyday portal software. Examples of portal maturity are the OASIS WSRP Producer and Consumer and JSR-168 standards, which allow portals to consume portlets from other places (one through Web Services and the other via Java). This is not a service-oriented integration approach to a composite application but does let a portal vendor simulate it.
Pay me now or pay me later - Portal-based applications are typically HTML-driven and can be counted on for fairly lean-and-small sets of 'code' driven to the user's browser. In contrast, composite applications have a larger upfront "load" since there can be a certain amount of code (like the Ajax runtime client described earlier in this article) as well as data that needs to be cached. After this initial load, however, well-designed composite applications can have developer-controlled incremental exchanges with the server that compare favorably with the complete page or frame loads/reloads common to portals.