Recently industry analysts, press, and bloggers have been writing about the state of service-oriented architecture (SOA) and whether it's "dead" or in the "trough of disillusionment."

These discussions have been fueled by surveys that suggest a decline in the number of organizations considering SOA, or that organizations have evaluated, but decided to shelve SOA for the time being. Or, even worse, that after an initial investment in implementing SOA architectures, organizations have been unable to capitalize on these investments.

So, what's happening? What were these companies expecting and where did things go wrong? More importantly, can anything be done to put things back on track?

Let's review the original promises of SOA and the most likely culprits in its perceived demise. And after all the doom and gloom, I'll suggest how practitioners can revive their SOA dreams and turn them into a new and valuable reality by introducing Agile technologies as a part of your SOA development methodologies. We'll call this Agile SOA.

The Promise of SOA
The promise of SOA is essentially the promise of agility for a company's IT landscape as defined in three main categories:

1. Flexibility: By using SOA, technology reuse of legacy and existing systems in the enterprise becomes possible. The premise is that by using SOA you can create flexibility (agility) in systems at the front-end while reusing existing legacy systems at the back-end.
2. Productivity: SOA will enable higher productivity by helping companies react faster to changes in their business processes and providing the possibility of delivering new applications more rapidly by reusing existing code.
3. Expandability: SOA creates the possibility of new business applications while reusing existing systems and ensuring that the business value encompassed in these legacy systems is (re)used and expanded. SOA transforms point-to-point integrated environments into a more service-oriented architecture that in turn encourages scalability.

So, Where Did Things Go Wrong?
Early adopters embraced SOA practices and forged ahead toward implementation with the promise of flexibility, productivity, and expandability as their goal. However, many of these companies failed in delivering on SOA and value to the business.

So, let's examine what might have gone wrong with some of these SOA projects. Here are the top four likely culprits:

1. Misconceptions About SOA
Ask a question and you're likely to get several different answers to "What is SOA?" According to one very broad definition:

A service-oriented srchitecture is essentially a collection of stateless services. These services communicate with each other. The communication can involve either simple data passing or it could involve two or more services coordinating some activity. Some means of connecting services to each other is needed.

However, some people only think of a Web Service (WSDL) when thinking of SOA, and because of this misconception, there can be a lot of stuff considered "SOA-enabled." Others believe that implementing an Enterprise Service Bus (ESB) brings you to a SOA state. And then there are the software vendors proclaiming their products to be SOA-ready, which typically means they've exposed some Web Services. However, these are often too fine-grained to be called services and are delivered as one size fits most to satisfy the general marketplace. These services are usually more oriented to building the technology infrastructure and, initially, don't deliver any new functionality to the business.

Organizations that purchase SOA-ready products must build a specific layer on top of the product's technology to make it align with their company - without doing this, the chances of the SOA project failing are high.

2. Lack of Business Commitment
Most SOA projects are driven by someone in the IT department. The CIO, enterprise architect, or IT manager believes that SOA might be a solution to their ever-present problems - slow delivery; ever-increasing project backlog; or enhancement requests to legacy applications. This person directs the developers to build applications based on SOA principles. The problem with this is that there's typically a lack of commitment from the business. SOA isn't a technology that can be introduced without visibility to the end user. SOA requires commitment from the business because it touches many different systems across multiple departments.

If any part of the business fails to sign up to the SOA way of thinking, the chain is broken and, once again, SOA fails to live up to its promise.

3. Lack of Funding
Starting a SOA project can be a costly exercise. First, there's the cost of technology. You'll need a platform with which to build your services, security, wrappers for existing systems, etc. Then, there's the cost of expertise and initial SOA projects that typically equate to steep learning curves and require high levels of commitment. This can be done by hiring experienced people or a consulting partner. Either way, you have to include this in your budget.

Remember, the first project is usually the project that puts the SOA architecture and its technologies in place; it doesn't deliver the promised business application services. Getting funding for projects with a high business value is difficult at the best of times - and getting funding without immediate return is almost impossible in today's economy.

Implementing SOA without the appropriate budget or support from the business for the expenditure is another cause of failure.

4. Project Execution
Most SOA projects are managed and executed using Waterfall-style project methods. This means defining a project whose duration is from nine-12 months in which the whole SOA infrastructure is defined, coded, and in the end implemented.

This top-down approach to implementing SOA delivers neither immediate value nor working business applications for two reasons. One, there's little end-user involvement, which increases the risk of failing to meet the true business needs of IT or the business. And, this approach doesn't take into account that the world, and the requirements, usually change over time. It's a common misconception that when all required services are defined upfront that the need for change will be minimal. This misconception existed for object orientation and is equally true for SOA.

Services will change, so the technologies used for a SOA implementation must be flexible and support change. If the architecture used to implement SOA is not agile enough and inflexible SOA will fail.

Can These Issues Be Resolved?
How do we solve these issues and get SOA users to climb the path to enlightenment? How do we get commitment from the business, free up funding, and make sure we have a flexible SOA that speeds up application delivery times?

The focus needs to shift away from issues of implementing a new technology architecture to delivering tangible benefits to the business for every project, namely:

• Improving alignment with the business
• Accelerating time-to-market of business applications
• Reducing the cost of building and managing business applications
• Providing flexibility to respond better to changing business requirements
• Reducing project backlogs

The answer is introducing Agile technologies as a part of your SOA development methodologies; we call this Agile SOA. Agile SOA is the bottom-up approach to SOA development.

Now, it may seem a bit strange since SOA architecture is widely seen as something that doesn't really fit well with an Agile approach. After all, the common perception of Agile is that it works best when there are lots of fuzzy requirements and user interaction. A SOA implementation seems to be just the opposite, it's definitive in its requirements and IT is usually the sole sponsor. In addition, there's a widespread perception that taking a bottom-up approach to a SOA project isn't practical.

Through our interactions with customers, OutSystems found that these perceptions are easily avoided. Based on our experience, success starts with setting the appropriate strategy for the SOA project:

1. Do not buy a product to SOA-enable your business. Every company has its own specific processes that need to be defined as services in their unique way.
2. Make sure that your base architecture will let you define services that aren't too fine-grained.
3. Spend time to find the right technology to support your ever-changing environment.

Once the technology and architecture are determined, the next most important issue is getting business commitment.

This means demystifying SOA for the business and transforming SOA from a technology into a business benefit. This can be achieved by keeping in mind two key points:

1. Perceived business value is high when real-world issues are solved. So, find a project that will solve a real business issue and contribute to a part of the SOA infrastructure layer. Try to find a project from the backlog that needs to (re)use data and business logic from existing systems.
2. Make sure the project can be delivered less than three months. This limits the budget risks for the user and ensures the business won't change too much during project execution. Plus it sets you up to run additional short-lived projects that can begin to leverage and reuse existing services. The business will rapidly see an improvement in time to delivery and perceive benefit from this thing called SOA!

All these things will help you get commitment and funding from the business and enable you to get started on implementing a solid SOA architecture.

There's one last thing.

To ensure that what you deliver provides the necessary functionality and is accepted without organizational blockage - you must involve the end user in the project. Then, and only then, are you doing Agile SOA!

Agile SOA will help you get commitment from the business, help you fund the project, and reduce the risk of failure. Agile SOA especially helps you realize the advantages of a SOA infrastructure when budgets are tight, while at the same time delivering business value. With an Agile SOA approach you'll avoid the troughs of disillusionment or the death of SOA and ascend straight up the slope of enlightenment with a successful SOA implementation.