Organic Information
Management — A Better Way to
Reengineer Information Systems

Sub-optimization and Stove-pipe Solutions

The goal of mechanical efficiency is almost impossible to fully realize. Information lifecycles and their associated production processes tend to be more expensive than the engineered ideal because the desire for maximum performance in one arena has often sacrificed the company via lowed performance, overall. From a mechanistic viewpoint, sub-optimization occurs because computer applications almost always target a subset of the information lifecycle and have a narrow corporate focus. In most organizations, the resulting stove-pipes of technology and information match the organization's functional hierarchy and hamper the efficiency of processes which must cross those functional boundaries.

From an organic viewpoint, sub-optimization occurs because the needs of individuals and groups cannot be completely subordinated to system designs. Accordingly, it is probably not possible to completely eliminate redundant software systems, unsynchronized parallel information streams, and duplicated development and maintenance costs. At the same time, open information standards such as SGML (see sidebars: Realizing Organic Value through Organic DTDs and Using SGML to Balance Mechanical and Organic Approaches) can help reduce some of the costly problems with sub-optimization: the over-reliance on paper-based information exchange, data integrity problems, and the need to recreate, reconcile, or perform expensive, labor-intensive conversions of information assets.

In a way, the commercial software industry is responsible for many of the biggest problems. Proprietary data encoding schemes provide individual vendors with competitive advantage, but they also help create the pockets of information that block information compatibility and process integration. In addition, most commercial applications retain, at their core, a paper paradigm — where the computer functions like a big, powerful pen. They draw very fast, can handle very complex images, and allow printing instructions to be revised, but they do little to manage the true meaning of the information. By doing so, they emphasize personal productivity, not overall organizational performance. In contrast, most open information management standards are better able to balance local variations and customization with the need to reuse and share information.

Lack of Flexibility and Customer Responsiveness

Accelerating economic and political change is forcing many companies to adapt and change faster than their information infrastructures allow them to do. Many of these organizations have begun to realize that software isn't very soft. The system won't let me do that is an increasingly common complaint. Strategic business initiatives are often delayed or hampered by trying to adapt or modify legacy information management systems that weren't designed for such changes. In short, you can't streamline smaller, isolated systems that cannot even talk to each other at the most basic level!

In his Harvard Business Review article: What Really Makes Factories Flexible? David M. Upton provides some examples of this phenomenon. Upton studied the relationship between Computer-Integrated Manufacturing (CIM) and operational flexibility within the fine paper industry. He found that in most cases, CIM reduced flexibility. The CIM systems operated the plants more conservatively than human operators, slowed changeovers, and produced more paper breaks than the computer's human counterparts. In addition, operator skills atrophied from lack of use after CIM systems were installed. The biggest detriment to flexibility, however, was traced to management behavior. By dumping the problem on technology and (directly or indirectly) technologists, the managers used CIM to dodge their responsibilities for defining flexibility objectives and managing the organization to meet those performance goals.

Dysfunctional Decision-making and Investment Strategies

Defining productivity solely in terms of physical inputs and outputs started to lose viability in the 1930's as quality — not quantity — became a significant measure of value. Since then, the information content of goods and services increased. In the 1970's with an even more noticeable shift from a manufacturing-based economy to a service-based economy, Taylor's productivity model lost almost all validity.

Accepted productivity models became invalid, in large part, because their methods of measuring value, such as return on assets (ROA) and return on investment (ROI), are based on the assumption that capital is the scarcest — and thus most valuable — input. Today, capital is a commodity item, crossing international borders at the speed of light. Human capital and knowledge are now the most valuable inputs. Individuals who can mobilize the productive capacities of others, protect the organization's future, and direct the efficient use of capital are more scarce than the capital resources — themselves — even though they are virtually invisible from productivity calculations.

Without productivity measures that reflect the changing value of information and human capital, however, organizations often find themselves trapped in dysfunctional decision-making processes that allocate resources poorly and make bad investment decisions.

For example, American productivity appeared to fall as white collar workers became an increasing portion of the labor force. By using the standard productivity models, many organizations counted the reduction of clerical and secretarial staff as a cost savings, even if their work was just shifted to higher paid managerial and professional personnel.

In The Business Value of Computers, Paul Strassman states that invalid productivity measures can force information management investments to be justified only in strictly financial terms, where—as with all other capital investments — the projected costs and benefits need to be estimated with some confidence. When applied to IT investments, however, these justification processes become counterproductive and prevent needed change. Even worse, requiring reliable savings estimates doesn't reduce the risk of projects as much as it biases projects towards the automation of routine clerical operations and away from those that also involve the upgrading of business processes.

In short, Strassman found that strategies to manage information technologies through project authorizations failed to improve organizational performance, increased risk — as long-overdue improvements were combined into larger and riskier projects — and failed to control costs, because initial purchase and implementation had become a shrinking percentage of lifecycle costs.

Derivatives: A parable for information managers

The financial derivative market has generated much attention. Kodak, Orange County, Procter & Gamble, Metallgesellschaft, and other high-profile organizations have suffered major losses or become bankrupt as a result of speculating on derivatives.

These problems result, in large part, from complexity. Derivatives have evolved so quickly and become so complex that few can explain or claim to understand exactly how many of these investments will behave in the market. Understanding — and thus much of the decision-making — has shifted to a small group of technicians. As a result many senior managers have effectively surrendered their policy-making roles.

After first singing the praises of the new breed of financial analysts dubbed rocket scientists, organizations that choose to trade in derivatives have started to realize that Financial decisions that were previously designed and implemented by specialists need to be monitored more closely from the very top of organizations.

A similar situation exists in the information technology arena. There too, senior managers have largely turned decision-making over to technologists. By selecting and implementing technologies in a relative vacuum, these technologists also implicitly define corporate policies. This is unfortunate, because the stakes of bad information technology investments are just as high, if not higher, than bad financial investments.

Unlike their financial counterparts, however, the impact of poor information management policies and investments isn't as observable or as measurable. The harm done by inappropriate solutions can remain unrealized for years or even decades, hampering organizational performance and competitiveness long after the key participants are gone. CIO turnover, outsourcing, lost opportunities, and downsizing are just a few of the results of poor information investment strategies.

Converting Political Resistance into Support

Information professionals tend to look upon politics with disdain — the enemy of rational decision-making. Politics has a negative connotation. Political forces are dangerous, precisely because few organizations have adopted methods for identifying and addressing critical political issues and communicating the resulting choices in the form of well-defined information policies.

As a result, projects are put at risk. DeMarco and Lister observe in Peopleware: Productive Projects and Teams that projects which require no real technical innovation (such as accounts receivable systems) are failing. Technology is not to blame in the vast majority of these failed projects. Instead, politics is the perceived culprit.

Strassman echoes similar frustrations in The Politics of Information Management, recounting his experiences from the late 1950s: ...I was never sure how my technology plans would turn out because of the problems that emerged every time I started installing well conceived and formally approved computerization programs. It always came as a surprise that well-laid plans could go awry for no apparently good reason except for something I kept dismissing as mere company politics. Today, Strassman has concluded that Managing information systems is primarily a matter of politics and only secondarily a matter of technology.

When viewed from an organic perspective, politics is a critical success factor, perhaps even the singular critical success factor. Uncontrolled, the political forces which result from divergent stakeholder interests introduce uncertainty and can disrupt the success of important efforts. Properly integrated, those same stakeholder interests can be converted from forces of resistance to forces of support. Such alignment not only is an important mechanism for ensuring that information management initiatives provide sufficient organizational value to justify the cost, but reduces the need to only base investment decisions on anticipated cost savings.

Formalizing Information Governance and Policy-making

Just as western culture has seen a steady progression of governance models, Strassman traces the evolution of computing as a political history, where power has shifted from centralized to decentralized institutions. Mainframe theocracies gave way to minicomputer hierarchies, which in turn were displaced by the microcomputer revolution and today's age of cooperative alliances.

Strassman argues that organizations today are posed to enter an age of information democracy, where decision-making is formally distributed throughout the organization using the federalist principles of Jefferson, Madison, and Adams to balance power among a central authority and a number of constituent political units. Specifically, Strassman suggests the establishment of a formalized governance process, which is built around an information management policy board that has sole authority over policy decisions. This board functions like a legislature to define policies involving standards, technologies, and implementation practices. Executive powers are split among a variety of enterprise, process, business, application, and local units that have operational responsibilities and are in the best position to decide how to apply information technology to improve business processes. An auditing organization is vested with judicial powers. In all cases, the representatives of the governance processes are all managers, because all management is information management.

Perhaps the most sensitive issues involve conflict resolution, and enforcement. Formalizing information management policies is of no value if they can be ignored, misinterpreted, or used for political gain. Although clear lines of accountability will resolve many of the conflicts that exist today, swift and competent interpretations are necessary to develop the precedents that future policy changes will need to be based on.

Reliable enforcement can also help to eliminate micro-management. Without an ability to trust that policy objectives will be fulfilled, managers often start directing the execution of work activities and fixating on minute operational details — in hopes of controlling performance and results.

According to Strassman, such a formalized governance process promises to put into place a framework by which individuals and organizational units can cooperate while minimizing the debilitating effects of contentious politics and formalize the information policies that are needed for improved organizational performance. Computer staff can safely deliver superior results only if they are executing an unambiguous information policy. Fuzzy objectives and ill-defined means lead to confused execution. Under such conditions, it may be a matter of luck by blundering into success if what is delivered matches what management alleges that they always wanted.

The Use of Values to Integrate Divergent Stakeholder Interests

Policy is the codification of an organization's desired balance among competing values. Thus the process of defining policies is a political process that pits the values, interests, and desires of groups and individuals against each other.

Political conflict often results from the polarizing effects of group decision-making. As Strassman points out, Politics is the art of the feasible and one of the primary challenges in group decision-making is to identify and define a shared vision for the future. Joe Beck, of the University of Eastern Kentucky, has concluded that values, not facts, form the basis for most decision-making and that if agreement can be reached on the values that should drive a decision, consensus can usually be found.

Beck has developed a predictability model called VIE to translate values, interests, and expectations into a range of acceptable options, effectively defining the conceptual space for politically viable solutions. Each individual brings with them a set of unacceptable solutions. When those are combined at a group level, the resulting range of tolerance is usually much smaller than for any given individual. When VIE analysis is applied to a diverse group of stakeholders in a facilitated setting, however, the resulting range of tolerance can be much broader than would be expected in political models that promote polarization. Beck's facilitation model provides a collaborative framework for learning and negotiating a shared consensus of how individual values should be integrated and balanced to define mutually-desired end states.

After achieving alignment of the stakeholders around a shared vision, the group can begin to identify mutual drivers and constraints, which are marked as key issues to be considered during the development of future strategies. This is similar to marking land mines in order to navigate around them. Dispute resolution occurs by focusing on mutual long term agreements rather than fixating on areas of disagreement.

Beck developed this approach to address the political issues associated with the creation and implementation of environmental policy, responding to the inability of traditional strategic planning to converge external and internal stakeholder input. He has tested the approach with over 100 groups during the past few years on high-conflict-potential projects and successfully aligned diverse and contentious groups of stakeholders dealing with a wide array of issues. These successes include the definition of new missions for international governmental agencies, the development of enterprise-level information policies, and the structuring treaty negotiations between antagonistic governments to resolve sensitive national security issues.

When used at the project level, VIE analysis integrates the diverse views of multiple stakeholders into the strategic planning activity, but leaves decisions to the true decision makers — not planners or technologists. This ensures that projects better reflect perceived requirements, have a supportive constituencies, and facilitate the learning and sharing of knowledge necessary for wise decisions.

Avoid Comprehensive, Ideal Solutions

Monolithic software engineering projects, especially as reflected in the data dictionary and enterprise architecture planning schools of thought are close to death. Relational database nirvana — all-encompassing relational database architectures — will not be reached. The main reason is time. Most organizations do not have time to wait. By the time comprehensive database architectures are designed, there's a good chance that the organization's understanding of their needs and the organization itself will have changed. By the time the system is implemented, the organization may not even exist.

Risk increases dramatically with the size of the effort and can be minimized by keeping projects and design teams small. While this tends to promote sub-optimization and violate many lifecycle information management principles, chances are that the solutions would have had mistakes regardless of how well-designed they were. The idea here is not to avoid mistakes, but to make as many mistakes as quickly as possible. This facilitates the learning and understanding which are necessary for the evolution of successful solutions. Also, a series of small mistakes is usually far less disruptive than large, expensive ones.

Monolithic systems are also unlikely to meet the needs of all users or be culturally acceptable. Davenport states that Some managers have always been distrustful of the information systems approaches of their companies, largely because they didn't understand them. In many cases, they were right to feel uneasy. As the diverse company experiences suggest, grand IT schemes that don't match what rank-and-file users want simply won't work.

Rely on Relationships

Organic designs do not abandon engineering all together, but shift the emphasis from comprehensively engineered systems to well-engineered components that can be integrated in a number of useful ways. This integration is performed by the users of the system — not the system designer.

Integrating information management components as needed through time reduces risk and improves organizational flexibility. Risk is decreased because the system designer is not challenged to identify all possible uses for the information objects and every needed software behavior during development. Likewise, component engineering is smaller in scope and shorter in duration than system engineering, and that also lessens risk.

Organizational flexibility is improved when humans to do what they do best: interpret situations, weigh alternatives, and apply information and tools to meet task objectives. Organic designs do not lock individuals into predefined behaviors, but give them the information tools that they need to solve unanticipated business problems and preserve their value to the organization. Both individual and organizational flexibility are enhanced when ...the system [is] designed from the outset to help workers make better decisions rather than to cut them out of the decision-making process. [Upton]

Allowing individuals and groups to define their own interfaces and link them to important data elements is another attribute of organic designs. Allowing individuals to preserve and document the relationships that they find useful also begins the transition to knowledge-based systems, where the both the system and the human become more knowledgeable the more that they interact. Shifting from formalized structures to user-defined relationships helps to preserve investments, especially when the information technology initiatives have strategic value and are intended to create destabilizing change, either to the way that the business operates or the way that people think.

Diagnose Before Prescribing

Like organisms, organizations get sick when they can't adapt to change. In The Stress of Life, Hans Seyle used the phrase General Adaption Syndrome to describe an organism's response to stress. By improving an organization's ability to adapt, and thus its well-being, computers can have therapeutic effects. Both mechanical and organic approaches to information management can have value, but both can also be misapplied. A specific therapy should vary with the condition of the patient and the cause of the malady. Thus, differentiating organic and mechanical goals, requirements, and deficiencies is a critical diagnostic process.

In most cases, an effective computer system balances mechanized and organic elements. Numerous information management components will need to be designed and engineered very carefully for an organic approach to work, but OIM-based systems are characterized by intentionally limiting such engineering to the component and sub-system levels. This leaves much of the final integration to be done by humans in the form of defining relationships between components or executing some of the behaviors that would otherwise be automated. Such an approach is likely to be more expensive than fully automated solutions when comparing isolated transactions, but it is imperative to maintain flexibility and resiliency. Thus, it is almost certain to lower long-term costs by preserving the value of information technology investments.

Reengineering has strong organic foundations

As described by Champy, reengineering is all about building a fundamentally better place to work. A reengineered organization should support profound questioning of organizational purpose and procedures and encourage radical experimentation. Reengineering is based on trust and a belief in an expansive human potential — that as jobs become more complex, people will be able to do the work. Jobs will become more rewarding and take better advantage the pluralistic thinking of multi-skilled individuals. A reengineered organization is built around multiple goals and integrates the diverse thinking of varieties of technical experts and stakeholders. Decision making is distributed and authority isn't derived from position in the org chart, but how individuals contribute to the success of the organization.

Reengineering is very much an organic process. So much so, that the primary challenge for an organization starting a reengineering program is one of being. That is, to understand its fundamental purpose and the realities of its position within its environment — improving organizational focus through shared vision and goals. Reengineering also acknowledges the idea that change changes change. New levels of efficiency, processes, and technologies will often change the very nature of what the business is all about and eliminate the fragmentation found in stove-pipe organizations by resynthesizing work along process flows.

But in practice, it is usually something quite different

In practice, reengineering is usually something very different from the organic approach described above. Organizations do a variety of things and call them reengineering. In many cases, it is simply a cover for attacking the productive capacity of the organization, replacing voluntary agreements that are mutually beneficial with exercises of power and forced self-sacrifice.

Meg Wheatley describes this phenomenon in Leadership and the New Science: Reengineering is the supernova of our old approaches to organizational change, the last gasp of efforts that have consistently failed. What is reengineering but another attempt, usually by people at the top, to impose new structures over old — to take one set of rigid rules and guidelines and impose them on the rest of the organization? It's a mechanical view of organizations and people — that you can `design' a perfect solution and then the machine will comply with this new set of instructions.... Reengineering doesn't change what needs to be changed most: the way that people at all levels relate to the enterprise. We need to be asking: Has the organization's capacity to change increased and improved?... Or have we just created a new structure that will atrophy as the environment shifts?

Even Champy is appalled by the way that reengineering is being used as a cover for destructive downsizing, having seen numerous organizations reduce the numbers of workers without changing the fundamentals of the work process. Organizations that reengineer by simply downsizing (leaving the remaining people to figure out how to cope) do little but reduce their functional capacity and create organizations that are populated by the walking wounded. In a January 1996 speech, Champy even expressed a fear that a generation has been taught to grow by cutting.

Why has Hammer and Champy's reengineering concept caused so much trouble?