Methodology

veryard projects > sebpc > methodology

on this page

notions of methodology
issues
methodology wars

two perspectives
quality criteria
references

on other pages

veryard projects home page

systems engineering for business process change

modelling

There are many different ways of thinking about methodologies. There are overlaps and conflicts between these different ways of thinking. On this page, we provide a brief summary of some of these ways of thinking.

academic study of method

methodical approach

collection or philosophically grounded collection

package

set of organizing principles

description or prescription

post hoc justification

skeleton or complete body

insurance policy

system

body of knowledge

body of practice

Notions of Methodology

veryard projects > sebpc > methodology > notions

Academic study

In some academic contexts, the word ‘methodology’ is sometimes reserved for the (theoretical) study of methods. However, among practitioners, the word has more practical significance, which we shall explore on this page.

Methodical approach

Some members of Working Group 8.1 of the International Federation of Information Processing have defined an information system methodology as ‘a methodical approach to information systems planning, analysis and design’. [Olle, p1]

Collection or philosophically grounded collection

A methodology can be defined by its contents. Methodologies may be described as structured sets of steps, techniques, design products and processes, components and perspectives. [Olle]
‘A methodology is a collection of procedures, techniques, tools, and documentation aids which will help the systems developers in their efforts to implement a new information system. A methodology will consist of phases, themselves consisting of sub-phases, which will guide the systems developers in their choice of techniques that might be appropriate at each stage of the project and also help them plan, manage, control and evaluate information systems projects. … But a methodology is more than merely a collection of these things. It is usually based on some philosophical view, otherwise it is merely a method, like a recipe.’ [Avison]

Package

Methodology vendors typically present their methodologies as a loosely bundled package of documents, training courses, software tools and other materials. They emphasize the uniqueness of their methodological offerings, yet appeal to almost identical concepts, techniques and arguments. This means that there are clusters of similar methodologies, where it is not clear to what extent the differences are significant, or merely a desire for commercial branding.

Set of organizing principles

A design methodology provides organizing principles or rules of thumb, together with what Rowe (citing Gadamer) calls ‘enabling prejudices’.[Rowe, p 36]

‘When a person is faced with an act of design, he does not have time to think about it from scratch. He is faced with the need to act, he has to act fast; and the only way of acting fast is to rely on the various rules of thumb which he has accumulated in his mind.’ [Alexander, pp 204-5]

These rules of thumb are sometimes known as heuristics. Koen attempts to produce a formal definition of the popular engineering term ‘state-of-the-art’ in terms of a set of heuristics. This suggests another definition of a methodology.

Description or prescription

We can regard a methodology as a generalized description of the activities of a series of design projects, together with a theory that explains why those projects were successful.

Thus a methodology may be seen as an abstraction from good practice. This is of course not a value-free description, since if we know what counts as good practice, this entails our having some criteria of goodness.

However, a methodology is usually presented not as a description but as a prescription - a recommendation that projects should follow the generalized task structure.

Post hoc justification

A methodology may provide a basis for justifying either the activities of a project, or its results, or both.

Such accounts are not required to be complete. If we did something, but it turned out in retrospect not to be useful, we need not include it in the post-hoc justification. The distinction between methodology as reconstruction and methodology as operational method is made by Iivari.

On this view, a methodology describes not how today’s projects actually proceed, but how their managers think they ought to proceed.

Skeleton or complete body

Some people think of a methodology as if it were a skeleton, on which a project can be hung. As a follower of the methodology, you flesh it out, apply the techniques suggested by it, make your own decisions prompted by it, produce some of the deliverables dictated by it. How strictly you adhere to the methodology depends on the project goals, and on your own common sense. On this view, a methodology is judged by its usefulness, in guiding an analyst or designer of a given background and ability, to work effectively in a given situation.

Alternatively, a methodology can be thought of as a full body of tasks, tools and techniques. Whereas on the skeleton view, the methodology supports a project, on this view it constrains it. Whatever the methodology omits, you omit. The project task structure is prescribed by the methodology. On this view, any gap in the methodology is a serious flaw.

Insurance policy

Information systems - and information system development projects - can fail in many different ways. Some risks are more considerable than others. A methodology can be regarded as an insurance policy, to reduce or eliminate these risks.

One useful source of input to a methodology is to analyse the failures to which projects within the target domain of the methodology are vulnerable. The methodology is then designed to address (and guard against) a specific set of possible failures. This line of research has been followed by Lyytinen.

This way of thinking about methodologies allows us to think about the coverage of the methodology. The implication is that the methodology only needs to contain guidelines for the risky or difficult design decisions, and need not provide support for things that are obvious (i.e. to the target audience of the methodology).

System

A methodology can be regarded as a system converting an input into an output. [Veryard 85] So-called hard methodologies start from a requirement and produce a product (e.g. an implemented computer application system). So-called soft methodologies start from an open situation, where specific problems and opportunities have not yet been identified or agreed, and produce a plan for improving the situation. The output from the latter may become the input for the former.

The system is a human-activity system, and is therefore not deterministic. Different people will achieve different results.

The unit of execution of a methodology (soft or hard) is called a project.

Regarding a methodology as a system provides us with a complete set of design criteria, together with an understanding of how they may conflict. A methodology can be made more powerful and flexible, at the cost of making it only accessible to highly trained experts; a methodology can be designed for bureaucrats to use, and only bureaucrats will enjoy using it; a methodology can contain many activities for cross-checking and confirmation, which increase its reliability but reduce its efficiency; and so on. When developing a new methodology, these conflicts - and the trade-offs between them - should be addressed explicitly.

Body of knowledge

The methodology represents knowledge of a certain kind. This knowledge is a combination of know-how and know-what.
Broadbent identifies four main components of a design methodology:

Technical knowledge - such as equations or laws of nature

Professional skills and techniques - including ways of thinking and modelling

Ideologies and values

Templates or exemplars - providing examples of good practice.

Regarding a methodology as a body of knowledge provides us with validity criteria. What makes the methodology true, and how can this be verified? What are the formal foundations of the methodology, what mathematical or philosophical arguments can be adduced in its support?

It is often not necessary for most users of the methodology to know specifically how the methodology has been tested, nor to know the formal foundations of the methodology, although they may draw some comfort from the fact that this work has been carried out. However, master practitioners may need to refer to these foundations when resolving difficult practical issues.

Furthermore, the foundations will be of interest to academics and theorists, as well as tool-makers.

Body of practice

A body of working practices, embedded in a community of practitioners. A discursive practice.

Issues

veryard projects > sebpc > methodology > issues

	Identity	When comparing projects using the same methodology or different methodologies, the methodologies need to be clearly identified. Is this a variant/dialect/interpretation of the same methodology, or an entirely different one?
	Espoused / InUse	There will often be significant differences between the methodology actually used on a project and the methodology the practitioners think they're using. After I wrote a paper suggesting that practitioners sometimes espoused a hard systems approach but unknowingly practised a soft systems approach [Veryard86], Checkland revised his account of Soft Systems Methodology to insist that a project doesn't count as SSM unless the project team explicitly intends to practice SSM.
	Expert / Common	A methodology may be able to achieve amazing results in the hands of its author, or an elite group of his chums. However, when reviewing the general powers of the methodology, it is appropriate to consider what it can achieve in the hands of ordinary users. The same is true of the power of a notation - even if Grady Booch can express practically anything in UML, that is perhaps not a fair test of UML's expressive power.
	Theory of Change	Many methodologies provide ways of diagnosing dysfunctional systems, processes and organizations (As-Is), and designing improvements (To-Be). But they do not adequately support the change from As-Is to To-Be. They do not provide any mechanism that might effect such a change, and they lack any notion of resistance to such a change. They therefore lack a robust approach to change management.

Methodology Wars

veryard projects > sebpc > methodology > wars

Once upon a time, methodologists disagreed with one another in public. These disagreements were known as "methodology wars". Nowadays they avoid any public sign of disagreement. Should we be fearful of a return (or resurfacing) of serious methodological debate?

Real wars are horrible. Nobody was killed, maimed or tortured in the methodology wars. A few pet concepts were mauled about a bit, and a few egos bruised, that's all. Robust competition, I call it.

So what do we have now? A kind of bland arrangement, where many people seem afraid of exposing any conflict, lest it put off
the punters. The conflicts haven't disappeared -- they've merely been buried inside large baroque umbrella languages and processes, whose concepts and notations are capable of widely various interpretations, and whose weaknesses can always be hidden by bolting on arbitrary extensions.

The methodology wars did, of course, indicate that many of the so-called experts didn't have any way of reasoning intelligently and practically about the differences between one method(-ology) and another, and often displayed rather strange ideas about what was important and what wasn't. Some people might interpret the present avoidance of warfare as a sign that this was still true. One of the lessons from history is that attempts to avoid warfare often result in worse wars.

Methodology experts often regret the disagreements of the past as if it were merely a breach of etiquette or marketing faux pas. But if there was nothing more going on than that, the rest of the world will conclude that there was never much real intellectual content to any of it. How often recently have you heard a methodology expert say that something was actually wrong?

As in: "Information Engineering was constructed on the premise that business processes were stable. We encouraged people to plan and build long-term systems architectures on this basis. We now recognize that this premise is no longer valid -- if it ever was -- and that the architectures we then encouraged people to build are no longer suitable. Sorry."

IMHO, a framework that can never be proved wrong is useless. Do we really believe that robust selling of one methodology against another would cause carnage? What's the alternative?

Two Perspectives on Software Process

veryard projects > sebpc > methodology > two perspectives

A methodology must be described both diachronically and synchronically. Users of a methodology typically prefer to start with a diachronic description. However, sophisticated use of the methodology (including customization and optimization) ideally requires access to the synchronic description. Furthermore, the synchronic description is essential to the people building tools to support the methodology.

If the tool builders work from a diachronic description, the resulting tools are likely to be sequentially restrictive. (A given tool may support the methodology extremely well - but only if you perform the tasks in the ‘right’ order.)

Something similar happens with many computerized clerical systems, which only work effectively if the documents arrived in the ‘right’ order.

Confusion between the synchronic and the diachronic descriptions is a common difficulty with the development of software processes, methods and tools.

Quality Criteria

veryard projects > sebpc > methodology > quality criteria

If we regard a methodology itself as a human activity system, we can then apply the same sort of quality criteria as we should apply to any such system, including information systems themselves. [Veryard 85]

From these quality criteria, we can derive the principles of methodology design, in terms of the quality characteristics expected of a good methodology. By being explicit about these characteristics, and the trade-offs between them, we can place any methodology in terms of these trade-off decisions.

Effective • Does the methodology work?
• Does it solve the problems, or produce the products, for which it is intended?
• Do projects that follow the methodology turn out successfully?

Efficient • Are all the tasks and activities prescribed by the methodology strictly necessary?
• Are all legitimate short cuts exploited?
• Is there any redundant effort?

Universally applicable
Comprehensive • Does the methodology work across the whole of a domain? Is this a general-purpose methodology or a specialized methodology?
• If there are any restrictions on the range of situations that the methodology can handle, are these restrictions well understood?
• Does the methodology work in any organization size or culture, or does it assume a particular organization or management style.
• Does the methodology have limits of the size or complexity of projects it can handle?

Reliable
Accurate • What risks are involved in using the methodology?
• How are the risks minimized?
• At what stage of a project can we be reasonably certain of success?
• What quality control procedures are there, and how do they work?

Stable
Robust
Flexible
Evolving • Is the methodology tolerant of minor errors and alterations?
• Does the methodology allow for human imperfection?
• Does the methodology contain a self-preservation mechanism, to maintain its relevance within the organization?
• Is the methodology capable of incremental change, to cope with new ideas or technological opportunities?
• Is the methodology capable of incorporating improvements learned from experience?

Simple & easy to learn and use
Acceptable to participants • Is the methodology targeted at a well-defined population?
• Is the methodology based on a coherent set of concepts and techniques?
• Are all the concepts and techniques strictly necessary?
• Does the methodology conform to the prevailing conceptual paradigms and values?
• Is it easy to motivate people to adhere to the methodology?
• Is the methodology scalable (in other words, does the complexity of the methodology grow in proportion to the complexity of the problem faced, or do you have to have complex solutions even to simple problems)?

Manageable • Does the methodology provide guidelines for the management environment of the project (including project management, inter-project coordination, risk assessment and quality management)?
• Does the methodology clearly state what it regards as success or failure for a project, and provide suitable measures (e.g. for productivity and quality)?
• Is the methodology self-monitoring? In other words, does it provide the project manager with information about the effectiveness of the process?

Visible
Comprehensible • Does the methodology make its reasoning clear and visible to the participants, so that they can intelligently judge the relevance and completeness of each piece of work?
• Do participants attribute their successes (if any) to the methodology?

Well supported • To what extent are relevant tools, skills and services currently available to support this methodology?
• What are the future prospects for the development and commercial dissemination of such tools, skills and services? In other words, is the methodology automatable?

References

veryard projects > sebpc > methodology > references

Christopher Alexander, The Timeless Way of Building (New York: Oxford University Press, 1979).
David Avison & Guy Fitzgerald, Information Systems Development: Methodologies, Techniques and Tools (Oxford: Blackwell Scientific Publications, 1988).
Geoffrey Broadbent, ‘Design and Theory Building’ (Design Methods and Theories 13 (3/4) 1979) pp 103-7.
Juhani Iivari, ‘A Methodology for IS Development as an Organizational Change’ IFIP WG 8.2 Conference on Information Systems Development for Human Progress in Organizations, Atlanta GA, May 29-31, 1987.
Billy Vaughn Koen, ‘The Engineering Method’ in Paul T. Durbin (ed) Critical Perspectives on Nonacademic Science and Engineering (Cranbury NJ: Associated University Presses, 1991) pp 33-59
Kalle Lyytinen & Rudi Hirschheim, “Information system failures - a survey and classification of the empirical literature' in P.I. Zorkoczy (ed), Oxford Surveys in Information Technology, Vol 4 (Oxford: Oxford University Press, 1987) pp 257-309.
Kalle Lyytinen, ‘Stakeholders, information system failures and soft systems methodology: an assessment’ (Journal of Applied Systems Analysis Vol 15, April 1988) pp 61-81.
William Olle et al, Information Systems Methodologies: A Framework for Understanding (2nd edition, Wokingham England: Addison-Wesley, 1991).
P.G. Rowe, Design Thinking (Cambridge MA: MIT Press, 1987)
Roger Tagg, ‘Too many methodologies’ in G. Baker (ed) Data Analysis Update (London: British Computer Society Database Group, 1983)
Richard Veryard, ‘What are methodologies good for?’ (data processing 27 (6) July/August 1985, pp 9-12
Richard Veryard, 'Computer systems analysis, models and prototypes: a reply to R.K. Miles' (Journal of Applied Systems Analysis, Vol 13, 1986) pp 89-93

veryard projects home page

Please make contact

	Effective	• Does the methodology work? • Does it solve the problems, or produce the products, for which it is intended? • Do projects that follow the methodology turn out successfully?
	Efficient	• Are all the tasks and activities prescribed by the methodology strictly necessary? • Are all legitimate short cuts exploited? • Is there any redundant effort?
	Universally applicable Comprehensive	• Does the methodology work across the whole of a domain? Is this a general-purpose methodology or a specialized methodology? • If there are any restrictions on the range of situations that the methodology can handle, are these restrictions well understood? • Does the methodology work in any organization size or culture, or does it assume a particular organization or management style. • Does the methodology have limits of the size or complexity of projects it can handle?
	Reliable Accurate	• What risks are involved in using the methodology? • How are the risks minimized? • At what stage of a project can we be reasonably certain of success? • What quality control procedures are there, and how do they work?
	Stable Robust Flexible Evolving	• Is the methodology tolerant of minor errors and alterations? • Does the methodology allow for human imperfection? • Does the methodology contain a self-preservation mechanism, to maintain its relevance within the organization? • Is the methodology capable of incremental change, to cope with new ideas or technological opportunities? • Is the methodology capable of incorporating improvements learned from experience?
	Simple & easy to learn and use Acceptable to participants	• Is the methodology targeted at a well-defined population? • Is the methodology based on a coherent set of concepts and techniques? • Are all the concepts and techniques strictly necessary? • Does the methodology conform to the prevailing conceptual paradigms and values? • Is it easy to motivate people to adhere to the methodology? • Is the methodology scalable (in other words, does the complexity of the methodology grow in proportion to the complexity of the problem faced, or do you have to have complex solutions even to simple problems)?
	Manageable	• Does the methodology provide guidelines for the management environment of the project (including project management, inter-project coordination, risk assessment and quality management)? • Does the methodology clearly state what it regards as success or failure for a project, and provide suitable measures (e.g. for productivity and quality)? • Is the methodology self-monitoring? In other words, does it provide the project manager with information about the effectiveness of the process?
	Visible Comprehensible	• Does the methodology make its reasoning clear and visible to the participants, so that they can intelligently judge the relevance and completeness of each piece of work? • Do participants attribute their successes (if any) to the methodology?
	Well supported	• To what extent are relevant tools, skills and services currently available to support this methodology? • What are the future prospects for the development and commercial dissemination of such tools, skills and services? In other words, is the methodology automatable?