Monday, February 20, 2012

Analytic Hierarchy Process for IT Project Selection

A couple weeks ago I had the opportunity to meet with the IT leadership at another university outside of the state where I work.  I really value opportunities like that because it gives me a chance to learn what approaches are being used without sitting in the echo chamber that sometimes forms among the usual set of schools that I talk to.  Much of the time we talked about IT Project Management and one question that came up in particular was how to prioritize which projects will be green lighted for the next development cycle when there are several candidates with equal or nearly equal priority.  That lead to an interesting discussion about the importance of having a steering committee made up of a variety of stakeholders and an open process for weighing projects.

Since then I've had more time to think about that open process for weighing projects, and I'd like to explore the use of Analytic Hierarchy Processing (AHP) to prioritize projects.  I'm not going to explain the entire AHP process here, but I'll give you the four sentence explanation.  AHP is similar to using a weighted matrix to prioritize items.  Just like with a weighted matrix, you're still going to decompose the decision into various items of concern and try to gauge the importance of each one.  And you're going to then try to decide on how well each alternative scores in each of those categories just like you would with a weighted matrix.  The magic of AHP is that each one of these decisions is arrived at mathematically by making a series of comparisons between two items.  That is much easier to do than looking at a long list of criteria and trying to assign weights all at once.  So you might consider both time and budget when prioritizing projects.  At some point you will be asked which of these two is more important and how much more important using an ordinal scale.  The AHP will develop the weights and the scores of each alternative and produce information about the logical consistency of the result.  Wikipedia has a great walk-through of the process:

This post is mainly about the criteria that should be used when comparing the projects.  I was just writing some stuff down the other day and I came up with this list.  I'm sure we could add more to it, but I think that if you even used these items verbatim you would have a process that is consistent, valid, repeatable, and defensible for selecting IT projects.  Especially if the process of rating the projects and the selection of criteria is driven by the steering committee that I talked about earlier.
  • Estimated time to complete the project - A%
  • Estimated cost of the project - B%
  • Alignment with the organization's strategic priorities - C%
    • Priority 1 (some fraction of C%)
    • Priority 2 (some fraction of C%)
    • Priority 3 (some fraction of C%)
  • Alignment with ITS priorities - D%
  • Urgency of need - E%
  • Number of stakeholders benefiting from the project - F%
  • Perceived absence of risk - G%
  • How long has this project been in the queue - H%

After going through the AHP  on each of the proposed projects you should have them ranked in such a way that you can tell which ones are the most important and how much more important they are.  Rather than a typical ordinal scale, the output of AHP is a ratio so a project that scores .2 is twice as "good" as a project that scores .1.  Now you can select which of the big projects that will take up most of the development cycle will get green lit.  It's also easy to see which of the smaller projects you should use to round out the development cycle.

If you want to play with this some more, check out the demo of Make it Rational's product for performing AHP.  The demo runs in a browser and seems to be fully featured except for the ability to save your work.

No comments: