The Community Blog for Business Analysts

Marc Thibault
Marc Thibault

Quantitative Risk Analysis

I may be overreaching but I include risk analysis as a proper subject of systems analysis. I've done enough TRAs to justify that position—at least to myself. So here's a risk analysis topic.

Toying with the idea of getting some certification I took a look at the CISSP and ISC Common Body of Knowledge.  One thing I found odd enough to exchange a few emails with ITSec gurus. They assured me that this was the state of the discipline. The offense lay in a particular statement, repeated in various documents:

Purely quantitative risk analysis is not possible because the method is attempting to quantify qualitative items.

That, in the words of Dr. Pauli, is not even wrong; "Nothing that matters is so intangible that it can't be measured," is almost a tautology.

If it matters, it has an effect. Observing that effect is measuring it. Drawing a distinction between its presence or absence is measuring it. Estimating a range of values or probability distribution for it is measuring it.

This isn't unimportant. No one can do a cost/benefit analysis that tells them how much they should spend mitigating a "medium-high risk". The effect is that a lot of people are overspending on security based on a "let's scare the pants off them" qualitative risk assessment.

Bottom line: one of an analyst's skills should be measuring the putatively immeasurable.

Any challenges?

 

This entry was published on Mar 07, 2010 / Marc Thibault. Posted in Analytical and Problem Solving Skills. Bookmark the Permalink or E-mail it to a friend.
Like this article:
  1 members liked this article

Related Articles

COMMENTS

zarfman posted on Wednesday, May 26, 2010 10:04 PM

Marc:

I just stumbled unto your post.

You wrote - Bottom line: one of an analyst's skills should be measuring the putatively immeasurable

I think I tend to agree with you. However, in my mind analysis either qualitative of quantitative are quite often plagued with uncertainties.

Some uncertainties that come to mind are; natural variability, modeling uncertainties, human and organizational uncertainties, and knowledge uncertainties.

Depending on the analysts skill level, it may not possible for the analyst to develop unambiguous definitions and evaluations of these uncertainties. If the analyst is not skilled in the particular art or science required for the analysis. I suspect knowledge uncertainty would render an analysis of questionable value.

I suspect the field of BA has a long way to go before it can catch up to the analytical prowess of finance, science and engineering.

Regards,

zarfman
zarfman
Marc Thibault posted on Thursday, May 27, 2010 9:06 AM
Zarfman:

You hit the nail on the head. We need to stop looking for simple numbers and start working with distributions that express the uncertainty in our data, carrying that uncertainty all the way to the results. Fuzz in Fuzz out. That's a good thing.

Regardless of his skill level, an analyst will always do better recognizing uncertainty than trying to ignore it. That's not to say they'll do well--just better.

The analytical prowess of finance led them straight into the 2008 crash. Their models ignored uncertainty and failed to tell them what their true risks were. VAR told them how bad their position could be nineteen out of twenty tomorrows, but gave them no hint of the catastrophe that could be the twentieth tomorrow. They should have been looking at distributions and scatter plots when they were hanging their clients' futures on one badly contrived number. We can only hope that they learn something. They didn't the last time, or on those many twentieth days where they skirted disaster and called surviving it a success.

Science and engineering and economics (not the same as finance) are working on the problem of characterizing complex adaptive (social) systems (see the Santa Fe Institute). What they learn will be useful to us someday. In the meantime, they have much to teach us about modeling and simulation. I'm trying to encapsulate that at http://goodplan.ca .

Marc Thibault
zarfman posted on Monday, May 31, 2010 12:53 AM

Hi:

Its been a long time since I thought about or used Monte Carlo simulation, statistics etc.

I was interested in your writing about Savage and Hubbard. Of particular interest is Hubbard and his book how to measure anything.

My question is, how does one measure the competency of an individual? Does Hubbard address this problem? An example of what I mean follows.

I have a friend who makes a very very good living cleaning up after modestly skilled DBA's. In fact, I've recommended him to a number of companies. after he has done his magic the performance increases of the DBMS's are dramatic. Processes that once took six hours to run now take less than five minutes.

I feel the accessing competency problem is further compounded if the individual attempting to determine the competency of another individual, is not not skilled in the art or science espoused by the individual under consideration.

I don't know if this makes in sense to you, if not I'll give it another try.

Regard,

Zarfman
zarfman
Marc Thibault posted on Monday, May 31, 2010 1:39 PM
Zarfman:

It makes a lot of sense. People are hard to measure. I think there are a couple of reasons that bear exploration.

First, people are not consistent. We have bad days; we have bad years.

Second, there's feedback. Tell me I'm performing badly and I may work to improve. I may give up. I may ignore you.

Third, calibrate me and I may use the calibration to improve. Change for the better isn't a change for the better if what you're looking for is consistency.

All that said, I still think it's worth the attempt. The main question is, "given a good performer and a poor performer, what's the difference?" How do you know that the one is better than the other? In the answer lies something you can measure.
Marc Thibault
Only registered users may post comments.

Modern Analyst Blog Latests

As we start a new year many of us will take the time to reflect on our accomplishments from 2012 and plan our goals for 2013. We can set small or large goals. goals that will be accomplished quickly or could take several years. For 2013, I think Business Analysts should look to go beyond our traditional boundaries and set audacious goals. Merriam-...
Recently, I was asked by the IIBA to present a talk at one of their chapter meetings. I am reprinting here my response to that invitation in the hope that it will begin a conversation with fellow EEPs and BAs about an area of great concern to the profession. Hi xx …. Regarding the IIBA talk, there is another issue that I am considering. It's p...
Continuing the ABC series for Business Analysts, Howard Podeswa created the next installment titled "BA ABCs: “C” is for Class Diagram" as an article rather than a blog post. You can find the article here: BA ABCs: “C” is for Class Diagram Here are the previous two posts: BA ABCs: “A” is for Activity Diagram BA ABCs: “B” is for BPMN

 



Blog Information

» What is the Community Blog and what are the Benefits of Contributing?

» Review our Blog Posting Guidelines.

» I am looking for the original Modern Analyst blog posts.

 




Copyright 2006-2024 by Modern Analyst Media LLC