The Community Blog for Business Analysts

Marc:

I just stumbled unto your post.

You wrote - Bottom line: one of an analyst's skills should be measuring the putatively immeasurable

I think I tend to agree with you. However, in my mind analysis either qualitative of quantitative are quite often plagued with uncertainties.

Some uncertainties that come to mind are; natural variability, modeling uncertainties, human and organizational uncertainties, and knowledge uncertainties.

Depending on the analysts skill level, it may not possible for the analyst to develop unambiguous definitions and evaluations of these uncertainties. If the analyst is not skilled in the particular art or science required for the analysis. I suspect knowledge uncertainty would render an analysis of questionable value.

I suspect the field of BA has a long way to go before it can catch up to the analytical prowess of finance, science and engineering.

Regards,

zarfman

Zarfman:

You hit the nail on the head. We need to stop looking for simple numbers and start working with distributions that express the uncertainty in our data, carrying that uncertainty all the way to the results. Fuzz in Fuzz out. That's a good thing.

Regardless of his skill level, an analyst will always do better recognizing uncertainty than trying to ignore it. That's not to say they'll do well--just better.

The analytical prowess of finance led them straight into the 2008 crash. Their models ignored uncertainty and failed to tell them what their true risks were. VAR told them how bad their position could be nineteen out of twenty tomorrows, but gave them no hint of the catastrophe that could be the twentieth tomorrow. They should have been looking at distributions and scatter plots when they were hanging their clients' futures on one badly contrived number. We can only hope that they learn something. They didn't the last time, or on those many twentieth days where they skirted disaster and called surviving it a success.

Science and engineering and economics (not the same as finance) are working on the problem of characterizing complex adaptive (social) systems (see the Santa Fe Institute). What they learn will be useful to us someday. In the meantime, they have much to teach us about modeling and simulation. I'm trying to encapsulate that at http://goodplan.ca .

Hi:

Its been a long time since I thought about or used Monte Carlo simulation, statistics etc.

I was interested in your writing about Savage and Hubbard. Of particular interest is Hubbard and his book how to measure anything.

My question is, how does one measure the competency of an individual? Does Hubbard address this problem? An example of what I mean follows.

I have a friend who makes a very very good living cleaning up after modestly skilled DBA's. In fact, I've recommended him to a number of companies. after he has done his magic the performance increases of the DBMS's are dramatic. Processes that once took six hours to run now take less than five minutes.

I feel the accessing competency problem is further compounded if the individual attempting to determine the competency of another individual, is not not skilled in the art or science espoused by the individual under consideration.

I don't know if this makes in sense to you, if not I'll give it another try.

Regard,

Zarfman

Zarfman:

It makes a lot of sense. People are hard to measure. I think there are a couple of reasons that bear exploration.

First, people are not consistent. We have bad days; we have bad years.

Second, there's feedback. Tell me I'm performing badly and I may work to improve. I may give up. I may ignore you.

Third, calibrate me and I may use the calibration to improve. Change for the better isn't a change for the better if what you're looking for is consistency.

All that said, I still think it's worth the attempt. The main question is, "given a good performer and a poor performer, what's the difference?" How do you know that the one is better than the other? In the answer lies something you can measure.