Applying Unknowns to Annualized Loss Expectancy

I've started putting together a presentation for a conference that I'm writing a proposal for about how you can use security metrics and statistics to make Annualized Loss Expectancy work a little better. The idea is to get a compromise between the ease of qualitative risk analysis and the accuracy of quantitative risk analysis. I started writing up an example of what I mean, and it turned out pretty good, so I thought I would share it here.

How much should we spend on hard disk encryption for our sales force?
Let's start with a simplifying assumption. How much do you spend on each record that you lose? Well that answer might be based on the type of data that is lost, but you can probably come up with a range of numbers that is reasonably accurate. For example, you know that you're probably going to pay for a stamp for each letter, so you know that it has to be more than 42 cents per record. You probably have to figure that you're going to put two hours into accounting for the records on each breach. You should assume that the employee who lost the laptop is also going to lose some time while he is being interviewed by you. You can expect that your legal department is going to lose a couple hours of time while they draft up a letter to everyone involved. So at an absolute minimum, you know that it is going to cost five or six hours of employee time plus 42 cents per record. A quick look on the web shows that companies that sell expensive software want you to believe that the average cost is $197 per record. That's probably a reasonable high end of the range.

Next it would be nice to know how many records are on the average sales laptop. That's really not too hard to do. Get a list of all the sales people and find out how many you would need to sample to be 95% certain of a 3% confidence interval. Let's say you've got 50 sales people. A quick Google search finds me a sample size calculator. When I plug in my numbers it says that I need to sample 48 of my 50 sales people if I want that kind of accuracy. Crap, I'm not going to do that. But I know from the rule of five from statistics that if I randomly sample 5 of them, I am 93% certain that the true median falls between the top and bottom of that range. So let's randomly sample five of them and check out what is on their laptops right now. In this hypothetical scenario, I found that the laptops had 250, 128, 64, 0 and 0 records. I don't want to run into a divide by zero error, so I'll set a floor of one record. So I can set a range of 1 to 250 records on each laptop.

Let's assume that HR comes back and tells us that the average employee salary is $54/hour with a standard deviation of $14/hour. We decided that each laptop was going to require at least 6 hours of staff time regardless of how many records are in place. So if there is one record on the laptop, then have a cost per record of ($54 * 6) + 42 cents. That's $324.42 per record. If there are 250 records then the cost is $1.72 per record. That $324 per record doesn't withstand my smell test, so I'm going to throw it out in favor of of the $197 per record cited above. That means that when a laptop is stolen from our sales force, we should expect that it will cost between $1.72 per record and $197 per record and that there will be between 1 and 250 records exposed. That gives us a cost range of $1.72 to $49,250 for a single loss.

There are ways that we could get tighter numbers if we really needed them. We could actually sit down and survey 48 of our 50 sales people so that we would be more confident about the number of records on each sales person's laptop computer. We could also run our numbers through a Monte Carlo simulation to see if we get a tighter distribution of costs. All that would be unnecessary, however, if we found that even based on our broad estimation of costs, the software was too expensive to implement. Based on the parameters that we established above, I ran a small simulation and came up with an average of $6.67 per record with a standard deviation of $11.96. We know that there is a 66% chance that the true average is within one standard deviation of the mean, but we should also expect that we still wont have a scenario where the cost dips below $1.72 per record. So now we can say with a fairly high degree of certainty that the average cost per record will be between $1.72 and $18.63. And that gives us a single loss expectancy between $1.72 and $4,657.

Now we're getting numbers that don't seem unreasonable, and you can really tell your managers that you didn't just make something up. There is real math behind using real numbers behind this. What if you were trying to justify spending $5000 on disk encryption for your sales team and you expect the software to be useful for five years? You would only need to have two laptop thefts in the next five years to make this a good decision. You can repeat the same processes above to get a strong estimate of the number of laptop thefts that you're likely to suffer.

Also, remember to bring all of your costs back to today's dollars using Net Present Value. The total cost of your software today should include the discounted cost of employee time next year and software maintenance for next year. Decide how many years you're going to assume for the useful life of the product. Same goes for your losses. If you assume a loss of $4,657 per year, and a discount rate of 4% then the present value of those losses is $20,372.14. Subtract the present value of the cost of your software project and you've got NPV. If NPV is positive, then you've got a good project. If not, then you should look at other ways you can improve the situation for less money.