The statistician and blogger Nate Silver correctly predicted the presidential election outcome for all 50 states this week. I recently had a chance to hear him speak about this new book The Signal and the Noise: Why So Many Predictions Fail – But Some Don’t at a conference.

In his talk he pointed out that predicting presidential elections is really hard because they only happen every four years. We simply don’t have much historical data to use! Instead he used a tremendous amount of related data to build the predictions. Businesses have a similar opportunity. Even if a retailer has never done a particular type of promotion before – they can use data about how similar promotions have performed. If a manufacturer has never built a particular product in a given color – they can look at how other products in that color have performed.

We’ve found that the simple act of examining your data and thinking about relationships in the data can open up dramatic new opportunities for business improvement through better use of information.

One of the most common business requests is for exception reports that identify unusual transactions. Often this takes the form of a request to show the Top x or the Bottom y of a particular set of transactions such as sales orders. Then a person is charged with manually scanning the report every day and verifying that the top and bottom transactions are reasonable.

There is a more efficient way to find the exceptional in a set of transactions that have a normal distribution. Calculate a mean and standard deviation for the set and then identify transactions that are more than two standard deviations above or below that mean. This will give you the truly exceptional 5% of transactions. This approach also gracefully deals with slow trends in transactions such as gradual increases in the sales volume of particular products.

We can take this technique just a little further and create a query or report that will show us just the exceptional activity for a particular day or time period.

Here is an example that compares today’s sales in each of thousands of product finelines to the historical distribution and identifies the handful of finelines with exceptional activity today. This would serve as the basis for a report that would provide drill-through to the transaction detail behind each line in this query. The first row of this table is showing that today’s sales were $423 which is well above the $251 upper bound.

Here is the histogram of the historical data from the first row of the table above with the 95% upper and lower bounds highlighted in red.

A little bit of basic descriptive statistics helps us efficiently discover information about exceptional activity in our business.

Turns out Microsoft had a little April Fool’s joke in store for me.

So here I am, perking along swimmingly (after installing VS 2010 SP1). Partway through the installation process, though, I receive an error message that the installer couldn’t locate the vs_setup.msi in the directory where it expected to find it.

Well.

Perkins Consulting is a Microsoft partner, receiving most of our software through MSDN subscriber downloads. We run Visual Studio 2010 Premium with our TFS installation, which is apparently the version SQL Server 2012 is looking for. However, to save time during our last new laptop swap, I used the web installer rather than downloading the entire ISO file. Apparently, the web installer doesn’t leave the vs_setup.msi behind. Ergo, nothing for SQL Server 2012 to locate on my hard drive.

When I cancelled out of that error window, the installer rolled back a part of the installation, but continued through to the end. The parts that failed installation were related to Management Tools (aka, the components that are closely related to Visual Studio).

I was able to download SQL Server Data Tools (there’s a link to the download site on the SQL Server setup documentation), but the tools didn’t appear to do everything I expected them to do. I remain suspicious and resigned.

Bottom line? I’m now waiting for that VS 2010 Premium download just so I can point to vs_setup.msi.

Whee.

So remember, boys and girls, before you try this at home and click that Install button on SQL Server 2012, make sure you’ve got your VS 2010 Premium-or-better installation media. Otherwise you may end up taking a header in a late-arriving April Fool’s pratfall.

Additional note:

In the MSDN social area, I found one thread regarding this issue. Apparently this has been identified as a bug, but the date for the fix wasn’t mentioned.

Ahmed Ibrahim from Microsoft offered this workaround in that thread:

1-Go to control panel programs and features select VS ultimate and then click Uninstall/Change
2-Select Add or Remove Features then click download since media source is missing. 
3-Select all VS ultimate features
4-Start download and install all features
5-Install Visual Studio 2010 SP1 
6-Install SQL server 2012.

“Data from the National Retail Federation shows that retailers accumulate so much excess inventory, that, markdowns have increased from 8 percent in 1970 to nearly 30 percent in the 1990s. Despite this glut of merchandise, consumers aren’t finding what they want.” – From The New Science of Retailing by Marshall Fisher and Ananth Raman pg.3

For mid-sized retailers the drive to help customers find what they want and sell it to them in a consistently profitable manner has continued to be challenging. The capital required to launch an Internet presence or compete with the store building capabilities of national chains is beyond their reach – especially in these times when capital is hard to come by.

One approach to improved revenue and profitability that isn’t out of reach is to leverage the information that they already have. In our experience we’ve seen three crucial information opportunities for mid-sized retailers:

• Really understanding profitability of products and customers allows a focus that leads to higher margins.
• Better forecasting of demand leads to better staff scheduling and improved inventory turns.
• Measuring employee performance and tying it to behavior allows better coaching for improved sales.

We’ve provided more information on each of these opportunities at our brand new retail focused website: retail360solutions.com

Take a look. Tell us what you think.

Abraham Lincoln once said, “Determine that the thing can and shall be done, and then we shall find the way.” Pretty good advice for President’s Day. He was speaking then as a member of the House of Representatives in support of an internal improvements bill. The opponents of the bill were bringing up reasons why internal improvements couldn’t be implemented. Lincoln was pointing out that the important thing was to first agree on whether internal improvements were a good idea.

Using information to improve business performance is similar. It is such a new approach for most organizations that it is easy to get stuck listing all the reasons why the right data isn’t available or can’t be trusted. Our Making Information Powerful methodology provides a roadmap around such obstacles. Our expertise from dozens of projects gets us quickly to the finish line.

It’s important to start by simply deciding that you can use information to improve business performance and that you will. We often conduct two day or one week Proof of Concept projects that deliver proof of the value of information and remove many of the imaginary roadblocks to success. Once you know that the thing can be done and determine that it shall be done – we can find the way.

Ordinarily we tend to recommend buying rather than building. Buying commercial off the shelf software is almost always cheaper and more effective than building custom applications. Hiring consultants with specific skills can get an organization very quickly to the cusp of improved business performance based on information.

But how to get beyond the cusp? How can organizations truly make information powerful?

There is a danger that the increased hype around information may drive some organizations to buy more technology than their organization can absorb.  This article in the Sunday New York Times implies that data is somehow new and mysterious and requires Yale MBA graduates. But here at Perkins we’ve been doing this work with clients since 1997. We believe that organizations need to build their information usage capacity internally over time by developing the analytical muscle they already have.

There are some simple questions that can identify analytical candidates that already work in the business. Putting those candidates through a guided development process to build their vocabulary and communications capabilities will produce analysts whose skills and background are aligned with the business. They’ll already have the relationships and communication channels. They’ll have the muscle in the organization to develop insights and also drive decisions and outcomes from those insights.

The successful combination is to use consultants on project work while continuing to build internal strategic alignment, confidence in information, communication skills and “insight to decisions” processes. Building on an  motivated and educated internal staff will truly make information powerful.

The Harvard Business Review had an excellent blog post this week on how to “Make Data Work Throughout Your Organization". One of the points they made was that in order to put data to work it has to be clearly defined and have high quality. We couldn’t agree more. The challenge we see is that heart of the information quality issue is beyond simple technical measurements of data quality. For example, we frequently speak with our clients about technical aspects such as uniqueness, relational quality and domain integrity.

But there can be no quality without the ability for humans to communicate about it. If an organization lacks the vocabulary, process and experience to communicate about basic aspects of information such as uniqueness, hierarchies and sets – can they ever really have trust in their information? If two people cannot quickly and efficiently determine that they are both talking about the same set of customers are they prepared to have a conversation about customer segmentation?

In our Making Information Powerful approach we believe that we have to teach more than just data manipulation technology. We have to provide a language and a vocabulary. Only with an ability to communicate can we achieve information quality.

Would you believe that a company that prided itself on customer service would have only the bare minimum of information on customers in its data warehouse? Would a company that focused on wide product selection not capture all of the available attributes of its products?

Every organization has a unique strategy, a unique answer to the question – why do we exist? We believe that our most successful clients use information in a way that reinforces that unique strength. They do this by having more than just the same generic data from their business systems that all their competitors have. They’ve supplemented that with measures and dimensional information that applies directly to their unique strategy.

One of our clients is a retailer that focuses on customer service and the expertise of their staff. Their data warehouse therefore contains not just the names and current positions of each staff member but also their position history over time, the number of touches they’ve had with customers and the detailed information on their activities on the floor and in the stock room.

This gives them more than just a minimum set of data from their business systems coupled with some shiny business intelligence software tools. They have a unique and powerful perspective on their unique strategy and how well they are executing on that strategy.

The dream of business forecasting would be to find a single easily accessible indicator that would predict your company’s future business demand with perfect accuracy. In the last few years we’ve been able to get more prospect behavior data than ever. Examples include whether email campaigns were actually opened and patterns of customer behavior on websites. These are certainly great leading indicators of potential customer interest in products or services.

But what if we could get these same indicators of customer interest without the effort and expense of promotional campaigns? A recent article explained how using Google Correlate helped an auto manufacturer create a forecast that was 18% more accurate. This toolset allows anyone to analyze the correlation between web searches for particular words or phrases and their business performance. This in turn can be used as an input to a more accurate fitted model for forecasting.

Our best demand forecasting models include broad indicators like Google Correlate as well as more geographically specific indicators. This combined with sophisticated statistical models of historical performance to capture trends and seasonality let us get closer and closer to the dream of business forecasting.

The Economist magazine recently published a fun article about reader suggested ideas for new leading indicators of the state of the economy. Their readers suggested ideas ranging from the use of veterinary services—which fall in a down economy--to the sales of stress related healthcare products such as antacids—which rise.

At Perkins, we encourage our clients to think about leading indicators that are more directly related to their business. Traditionally, most organizations have looked at their sales pipeline as their key leading indicator, but our clients have found that thinking outside the box leads them to valuable insights at different time horizons. We’ve helped them look at demographic change data by Metropolitan Statistical Area (MSA), local weather forecasts and building permit counts--all useful leading indicators at relatively short time horizons.

Another example might be to use easily accessible public data to track specific financial indicators for a basket of public stocks representing your customers. Seeing their inventories go down while their sales go up might be a valid leading indicator that they will need to place additional orders. As always, the best way to build demand models is by doing low-cost testing. At Perkins we are always testing for better leading indicators of unconstrained demand for our clients products and services.