In a recent exchange in the Fed KM Google group, one of my correspondents thought that it would be very helpful to a National KM Center to have access to benchmarking-derived “lessons learned,” “best practices,” and “knowledge architecture” from APQC. In reply, I indicated my agreement provided that these ideas were construed appropriately, but then I also expressed doubt about whether APQC excelled in measurement theory and methodology, or whether they had a very good knowledge processing framework on which to base their indicators. And I also expressed doubts about the adequacy of their measures of impact.
My correspondent replied by saying that APQC’s practices and methodologies “are adequate for the job APQC performs as a benchmarking and consulting service provider,” and then suggested that they ought to be evaluated “. . . against the standard appropriate to their business . . .” This reply seems fair and reasonable, but only if one adopts a relative standard of excellence based on APQC’s acceptance in the market place, rather than a standard based on KM’s needs and requirements. Since APQC is in the end driven by its benchmarking of existing Best Practices, its methodological approaches appear to be limited to what already exists in KM, is being practiced somewhere, and has also been made available to APQC benchmarking activities.
This is, if you will, an internal, industry and market-based approach to methodology; an approach that confounds what is, with what ought to be. But the issue I raised earlier, is the issue of the adequacy or validity of APQC’s approach as a guide to measuring impact, and not the issue of whether its practices are adequate for its consulting goals, or for industry or market place success. Best practices and lessons learned from benchmarking are not necessarily the best practices and lessons learned we need to do a good job of measuring KM impact. My conjecture about the inadequacy of APQC measurements, methodologies, and conceptual frameworks might well be correct, regardless of the quality of their benchmarking, if impact measurement best practices within the KM industry are themselves inadequate. Based on my own very long experience in the social sciences, and exposure to many different kinds of measurement methodologies, and research on measurement practices in use within KM, I believe that these practices are inadequate. As long as APQC’s methodologies reflect only the best of these, they too will not meet the need of Federal KM to measure the impact of KM interventions in such a way that KM programs can be accountable. More is necessary than simply the best that can now be found in industry.
There’s an old saying that I ran into years ago at, of all places, the US Census Bureau, which says: “It’s good enough for Government work.” This is a saying that the Government needs badly to eliminate from American business folklore. The only way to do that is to make Government work so high in quality that it often leads work done in the private sector. To have that happen in KM, we have to do a better job of measuring KM impact than we currently find in private sector best practices, and that will require better measurement methodology than is currently being practiced there.