This initiative examined systematically the extent to which a large set of archival research findings generalizes across contexts. We repeated the key analyses for 29 original strategic management effects in the same context (direct reproduction) as well as in 52 novel time periods and geographies; 45% of the reproductions returned results matching the original reports together with 55% of tests in different spans of years and 40% of tests in novel geographies. Some original findings were associated with multiple new tests. Reproducibility was the best predictor of generalizability—for the findings that proved directly reproducible, 84% emerged in other available time periods and 57% emerged in other geographies. Overall, only limited empirical evidence emerged for context sensitivity. In a forecasting survey, independent scientists were able to anticipate which effects would find support in tests in new samples.
Under general conditions, the asymptotic distribution of degenerate second-order $U$- and $V$-statistics is an (infinite) weighted sum of $\chi^2$ random variables whose weights are the eigenvalues of an integral operator associated with the kernel of the statistic. Also the behavior of the statistic in terms of power can be characterized through the eigenvalues and the eigenfunctions of the same integral operator. No general algorithm seems to be available to compute these quantities starting from the kernel of the statistic. An algorithm is proposed to approximate (as precisely as needed) the asymptotic distribution and the power of the test statistics, and to build several measures of performance for tests based on $U$- and $V$-statistics. The algorithm uses the Wielandt–Nyström method of approximation of an integral operator based on quadrature, and can be used with several methods of numerical integration. An extensive numerical study shows that the Wielandt–Nyström method based on Clenshaw–Curtis quadrature performs very well both for the eigenvalues and the eigenfunctions.