Arthur Diamond comments that "it is not clear how a donor distributes money through Hanson's market". Let me try again to be clear. Imagine David Levy were to seek funding for the regression he suggests in his comments, on the relative impact of sports versus science spending on aggregate productivity. Consider what might happen under three different funding institutions.
First, what might happen today, under proposal peer review? Levy would write a proposal describing how he might investigate the relative impact of sports versus science spending on aggregate productivity. Levy would then submit his proposal to some science patron, who would forward it to a community of experts for review, experts carefully selected and monitored by that patron. The patron would also monitor Levy if his proposal were funded.
Next, what might happen if prizes dominated science funding? Levy might suggest to a science patron that some prize amount be awarded for the first "significant" statistical evidence on sports versus science productivity. (As a reward, Levy might be paid directly for his suggestion.) If Levy then did the regression that he proposed, he might win this prize.
As with peer review, the patron would have to select and monitor experts, who would now approve prize proposals and decide prize winners. However, if research quality is clearer in hindsight than in foresight, then the patron needn't be as careful in selecting and monitoring these experts, and the patron needn't monitor prize competitors at all. Also, if no one ever claimed the prize, the patron would lose only costs incurred to offer the prize.
Finally, what might happen under idea futures? Levy might suggest to a science patron that she fund betting markets on actual future productivity, conditional on actual future spending on sports and science. The patron would again need to select and monitor experts, this time to approve funding suggestions and to declare final results (such as actual sports spending). A "donor distributes money" by creating automated market-makers, who always offer to make conditional trades at prices monotonic in their current inventory. Market prices would then offer a direct consensus estimate of the conditional probability of productivity given sports and science spending.
If Levy did his regression, and found his results to be at odds with current market prices, he could trade with this market maker, and expect to profit even if no one else ever traded in these markets. Others would always have the option to trade, and to share in the profits, but their participation would not be required. Thus the market-maker bankroll can be thought of as an "information prize",l which is awarded to those who arrive first with relevant information on some question, and which can be flexibly divided among many prize-winners.
As a kind of prize, idea futures share many of the advantages and disadvantages of prizes as a science funding mechanism. Like prizes, idea futures require less expertise and monitoring by patrons, but require more up-front commitment about what will be considered to be a valuable research contribution. In fact, most of the objections raised against idea futures are actually objections against all other types of science prizes.
Viewing idea futures as information prizes, I think that we are less tempted to evaluate their feasibility by struggling, as Stephen Dukes does, to map standard rules of thumb regarding when commodity futures markets will exist onto idea futures. It need not be asked, for example, if a reliable clearinghouse "on the scale of the Chicago Board of Trade" could operate profitably given unstandardized con- tracts "as numerous as assistant professors struggling to get tenure". The funding process described above (and in my paper) directly creates a reliable clearing-house for each contract. These markets would not be on the scale of the largest markets we know, but markets need not be huge to be feasible and useful.
The relevant comparison, I think, is between idea futures and other ways of funding any given kind of research, not between idea futures and corn futures. It is likely, as Dukes claims, that idea futures contracts would be less standardized, have higher transaction costs, and require traders to make do with less standardized data. However, this just confirms that research is a more specialized line of work than most. If a prize (or payment upon delivery) for getting your lawn mowed was compared to an ordinary prize for getting some specific research result, less standardized contracts in research would similarly be expected. This does not mean that ordinary prizes fail for research.
The question is not whether idea futures could "work"; I am confident that information prizes could be offered, sometimes be won, and sometimes induce relevant research. The real question is whether they can be typically cost-effective relative to other funding methods, i.e. whether the benefits of better incentives would outweigh any extra overhead costs for researchers to phrase and judge questions, keep track of current prices, etc. I admit that this is an open question.
Dukes worries about the "lack of contemporaneity between the inception of the idea and the profitability" of rewards, and Diamond worries about the uncertainty of the reward time. However, long and uncertain times until clear payoffs are just a fact of research life, so there is a risk that someone must bear under any funding system. At one extreme, researchers directly on the patron payroll leave almost all the risk to their patrons. Such researchers would have the worst incentives. At the other extreme, patrons could make research laboratories take on the most risk by using prizes based on fundamental milestones. Patrons would be relatively safe, paying only for clear progress.
Since there is a fundamental tradeoff in incentive contracts between risk-sharing and the quality of incentives, and since large research laboratories with access to capital markets can plausibly bear maximal research risk at low cost, I lean toward exploring this option. Idea futures and other types of prizes can also operate be- tween the extremes, if patrons base rewards on more frequent but less fundamental research milestones.
I disagree with Diamond that my proposal requires that "donors themselves must have ... good judgement about which of the important problems is currently solvable", a skill that is now the "hallmark of the good scientist". Donors can reasonably take a shot-gun approach and fund prizes on all the important problems that they can think of, regardless of which are currently solvable. Little is lost if a prize is declared but never won.
Diamond also suggests that "a shrewd scientist ... would want to place his bets slowly and quietly" so as not "to reveal his hand to ... technical analysts who look at trends in betting", and thus suggests that the problem of secretive researchers is worse than I imagine. However, it is a simple matter to allow people to make arbitrarily large trades with the funding market makers, at no penalty relative to a series of small trades. Thus if a scientist is content to take profits only from the patron, rather than from other foolish traders, he need not place himself at such risk.
Diamond asks "are not the scientists with the highest salaries ... usually just the same privileged elite whom Hanson is criticizing as having too much power in the present system? Wouldn't they still have the most resources to influence the odds?" It is not at all clear that the scientists who now make the most would make the most in an alternative funding system. I have no complaint about unequal power per se, but I am concerned that wages be related to true productivity. I agree with Diamond that a demonstration market is a logical next step; please contact me if you know of a sympathetic patron.
I am glad that Levy took the time to stress something I mention "only in passing: the relation between gambling and the interest of spectators". Science gambling could indeed increase popular interest in science, and thereby increase resources devoted to science. In truth, the potential for broader participation in our quest to understand the mysteries of nature and life is probably what excites me most about idea futures.
Will I be viewed as a "naive pre-Kuhnian by the jaded constructivists of much modern sociology of science", as Diamond suggests? I hope not. I strongly applaud constructivist reluctance to simply accept scientist's self-serving stories about the scientific process. Not only has this skepticism enabled exciting insights into the social processes of science, but I hope that it can translate into skepticism regarding scientist's claims about the optimality of their existing institutions.
I do postulate a fairly robust long-term convergence of opinion across widely varying social contexts, a convergence that some may deny exists. However, I have not heard constructivists claim that their view would be "disproved" if such a convergence were observed. Constructivist sociology is largely silent on the normative question of what we might want from science institutions. I have not heard constructivists claim that the process of social construction of science is bad per se, and thus my goal of speeding up the process is not anti-constructivist per se. I have heard constructivists advocate broader participation, but this goal I heartily endorse.
Diamond says that "the odds of Hanson's scheme working will be increased if the propositions in the science market are as little theory-laden as possible", i.e. if "scientific advance ... does not ... render old hypotheses meaningless". I agree that idea futures would work more smoothly if more old hypotheses retain meaning. However, I again want to caution that the relevant comparison is between different funding methods; perhaps meaning changes make all funding methods more difficult.
Thanks to Levy, Diamond, and Dukes for their thoughtful comments, and to Steve Fuller for providing this forum.