5. Hoji 2015
5.1. Introduction
　　As noted above, the concerns addressed in the papers collected in this volume and subsequent research have led to the methodological proposal in Hoji 2015, which explore how we can aspire to accumulate knowledge about the language faculty in line with Feynman's statement "The test of all knowledge is experiment," as noted above. The two pillars of the proposed methodology for language faculty science are the internalist approach advocated by Chomsky and what Feynman calls the "Guess-Compute-Compare" method. Taking the internalist approach, the book is concerned with the I-language of an individual speaker. Adopting the Guess-Compute-Compare method, it aims at deducing definite predictions and comparing them with experimental results.
　　It is hypothesized, in Chomsky 1986 among many other places, that the language faculty in its initial state is uniform across the members of the species and that, in its steady state, where its non-trivial "growth" has stopped, it varies in accordance with one’s linguistic experience, within the limit imposed by the genetic endowment. Given this, it follows that our hypotheses about the language faculty are of two types: one is about its initial state and the other is about its steady state. The initial state of the language faculty is uniform across the members of the species; hence, we refer to hypotheses about it as universal hypotheses. The steady state of the language faculty varies based on one's linguistic experience, as just noted, but it is hypothesized that most of the properties of its initial state remain unchanged in the steady state. Hypotheses about the steady state of the language faculty of an individual speaker must therefore consist of universal hypotheses and hypotheses about the particular consequences of the linguistic maturation that the individual has undergone. If we grossly simplify and assume, as is done in Hoji 2015 without addressing the issue, that a group of speakers of a particular language have undergone the same linguistic maturation, we can call the latter type of hypotheses language-particular hypotheses.
　　The object of inquiry in language faculty science is the language faculty. The language faculty, however, is not directly observable. Not only that, the language faculty as an independent module of the mind is itself a hypothesized concept/object. Language faculty science thus aspires to find out about a hypothesized object by putting forth hypotheses about this hypothesized object. As remarked in Hoji 2015: p. 5, this makes language faculty science "an extreme case of a theory-laden research program even at its very early stage of development."
　　The major challenge we face is, therefore, how to attain rigorous testability when dealing with something that is not directly observable. Among the crucial issues is what can be regarded as being revealing about the properties of the language faculty. It should be something that we can deduce as a definite prediction and identify in our experiments as being definite because otherwise we would not be able to compare it with our definite predictions.
　　Since, by hypothesis, the language faculty relates linguistic sounds and meaning, the most elementary form of an experiment in language faculty science seems to be such that the informant is asked whether a given sentence is acceptable under a specified interpretation. As remarked on Hoji 2015: 5, however, "One may wonder how we can make definite and categorical predictions about the judgment of an individual speaker of a particular language as a reflection of universal properties of the language faculty and how we can attain experimental results in accordance with such predictions." In Hoji 2015, I "provide answers to these and related questions and illustrate them by making reference to actual experiments." Hoji 2015 is thus an attempt to show how we can make language faculty science a rigorous empirical research program despite its inherently theory-laden nature. According to the proposed methodology, we check hard predictions with hard facts and state the hard facts in a theory-neutral way, although they are identified as such by being predicted by hypotheses. "Hard" in "hard predictions" and "hard fact" here is borrowed from Feynman (1999: 198–199):


　　In the strong nuclear interaction, we have this theory of colored quarks and gluons, very precise and completely stated, but with very few hard predictions. It’s technically very difficult to get a sharp test of the theory, and that's a challenge. I feel passionately that that’s a loose thread; while there's no evidence in conflict with the theory, we're not likely to make much progress until we can check hard predictions with hard numbers.


In other words, Hoji 2015 is "an attempt to show how we can deduce hard predictions and how we can identify hard facts in language faculty science." In summary, Hoji 2015 offers a conceptual articulation of how we deduce definite predictions about the judgments of an individual speaker on the basis of universal and language-particular hypotheses and how we obtain experimental results precisely in accordance with such predictions.

5.2. The key to deducing definite and categorical predictions
　　With regard to what should count as evidence for or against our hypotheses about properties of the language faculty, Hoji 2015 proposes that we should focus on what is predicted to be impossible and check whether we obtain informant judgments in line with such a prediction in a reproducible manner. It is argued in Hoji 2015: Chapters 2 and 3 that the key to deducing definite and categorical predictions about the informant judgment is the recognition of the fundamental asymmetry in [P-a] and [P-b].

[P]The fundamental schematic asymmetry
a.The *Schema-based prediction:
Every example sentence instantiating a *Schema is unacceptable with the specified interpretation pertaining to two expressions.
b.The okSchema-based prediction:
Some example sentences instantiating an okSchema are acceptable at least to some extent with the specified interpretation pertaining to two expressions.

[P-a] is a universal statement but [P-b] is an existential one. [P-a] can be disconfirmed but it cannot be confirmed while [P-b] cannot be disconfirmed but it can be confirmed.
　　Without recognizing this asymmetry, it would not be possible to deduce definite and categorical predictions about the informant judgment and expect them to be supported experimentally. According to Hoji 2015, definite and categorical predictions in language faculty science are about the complete unacceptability of example sentences that instantiate a *Schema, in contrast to those instantiating its corresponding okSchema.
　　The combination of a *Schema-based prediction [P-a] and its corresponding okSchema-based prediction [P-b] is called a predicted schematic asymmetry. When the *Schema-based prediction has survived a rigorous attempt at disconfirmation and the corresponding okSchema-based prediction has been confirmed, the reported judgments by the informants on the relevant *Examples and okExamples are said to constitute a confirmed predicted schematic asymmetry. It is suggested in Hoij 2015 that the confirmed predicted schematic asymmetry is the smallest unit of fact in language faculty science.

5.3. The key to obtaining definite and categorical experimental results as predicted
　　As discussed in Hoji 2015: Chapter 4, the key to obtaining definite and categorical experimental results in accordance with our predictions (in the form of predicted schematic asymmetries) is a clear understanding of the structure of our prediction-deduction, i.e., what universal and language-particular hypotheses give rise to the predictions in question.
　　The relevant considerations have led to the recognition in Hoji 2015 that an experiment in language faculty science must consist of a Main-Experiment and its Sub-Experiment(s). We must also clearly understand what the informant's reported judgments mean for the validity of each of the hypotheses that have given rise to the prediction in question. A Main-Experiment tests for each informant the validity of the Main-Hypotheses of a predicted schematic asymmetry. Sub-Experiments test for each informant (i) the validity of Sub-Hypotheses of a predicted schematic asymmetry and/or (ii) the reliability of the design of the Main-Experiment such as how we convey the intended dependency interpretation to our informants.
　　In order to effectively assess the validity of the Main-Hypotheses tested in the Main-Experiment, it is necessary to interpret its results by focusing on the informants whose judgments in the Main-Experiment are significant with regard to the validity of its Main-Hypotheses, i.e., those (i) for whom the Sub-Hypotheses of the predicted schematic asymmetry seem valid and (ii) who clearly understand the instructions, including the intended dependency interpretation.Our predictions are not about any informant. It is about those informants who are deemed reliable for the purpose of testing the Main-Hypothesis/ses in the Main-Experiment. Crucial reference to the results of Sub-Experiments is for the purpose of making the result of the Main-Experiment as significant as possible with respect to the validity of the Main-Hypotheses tested in the Main-Experiment, and that is analogous to enhancing the reliability and the precision of the experimental device in a physical science. What has led us to recognize Main-Hypotheses and Sub-Hypotheses as well as Main-Experiments and Sub-Experiments is the desire to be able to focus on the validity of (a) particular hypothesis/ses among those that give rise to the predicted schematic asymmetry. It stems from our desire to assign maximal significance to our experimental result with respect to such (a) hypothesis/ses. We want our experimental result to be as significant as possible. This is regardless of whether it turns out to be in line with our definite and categorical prediction.
　　The key to obtaining definite and categorical experimental results is thus ensuring the reliability of the experimental device as much as possible. It is imperative that we pay close attention to the effectiveness and the precision of the experimental device in language faculty science, just as it is imperative to do so in a physical science. Unlike a physical science, however, we do not, at least at the moment, have a physical experimental device. We cannot, therefore, check the reliability of the design, construction and operation of a physical experimental device. What then is an experimental device in language faculty science? It seems reasonable to consider our informants and our instructions to be part of our experimental device in language faculty science. Once we recognize this, it follows that we must pay close attention to the reliability and the effectiveness of our informants and our instructions.
　　What Hoji 2015 suggests is as follows: We can consider the result of our Main-Experiment revealing about the validity of its Main-Hypotheses only if we focus on the informants for whom the instructions are effective and for whom the Sub-Hypotheses seem valid, judging from the results of the Sub-Experiments. Interpreting the result of the Main-Experiment without reference to those of its Sub-Experiments would be like conducting experiments without taking necessary care and without necessary checks; see the Feynman quote given in (11).
　　As noted above, unless we use certain types of expressions for A and B in BVA(A, B), we cannot expect to obtain robust informant judgments as indicated in (1). This is as expected if there are more than one source of BVA(A, B), and the choice of A and B affects the possibility of the BVA(A, B) of different sources, as discussed in Paper 1 and more in depth in Ueyama 1998. Likewise, as discussed in Papers 2-6, there are more than one sources of the sloppy-identity reading, and the relevant lexical choice affects how the sloppy-identity reading can arise.
　　As also noted above, it seems reasonable to understand Hoji 1985 as an attempt to identify the informant intuitions that are necessarily based on the satisfaction of the c-command condition. One might suggest that Papers 1-6 were concerned with the nature of BVA and the sloppy-identity reading that are based on LF c-command, and especially with the identification of the expressions whose use necessarily results in the BVA or the sloppy-identity reading that is based on LF c-command.
　　We can regard Papers 1-7 as research that is concerned with analyzing linguistic phenomena in terms of theoretical concepts such as LF c-command, anti-locality, and the lexical property in question. We can take the conditions noted above to be on BVA and the sloppy-identity reading, or more strictly, on the type of BVA and on the type of sloppy-identity reading whose availability is contingent upon the satisfaction of those conditions.
　　As indicated in Papers 1-7, however, I was actually pursuing the possibility that those conditions are on a theoretical/hypothesized/formal object (FD), rather than on BVA or the sloppy-identity reading. I was not fully aware that I was doing so while preparing Papers 1-7. But while preparing Hoji 2015 I have come to a clear understanding that I am investigating the properties of FD (and eventually, what underlies FD and other theoretical/hypothesized/formal objects like FD). Close examination of BVA and the sloppy-identity reading is for the purpose of finding out about FD. As it has in fact turned out, the particular choice of A and B for BVA(A, B) does not necessarily result in robust judgments for every speaker although it does for most speakers, and there are judgmental fluctuations among speakers and even within a single speaker. If linguistic phenomena are our object of inquiry, so to speak, it seems impossible to deduce definite predictions about the individual informant's judgments about the phenomena in question and expect them to be supported experimentally.

5.4. Pursuing rigorous testability and identifying facts in language faculty science
　　Through my research subsequent to Hoji 1985, I have come to think that much of the research in the field of generative grammar does not pursue rigorous testability. This seems to me to have resulted over the years in the general lack of a clear sense of progress in the field. I had thought for some time that such a state of affairs was due to the lack of intellectual rigor on the part of the practitioners, including myself. Upon reading Feynman's "Cargo Cult Science" several years ago (included in Feynman 1985), however, I came to think that one of the reasons for what one might call the absence of intellectual rigor and integrity in question is that we do not have a means to determine what the facts are. If we do not know what the facts are, it may not be entirely clear how to be honest and how not to fool ourselves; see the Feynman quotes given in (4) and (5).
　　I provide some quotations of Feynman's remarks here in hopes that they might give the reader a general idea about the intended points. For a fuller discussion, the readers are referred to Hoji 2015.

(4)　　"Now it behooves me, of course, to tell you what they're missing. But it would be just about as difficult to explain to the South Sea islanders how they have to arrange things so that they get some wealth in their system. It is not something simple like telling them how to improve the shapes of the earphones. But there is one feature I notice that is generally missing in cargo cult science. That is the idea that we all hope you have learned in studying science in school―we never say explicitly what this is, but just hope that you catch on by all the examples of scientific investigation. It is interesting, therefore, to bring it out now and speak of it explicitly. It's a kind of scientific integrity, a principle of scientific thought that corresponds to a kind of utter honesty―a kind of leaning over backwards. For example, if you're doing an experiment, you should report everything that you think might make it invalid―not only what you think is right about it: other causes that could possibly explain your results; and things you thought of that you've eliminated by some other experiment, and how they worked―to make sure the other fellow can tell they have been eliminated.

　　Details that could throw doubt on your interpretation must be given, if you know them. You must do the best you can―if you know anything at all wrong, or possibly wrong―to explain it. If you make a theory, for example, and advertise it, or put it out, then you must also put down all the facts that disagree with it, as well as those that agree with it. There is also a more subtle problem. When you have put a lot of ideas together to make an elaborate theory, you want to make sure, when explaining what it fits, that those things it fits are not just the things that gave you the idea for the theory; but that the finished theory makes something else come out right, in addition." (From "Cargo Cult Science," included in Feynman 1985 Surely You're Joking Mr. Feynman ) (p. 340-341).


(5)　　"The only way to have real success in science, the field I'm familiar with, is to describe the evidence very carefully without regard to the way you feel it should be. If you have a theory, you must try to explain what's good and what's bad about it equally. In science, you learn a kind of scientific integrity and honesty.
In other fields, such as business, it's different. For example, almost every advertisement you see is obviously designed, in some way or another, to fool the customer: the print that they don't want you to read is small; the statement are written in an obscure way. It is obvious to anybody that the product is not being presented in a scientific and balanced way. Therefore, in the selling business, there's a lack of integrity." (Feynman 1988: 217-218)


(6)　　"In the strong nuclear interaction, we have this theory of colored quarks and gluons, very precise and completely stated, but with very few hard predictions. It’s technically very difficult to get a sharp test of the theory, and that’s a challenge. I feel passionately that that’s a loose thread; while there’s no evidence in conflict with the theory, we’re not likely to make much progress until we can check hard predictions with hard numbers." (Feynman 1999: 199)


(7)　　"Another thing I must point out is that you cannot prove a vague theory wrong. If the guess that you make is poorly expressed and rather vague, and the method that you use for figuring out the consequences is a little vague―you are not sure, and you say, “I think everything’s right because it’s all due to so and so, and such and such do this and that more or less, and I can sort of explain how this works …”, then you see that this theory is good, because it cannot be proved wrong! Also if the process of computing the consequences is indefinite, then with a little skill any experimental results can be made to look like the expected consequences." (Feynman 1965/94: 152–153)


(8)　　"The principle of science, the definition, almost, is the following: The test of all knowledge is experiment. Experiment is the sole judge of scientific ‘truth’." (The Feynman Lectures on Physics: 1-1, reproduced in Feynman 1963: 2).


(9)　　"In general, we look for a new law by the following process. First we guess it. Then we compute the consequences of the guess to see what would be implied if this law that we guessed is right. Then we compare the result of the computation to nature, with experiment or experience, compare it directly with observation, to see if it works. If it disagrees with experiment, it is wrong. In that simple statement is the key to science. It does not make any difference how beautiful your guess is. It does not make any difference how smart you are, who made the guess, or what his name is―if it disagrees with the experiment, it is wrong. That’s all there is to it." (Feynman 1965/94: 150)


(10)　　"It is true that one has to check a little to make sure that it is wrong, because whoever did the experiment may have reported incorrectly, or there may have been some feature in the experiment that was not noticed, some dirt or something; or the man who computed the consequences, even though it may have been the one who made the guesses, could have made some mistake in the analysis. These are obvious remarks, so when I say if it disagrees with experiment it is wrong, I mean after the experiment has been checked, the calculations have been checked, and the thing has been rubbed back and forth a few times to make sure that the consequences are logical consequences from the guess, and that in fact it disagrees with a very carefully checked experiment." (Feynman 1965/94: 150–151)


(11)　　"Because of the success of science, there is, I think, a kind of pseudoscience. Social science is an example of a science which is not a science; they don’t do [things] scientifically, they follow the forms―or you gather data, you do so-and-so and so forth but they don’t get any laws, they haven’t found out anything. They haven’t got anywhere yet―maybe someday they will, but it is not very well developed … I may be quite wrong, maybe they do know all these things, but I don’t think I’m wrong. You see, I have the advantage of having found out how hard it is to get to really know something, how careful you have to be about checking the experiments, how easy it is to make mistakes and fool yourself. I know what it means to know something, and therefore I see how they get their information and I can’t believe that they know it, they haven’t done the work necessary, haven’t done the checks necessary, haven’t done the care necessary. I have a great suspicion that they don’t know, that this stuff is [wrong] and they’re intimidating people. I think so. I don’t know the world very well but that’s what I think." (Feynman 1999: 22)


　　The language faculty is our object of inquiry. But it is what we hypothesize to be underlying our ability to relate linguistic sounds and meaning. The fact that the language faculty, our object of inquiry, is a hypothesized object makes language faculty science an extreme case of a theory-laden research program, even at the earliest stages of its development, as pointed out in Hoji 2015: Chapter 1. One of the concrete proposals in Hoji 2015 is about how to identify facts in a research program that aims at discovering properties of the language faculty by following Feynman's "Guess-Compute-Compare" method. In other words, Hoji 2015 proposes how we can pursue rigorous testability and reproducibility in language faculty science, despite its highly theory-laden nature.