When you have put a lot of ideas together to make an elaborate theory, you want to make sure, when explaining what it fits, that those things it fits are not just the things that gave you the idea for the theory; but that the finished theory makes something else come out right, in addition.

Well, at least, we want to make sure that the finished theory makes a testable prediction beyond the things that gave you the idea for the theory. (That property sort of corresponds to 'being theoretically progressive' in Lakatos' discussion of problemshifts, as discussed in chapters 3 and 5 of my book draft. If the prediction turns out to be correct, i.e., if the finished theory indeed makes something else come out right, in addition, that would sort of corresponds to 'being empirically progressive'.) If not, the 'finished theory' is just a description of the things that gave you the idea for the theory. This is the point that seems to be understood rather poorly in generative grammar. But come to think of it, that is understandable, given what seems to me to be the general failure of the field to recognize the significance of establishing repeatable phenomena and proceeding on the basis of them.

[This posting is in relation to [35922]. The old URL given there has since been replaced by a new one. Furthermore, the URL given below does not seem to work as of 7/2/2014. But I am keeping this posting just to keep the point that there seem to be different versions of "Cargo Cult Science" available on-line. 7/2/2014.]
http://www.columbia.edu/itc/applied/wiggins/Classes/E4903/Fall2003/cargo.pdf
"Cargo Cult Science"
Richard Feynman
From a Caltech commencement address given in 1974

If you go to:

http://www.kyb.tuebingen.mpg.de/bethge/reading/Feynman_1974.pdf

you have a slightly more expanded, or slightly less shortened, version of his 1974 address than what is given at the first URL given above. Perhaps the version at the first URL is what has appeared in the book Surely You're Joking, Mr. Feynman! and what is given at the second URL is what appeared originally in "Engineering and Science" (June 1974), which might have been (or might be) a Cal-Tech-internal publication.

I highly recommend this; it is only 5 pages.
If you are a linguist, it may be an interesting exercise to ask yourself how much of what Feynman says a good scientist should do you are trying to practice in your own research. When you read my book, you can also ask how much of what Feynman has noted you think I have tried to do in my research as reported in my book.

[Revised on 3/28/3014]

I find Feynman on Social Sciences very interesting. You seem to have to open the video in a New Window. The video was last accessed on 3/28/2014. It is only less than 2 minutes.

If it does not work, you can search "Feynman on Social Sciences" in YouTube.

[Modified on 7/2/2014]
The URL given in [35922] works as of 7/2/2014.

This link also works as of 7/2/2014. As noted in [35922], [y]ou can get to its content at various places. Just Google "Cargo Cult Science."

Although the chapter has not been written in that way so far, it seems worthwhile to try to write the introductory chapter of my book making reference to chapter one of Chomsky's Aspects. Much of Chomsky 1965: chap. 1, especially, the discussion in sections 1-4, is directly related to what is addressed in my book. Introductory remarks, if successfully written, will make it clear that the work presented in my book is a direct continuation of Chomsky 1965, at least in terms of the methodological concerns expressed there. It will also be useful, I think, to clarify how the shift from the Aspects model (and more in general the model that has Phrase Structure Rules) to the model with Merge affects the methodological discussion in Chomsky 1965: chap. 1 in terms of what must have been intended and how it is actually expressed there. The issue is addressed briefly in A. Ueyama's forthcoming paper in the Proceedings of the JK Conference last year, which can be downloaded in the "Works by Ayumi Ueyama" page.

Chomsky (1965: 4) remarks that "linguistic theory is mentalistic, since it is concerned with discovering a mental reality underlying behavior" and in note 1 (p. 193) appended there states that "[m]entalistic linguistics is simply theoretical linguistics that uses performance as data (along with other data, for example, the data provided by introspection) for determination of competence, the latter being taken as the primary object of its investigation." The research reported in this book is mentalistic linguistics in the above sense, and the book's aim is to put forth and defend a means to determine, with some confidence, when to take a certain set of informant judgments as a reflection of properties of the Computational System.

[Here is a somewhat more expanded version of [34795].]

Chomsky (1965: 4) remarks that "linguistic theory is mentalistic, since it is concerned with discovering a mental reality underlying behavior" and in note 1 (p. 193) appended there states that "[m]entalistic linguistics is simply theoretical linguistics that uses performance as data (along with other data, for example, the data provided by introspection) for determination of competence, the latter being taken as the primary object of its investigation." The research reported in this book is mentalistic linguistics in the above sense, and the book's aim is to put forth and defend a means to determine, with some confidence, when to take a certain set of informant judgments as a reflection of properties of the Computational System, hypothesized to be at the center of the language faculty. The book addresses how we can approach, and formulate our hypotheses, about the Computational System in a way empirically testable, presents a conceptual basis for its proposal and provides empirical illustration of the methodological proposals.
Among the general theses put forth and defended in this book are:

--The empirical justification for our hypotheses about the Computational System as to whether and in what way they are linked to native speakers' linguistic intuitions.^{Note 1}
--It is not a priori clear what linguistic intuitions are likely to be a reflection of, and hence revealing about, properties of the Computational System; given our goal, identifying such linguistic intuitions is in fact part of the task of the researcher.
--There is a way to identify such linguistic intuitions.
--Once we adopt a means to identify such linguistic intuitions, we can capitalize on it in evaluating the empirical consequences of our hypotheses about the Computational System.

One of the main methodological points of the book is its claim that it is possible to, and we in fact should proceed so as to be able to, learn something about the Computational System from the disconfirmation of a certain type of predictions.

Note 1: Cf. Einstein and Infeld's (1938: 294) "Science is not just a collection of laws, a catalogue of unrelated facts. It is a creation of the human mind, with its freely invented ideas and concepts. Physical theories try to form a picture of reality and to establish its connection with the wide world of sense impressions. Thus the only justification for our mental structures is whether and in what way our theories form such a link." (in 'Physics and Reality', the last section of The Evolution of Physics by Einstein and Infeld, originally published in 1938. The page reference is to the 2007 edition, Simon & Schuster, New York.) See also "Summary" of "Physics and Reality" in Einstein 1936/1954, p. 322. Bridging statements, to be introduced in chapter 3, are none other than an attempt to link our hypotheses about the properties of the Computational System (CS) with the 'sense impressions' as hypothesized to arise from the postulated properties of the CS.

The research reported in this book is mentalistic linguistics in the above sense, and the book's aim is to put forth and defend a means to determine, with some confidence, when to take a certain set of informant judgments as a reflection of properties of the Computational System, hypothesized to be at the center of the language faculty.

It may be important to note that while it is often stated that one of the goals of generative grammar is to characterize and predict "the possible (actual and potential) occurrence of all and only the grammatical sentences of a given language," such is not a concern of the present work as it is concerned with the language faculty (more precisely, the Computational System) rather than with language (more precisely a particular language or another) although we do draw from informant judgments on sentences of a particular language.

The research reported in this book is mentalistic linguistics in the above sense, and the book's aim is to put forth and defend a means to determine, with some confidence, when to take a certain set of informant judgments as a reflection of properties of the Computational System, hypothesized to be at the center of the language faculty.

It may be important to note that while it is often stated that one of the goals of generative grammar is to characterize and predict "the possible (actual and potential) occurrence of all and only the grammatical sentences of a given language," such is not a concern of the present work as it is concerned with the language faculty (more precisely, the Computational System) rather than with language (more precisely a particular language or another) although we do draw from informant judgments on sentences of a particular language.

In the terms of Chomsky 1986, I am concerned with I-language, not E-language. As Chomsky states here and there, it seems to be a consequence of some unintended historical "accident" (i.e., the way Syntactic Structures was prepared and published, "in place of" Logical Structure of Linguistic Theory) that the false conception of the goal of generative grammar as noted above spread. The field of generative grammar seems to me to have been (badly) influenced by this over the years.

While we emphasize the significance of across-informant repeatability, it should be made clear that how much across-informant repeatability obtains 'statistically' would in principle be of little significance if we did not obtain within-informant repeatability, as will be discussed in chapter 3. We will consider in chapter 3 the idealized researcher-informant whose judgments would not be affected by any difficulty imposed by parsing or pragmatic considerations, very much along the lines of the idealization in Chomsky 1965: 3 in regard to "an ideal speaker-listener, in a completely homogeneous speech-community, who knows its language perfectly and is unaffected by such grammatically irrelevant conditions as memory limitations, distractions, shifts of attention and interest, and errors (random or characteristic) in applying his knowledge of the language in actual performance."

Chomsky 1995: 6-7 states as in (i).

(i) This way of formulating the issues, within the P&P model, brings out clearly a crucial inadequacy in the characterization of language as a state of the language faculty. The latter [which seems to refer to 'a state of the language faculty', HH] can hardly be expected to be an instantiation of the initial state with parameter values fixed. Rather, a state of the language faculty is some accidental product of varied experience, of no particular interest in itself, no more so than other collections of phenomena in the natural world (which is why scientists do experiments instead of recording what happens in natural circumstances). My personal feeling is that much more substantial idealization is required if we hope to understand the properties of the language faculty, but misunderstandings and confusion engendered even by limited idealization are so pervasive that it may not be useful to pursue the matter today. Idealization, it should be noted, is a misleading term for the only reasonable way to approach a grasp of reality. (The emphasis is as in the original.)

Chomsky 1995: 6-7 states as in (38).

(38)　　This way of formulating the issues, within the P&P model, brings out clearly a crucial inadequacy in the characterization of language as a state of the language faculty. The latter [I think this refers to 'a state of the language faculty', HH] can hardly be expected to be an instantiation of the initial state with parameter values fixed. Rather, a state of the language faculty is some accidental product of varied experience, of no particular interest in itself, no more so than other collections of phenomena in the natural world (which is why scientists do experiments instead of recording what happens in natural circumstances). My personal feeling is that much more substantial idealization is required if we hope to understand the properties of the language faculty, but misunderstandings and confusion engendered even by limited idealization are so pervasive that it may not be useful to pursue the matter today. Idealization, it should be noted, is a misleading term for the only reasonable way to approach a grasp of reality. (The emphasis is as in the original.)

What is meant by "this way of formulating the issues, within the P&P model" is the thesis expressed in (39).

(39) 　(Chomsky 1995: 6)
　a. 　Language differences and typology should be reducible to choice of values of parameters.
　　b. 　[The options in question] are restricted to formal features of functional categories...
　　c. 　In this context, language acquisition is interpreted as the process of fixing the parameters of the initial state in one of the permissible ways. A specific choice of parameter settings determines a language in the technical sense that concerns us here...

The view expressed above, in particular "much more substantial idealization is required if we hope to understand the properties of the language faculty," seems to be quite consistent with what we are pursuing. First all, "predict[ing] the possible (actual and potential) occurrence of all and only the grammatical sentences of a given language" is no longer "the minimum requirement' of a successful generative grammar, i.e., of a theory of the Computational System, or even of a 'theory' of a particular grammar if that is meant to be a combination of the Computational System and the mental Lexicon of the speaker of the language once we accept the conception of the goal of generative grammar and the methodological proposal about how we can assess our hypotheses about the Computational System. Once we have liberated ourselves from the 'minimum requirement', our research seems to be indeed making "much more substantial idealization" than, i.e., beyond, assuming "the idealized speaker-hearer," which we can understand is quite close, if not identical, to our "idealized informant-researcher." The notion of "the idealized speaker-hearer" is not addressed specifically with respect to experiments in Chomsky 1965, unlike "the idealized informant" in my book, and that is perhaps the only crucial difference between Chomsky's "idealized speaker-hearer" and my "idealized informant."
　　Our idealization can be seen, for example, in our answer to (40a) that it is a repeatable phenomenon, which can be understood, roughly, as a generalization whose defining characteristic is a contrast between total unacceptability and the lack thereof, rather than a mere 'statistically significant' contrast. (I will present the essential idea behind repeatable phenomena and illustrate the notion in chapter 2, and provide further articulation of the conceptual basis of repeatable phenomena and related issues in chapter 3.)

(40)　Two specific concerns stemming from (8):
a.What qualifies as data for research concerned with the properties of the Computational System?
b.How could we evaluate our hypotheses about the Computational System?

(8)　How should we proceed in order to ensure progress toward the goal of discovering the properties of the CS?


I might note, in relation to (41), that it indeed seems to be the case that interesting tests can be carried out only by involving a fairly involved experiment, as is illustrated in my book in some depth.

(41)　(Chomsky 1965: 21)
Perhaps the day will come when the kinds of data that we now can obtain in abundance will be insufficient to revolve deeper questions concerning the structure of language. However, many questions that can realistically and significantly be formulated today do not demand evidence of a kind that is unavailable or unattainable without significant improvements in objectivity of experimental technique.

Chomsky (1965: 4) states, "To study actual linguistic performance, we must consider the interaction of a variety of factors, of which the underlying competence of the speaker-hearer is only one. In this respect, study of language is no different from empirical investigation of other complex phenomena." This seems to be Chomsky's consistent position over the years. Chomsky (1988), for example, states as in (66) and Schütze (1996) reports as in (67).

(66)As for my own methods of investigation, I do not really have any. The only method of investigation is to look hard at a serious problem and try to get some ideas as to what might be the explanation for it, meanwhile keeping an open mind about all sorts of other possibilities. Well, that is not a method. It is just being reasonable, and so far as I know, that is the only way to deal with any problem, whether it is a problem in your work as a quantum physicist or whatever. (Chomsky 1988: 190)

(67)Chomsky (personal communication) believes that research practice in linguistics ought to follow that in the natural sciences, where (in contrast to the social sciences) "almost no one devotes attention to 'methodology'." Obviously, I disagree. (Schütze 1996: 210, note 1)

In chapter 5 of my book, I present my own view in relation to Chomsky's view stated above.

[Here is the table of contents, as of 11/30/2008.]

A Foundation of Generative Grammar as an Empirical Science
by Hajime Hoji

Chapter 1: Introduction
1. Introduction 1
2. Chomsky's (1965) challenge 4
3. The goal and the main claims 7
4. Outline of chapters 2-5 12
4.1. Chapter 2 12
4.2. Chapter 3 12
4.3. Chapter 4 13
4.4. Chapter 5 13
5. Answering Chomsky's challenge 14

Chapter 2: Repeatable Phenomena
1. Introduction 1
2. Repeatable Phenomena 2
2.1. *Schemas, *Examples, okSchemas, okExamples and repeatable phenomena 3
2.2. The asymmetry between*Schemas and okSchemas 8
2.3. Across-speaker repeatability and within-speaker repeatability 10
2.4. Repeatable phenomena, hypotheses, and progress in generative grammar 11
2.5. Summary 13
3. Some Illustration 14
3.1. Introduction 14
3.2. Hypotheses not backed up by a repeatable phenomenon in Japanese 16
3.2.1. Zibunzisin 16
3.2.1.1. Experimental design 17
3.2.1.2. Results 20
3.2.2. Otagai 22
3.2.2.1. Experimental design 23
3.2.2.2. Results 27
3.3. Repeatable Phenomena in Japanese 29
3.3.1. Introduction 29
3.3.2. So-NPs vs. A-NPs 29
3.3.2.1. Background: the demonstratives in Japanese 29
3.3.2.2. So-NPs vs. a-NPs 31
3.3.2.3. Experimental design 34
3.3.2.4. Results 36
3.3.3. Weak crossover, reconstruction and the OS Construction in Japanese 36
3.3.3.1. The Initial Observation 36
3.3.3.2. Two types of dependency in Ueyama 1998 38
3.3.3.3. *Schemas and an okSchema 41
3.3.3.4. More okSchemas based on the OS constructions in Japanese 42
3.3.3.5. LF-c-command-based BVA Paradigms 45
3.3.3.6. Results 47
3.3.3.7. Precedence-based BVA 50
3.3.3.8. Results 54
4. Concluding remarks 56

Chapter 4: Theory of Judgment Making and Its Consequences
1. Introduction 1
2. Theory of judgment making 1
2.1. Introduction 1
2.2. The model of judgment making and the CS 3
2.2.1. How the theory of the CS is embedded in the model of judgment making 3
2.2.2. Difficulty 5
2.2.3. The major subject construction 6
2.2.4. The OS (Object Subject) construction 7
2.2.5. Summary 9
2.3. The judgment by the informant 9
2.4. The informant's sensitivity and resourcefulness 10
2.4.1. String/meaning sensitivity 10
2.4.2. Structural/contextual resourcefulness 11
2.5. Summary 12
3. Some consequences 13
3.1. The asymmetry between *Schemas and okSchemas 13
3.2. Focusing on *Examples 14
3.3. Single and multiple-informant research 17
4. Bridging statements and predictions 18
4.1. Making hypotheses testable 18
4.1.1. Bridging statements 19
4.1.2. Bridging statements and the asymmetry between *Schemas and okSchemas 20
4.2. Predictions 22
4.2.1. Two types of predictions 22
4.2.2. Confirmation and disconfirmation of Predictions 24
4.2.3. Single and multiple-informant research 25
4.3. Summary 28
5. Repeatable phenomena and the significance of the informant judgment 29
5.1. The asymmetry between a *Schema and an okSchema 29
5.2. When do we obtain a repeatable phenomenon? 31
5.3. Single-informant and multiple-informant research 32
5.4. Summary 34
6. Predictions, auxiliary hypotheses and heuristics 35
6.1. Introduction 35
6.2. The model of prediction making 35
6.3. When a prediction fails 39
6.3.1. Auxiliary hypotheses 39
6.3.2. When a *Schema-based prediction gets disconfirmed 41
6.3.3. When an okSchema-based prediction fails to be confirmed 43
6.4. Heuristics 45
6.4.1. Progressive problemshift 45
6.4.2. Heuristics for single-informant research and multiple-informant research 46
6.4.3. Summary 48
7. Summary 49

Chapter 4: Empirical Illustration: Anaphoric Relations in Japanese
1. Introduction 1
2. The BVA (Bound variable anaphora) 1
2.1. The linguistic intuition BVA 1
2.2. The bridging statement 6
2.2.1. The Lexical condition 6
2.2.2. The Structural condition 7
2.3. The Model of prediction making 14
2.4. Modification 17
2.5. Further modification 23
2.6. Some methodological clarification 26
3. BVA and "Principle B" Effects 29
3.1. A Problem 29
3.2. Modification regarding pf-LF correspondences 34
3.3. Further modifications 38
3.3.1. A new repeatable phenomenon 38
3.3.2. An account 42
3.3.3. A new prediction 46
3.3.4. Further predictions 51
4. "Principle B" effects for coreference: a further prediction 53
5. The Hypotheses about zibunzisin and otagai, revisited 56
5.1. Eliminating a *Schema-based prediction and adding a new one 56
5.2. Attempts to 'save' the zibunzisin-as-a-local-anaphor hypothesis 58
5.2.1. The initial formulation 58
5.2.2. Modification attempted 60
5.3. Attempts to 'save' the otagai-as-a-local-anaphor hypothesis 64
5.4. Summary 65
6. Summary 66
Appendix: Hoji 1995

Chapter 5: The Essentials of the Proposal
1. The Proposal stated in terms of Lakatos 1970/1978 1
2. The Model of judgment making 4
2.1. Judgments on the acceptability of sentence αwith interpretation γ(a, b) 4
2.2. Judgments on the acceptability of sentence alpha 13
3. The Model of prediction making and Duhem's problem 19
3.1. The model of prediction making 19
3.2. Duhem's problem and reference to γ(a, b) 21
4. The Reinhartian heuristic 23
5. Summary and remaining issues 24

[Here is the table of contents as of 1/9/2009.]

A Foundation of Generative Grammar as an Empirical Science
by Hajime Hoji

Chapter 1: Introduction
1. Introduction1
2. Chomsky's (1965) challenge5
3. The goal and the main claims8
4. Outline of chapters 2-5 13
4.1. Chapter 2 13
4.2. Chapter 3 13
4.3. Chapter 4 13
4.4. Chapter 5 14
5. Answering Chomsky's challenge15

Chapter 2: Repeatable Phenomena
1. Introduction2
2. Repeatable Phenomena3
2.1. *Schemas, *Examples, okSchemas, okExamples and repeatable phenomena3
2.2. The asymmetry between *Schemas and okSchemas7
2.3. Across-speaker repeatability and within-speaker repeatability8
2.4. Repeatable phenomena, hypotheses, and progress in generative grammar8
2.5. Summary10
3. Some Illustration10
3.1. Introduction10
3.2. Hypotheses not backed up by a repeatable phenomenon in Japanese12
3.2.1. Zibunzisin12
3.2.1.1. Experimental design12
3.2.1.2. Results of experiments14
3.2.2. Otagai16
3.2.2.1. Experimental design16
3.2.2.2. Results19
3.3. Repeatable Phenomena in Japanese20
3.3.1. Introduction20
3.3.2. So-NPs vs. A-NPs20
3.3.2.1. Background: the demonstratives in Japanese20
3.3.2.2. So-NPs vs. a-NPs21
3.3.2.3. Experimental design24
3.3.2.4. Results25
3.3.3. Weak crossover, reconstruction, and the OS Construction in Japanese25
3.3.3.1. The Initial observation25
3.3.3.2. Two types of dependency in Ueyama 199827
3.3.3.3. *Schemas and an okSchema28
3.3.3.4. More okSchemas based on the OS constructions in Japanese29
3.3.3.5. LF-c-command-based BVA31
3.3.3.6. Results33
3.3.3.7. Precedence-based BVA35
3.3.3.8. Results37
4. Concluding remarks39
5. Appendix42

Chapter 3: Theory of Judgment Making and Its Consequences
1. Introduction2
2. Theory of judgment making3
2.1. Introduction3
2.2. The model of judgment making and the CS5
2.2.1. How the theory of the CS is embedded in the model of judgment making5
2.2.2. Difficulty9
2.2.3. The major subject construction10
2.2.4. The OS (Object Subject) construction12
2.2.5. Summary14
2.3. The model of quantifying the informant judgment15
2.4. The informant's sensitivity and resourcefulness17
2.4.1. String/meaning sensitivity17
2.4.2. Structural/contextual resourcefulness17
2.5. Summary20
3. Some consequences21
3.1. The asymmetry between *Schemas and okSchemas21
3.2. Focusing on *Examples22
3.3. The advantage of single-informant experiments26
4. Bridging statements and predictions29
4.1. Making hypotheses testable29
4.1.1. Bridging statements29
4.1.2. Bridging statements and the asymmetry between *Schemas and okSchemas31
4.2. Predictions34
4.2.1. Two types of predictions34
4.2.2. Confirmation and disconfirmation of Predictions36
4.2.3. Single and multiple-informant experiments38
4.3. Summary43
5. Repeatable phenomena and the significance of the informant judgment45
5.1. The asymmetry between a *Schema and an okSchema45
5.2. When do we obtain a repeatable phenomenon?47
5.3. Single-informant and multiple-informant experiments49
5.4. Summary52
6. Predictions, auxiliary hypotheses and heuristics53
6.1. Introduction53
6.2. The model of prediction making54
6.3. When a prediction fails60
6.3.1. Auxiliary hypotheses60
6.3.2. When a *Schema-based prediction gets disconfirmed63
6.3.3. When an okSchema-based prediction fails to be confirmed66
6.4. Heuristics69
6.4.1. Progressive and degenerating problemshifts69
6.4.2. Heuristics for single-informant and multiple-informant experiments72
6.4.3. Summary73
7. Summary75

Chapter 4: Anaphoric Relations in Japanese
1. Introduction1
2. The BVA (Bound variable anaphora)2
2.1. The linguistic intuition BVA2
2.2. The bridging statement6
2.2.1. The Lexical condition6
2.2.2. The Structural condition8
2.3. The Model of prediction making16
2.4. Modification18
2.5. Further modification24
2.6. Some methodological clarification27
3. BVA and "Principle B" Effects30
3.1. A Problem30
3.2. Modification regarding pf-LF correspondences35
3.3. Further modifications39
3.3.1. A new repeatable phenomenon39
3.3.2. An account44
3.3.3. A new prediction49
3.3.4. Further predictions53
4. "Principle B" effects for coreference: a further prediction55
5. The Hypotheses about zibunzisin and otagai, revisited59
5.1. Eliminating a *Schema-based prediction and adding a new one59
5.2. Attempts to 'save' the zibunzisin-as-a-local-anaphor hypothesis60
5.2.1. The initial formulation60
5.2.2. Modification attempted62
5.3. Attempts to 'save' the otagai-as-a-local-anaphor hypothesis67
5.4. Summary68
6. Summary68
7. Appendix: Hoji 1995 70

Chapter 5: The Essentials of the Proposal
1. The Proposal stated in terms of Lakatos 1970/19781
2. The Model of judgment making4
2.1. Judgments on the acceptability of sentence α with interpretation γ(a, b)4
2.2. Judgments on the acceptability of sentence α13
3. The Model of prediction making and Duhem's problem20
3.1. The model of prediction making20
3.2. Duhem's problem and reference to γ(a, b)22
4. The Reinhartian heuristic24
5. Summary and remaining issues26

Appendix: Comparison with Schütze's (1996) Model of Judgment Making
1. Introduction1
2. The Models of Judgment Making2
2.1. The Schütze 1996 model and the Ueyama model2
2.2. The main differences5
2.3. More details8
3. Summary11
3.1. Initial concerns11
3.2. The use of the informant judgments11
3.3. Gradient judgments12
3.4. The use of the researcher as the informant14
3.5. Repeatable phenomena15

If you would like to take a look at the draft, please email me.

The methodological proposals in my book can be further illustrated by discussing the following.

1. a. The use of so of so-no tyosya (intended as 'its author' not 'that author') as B of BVA(A, B). (We can show that the *Schema-based prediction fails quite badly; i.e., we already have robust results of some experiments, as discussed in Hoji 2006 (Assessing Hypotheses))
b. An 'account' of the above and how the account extends to 'quirky binding' in relation to Principle B effects in the OS as well as the SO context as discussed in chapter 4 of my book. (I had a section on (1b) in chapter 4, but I have decided not to include it in the book.)

2. Resumption and the OS construction. (Although we have not conducted an experiment that directly addresses the issue, some experiments on Negation-related issues dealt with resumption and their results were quite robust, precisely as predicted).

3. The long-distance-OS-related claim made in Saito's (2003, for example) work and the disconfirmation of its *Schema-based prediction with do-no-NP as A of BVA(A, B). This is discussed to some extent in Hoji 2006 (Assessing Hypotheses) and also in chapters 2 and 4 of my book, but not in any depth especially in light of the methodological proposals in my book.

4. Miyagawa's claim on the basis of the interaction between NEG and zen'in. (We have already conducted experiments on this and have gotten quite robust results; the *Schema-based prediction has been disconfirmed quite badly. Since the empirical claim in question seems to have played a significant role in recent years in relation to 'floating numerals' and 'scrambling', it may not be a bad idea to discuss this in the public domain for the record.)

Since the manuscript has become long enough, I plan to discuss the above in separate work(s).

The methodological proposals in my book can be further illustrated by discussing the following.

1. a. The use of so of so-no tyosya (intended as 'its author' not 'that author') as B of BVA(A, B). (We can show that the *Schema-based prediction fails quite badly; i.e., we already have robust results of some experiments, as discussed in Hoji 2006 (Assessing Hypotheses))
b. An 'account' of the above and how the account extends to 'quirky binding' in relation to Principle B effects in the OS as well as the SO context as discussed in chapter 4 of my book. (I had a section on (1b) in chapter 4, but I have decided not to include it in the book.)

2. Resumption and the OS construction. (Although we have not conducted an experiment that directly addresses the issue, some experiments on Negation-related issues dealt with resumption and their results were quite robust, precisely as predicted).

3. The long-distance-OS-related claim made in Saito's (2003, for example) work and the disconfirmation of its *Schema-based prediction with do-no-NP as A of BVA(A, B). This is discussed to some extent in Hoji 2006 (Assessing Hypotheses) and also in chapters 2 and 4 of my book, but not in any depth especially in light of the methodological proposals in my book.

4. Miyagawa's claim on the basis of the interaction between NEG and zen'in. (We have already conducted experiments on this and have gotten quite robust results; the *Schema-based prediction has been disconfirmed quite badly. Since the empirical claim in question seems to have played a significant role in recent years in relation to 'floating numerals' and 'scrambling', it may not be a bad idea to discuss this in the public domain for the record.)

Since the manuscript has become long enough, I plan to discuss the above in separate work(s).

My current plan is to prepare a book that will include (2) and another book that will include (1), (3) and (4).

the draft of the book I hope I will finish soon.

I did finish the manuscript and I did submit it to a publisher, sometime in 2009.

Upon receiving a review of the manuscript, I decided to revise it substantively and ended up working on a "new book." Because of the methodological articulation and the improvement of the general experimental design (that we use) that have been made subsequent to the completion of the 2009 book manuscript, I am glad that I did not try to make minor changes to the manuscript for publication.

In the meantime, works have continued to be published which seem to me to have little, if any, concern about making progress on the basis of rigorous testability. Such work continues to use what falls FAR SHORT of a solid generalization -- which I now call a confirmed schematic asymmetry -- as an empirical basis of their proposals, and their citations among themselves continue, leading uninformed and uncritical readers the impression that something significant is really being addressed.

I once had a number of postings on Popper/Lakatos/etc., but they have subsequently been removed. I have just realized that a few of those postings are referred to elsewhere at this HP. Under this posting, therefore, I will reintroduce such postings, along with the info. as to where each posting is mentioned.

[This posting is referred to at the end of the remarks under "Otagai" (Hoji 2006) in the Downloadable Papers page.]

Popper 1983: p. xx (i.e., the 20th page of "Introduction 1982," HH) contains (i).

(i) But when is a statement falsifiable? It is of great importance to current discussion to notice that falsifiability in the sense of my demarcation criterion is a purely logical affair. It has to do only with the logical structure of statements and of classes of statements. And it has nothing to do with the question whether or not certain possible experimental results would be accepted as falsifications.

Popper, K. 1983. Realism and the Aim of Science: From the Postscript to the Logic of Scientific Discovery, Routledge, New York.

If one only reads Kuhn (and/or some work that cites Kuhn), one might "classify" Popper as a naive falsificationist.

Popper's remarks as those found in Popper 1983, as cited in [27158], and in many other places, including Popper 1983: xxxi-xxxv, should at least make one think twice before one mentions Popper as a "naive falsificationist," or even as a "falsificationist."

　　　A fundamental working hypothesis in generative grammar is the existence of the language faculty, understood as an algorithm whose input is a set of items taken from the mental Lexicon of the speaker and whose output is a pair of mental representations – one underlying 'meaning' and other 'sounds'. The main goal of generative grammar can thus be understood as demonstrating the existence of such an algorithm and discover its properties. Construed this way, the object of inquiry is not language as an external 'object' but the language faculty; in the terms of the distinction made by Chomsky in the 1960s, it is not performance but competence that constitutes the core goal of generative grammar.
　　　The data in actual research activities in generative grammar, however, is based on acceptability judgments on a given sentence, whether they are introspective judgments by the researchers or their informants or observation of various other types of reactions by 'subjects' in an experimental setting. I.e., what we deal with in an attempt to discover the properties of the speaker/hearer's competence (the Computational System) is his/her performance (i.e., language use, in a broad sense). This makes it crucial, in the context of generative grammatical inquiry as construed above, to articulate how we can extract from performance data evidence for a hypothesis about the properties of the Computational System. The absence of a minimally satisfactory articulation of how to do this is likely to lead to a situation where different (groups of) researchers base their proposals about the Computational System on different sets of speaker judgments, collected in a variety of ways, that are not necessarily uniform or robust, being subject to a great deal of fluctuation and variation not only among speakers but also within a single speaker. This makes it difficult to evaluate competing proposals in a reliable and objective manner. In light of recent debates in leading journals (e.g., Language, Lingua and Natural Language and Linguistic Theory) about what should qualify as data, about the use of introspective judgments as crucial evidence for or against a theory, etc., one might in fact conclude that we have not yet developed a means to evaluate the empirical bases of hypotheses in generative grammar that is compelling enough to the majority of the practitioners. An evaluation of a given hypothesis thus tends to have an arbitrary aspect to it, influenced by such factors as whether or not the terms and concepts utilized are of a theory currently in fashion and whether or not it endorses the standard view concerning the validity of alleged empirical generalizations, regardless of how much or how little 'repeatability' obtains in regard to the predicted speaker judgments on the crucial sentences.
　　　When one aims at discovering the properties of the language faculty as construed above, one must recognize the following: the Computational System's yielding something as its output does not guarantee that the speaker finds it (more precisely, finds its surface manifestation) to be acceptable; after all, non-grammatical as well as grammatical factors must contribute to the ultimate acceptability judgment by the speaker on a given sentence form under a specified interpretation (e.g., one's knowledge about the world, one's belief system, and the like). The Computational System's failure to yield something as its output, on the other hand, should necessarily mean that the 'sentence form' corresponding to such a 'failed representation' should be judged unacceptable under the specified interpretation. If something is predicted to be impossible due to the hypothesized formal properties of the Computational System under discussion -- i.e., if the Computational System never yields what would be required for the interpretation in question --, how can some pragmatic adjustment save it? Only by taking this point to heart and by putting forth a hypothesis so as to yield a negative prediction (the prediction that something is impossible (under a specified interpretation)), do we have a hope of making generative grammar an empirical science or of making it a progressive research program in the sense of Lakatos 1970.

Lakatos, I. 1970. "Falsification and Methodology of Scientific Research Programmes," in I. Lakatos and A. Musgrave (eds.), Criticism and the Growth of Knowledge. Cambridge University Press. pp. 91-195.

The major concerns underlying my research include (1).

(1)　a. 　How can we try to ensure and measure progress in what we do in generative grammar?
　　b. 　How can we tell whether or not given intuitions of ours are likely to be a reflection of the Computational System?

The absence of the concern in (1b) in much of the field seems to have resulted in the state of affairs depicted in (2a) and remarks such as (2b).

(2)　a.　Alleged generalizations that have long been shown to be invalid keep on being used in theoretical discussion.
　　b.　"Okay, you have provided some examples that go against the generalization I have proposed/adopted. But what is your alternative account of the contrast that I have been discussing? I have an account but you do not. I would give up my account only if you provided an alternative account of the contrast that I have been discussing, which, as I noted, is shared by at least some native speakers of the language under discussion."

Not every observation qualifies as something that must be accounted for by a theory about the Computational System; it must first be demonstrated that it is most likely a reflection of the Computational System. As noted in [29073], to do so would require the recognition of the significance of negative predictions insofar as the research in question is aimed at demonstrating the existence of and discovering the properties of the Computational System.

The major concerns underlying my research include (1).

(1)　a. 　How can we try to ensure and measure progress in what we do in generative grammar?
　　b. 　How can we tell whether or not given intuitions of ours are likely to be a reflection of the Computational System?

The absence of the concern in (1b) in much of the field seems to have resulted in the state of affairs depicted in (2a) and remarks such as (2b).

(2)　a.　Alleged generalizations that have been shown to be invalid keep on being used in theoretical discussion.
　　b.　"Okay, you have provided some examples that go against the generalization I have proposed/adopted. But what is your alternative account of the contrast that I have been discussing? I have an account but you do not. I would give up my account only if you provided an alternative account of the contrast that I have been discussing, which, as I noted, is shared by at least some native speakers of the language under discussion."

Not every observation qualifies as something that must be accounted for by a theory about the Computational System; it must first be demonstrated that it is most likely a reflection of the Computational System.

One example of (2a) is the continued adoption by some researchers of the claim that zibunzisin in Japanese has the same formal properties as the so-called anaphors in English such as himself, herself, myself, etc. The results of the preliminary surveys, portions of which are available at

http://www.gges.xyz/hoji/cfj-results/CFJ-37_Zibunzisin.pdf
http://www.gges.xyz/hoji/cfj-results/CFJ-47_LocalAnaphorsEnglish.pdf

show clearly that such a claim is not justified at all; yet the assumption in question is still adopted and used widely, and in fact crucially, in the field and continues to be referred to by researchers who do not have direct knowledge of the language (as well as by some who do).

Consider the claim in (i).

(i) Zibunzisin has the same formal property as English reflexives (e.g., himself) in terms of the requirement of having its 'antecedent' in a particular structural position.

While there are environments where English reflexives do not behave like a 'local anaphor', it has been fairly widely agreed upon in the field that when they occur in a direct object position, their use is felicitous only if there is an NP/DP in a position that c-commands it within the same clause.

So, when native speakers of English are asked to report how (un)acceptable they find each of the examples below, most speakers seem to have clear judgments and mark the (a) examples "acceptable" and the (b) examples "unacceptable."

(1) 　a. 　John loves himself.
　　b. 　John thinks Mary loves himself.
(2) 　a. 　John recommended himself for that position.
　　b. 　John thought that Mary had recommended himself for that position.
(3) 　a. 　Mary, who had firmly believed that Chomsky would recommend her, was shocked to death when she found out that Chomsky recommended Bill instead. (where her = Mary)
　　b. 　Mary, who had firmly believed that Chomsky would recommend herself, was shocked to death when she found out that Chomsky recommended Bill instead.

Although there are slight variations among the speakers, their reactions are fairly uniform, and I have not encountered any native speakers who would say "Fine" to the (b) examples above without any hesitation, whether they are linguists or not.

Given the claim in (i) above, the Japanese analogues of the (b) examples in (1)-(3), with zibunzisin in place of him/herself, are predicted to be unacceptable due to the same grammatical/formal property that underlies the native speaker intuitions on the (b) examples in English (1)-(3).

When native speakers of Japanese are asked the same questions about the Japanese analogues of (1)-(3), however, their reactions are qualitatively different from the reactions by native speakers of English to (1)-(3). Many speakers readily accept the JP versions of the (b) examples in (1)-(3). If they do answer "unacceptable," such a response often/usually comes only after a great deal of hesitation, in sharp contrast to how the native speakers of English typically respond as "unacceptable" to the (b) examples in (1)-(3) – there typically is no such hesitation.

It may be of interest to note that all of the 7 non-linguist informants I have checked with find the JP version of (3b), for example, almost fully acceptable, while the 32 linguist informants' judgments are more 'conservative'. Yet, their average response is more 'acceptable' than 'unacceptable'. If we compute 'markings' as Good to Bad on a 5-point scale, as -2, -1, 0, +1, +2, which we might say more or less correspond to customary *, *?, ??, ?, okay, the average 'score' by the 7 non-linguists on the JP version of (3b) was +1.57, and the average score by the 32 linguist informants was +0.47.

It is actually quite difficult to find speakers who systematically find examples like the JP versions of the (b) examples in (1)-(3) totally unacceptable.

As noted before, if zibunzisin shared with English reflexives the formal property that is responsible for their local-anaphor nature, the Japanese version of (3b) should be ruled out in the same way as (3b) is ruled out.

The same goes with so-called reciprocal anaphor otagai. It was pointed out more than 10 years ago and was noted in Ueyama 1998 that the generalization under the thesis/assumption that otagai is a local anaphor is far from being valid. Compare the English example in (4) and the Japanese example in (5).

(4) 　*The nice spring breeze made each other feel very happy.
(5) 　Haru-no atatakai kaze-ga otagai-o totemo siawasena kimoti-ni sita.
　　Spring-gen warm breeze-nom otagai-acc much happy feeling-to did/made
　　'The warm spring breeze made them feel very happy.'

The native speakers of English cannot even seem to process (4). On the other hand, (5), which is a pretty close rendition of English (4) is quite acceptable. I found the example in (5) in the Mainichi Newspaper about 10 years ago, and I have not met any native speaker of Japanese who finds it unacceptable. As the English translation of (5) indicates, otagai in (5) corresponds to 'them'. One might thus suggest that the local-anaphor property under discussion can be detected with otagai only under its 'reciprocal interpretation'. As discussed in some depth in my "Otagai" paper, however, such a move would not save the thesis/assumption that otagai is a local anaphor since the distribution of 'otagai' 'with a reciprocal interpretation' does not exhibit the predicted locality, either, even if we focus on otagai appearing in 'argument positions'. (The paper "Otagai" can be downloaded at "Downloadable papers.") Yet, the refuted thesis keeps on being used, and responses such as (2b) are rather common. What underlies such research attitude, I contend, is the failure to understand the point made in [29073] at this board, the posting "dominating" this..

Please see Generalization [29124] for further remarks/discussion .

More recent methodological articulation is provided under the thread Methodology [42043].

"A Review of Japanese Syntax and Semantics by S.-Y. Kuroda," (1998) in Language 74, pp. 146-152. contains (i).

(i) Since our sense experiences, such as introspective judgments about a given sentence in a given language, most likely reflect more than the language faculty proper, such a task necessarily involves hypotheses about the nature of the relevant sense experiences, in particular, hypotheses as to which aspects of our sense experiences under discussion are reflections of our grammar, and in what theoretical terms they are to be expressed.

Once we start addressing the above issue, we must recognize (ii).

(ii) a. The grammar's yielding something as its output does not guarantee that the speaker finds it to be acceptable; after all, non-grammatical as well grammatical factors must contribute to the ultimate judgment of the speaker on the 'acceptability' of a 'sentence' under a specified interpretation.
b. The grammar's failure to yield something as its output, on the other hand, should necessarily mean that the 'sentence form' corresponding to such a 'failed representation' should be judged unacceptable under the specified interpretation.

(ii-b) is the basis for our emphasis on the significance of negative predictions in generative grammar. If something is predicted to be impossible due to the hypothesized formal properties under discussion, how can some pragmatic adjustment save it?

There has been much debate in regard to the validity of Popper's demarcation criterion; see Lakatos' critique, for example). But if the goal of generative grammar is to demonstrate the existence of the language faculty as an autonomous system and discover its properties, and if we adopt the working hypothesis that the language faculty, as it manifests itself in the mind of an individual speaker, consists of the grammar and a lexicon, and the grammar is a computational system, which, given an input (i.e., a set of items from the lexicon), mechanically yields two representations as its output, one having to do with 'sound' and the other with 'meaning', (ii) seem to me to be unavoidable, regardless of the ultimate verdict on Popper's demarcation criterion. Only by taking the points in (ii), especially (ii-b), to heart and by putting forth a hypothesis so as to yield a negative prediction, do we have a hope of making generative grammar an empirical science or making it a progressive research program in the sense of Lakatos 1970. If we did not take (ii) seriously, we would not be able to realize the difficulty we are faced with in identifying what should qualify as data in generative grammar. As suggested above, such identification goes hand in hand with theory construction; and that makes it even more important to recognize the significance of (ii).

And most of the work in generative grammar, at least in so-called generative grammatical studies of Japanese, seems to me to have abandoned the hope just noted. Otherwise, how would it be possible to maintain hypotheses whose negative predictions are so clearly disconfirmed, and demonstrated as such in published works, and to keep on using them in a crucial way?

If you are interested, please check section 3 of "Assessing Competing Analyses: Two Hypotheses about 'Scrambling' in Japanese" (2006) in Ayumi Ueyama, ed., Theoretical and Empirical Studies of Reference and Anaphora?Toward the establishment of generative grammar as an empirical science, pp. 139-185.

which is downloadable at the "Downloadable Papers" page.

And most of the work in generative grammar, at least in so-called generative grammatical studies of Japanese, seems to me to have abandoned the hope just noted. Otherwise, how would it be possible to maintain hypotheses whose negative predictions are so clearly disconfirmed, and so demonstrated in published works, and to keep on using them in a crucial way?

As noted in [28637], further discussed in the document mentioned there, what is most crucial is the recognition that there is a qualitative difference in significance between judgmental variation on (i) what is predicted to be impossible for a grammatical reason and (ii) what is not so predicted. The lack of understanding of this leads to a practice in the field, not uncommon unfortunately, to take it to be significant that some (or even many) speakers find [the examples that are predicted to be impossible due to some property of the Computational System] to be highly marginal. After all, mere complexity of a particular example under a specified interpretation might be a big enough factor to negatively affect the speaker judgment. Much more significant would be the speaker judgment that the examples that are predicted to be impossible are acceptable especially when the speakers in question consistently detect a predicted contrast in paradigms that are independent from but are crucially related to the paradigm in question; see Hoji and Ishii 2004: note 4.

"What Gets Mapped to the Tripartite Structure of Quantification in Japanese," (2004) Hoji & Ishii WCCFL 23. Downloadable at this HP.

When faced with such disconfirmation of the negative prediction in a crucial experiment, one might try to save the hypothesis by introducing an additional assumption, a common practice in natural sciences. "If we assume such and such, the unacceptability of the examples under discussion (with the specified interpretation) is no longer predicted and hence the fact that some speakers accept them would no longer pose a problem to the proposal under discussion. Unless the introduction of the new assumption leads to new discovery/insight presumably by making new negative predictions, this would be what we might call, slightly adapting Lakatos' (1978) terminology, a 'degenerative' move, rather than a 'progressive' move. The situation is actually quite serious if the negative prediction that appears to have been disconfirmed is the only negative prediction that the proposal under discussion makes and can be tested at the moment.

Miyagawa and Arikawa 2005: footnote 6 seems to provide a nice illustration of (i) the lack of understanding of the significance of negative predictions and (ii) a 'degenerative' move of the sort noted above.

Miyagawa and Arikawa 2005 LOCALITY IN SYNTAX AND FLOATED NUMERAL QUANTIFIERS. http://web.mit.edu/miyagawa/www/pdfs/FNQFINAL.pdf

Further discussion would have to address issues such as the following, and it will have to take place in the "Further Discussion" board instead of this board.

(iii) how to evaluate reports of different speaker judgments, in terms of their theoretical significance, repeatability, etc.
(iv) whether or not the proposal under discussion, e.g., that in Miyagawa and Arikawa 2005, makes any negative predictions other than the one that is being discussed in their footnote 6 for which a crucial experiment can be conducted.

this would be what we might call, slightly adapting Lakatos' (1970, 1978) terminology, a 'degenerative' move, rather than a 'progressive' move.

Lakatos (1978: 1-7) distinguishes a progressive research programme and a degenerating research programme, as in (i).

(i) "... Thus, in a progressive research programme, theory leads to the discovery of hitherto unknown novel facts.
In degenerating programmes, however, theories are fabricated only in order to accommodate known facts."

Lakatos (1978, and earlier works, including his 1970 paper in Criticism and the Growth of Knowledge, edited by Lakatos and Musgrave, 1970, pp. 91-195) discusses these notions in relation to how a theory is saved from falsification. He mentions "a progressive research programme" when he talks about an auxiliary hypothesis not only saving the theory from falsification, but also making new predictions (that will later be confirmed). "A degenerating research programme" is mentioned when an auxiliary hypothesis is introduced only to save the theory from falsification. Popper (1974) calls the first type of 'saving hypothesis' an auxiliary hypothesis and the second type an ad hoc hypothesis.

In the context of generative grammar, we can certainly distinguish the two types of research attitude. Since we do not know what the facts are in generative grammar -- i.e., what linguistic judgments are directly due to properties of the Computational System, however, I would be inclined to say that we may in fact have (ii) and (iii), adopting the basic distinction between 'progressive' vs. 'degenerating' noted in (i) above.

(ii) a progressive way of establishing and dealing with generalizations
(iii) a degenerating way of dealing with generalizations

Suppose one comes up with a generalization G, either as something deduced from one's proposal or as a 'descriptive generalization'. The considerations in [28637] impose the following fundamental property on a generative generalization: a generalization must contain the 'specification' of (i) an Eg* (the schematic form of examples that are predicted to be unacceptable (with the specified interpretation) due to some property of the Computational System) and (ii) at least one Eg (the schematic form of examples that minimally differ from the Eg* with respect to the factor that is hypothesized to be responsible for the status of the Eg* examples).

Suppose someone shows the following.

(iv) The speaker judgments on some Eg* examples are not nearly as uniformly/systematically bad (at all) as claimed by G.

The researcher of the 'progressive attitude' would explore what is responsible for (iv), still at this point not giving up on the original generalization G or perhaps more accurately what G was intended to express. More specifically, s/he will try to identify a factor that must be controlled to obtain the judgments predicted under G. S/he may propose that a factor F is responsible for the 'unexpected' judgments on the Eg* examples. If F is grammatical in nature, there is a good chance that this would give rise to an independent negative prediction, on the basis of interaction between F and what is responsible for G, for example. Clearly, this would be a/the defining characteristic of a 'progressive research programme' of Lakatos, as it is 'applied to' the stage where one is trying to establish a 'generative generalization'. If F is not grammatical in nature, at least to the extent that we still cannot demonstrate that it is, we cannot make a negative prediction based on F. But the researcher with the 'progressive attitude' tries to see if F contributes to making some independent empirical discoveries, by serving as a non-grammatical factor that should and hopefully could be controlled in our syntactic experiments.

In summary, the 'progressive attitude' would be such that the researcher tries to establish new generalizations while making his/her attempt to save the original generalization G.

What about the 'degenerating attitude'? The researcher with this attitude would not be concerned with establishing new generalizations as part of his/her attempt to save the original generalization G. S/he would be content as long as G can be saved. Hence s/he would be content with coming up with some factor F that may be responsible for the unexpected judgments on the Eg*, and would not feel compelled to spell out what independent consequences F might have. S/he may refer to what appears to be the effects of F in other areas of the language under discussion or in other languages, and tries to make it plausible for us to accept F as the relevant factor in question, perhaps making reference to what is possible rather than what is impossible. S/he may also make remarks such as those in (v):

(v) a. As long as there is a contrast in judgment for some people, we can focus on that, at least for now, and proceed. Science progresses in the ocean of anomalies, anyway.
b. Judgments are always murky and we must focus on certain judgments in accordance with what the theory tells us in regard to what should be the most interesting and relevant set of judgments, among all the judgments we might get.

And at that point we wonder how the proposal can in principle be falsified once we have adopted the thesis that F is responsible for the unexpected judgments on the Eg* examples under discussion. This is of course independent from whether or not the advocates of the proposal would indeed abandon it in the face of the disconfirmation of the negative prediction; that would not be a logical matter, as acknowledged by Popper many years ago.

Lakatos 1978: "Science and Pseudoscience" by Imre Lakatos
is available at http://www.lse.ac.uk/collections/lakatos//Default.htm. The paper is published in Lakatos 1978, as "Introduction." Lakatos, Imre. 1978. The Methodology of Scientific Research Programmes: Philosophical Papers Volume 1, edited by John Worrall and Gregory Currie, Cambridge University Press.)

Popper 1974: Chapter 2, 'The Problem of Demarcation', sections 5-8 of "Replies to my Critics" in Part Three, 'The Philosopher Replies', of The Philosophy of Karl Popper, ed. by P.A. Schilpp, 1974, Vol. 2, pp. 961-1197.