Models of Language Change

Kenneth Konopka


Language Change
Social Network Model

 

In this project I have designed and implemented a NetLogo model of language change. There are several models in the Linguistics literature that posit grammar competition as the driving force behind language change, much as seen in biological systems. Kirby’s (1999) language change model simulates the case of an acquirer who possesses two innately specified possible grammars that can be used to parse sentences heard in the input from the population (referred to as the “arena of use”). Based on the utterances that the learner hears from the arena of use, a grammar is acquired over time based on the probabilistic distribution of the grammars that those utterances represent. By introducing a psycholinguistically based functional bias, Kirby demonstrated how the favorably biased grammar will replace the other grammar over time. As cycles of speaking and acquiring proceed, the functional bias ensures a higher proportion of the biased grammar each generation. This replacement results in an S-shaped curve for the course of language change, as is seen in the empirical data from historical linguistics (Kroch, 1989).

Yang (2002) also presents a model of language change driven by the competition between grammars. His model is applied to the situation where multiple innately specified grammars can parse overlapping sets of utterances from the arena of use. They are overlapping in the sense that a given grammar may parse certain utterances of another grammar. The model does not consider the actual words that make up the utterance (the lexicon), but is sensitive only to the word-order of the grammars. The grammar which is able to parse all the sentences over time wins out over those that cannot. This model is based on an incrementally updated learning algorithm which obviates the need for cognitive resources to calculate the distribution of the utterances heard over the entire acquisition process. In this way, unlike the functional bias model, the learner is not required to hold a large number of utterances in memory in order to calculate a distribution. This is an important feature since a model of language change must posit a minimum of resources available to the learner (i.e. an infant).

Each of the extant theories also has its failings. The functional bias model cannot account for certain features of the empirical data from Old English. For example, although intra-speaker variation is found in the historical texts (Pintzuk 1999, Clark 2004) (i.e. the same scribe uses different grammars in his writing), the model cannot account for this feature of the speaker. The functional bias model allows only one grammar per speaker – an all-or-none situation in which the acquirer will eventually acquire one grammar exclusively. Another problem is the model’s probabilistic acquisition of a grammar by a learner. In effect, the model states that a learner hearing 90% of one grammar type in the input and 10% of another type will learn the underrepresented grammar 10% of the time. A preferable model is one in which learners acquire their grammars deterministically so that any learner exposed to the exact same input will acquire the same grammar.

The variational model on the other hand does not account for the S-shaped trajectory of language replacement, and does not demonstrate how one grammar replaces the other if the two grammar types are mutually exclusive in the utterances that they parse (non-overlapping grammars).

As we see from the above, each of these approaches has its strengths and weaknesses, and by hybridizing these models I have developed a model that accounts for the empirical evidence and posits a learner that is more in accord with language acquisition research. In consideration of the above issues, a checklist can be compiled which will be used to evaluate the viability of the models that are presented:

Checklist of desirable features:
• intraspeaker variation vs. discrete grammar states
• S-shaped grammar replacement
• batch vs. incremental learner
• probabilistic vs. deterministic learner

In addition to developing a viable model in terms of these criteria, the four models presented can be observed for their behavior when certain conditions are manipulated. By mapping their parameter spaces, we can see the interaction of bias, sampling rate, and initial state on the course of change.

Functional Bias Model: As shown in the chart below, the functional bias model does indeed produce S-shaped curves for language change when the bias (alpha) is introduced. The chart shows five representative runs of the model at each alpha setting. The y-axis represents the average of the grammar values present in the arena, with iterations (i.e. time) represented on the x-axis. Note that the grammar value for the arena is an average of the two possible states for individuals: 1 and 0, and does not represent any single speaker. As seen in the chart, a higher alpha speeds the change to the biased grammar. Also, we can see that setting the bias to zero results in an average value that reflects the initial state.



The chart below is a mapping of the parameter space for alpha, the initial setting for the arena (seedProb), and delta, which is a calculation of the rate of change. Each column represents the average of ten runs at the relevant settings. As indicated above, increasing alpha speeds the rate of change. Additionally, we see that the initial setting for the proportion of the biased grammar in the arena does not affect the rate of change. The variation in the columns is due to the manner in which delta is calculated, and does not reflect any systematic variation. This calculation problem will be addressed in the next version.



Deterministic Model: This model seeks to address the problem of the probabilistic learner mentioned earlier. This model, based on the functional bias model, alters the acquisition scheme by positing an all-or-none learner. After accumulating utterances from the arena, and filtering through the functional bias mechanism, the learner acquires the grammar that is most represented in the data. The learner then acquires either of the two grammars based on majority rule.

The charts below show that the deterministic model produces the desirable S-shaped curve for language change, but note the difference in behavior from the probabilistic model. Not only do individual runs vary in their point of instantiation, but we see that at the given settings not all runs go to the biased grammar.



Variational model: In order to further address our checklist of desirable features we turn to the variational model. This model, an instantiation of Yang (2002) incorporates the incremental learning algorithm to effect change. As we see in the chart below, this model does not show change across the population since the grammars are non-overlapping in the utterances parsed. The acquirers however have been implemented such that they can embody more than one grammar.



Hybrid model: The final model of this series is the hybrid model. This model incorporates the bias as in the functional bias models to bias the perception of an acquirer. This bias is applied to the input utterances as they are processed from individual speakers as sampled. The chart below shows the results for the hybrid model. As the chart shows, the hybrid model produces the S-shaped curve, with the bias value affecting the rate of change.



Mapping the parameter space for the variables as in the functional bias model, we see similar behavior across the range of variables. The chart below shows the parameter space with each column representing ten runs of the model.



While both the functional bias model and the variational model offer explanations for language change based on grammar competition, they vary on the features employed and the assumptions made about the learner. A summary of the checklist evaluation of the four models implemented in this study is given in Table (1). The performance of each model is based on the checklist of features presented earlier. In the table ( + ) indicates that the model displays the desirable feature, while ( – ) indicates the lack of the desirable feature.





By implementing and verifying the behavior of the biased variational model we have provided a plausible theory which builds upon Kirby’s use of a functional bias in explaining diachronic change in a syntactic variable. This model preserves the S-shaped course of change by retaining the processing bias for one grammar over another. It also addresses the problematic issue of intraspeaker variation found in the empirical data by positing a learner that may be represented by more than a single grammar state. In so doing, the model does not require a mechanism for batch learning. Instead, it posits an incremental learner capable of maintaining multiple grammars as proposed in the Yang model. The biased variational model is thus able to preserve the central idea of both models – grammar competition – in a way that is supported by the empirical data on historical change, while making reasonable assumptions about the learner.

References

Clark, Brady Z. (2004). A stochastic optimality theory approach to syntactic change. Stanford University dissertation. Stanford, CA.

Kirby, Simon (1999). Function, selection, and innateness: The emergence of language universals. Oxford University Press Inc. New York.

Kroch, Anthony (1989). Reflexes of grammar in patterns of language change. Language variation and Change. (199-244).

Nettle, Daniel (1999). Using Social Impact Theory to simulate language change. Lingua 108, 95-117.

Pintzuk, Susan (1999). Phrase structures in competition: Variation and change in Old English word order. Garland Publishing, Inc. New York.

Yang, Charles D. (2002). Knowledge and learning in natural language. Oxford University Press Inc. New York.