 |
Shizgal, Peter (1997) Neural Basis of Utility
Estimation. Current Opinion in Neurobiology, 7 (2),
198-208,
© Current Biology Ltd.
Peter Shizgal Center for Studies in Behavioral Neurobiology Concordia University
This article appeared originally in Current Opinion in Neurobiology, 1997, 7(2), 198-208, © Current Biology Ltd.
The author is grateful to Andreas Arvanitogiannis, Kent Conover, Jane Stewart, and John Yeomans for their helpful comments and to the "Fonds pour la Formation de Chercheurs et l'Aide à la Recherche du Québec" (grant ER0124), the Medical Research Council of Canada (grant MT-8037), and the Natural Sciences and Engineering Research Council of Canada (grant A0308) for research support.
The allocation of behavior among competing activities and goal objects depends on the payoffs they provide. Payoff is evaluated among multiple dimensions including intensity, rate, delay, and kind. Recent findings suggest that by triggering a stream of action potentials in myelinated, medial forebrain bundle axons, rewarding electrical brain stimulation delivers a meaningful intensity signal to the process that computes payoff.
In his nihilistic novel, "The End of the Road," John Barth introduces a protagonist who falls prisoner to his own indecision. To Jacob Horner, the predicted consequences of all actions have become equivalent. Thus, he sits, immobile, on a railroad station bench. Frozen there throughout an entire night, Horner is eventually rescued by an unorthodox therapist who shakes him free of his paralysis and provides him with three rules designed to prevent a recurrence. "Sinistrality" selects the leftmost of a set of items, "antecedence" selects the item first encountered, and "alphabetic priority" selects the item whose name appears first in an alphabetically sorted list. By applying these rules sequentially, Horner will be able to choose a course of action in a wide set of circumstances. Given his professed indifference to consequence, it need not matter to him where the application of these arbitrary choice rules leads.
Barth's deft portrait of anomie stands in sharp contrast to the Darwinian absolutism of the natural world, where selection has imposed an ultimate goal: making grandchildren. Unlike Jacob Horner, animals in the natural world behave as if they are anything but indifferent to the consequences of their actions. Their choices are guided, not by whimsical rules, but rather by systematic assessment of the current stream of sensory input, of the recorded consequences of past actions and of physiological and ecological conditions. The challenge for the cognitive and behavioral neurosciences is to understand the processes of evaluation and decision and to uncover how they are implemented in the brain.
Following modern usage in economics, utility is used here to represent the basis for choice. If an animal consistently chooses option x in a given set of circumstances, then that option is said to have a higher utility than the competing alternatives at the time of decision. Insofar as choice is adaptive, the utilities of goal objects and activities can be conceived as subjective estimates of potential contribution to fitness.
This usage of utility is more narrow than the one intended by the British philosopher, Jeremy Bentham, who first used the term as both a normative and explanatory principle [1]. To Bentham, utility reflected both the bipolar dimension of pleasure and pain and the basis for decision. Bentham's proposal that hedonic experience is the basis of choice is incorporated in some modern biological theories [2]. Nonetheless, the argument developed here makes no reference to hedonic experience. This is due to the absence of a well-validated and general method for measuring the feelings of non-human animals and to skepticism that the information determining choice must necessarily pass through the bottleneck of consciousness. Thus, the utility values discussed here are merely inferences drawn from the choices animals are observed to make. It is assumed that these utility values are represented in the state of neural circuitry governing goal-directed behavior. If so, a theory attributing a particular role in utility computation to a given population of cells is testable by means of conventional psychophysical and electrophysiological methods. The question of when and if signals in this circuitry breach the waterline of awareness is left open.
Utilities vary in sign and magnitude. Resources that an animal strives to procure, such as food and water, or activities in which the animal strives to engage, such as exploration, have positive utilities. Negative utilities are inferred from escape or avoidance. This review is focused on the computation of positive utility, on several of the dimensions along which inputs to this computation are arrayed, and on what can be learned about the computational process from studying the rewarding effects of electrical brain stimulation.
The computation of utility involves sensory, perceptual, motivational, and cognitive mechanisms. How can such a process be brought under tight experimental control so as to link particular components to the activity of identifiable neural populations? One approach is to replace or supplement natural stimuli with direct electrical stimulation of the brain. Vertebrates from goldfish to humans will work vigorously in order to trigger stimulation of certain brain regions, as if the stimulation were capable of injecting a signal that is transformed into a positive utility. It is argued below that such rewarding electrical brain stimulation mimics some, but not all, of the properties of natural goal objects. If so, analysis of the phenomenon of brain stimulation reward (BSR) can isolate components of the computational apparatus so that they can be characterized at both the behavioral and neural levels.
Goal objects (e.g., food) or activities (feeding) that maintain and direct the performance of other activities (e.g., running down an alley, pressing a lever, flying to a bird feeder) are called "reinforcers;" thus, the reinforced behavior can be said to be controlled by its consequences. The question of how BSR is related to the effects of natural reinforcers was the subject of numerous experiments in the 1960s and has not yet been resolved convincingly. Recently, Conover, Woodside, and Shizgal reopened the issue, using newer techniques for measurement, stimulus delivery, and control of postingestive effects [3, 4, 5]. Rats were equipped with an intraoral cannula, an intragastric cannula, and a lateral hypothalamic stimulating electrode. In one experiment, a natural reward, an intraoral infusion of a highly concentrated sucrose solution, was pitted against the electrical reward in a forced-choice paradigm. When the electrical stimulation was weak, the rats chose the sucrose; increasing the stimulation frequency reversed the preference. Most importantly, the presence of the sucrose led the rats to forgo stimulation trains for which they had worked vigorously in the absence of the sucrose [3]. Thus, the rats behaved as if they had simply selected the larger of two rewards. Similar results were obtained in sodium-depleted subjects when a saline solution was substituted for the sucrose [4].
To select the larger of two signals in a forced-choice situation, both inputs must be "boiled down" to single quantities arrayed on a common dimension. Thus, the behavior of the rats in the competition test implies that the natural and artificial rewards were subjected to a common evaluation on a unidimensional scale. This conclusion is strengthened by the results of a related set of experiments in which a train of BSR was pitted against a compound reward consisting of an intraoral infusion of a sucrose or saline solution plus an equally preferred train of BSR. In rats that had undergone mild food deprivation or sodium depletion, the gustatory and electrical rewards summated: the compound reward was chosen in preference to its electrical component alone [3, 4]. Summation is possible only if the inputs share a common property that is registered by the system of measurement, and a behavioral paradigm measuring a forced-choice between two rewards can be construed as registering relative utility. Thus, summation is further evidence that the natural and artificial rewards are assessed using a common scale of utility.
Further experiments revealed an important difference between the electrical and gustatory rewards. Sodium depletion increased the utility of a saline reward but left the utility of the electrical reward unchanged [4]. When the gastric cannula was closed, and a concentrated sucrose solution ingested by the rat was allowed to accumulate in the gut, the utility of this solution was degraded and, in some cases, reversed so that following substantial intake of sucrose, the BSR alone was preferred to the combination of sucrose and the same BSR train. In contrast, the accumulation of prodigious quantities of concentrated sucrose in the gut failed to increase the threshold for BSR in one subject and produced only modest increases in others [5].
The results obtained following sodium depletion or accumulation of sucrose in the gut suggest that the effects of the BSR and the gustatory reward are combined downstream from the point where physiological feedback signals modulate gustatory value. Analogous results were obtained in a study of hormonal modulation. A regimen of ovarian hormone administration that altered body weight and sucrose preference failed to alter the BSR threshold at a lateral hypothalamic site [6*]. Although another group of investigators has reported a statistically significant effect of ovarian hormones on the reward effectiveness of lateral hypothalamic stimuluation [7*], the size of this effect is very small. Indeed, the shifts in the representative reward-summation functions shown fall well within the range of variation used as the criterion for within-session stability. Larger shifts have been produced by performance manipulations such as adding weight to the lever [8].
The notion that BSR and gustatory reward are combined downstream from the point where physiological feedback signals modulate gustatory value [4] is not easily reconciled with demonstrations that severe food restriction lowers the threshold for BSR [9*, 10, 11]. Additional work will be required to address the issue. It remains to be determined whether the effect of severe food restriction on BSR is due to a specific modulation of the neural circuitry that determines the intensity of gustatory reinforcement or to some more general potentiation of appetitive behavior. The possibility that the potentiating effect of food restriction depends on electrode placement also merits investigation.
One of the most puzzling aspects of BSR is why the rat treats the effect of such artificial stimulation as if it were a meaningful signal. In typical BSR experiments, macroelectrodes are used to deliver pulses at currents and durations likely to excite neurons at an appreciable distance from the electrode tip [12], and a rigidly periodic cadence of activity is imposed on the stimulated cells. As Shizgal and Conover have argued [13**], such stimulation is unlikely to simulate a naturalistic signal in a system, such as the auditory nerve, that employs a spatiotemporal code to represent multiple stimulus dimensions. Even more problematic for the notion that the directly-stimulated stage of the system encodes multidimensional information is the finding that the utility of BSR can be held constant as the current and frequency of the stimulation are varied in a compensatory fashion [14, 15]. A high-current, low-frequency train can be adjusted so that it is preferred equally to a low-current, high-frequency train. The simplest explanation of this very robust trade-off is that the directly stimulated stage of the system employs an aggregate rate code to represent a single dimension of utility. According to this view, it is the total impulse flow in the stimulated population within a given time window that determines the utility of the BSR. If so, to what dimension of utility does the stimulation-evoked activity correspond?
To a hungry animal, a 1.0 molar solution of sucrose has higher utility than a 0.1 molar solution: the animal will work harder for the higher concentration and will select it in preference to the lower concentration. Let us call the dimension along which the utilities of these two solutions differ the "intensity of reinforcement." In addition to its dependence on stimulus strength, the intensity of reinforcement depends on physiological state. Although the 1.0 molar sucrose solution has a higher utility than a 0.1 molar solution to a hungry animal, a sated animal is likely to treat the two solutions with indifference, and an overfed animal will resist attempts to induce it to ingest either solution. Thus, the intensity of reinforcement is derived from a weighting of sensory input by signals reflecting physiological state.
The utility of the gustatory stimuli in the experiments of Conover et al. depended on physiological state, and, although this was not tested explicitly, utility would surely have depended as well on concentration. One way to account for the summation between the gustatory and electrical rewards is to propose that at some level of neural processing, both rewards elicit signals representing the intensity of reinforcement, signals that are mapped ultimately into utilities.
Perhaps due to the influence of hedonic theories, most research on the neurobiology of reward has focused on the intensity of reinforcement rather than on other dimensions that contribute to utility. Nonetheless, the rate of reinforcement is no less crucial a determinant of utility than stimulation strength. The utility of rewarding brain stimulation appears to depend on reinforcement rate in the same manner as the utility of natural reinforcers such as food [16, 17].
Herrnstein's matching law [18, 19, 20] provides a means of linking behavioral output to the rate at which reinforcers are harvested. According to the strict form of the matching law, animals allocate time or responses in proportion to the ratios of the payoffs provided by available reinforcers; if delay, amount, and kind of reinforcement are held constant, payoff is treated as the product of the intensity and rate of reinforcement [21]. If so, then ratio-scaled measurements of the intensity of reinforcement can be derived from observation of behavioral allocation and the obtained rate of reinforcement. Such measurements were taken over 20 years ago in pigeons presented with different types of seed [22]. In recent years, Gallistel and his students have exploited this strategy to measure how the intensity of BSR grows as a function of stimulation strength (the "reward-growth" function). Thus, the principal focus of recent work on the rate of electrical reinforcement has been to use this dimension as a means of scaling the intensity of the reinforcing effect.
Using matching-based methods, Gallistel and his co-workers have shown that as the stimulation frequency is increased above threshold, the intensity of BSR grows rapidly at first and then decelerates, eventually approaching asymptote [15, 23, 24, 25]. At high stimulation currents, reinforcement intensity levels off at relatively low frequencies, well below the frequency-following capacity of the directly-stimulated fibers [15]. This suggests that the process responsible for spatiotemporal integration of the stimulation-evoked activity saturates when the aggregate rate of firing in the directly activated cells exceeds some critical level. These results have important implications for the interpretation of BSR data, particularly in applications where it is critical how the change in reinforcing intensity is scaled, such as measurement of the effects of drugs, lesions, and physiological manipulations. A note of caution: investigators who seek to replicate or extend these significant findings should take pains to determine whether the allocation of behavior by their subjects is as sensitive to changes in the intensity or rate of reinforcement as the strict form of the matching law predicts (i.e., do the subjects match, "undermatch," or "overmatch [26]?").
Gallistel and his co-workers have used matching to re-assess conclusions of earlier experiments carried out with weaker measurement methods. Simmons and Gallistel measured preference between BSR trains delivered at different currents and frequencies. Like the older results [14], the new data support the "counter model," the notion that reinforcement intensity is determined by the total impulse flow induced by the stimulation during a fixed time window [15]. In another matching experiment, Mark and Gallistel [25] confirmed a striking feature of temporal integration in the BSR substrate [27]. With stimulation frequency held constant, intensity of reinforcement grows only during a period of 1-2 seconds and shows little or no further increase with prolonged stimulation. Such "duration neglect" has also been found in the case of a variety of aversive stimuli in humans, and may be a general feature of encoding mechanisms that store a compressed record of ongoing experience in memory [28].
In another line of recent work, Gallistel's group have shifted their focus to the coding of the rate of reinforcement rather than simply using the effect of rate to scale the growth of intensity. The issue they have addressed is the kinetics of the process by which the rat adjusts its subjective estimate of rate to conform to changes in the objective rate [29]. A rat working on a variable-interval schedule whose mean value sometimes changes is faced with a difficult statistical problem. When the rat encounters a long inter-reinforcement interval or a series thereof, how does it determine whether it has merely encountered a sample drawn from the tail of a stationary distribution of intervals or whether the mean of the distribution has changed? On the basis of the results of an initial experiment, Gallistel and Mark have argued that the rat may continuously adjust its estimate of the mean on an interval-by-interval basis [23, 29]. The generality of this finding remains to be determined, but the issue raised by the experiment is of clear significance. Working out the processes by which an animal judges stationarity and adjusts its predictions of inter-reinforcement intervals would seem to be essential to understanding the temporal dynamics of goal-directed behavior in a changeable world.
The utility of a natural reinforcer has been shown to decrease hyperbolically as a function of the delay between the reinforced response and the delivery of the reinforcer [30]. The delay-discount function for BSR appears to have the same hyperbolic form as the function for natural reinforcers [31]. Thus, however impoverished the stimulation-induced signal, it would seem to be sufficient to provide a normal input to the mechanism that measures response-reinforcement intervals. There is a substantial and interesting literature on the neurobiological basis of the stopwatch-like interval timer that appears to carry out such assessments and which may also serve to measure inter-reinforcement intervals [32*, 33, 34]. To build a satisfactory neurobiological theory of utility computation, it will be necessary to understand how the interval timer interacts with the circuitry that computes the intensity of reinforcement.
If, contrary to the proposal of Shizgal and Conover, the electrical stimulation in a BSR experiment does recreate the sensory experience of a goal object, then one would expect that the rewarding stimulation would not substitute equally well for reinforcers of different kinds. For example, water is a poor substitute for food in operant experiments [35, 36]. If BSR mimicked the sensory properties of a particular food, it should substitute poorly for water but should substitute well for the food whose properties it mimics.
Substitution effects are measured by offering the subject two reinforcers, each on a different ratio schedule of reinforcement. The "budget" is fixed by the experimenter so that the subject has a constant number of responses to "spend" but is free to allocate this budget among the two reinforcers in any way it wishes. The "prices" of the reinforcers are varied by altering the ratio requirements. Thus, water may "cost" ten presses per drop and food twenty presses per pellet. The degree to which the reinforcers are substitutes or complements (the opposite of substitutes) is determined by altering the prices of the reinforcers while adjusting the budget so that if the subject so chooses, it can continue to "purchase" the same amounts of both reinforcers [35]. If the two reinforcers are perfect substitutes, then the subject's new purchases will reflect the change in relative prices; increasing the relative price of reinforcer a will shift consumption toward reinforcer b. However, if the two reinforcers are perfect complements, the subject will continue to purchase the two goods in the same ratio, regardless of relative price. Conventional examples of perfect complements in economics are left and right shoes or bicycle frames and bicycle wheels.
Goods that satisfy different physiological requirements, such as food and water, are treated as poor substitutes. Given that no amount of one can satisfy the requirement for the other, choice is insensitive to relative price. Thus, operant measurement of substitution is a means of determining whether BSR is perceived as a reinforcer of the same kind as food, water, or another resource. Using lateral hypothalamic stimulation in rats, Green and Rachlin [35] have carried out such an experiment and have obtained a clear result that anticipates some of the later findings of Conover et al. Although, as expected, food and water were poor substitutes, BSR was an effective substitute for both. Thus, the signal injected by the electrode is unlikely to have recreated the sensory experience corresponding either to food, water, or their combination. Rather, the electrode appears to have simulated some common aspect of the effect produced by food and water.
Perhaps the common effect of food, water and BSR is a neural signal representing the intensity of reinforcement. If so, such a signal is but one of many that compose the representation of a natural goal object. In this view, the stimulation does not cause the rat to hallucinate a food pellet or a draught of water. Rather, the stimulation merely injects a unidimensional intensity signal that normally contributes to determining what resources such as food or water are worth. This distinction is reminiscent of one that emerges from the findings of Newsome and co-workers in another type of decision task. They have documented how microstimulating motion-sensitive cells in area MT of the monkey cortex alters perceptual judgment of apparent motion [37]. Microstimulation of area MT alone is unlikely to evoke the image of a moving dot, because such an image would require simultaneous activity in the additional visual areas that process attributes of the dot (e.g., color and form) other than direction of movement. Newsome speculates that the microstimulation in area MT conveys only an aspect of the experience produced by the apparent motion display: an impression of movement, such as the opponent motion aftereffect one experiences after watching a waterfall for some time and then looking at a stationary scene [38].
Figure 1 summarizes the hypothesis developed above concerning the computation of the utility of BSR. The counter model of spatiotemporal integration [14, 15] is represented on the left. The unidimensional output of the counter is transformed into the intensity of reinforcement by a non-linear function similar to that described by Gallistel's group (depicted here as a logistic on double logarithmic coordinates). The intensity of reinforcement is then combined, by multiplication, with the rate of reinforcement. The response-strength equation in Figure 1 is Herrnstein's [19, 20]; time is allocated to working for BSR as a function of the ratio of the utility produced by the brain stimulation (UB) and the utility of all the available reinforcers (UB + UE , where UE represents the sum of the utilities of "everything else:" alternate activities such as grooming, exploring, and resting). In future work, it will be important to determine how other dimensions of utility, such as delay and cost, should be accommodated. Scalar expectancy theory provides one framework for combining the effects of delay and rate [34, 39].
|
| A model of how the utility of rewarding medial forebrain bundle stimulation is computed and translated into behavioral allocation. The action potentials triggered in the directly-stimulated neurons are accumulated by a counter. The spike count is then transformed non-linearly into a signal representing the intensity of reinforcement. The rate of reinforcement is assessed by an interval timing mechanism (not shown) and then multiplied by the intensity of reinforcement to yield the utility of the brain stimulation (UB). Behavior is allocated to obtaining additional stimulation according to the matching law, which compares the utility of the brain stimulation to the total utility of all available reinforcers (UB + UE, where UE represents the utility of "everything else" in the test environment, for example, the reinforcement available from activities such as grooming, exploring, and resting). |
The results of matching experiments suggest that a computation analogous to that described in Figure 1 is carried out in the case of a single kind of natural reinforcer. When reinforcers of different kinds are available, the degree to which they act as substitutes or complements must be incorporated into the computation [36].
The input to the model in Figure 1 is a stream of observable action potentials propagating away from a known location in the central nervous system: the tip of the stimulating electrode. This renders the process that computes the utility of BSR highly amenable to analysis at the level of cells and circuits. A logical starting point for such an analysis is the population of neurons (depicted at the left of Figure 1) that are directly excited by the electrode and give rise to the signal that ultimately produces the rewarding effect. Given the evidence for summation between BSR and the rewarding effects of natural stimuli [3, 4, 13**], finding these cells, tracing their inputs and outputs, and recording their activity in conscious subjects is likely to shed light on the processes that compute the utility of natural reinforcers.
Rolls [40], Ono [41], and their colleagues have carried out pioneering work on recording responses to natural reinforcers in single neurons activated by rewarding brain stimulation. In future work, it will be important to pursue the course they have set while taking advantage of recent advances in measuring the intensity of reinforcement produced by both gustatory and electrical stimuli [3, 24, 25] and in characterizing physiological and anatomical properties of neurons subserving BSR [42*, 43, 44*].
Directly-stimulated neurons. The structure investigated most intensively in research on BSR is the medial forebrain bundle (MFB), which will serve as the focus for this review of recent work at the level of cells and circuits. Nonetheless, it is important to note that electrical stimulation of a large and disparate set of additional sites [45*] can produce rewarding effects. The broad dispersion of effective sites and the functional heterogeneity that may well underlie it must ultimately be addressed. The best characterized of the sites outside the MFB are distributed along the neuraxis throughout much of the brainstem core [42*, 46]. This region is traversed by long reward-related fibers that extend at least as far as the ventral tegmental pole of the MFB [42*].
The experiments that gave rise to the model in Figure 1 were carried out with MFB electrodes. Much recent work has been aimed at mapping the icon representing the directly stimulated cells in Figure 1 onto real neurons with known nuclei of origin, axonal trajectories, and terminal fields. On the basis of behavioral measurements of recovery from refractoriness [44*, 47, 48], collision block [44*, 49, 50], and anodal hyperpolarization block [51], it has been proposed that at least some of the directly-stimulated fibers responsible for the rewarding effect of MFB stimulation have fine, myelinated axons in which the behaviorally relevant direction of conduction is rostro-caudal. The simplest anatomical arrangement compatible with this model places the somata of origin in, or rostral to, the lateral hypothalamus (LH) and the synaptic terminals in, or caudal to, the ventral tegmental area (VTA). This model has been dubbed the "descending-path" hypothesis.
Recent evidence of several different types is consistent with the descending path hypothesis. Electrolytic [52, 53*] and excitotoxic lesions [54**] of a region that includes the rostral LH, the lateral preoptic area, the substantia innominata, and parts of the bed nucleus of the stria terminalis have been shown to increase the threshold for self-stimulation of more caudal sites arrayed along the descending MFB projections from the lesioned area. Although the electrolytic lesions produced substantial transient and/or sustained increases in threshold at about half the tested LH and VTA sites, only one of eight subjects with excitotoxic lesions showed an increase in the VTA threshold. Thus, it is possible that the location of the somata that project only to the caudal LH differs from the location of somata that project either to the VTA or to both the caudal LH and VTA. Fos immunostaining shows that somata in the region of the effective lesions are activated by rewarding stimulation of the LH [55*, 56] and VTA [56]. Single-unit recording [57*] shows that in the basal forebrain region where lesions increase the threshold for self-stimulation of more caudal MFB sites, somata can be driven antidromically by stimulation of the LH and VTA; estimates of the axonal refractory periods and conduction velocities of these cells overlap behavioral estimates for the directly stimulated fibers subserving self-stimulation of the LH and VTA.
A puzzling aspect of the lesion results is the variation in effect size across subjects. Even when lesions and stimulation sites appear to be similarly placed, only some subjects show a decrease in reward effectiveness [53*, 54**]. Arvanitogiannis et al. [54**] have proposed a model that involves separate post-synaptic integration of activity in multiple sub-populations of directly stimulated cells and have shown that when the non-linear reward-growth function described by Gallistel's group is incorporated, this model can account for large across-subject differences in the effectiveness of lesions. To test the model, lesion effects must be measured as a function of both stimulation strength and reinforcement rate.
Other investigators have taken issue with the descending path hypothesis. Gallistel and co-workers have demonstrated that some strikingly large rostral lesions failed to reduce the frequency threshold for self-stimulation of the caudal MFB or did so only at low stimulation currents [58**]. In contrast, consistent and substantial increases in the threshold for LH self-stimulation were observed after lesions of the VTA. They concluded that the descending path hypothesis could not account for their results and proposed instead that the directly stimulated substrate consists of extensively collateralized brainstem neurons with sparse, ascending MFB projections and rich brainstem projections. In deriving this conclusion, they did not address the competing model proposed by Arvanitogiannis et al. [54**]. Nonetheless, their model is an interesting alternative and merits investigation via anatomical and physiological means. It remains to be determined whether cells such as those proposed as the directly stimulated substrate are indeed activated by rewarding MFB stimulation and if so, whether these cells have physiological properties compatible with the behaviorally derived estimates for the directly stimulated neurons subserving MFB self-stimulation. It is also important to keep in mind that the model proposed by Gallistel et al. does not account for the dozens of published cases where a rostral MFB lesion has produced a sustained or transient increase in the threshold for self-stimulation at a more caudal MFB site [52, 53*, 54**, 59]. Moreover, Gallistel et al. did not discuss the possible contribution to their results of damage to midbrain dopamine neurons, which offer an alternate means of accounting for the consistency and magnitude of the effects produced by the caudal lesions [60]. The element of their case most likely to elicit general agreement at this juncture is the claim that the failure of many MFB lesions to alter thresholds for high-current stimulation elsewhere in the MFB constitutes an important and puzzling finding in search of a convincing and satisfying explanation.
Malette and Miliaressis [61**] have proposed additional morphologies for neurons that contribute to the directly-stimulated substrate for self-stimulation of the MFB. In a meticulously executed experiment, they studied bilateral summation of BSR. The stimulation consisted of trains of pulse pairs, with one member of each pair delivered to the left MFB and the second member delivered to the right MFB. At some pairs of sites, they found that the level of bilateral summation was independent of the temporal offset between the two pulses in each pair. However, at many sites, summation was low at short offsets and high at longer ones. When the stimulation was delivered via electrodes at the same rostro-caudal level, the order in which the two hemispheres were stimulated did not matter. In contrast, when the electrodes were located at different levels of the neuraxis, the increase in summation occurred at shorter offsets when the first pulse in each pair was delivered to the more rostral electrode.
These results are interpreted in the context of a model based on branchpoint failure. The neurons responsible for the offset-dependent summation effects are posited to give rise to a minimum of three branches, at least one of which crosses the midline. It is assumed that antidromic action potentials elicited in either of the stimulated branches fail to invade the second stimulated branch but succeed in invading the third branch (the "central trunk"). Order effects arise due to differences between the conduction times from the two stimulation sites to the branchpoint. Malette and Miliaressis propose that the directly stimulated MFB substrate may include neurons with several different branching patterns as well cells with projections corresponding to the descending-path hypothesis.
The three-branch model proposed by Malette and Miliaressis represents a creative effort to deal with a very systematic and provocative set of data. However, their model can be questioned on the grounds of plausibility, and a simpler alternative can account for the results. Conduction failure is anticipated when an action potential propagating along a high-impedance (thin) branch arrives at a junction with a lower-impedance (thicker) branch; the thin branch may be unable to source enough current to drive the larger load imposed by the thicker branch. In the model proposed by Malette and Miliaressis, the stimulated collaterals are thin, and the common segment is thick. Such a neuron would be unlikely to behave as they have proposed. It would be difficult for antidromic spikes initiated in one of the thin collaterals to invade the thick common segment. However, if invasion did occur, then the thick common segment should easily initiate a spike in the second thin collateral. This is the reverse of the behavior required to account for the data. In order for the principle of impedance mismatch to generate the desired behavior from the model of Malette and Miliaressis, the two stimulated collaterals must be of larger caliber than the common segment from which they branch. This would seem to be an odd way to construct a neuron, because orthodromic action potentials that arose normally in the soma would be unlikely to invade the thick collaterals.
Spatial summation at strong, high-fidelity synapses provides an alternative means of accounting for the inter-hemispheric summation data reported by Malette and Miliaressis. Buckenham and Yeomans' have described a neural circuit [62, 63*] in which two sets of stimulated neurons are linked by a monosynaptic connection sufficiently strong to allow the post-synaptic neurons to follow presynaptic spikes on a one-to-one basis and with stable latency. Assume that an electrode in each hemisphere activates axons that converge on a common postsynaptic target and that the transmission characteristics at the synapses are as Buckenham and Yeomans have described. Summation will be lower at very short offsets than at longer ones due to the refractory period of the intial segment of the post-synaptic neurons. When the conduction times from both stimulation sites to the synapses are equal, the increase in summation will be observed at the same offset, regardless of which electrode delivers the first pulse of each pair. In contrast, when the conduction times are different, the offset required to produce the increase in summation will be shorter when the first pulse is delivered by the electrode that drives the post-synaptic response with the shorter latency. Curves of the form reported by Malette and Miliaressis would be produced if the following inequality were satisfied:
(B+C) > A > B
where A = conduction time from the caudal electrode to the
site of spike initiation in the post-synaptic neurons,
B = conduction time from the rostral electrode to the site of
spike initiation in the post-synaptic neurons,
and C = the refractory period at the site of spike initiation in
the post-synaptic neurons.
One way to achieve these conditions would be to place the somata of the post-synaptic neurons rostral to the VTA electrode. If so, the behaviorally relevant direction of conduction in the neurons responsible for the offset-dependent bilateral summation would be ascending at the level of the VTA, as in the model of inter-hemispheric summation proposed by Malette and Miliaressis.
Postsynaptic elements. It has long been established that MFB self-stimulation is dependent on neurotransmission in the dopaminergic mesoaccumbens projection [45*, 60]. However, it is unlikely that the directly-stimulated substrate is composed principally of dopaminergic neurons. A recent study of the electrophysiological properties of dopaminergic fibers in the MFB [64*] confirms earlier results [65]: refractory periods of the dopaminergic fibers are too long and the conduction velocities too low to account for the excitability and conduction properties of the directly stimulated neurons subserving MFB self-stimulation [49, 50]. Although some contribution of dopaminergic neurons to the later portions of the recovery from refractoriness and collision block cannot be ruled out, the high thresholds of the dopaminergic fibers [64*, 65] makes it unlikely that many would be stimulated directly in BSR experiments given the pulse durations, currents, and electrode tip dimensions commonly employed [12].
Opinion is divided about the nature of the dopaminergic contribution to BSR. In some contemporary theories, dopaminergic neurons are in series with the reward-related signals. For example, Yeomans has proposed that the rewarding effect of MFB stimulation arises from directly stimulated activity in descending MFB fibers that is relayed to midbrain dopaminergic cells by brainstem cholinergic neurons [66]. A study by Miliaressis, Emond, & Merali [67] challenges this view of the dopaminergic contribution. Release of dopamine in the nucleus accumbens, as measured by in-vivo dialysis, varied across pairs of currents and pulse durations that appeared to produce equivalent rewarding effects at either LH or VTA stimulation sites. At face value, this finding does not fit gracefully with the notion [68] that mesoaccumbens dopamine neurons either constitute the output stage of the neural circuit subserving the rewarding effect or relay the output of the directly stimulated substrate to efferent stages of the circuit. However, there is evidence that directly stimulated fibers of substantially different excitability contribute to the rewarding effect of MFB stimulation [43, 48]. If so, the size of the stimulated region would likely vary across a set of equally rewarding pairs of currents and pulse durations, and thus, the spatial distribution of dopamine release might vary as well.
In several recent formulations, a less direct role in the rewarding effect has been proposed for the dopaminergic neurons. For example, it has been argued that dopaminergic neurotransmission is necessary to prepare the animal for goal-directed action [69], to compare experienced and expected rewards [70*, 71], or to transfer information about recently experienced rewards to memory [72]. In such formulations, the computation of primary reward is the province of non-dopaminergic cells, such as the ones proposed as the directly activated substrate for BSR, but dopaminergic cells are crucial for learning about reinforcers or for directing behavior towards obtaining them. A major priority for future research on BSR is to clarify the contribution of the dopaminergic projections.
Another focus for ongoing work is the cholinergic projection from the pedunculopontine nucleus and the laterodorsal tegmental nucleus to the VTA and substantia nigra. Yeomans and co-workers have shown that blockade of muscarinic receptors in the VTA produces very large increases in the threshold for LH self-stimulation [73*, 74] and that smaller increases are seen following the administration of nicotinic blockers [73*]. Moreover, in vivo dialysis reveals that rewarding LH stimulation releases acetylcholine in the VTA; reverse dialysis of atropine via the same probes blocks self-stimulation [75]. Thus, the brainstem cholinergic projections would appear to compose a key element of the neural substrate for the rewarding effect. Recent lesion results [76*] can be interpreted to provide at least partial support of this view.
Let us return to Jacob Horner, frozen in a nocturnal trance on a bench in a Baltimore railroad station. The outlook for Horner is bleak. Although he is about to learn a set of rules that can deliver him from paralysis, neither sinistrality, antecedence, nor alphabetic priority is likely to steer Horner reliably towards salubrious activities and away from noxious ones.
The outlook is much better for the rat that scurries past Horner's feet en route to retrieving a half-eaten hot dog discarded by a thoughtless traveler. The rat eschews arbitrary rules, such as sinistrality, in favor of the dexterous application of systematic ones. In lieu of employing antecedence as a decision rule, the rat focuses on the likely consequences of available actions. Its priorities are ranked according to sober assessment of potential payoffs, not according to a whimsical sort. The business-like fashion in which the rat conducts its foraging rounds has much to do with the smooth operation of neural circuitry that evaluates sensory inputs as a function of physiological state, records the temporal and spatial density of resources in the environment, assesses the kinds and costs of these resources, and allocates behavior accordingly. Research on BSR suggests that certain myelinated MFB axons may make a vital contribution to the evaluative process. According to the hypothesis presented here, the activity of these myelinated MFB axons participates in encoding the intensity of reinforcement, one of the fundamental dimensions that enter into the computation of utility. Intensive research will be required to test this hypothesis, to identify the directly stimulated cells responsible for the rewarding effect of MFB stimulation, and to understand the function and structure of the circuitry in which these neurons are embedded.
* denotes a paper published within the
previous two years that was deemed to be "of special
interest."
** denotes a paper published within the previous two years
that was deemed to be of "outstanding interest."
1. Bentham J: An introduction to the principles of morals and
legislation. Edited by Burns JH, Hart HLA. Oxford: Clarendon
Press; 1996 (originally published: 1789).
2. Cabanac M: Pleasure: the common currency. J Theor Biol 1992, 155:173--200.
3. Conover KL, Shizgal P: Competition and summation between rewarding effects of sucrose and lateral hypothalamic stimulation in the rat. Behav Neurosci 1994, 108(3):537--548.
4. Conover KL, Woodside B, Shizgal P: Effects of sodium depletion on competition and summation between rewarding effects of salt and lateral hypothalamic stimulation in the rat. Behav Neurosci 1994, 108(3):549--558.
5. Conover KL, Shizgal P: Differential effects of postingestive feedback on the reward value of sucrose and lateral hypothalamic stimulation in the rat. Behav Neurosci 1994, 108(3):559--572.
*6. Woodside B, Renaudin A, Shizgal P: Adminstration of ovarian steroid hormones does not change the reward effectiveness of lateral hypothalamic stimulation in ovariectomized rats. Psychobiol 1996, 24(3):202--210.
Hormonal manipulations that alter sucrose preference and body weight fail to produce meaningful changes in the threshold for self-stimulation of the medial forebrain bundle. This result is consistent with the findings of Conover and co-workers showing that self-stimulation of the medial forebrain bundle is relatively unaffected by physiological manipulation that profoundly alter the rewarding properties of tastants.
*7. Bless EP, McGinnis KA, Mitchell AL, Hartwell M, Mitchell JB: The effects of gonadal steroids on brain stimulation reward in female rats. Behav Brain Res 1997, 82:235-244.
Curves relating the rate of lever pressing to the frequency of lateral hypothalamic stimulation were collected from female rats. The curves obtained during estrus were shifted slightly leftward from those collected during diestrus. Similar shifts were observed when curves obtained from ovariectomized rats given estradiol and progesterone were compared to curves obtained from ovariectomized controls. These results were interpreted as evidence that ovarian hormones potentiate the rewarding effect of the lateral hypothalamic stimulation. However, the size of the shifts in the representative curves shown is very small, falling well within the range of variation that served as the criterion for within-session stability. Moreover, larger shifts than this have been produced by performance manipulations such as adding weight to the lever.
8. Fouriezos G, Bielajew C, Pagotto W: Task difficulty increases thresholds of rewarding brain stimulation. Behav Brain Res 1990, 37:1-7.
*9. Abrahamsen GC, Berman Y, Carr KD: Curve-shift analysis of self-stimulation in food-restricted rats: relationship between daily meal, plasma corticosterone and reward sensitization. Brain Res 1995, 695:186--194.
A demonstration that severe food restriction lowers the threshold for self-stimulation of the lateral hypothalamus. In their present form, the models proposed by Conover et al. do not account for this finding. One avenue for investigation in future work is to determine whether differences in electrode placement in the studies of Carr and co-workers and those of Conover et al. might account for the differential sensitivity to physiological modulation. Another issue that merits attention is whether the effect reported by Carr and co-workers reflects a specific enhancement of gustatory reward or a more generalized potentiation of appetitive behavior.
10. Rossi J, Panksepp J: Analysis of the relationships between self-stimulation sniffing and brain-stimulation sniffing. Physiol Behav 1992, 51:805--813.
11. Carr KD, Wolinsky TD: Chronic food restriction and weight loss produce opioid facilitation of perifornical hypothalamic self-stimulation. Brain Res 1993, 607:141--148.
12. Yeomans JS: Principles of Brain Stimulation. Edited . New York: Oxford University Press; 1990.
**13. Shizgal P, Conover K: On the neural computation of utility. Curr Dir Psychol Sci 1996, 5(2):37--43.
This paper extends the proposal of Conover, Woodside, and Shizgal concerning a common currency for the evaluation of tastants and rewarding electrical brain stimulation. It is argued that the rewarding electrical stimulation is able to produce a meaningful neural signal because of the unidimensional nature of coding in the stimulated system. The weighting of the gustatory stimulus by physiological state appears to occur prior to the combination of the electrical and gustatory reward signals.
14. Gallistel CR, Shizgal P, Yeomans JS: A portrait of the substrate for self-stimulation. Psychol Rev 1981, 88:228--273.
15. Simmons JM, Gallistel CR: Saturation of subjective reward magnitude as a function of current and pulse frequency. Behav Neurosci 1994, 108:151--160.
16. Heyman GM, Monaghan MM: Reinforcer magnitude (sucrose concentration) and the matching law theory of response strength. J Exp Anal Behav 1994, 61:505-516.
17. Hamilton AL, Stellar JR, Hart EB: Reward, performance, and the response strength method in self-stimulating rats: validation and neuroleptics. Physiol Behav 1985, 35:897--904.
18. Herrnstein RJ: Relative and absolute strength of response as a function of frequency of reinforcement. J Exp Anal Behav 1961, 4:267--272.
19. Herrnstein RJ: On the law of effect. J Exp Anal Behav 1970, 13(2):243--266.
20. Herrnstein RJ: Formal properties of the matching law. J Exp Anal Behav 1974, 21:159--164.
21. Baum WM, Rachlin H: Choice as time allocation. J Exp Anal Behav 1969, 12:861--874.
22. Miller HL: Matching-based hedonic scaling in the pigeon. J Exp Anal Behav 1976, 26:335--347.
23. Gallistel CR: Foraging for brain stimulation: toward a neurobiology of computation. Cognition 1994, 50:151--170.
24. Gallistel CR, Leon M: Measuring the subjective magnitude of brain stimulation reward by titration with rate of reward. Behav Neurosci 1991, 105(6):913--925.
25. Mark TA, Gallistel CR: Subjective reward magnitude of medial forebrain stimulation as a function of train duration and pulse frequency. Behav Neurosci 1993, 107(2):389--401.
26. Davison M, McCarthy D: The Matching Law. Edited . Hillsdale, NJ: Lawrence Erlbaum Associates; 1988.
27. Shizgal P, Matthews G: Electrical stimulation of the rat diencephalon: Differential effects of interrupted stimulation on on- and off-responding. Brain Res 1977, 129:319-333.
28. Kahneman D, Fredrickson BL, Schreiber CA, Redelmeier DA: When more pain is preferred to less: adding a better end. Psychol Sci 1993, 4(6):401--405.
29. Mark TA, Gallistel CR: Kinetics of Matching. J Exp Psychol 1994, 20(1):1--17.
30. Mazur JE: Choice between single and multiple delayed reinforcers. J Exp Anal Behav 1986, 46:67--78.
31. Mazur JE, Stellar JR, Waraczynski M: Self-control choice with electrical stimulation of the brain. Behav Processes 1987, 15:143--153.
*32. Meck WH: Neuropharmacology of timing and time perception. Cog Brain Res 1996, 3:227--242.
A review of the psychological, pharmacological, and anatomical properties of the interval timer that likely measures inter-reinforcement intervals and response-reinforcement delays.
33. Church RM: Properties of the internal clock. In Timing and Time Perception, Edited by Gibbon J, Allan L. New York: New York Academy of Sciences; 1984:566--582.
34. Gibbon J: Scalar expectancy theory and Weber's law in animal timing. Psychol Rev 1977, 84(3):279--325.
35. Green L, Rachlin H: Economic substitutability of electrical brain stimulation, food, and water. J Exp Anal Behav 1991, 55:133--143.
36. Rachlin H, Kagel JH, Battalio RC: Substitutability in time allocation. Psychol Rev 1980, 87:355--374.
37. Salzman CD, Murasugi CM, Britten KH, Newsome WT: Microstimulation in visual area MT: effects of direction discrimination performance. J Neurosci 1992, 12(6):2331--2355.
38. Newsome WT, Salzman CD: The neuronal basis of motion perception. Ciba Foundation Symposium 1993, 174:217--246.
39. Gibbon J, Church RM, Fairhurst S, Kacelnik A: Scalar expectancy theory and choice between delayed rewards. Psychol Rev 1988, 95(1):102-114.
40. Rolls ET, Burton MJ, Mora F: Neurophysiological analysis of brain-stimulation reward in the monkey. Brain Res 1980, 194:339--357.
41. Ono T, Nakamura K, Nishijo H, Fukuda M: Hypothalamic neuron involvement in integration of reward, aversion, and cue signals. J Neurophysiol 1986, 56:63--79.
*42. Boye SM, Rompré PP: Mesencephalic substrate of reward: axonal connections. J Neurosci 1996, 16:3511--3520.
Using a behavioral adaptation of the collision method, the authors show that reward-related fibers directly link self-stimulation sites in the caudal mesencephalon and the ventral tegmental area. At least some of these fibers appear to extend as far as the lateral hypothalamus.
43. Murray B, Shizgal P: Evidence implicating both slow- and fast-conducting fibers in the rewarding effect of medial forebrain bundle stimulation. Behav Brain Res 1994, 63:47--60.
*44. Murray B, Shizgal P: Behavioral measures of conduction velocity and refractory period for reward-relevant axons in the anterior LH and VTA. Physiol Behav 1996, 59(4/5):643--652.
An extension of earlier work tracing the trajectory of reward-related fibers in the medial forebrain bundle by means of a behavioral adaptation of the collision method. The results suggest that reward-related fibers directly link the rostral portion of the lateral hypothalamus to the ventral tegmental area.
*45. Wise RA: Addictive drugs and brain stimulation reward. Ann Rev Neurosci 1996, 19:319--340.
A review of the relationship between the rewarding effects of drugs of abuse and electrical brain stimulation.
46. Rompré PP, Miliaressis E: Pontine and mesencephalic substrates of self-stimulation. Brain Res 1985, 359(1-2):246--259.
47. Yeomans JS: Quantitative measurement of neural post-stimulation excitability with behavioral methods. Physiol Behav 1975, 15:593--602.
48. Yeomans JS: Absolute refractory periods of self-stimulation neurons. Physiol Behav 1979, 22:911--919.
49. Bielajew C, Shizgal P: Behaviorally derived measures of conduction velocity in the substrate for rewarding medial forebrain bundle stimulation. Brain Res 1982, 237:107--119.
50. Shizgal P, Bielajew C, Corbett D, Skelton R, Yeomans J: Behavioral methods for inferring anatomical linkage between rewarding brain stimulation sites. J Comp Physiol Psychol 1980, 94:227--237.
51. Bielajew C, Shizgal P: Evidence implicating descending fibers in self-stimulation of the medial forebrain bundle. J Neurosci 1986, 6(4):919--929.
52. Murray B, Shizgal P: Anterolateral lesions of the medial forebrain bundle increase the frequency threshold for self-stimulation of the lateral hypothalamus and ventral tegmental area in the rat. Psychobiol 1991, 19(2):135--146.
*53. Murray B, Shizgal P: Attenuation of medial forebrain bundle reward by anterior lateral hypothalamic lesions. Behav Brain Res 1996, 75:33--47.
A further demonstration that lesions rostral to medial forebrain bundle self-stimulation sites can increase the threshold for the rewarding effect.
**54. Arvanitogiannis A, Waraczynski M, Shizgal P: Effects of excitotoxic lesions of the basal forebrain on MFB self-stimulation. Physiol Behav 1996, 59(4/5):795--806.
A key finding in support of the "descending-path" hypothesis: excitotoxic lesions in the basal forebrain can produce large increases in the threshold for self-stimulation of the lateral hypothalamus. To account for the large across-subject variance in the magnitude of the lesion effects, the authors propose a model that incorporates multiple subpopulations of reward-related cells and a non-linear reward-growth function.
*55. Arvanitogiannis A, Flores C, Pfaus JG, Shizgal P: Increased ipsilateral expression of Fos following lateral hypothalamic self-stimulation. Brain Res 1996, 720:148--154.
Using expression of the immediate-early gene product, Fos, as an index of neural activation, this paper demonstrates that rewarding electrical stimulation of the lateral hypothalamus produces neural activation in a variety of forebrain regions, predominantly, but not exclusively, in the stimulated hemisphere. Among the areas where the stimulation triggered expression of the Fos protein is the region where lesions increase the threshold for self-stimulation of the medial forebrain bundle.
56. Flores C, Arvanitogiannis A, Shizgal P: Fos-like immunoreactivity in forebrain regions following self-stimulation of the lateral hypothalamus and the ventral tegmental area. Behav Brain Res 1997, in press.
*57. Murray B, Shizgal P: Physiological measures of conduction velocity and refractory period for putative reward-relevant MFB axons arising in the rostral MFB. Physiol Behav 1996, 59(3):427--437.
In this study, single-unit recording is employed to measure excitability and conduction properties of fibers arising in the basal forebrain region where lesions reduce the reward effectiveness of stimulation delivered at more caudal sites along the medial forebrain bundle. The axonal refractory periods and conduction velocities of the antidromically activated neurons overlap the values that have been estimated for reward-related fibers by behavioral means. Such cells could compose part of the "descending path" hypothesized to link the lateral hypothalamic and ventral tegmental levels of the medial forebrain bundle via reward-related fibers.
**58. Gallistel CR, Leon M, Lim BT, Sim JC, Waraczynski M: Destruction of medial forebrain bundle caudal to the site of stimulation reduces rewarding efficacy but destruction rostrally does not. Behav Neurosci 1996, 110(4):1--25.
The authors report that lesions of the ventral tegmental area produce relatively large and consistent increases in the threshold for lateral hypothalamic self-stimulation, whereas lateral hypothalamic lesions produce smaller and much less consistent increases in the threshold for self-stimulation of the ventral tegmental area and/or caudal medial forebrain bundle. These results are interpreted as evidence against the descending-path hypothesis, and an imaginative alternative is proposed. However, the new model cannot account for the dozens of cases in which rostrally placed lesions have increased the self-stimulation threshold at more caudal sites along the medial forebrain bundle. Moreover, the authors appear to disregard damage to dopaminergic neurons as a factor contributing to the effects of their ventral tegmental area lesions.
59. Waraczynski M, Conover K, Shizgal P: Rewarding effectiveness of caudal MFB stimulation is unaltered following DMH lesions. Physiol Behav 1992, 52:211--218.
The principal message of this paper is the failure of dorsomedial hypothalamic lesions to alter self-stimulation of the caudal medial forebrain bundle. However, in one subject, the lesion was in the lateral hypothalamus, and it produced very large increases in the threshold for self-stimulation of the caudal medial forebrain bundle. This striking result is among those that are not explained by the model proposed by Gallistel et al.
60. Wise RA, Rompré PP: Brain dopamine and reward. Ann Rev Psychol 1989, 40:191--225.
**61. Malette J, Miliaressis E: Interhemispheric links in brain stimulation reward. Behav Brain Res 1995, 68:117--137.
Using stimulation trains consisting of pulse pairs delivered to the medial forebrain bundle in each hemisphere, bilateral summation of rewarding effects was studied. Contrary to previous reports, summation at many pairs of sites depended on the pulse-pair interval. With electrodes at different levels of the neuraxis, the pulse-pair interval at which summation increased often depended on the order in which the two sites were stimulated. These interesting effects are interpreted in the light of a model of branchpoint failure in neurons with multiple axonal branches. However, key assumptions underlying the branchpoint model appear implausible, and a simpler model, based on synaptic convergence, can account for the results. In the simplest anatomical arrangements compatible with both models, the behaviorally relevant direction of conduction is ascending at the level of the VTA.
62. Buckenham K, Yeomans JS: An uncrossed tectopontine pathway mediates ipsiversive circling. Behav Brain Res 1993, 54:11--22.
*63. Yeomans JS: Electrically evoked behaviors: axons and synapses mapped with collision tests. Behav Brain Res 1995, 67:121--132.
A review showing how pulse-pair stimulation can be used to trace both axonal and synaptic connections in the neural circuitry subserving behavioral effects of electrical brain stimulation.
*64. Anderson RM, Fatigati MD, Rompé P-P: Estimates of the axonal refractory period of midbrain dopamine neurons: their relevance to brain stimulation reward. Brain Res 1996, 718:83--88.
A carefully executed study in which the refractory periods of midbrain dopaminergic axons were estimated. Unlike conventional estimates, the ones obtained by the method employed in this study are relatively unaffected by the properties of the soma and initial segment. Thus, these values can be compared meaningfully to those derived from behavioral experiments on self-stimulation of the medial forebrain bundle, which also are likely to reflect axonal properties.
65. Yeomans JS, Maidment NT, Bunney BS: Excitability properties of medial forebrain bundle axons of A9 and A10 dopamine cells. Brain Res 1988, 450:86--93.
66. Yeomans JS, Mathur A, Tampakeras M: Rewarding brain stimulation: role of tegmental cholinergic neurons that activate dopamine neurons. Behav Neurosci 1993, 197:1077--1087.
67. Miliaressis E, Emond C, Merali Z: Re-evaluation of the role of dopamine in intracranial self-stimulation using in vivo microdialysis. Behav Brain Res 1991, 46:43--48.
68. Wise RA: Action of drugs of abuse on brain reward systems. Pharmacol Biochem Behav 1980, 13:213-23.
69. Blackburn JR, Pfaus JG, Phillips AG: Dopamine functions in appetitive and defensive behaviours. Prog Neurobiol 1992, 39:247--279.
*70. Montague PR, Dayan P, Sejnowski TJ: A framework for mesencephalic dopamine systems based on predictive Hebbian learning. J Neurosci 1996, 16(5):1936--1947.
An interesting model in which dopamine neurons signal the difference between experienced and expected rewards. If this model is correct, then the primary reward signal has already been computed before it is relayed to midbrain dopamine neurons.
71. Schultz W, Apicella P, Ljungberg T: Responses of monkey dopamine neurons to reward and conditioned stimuli during successive steps of learning a delayed response task. J Neurosci 1993, 13:900--913.
72. Gallistel CR: The role of the dopaminergic projections in MFB self-stimulation. Behav Brain Res 1986, 20:313--321.
*73. Yeomans J, Baptista M: Both nicotinic and muscarinic receptors in ventral tegmental area contribute to brain-stimulation reward. Pharmacol Biochem Behav In press.
Evidence that cholinergic input to the ventral tegmental area can modulate the rewarding effect of medial forebrain bundle stimulation.
74. Kofman O, McGlynn SM, Olmstead MC, Yeomans JS: Differential effects of atropine, procaine and dopamine in the rat ventral tegmentum on lateral hypothalamic rewarding brain stimulation. Behav Brain Res 1990, 38:55--68.
75. Rada P, Gibbs G, Yeomans J, Hoebel B: Soc Neurosci Abstr 1996, 22:683.
*76. Lepore M, Franklin KBJ: N-Methyl-D-Aspartate lesions of the pedunculopontine nucleus block acquisition and impair maintenance of responding reinforced with brain stimulation. Neuroscience 1996, 71(1):147-155.
Performance of a task in which lever pressing was rewarded by a frequency-modulated train of lateral hypothalamic stimulation was monitored following excitotoxic lesions of the pedunculopontine region; the stimulation was patterned to mimic the waxing and waning of drug action during absorption and elimination. That bilateral lesions blocked acquisition of the task in naive subjects and attenuated performance for BSR in rats trained prior to the lesion is consistent with Yeomans' hypothesis that cholinergic neurons in this region constitute a critical link in the neural circuitry responsible for the rewarding effect. Nonetheless, it is not clear why unilateral lesions were ineffective whereas unilateral administration of drugs that alter the firing of cholinergic somata in the pedunculopontine region has been shown to alter BSR [66]. The use of response rate as the behavioral measure of reward in the study of Lepore et al. complicates interpretation of the findings. Response rates can be insensitive to changes in reward and may be altered by manipulations that change performance capacity rather than the intensity of reinforcement.