Inbreeding depression is high in a self‐incompatible perennial herb population but absent in a self‐compatible population showing mixed mating

High inbreeding depression is thought to be one of the major factors preventing evolutionary transitions in hermaphroditic plants from self‐incompatibility (SI) and outcrossing toward self‐compatibility (SC) and selfing. However, when selfing does evolve, inbreeding depression can be quickly purged, allowing the evolution of complete self‐fertilization. In contrast, populations that show intermediate selfing rates (a mixed‐mating system) typically show levels of inbreeding depression similar to those in outcrossing species, suggesting that selection against inbreeding might be responsible for preventing the transition toward complete self‐fertilization. By implication, crosses among populations should reveal patterns of heterosis for mixed‐mating populations that are similar to those expected for outcrossing populations. Using hand‐pollination crosses, we compared levels of inbreeding depression and heterosis between populations of Linaria cavanillesii (Plantaginaceae), a perennial herb showing contrasting mating systems. The SI population showed high inbreeding depression, whereas the SC population displaying mixed mating showed no inbreeding depression. In contrast, we found that heterosis based on between‐population crosses was similar for SI and SC populations. Our results are consistent with the rapid purging of inbreeding depression in the derived SC population, despite the persistence of mixed mating. However, the maintenance of outcrossing after a transition to SC is inconsistent with the prediction that populations that have purged their inbreeding depression should evolve toward complete selfing, suggesting that the transition to SC in L. cavanillesii has been recent. SC in L. cavanillesii thus exemplifies a situation in which the mating system is likely not at an equilibrium with inbreeding depression.     

The divergence history of the perennial plant Linaria cavanillesii confirms a recent loss of self‐incompatibility

Many angiosperms prevent inbreeding through a self‐incompatibility (SI) system, but the loss of SI has been frequent in their evolutionary history. The loss of SI may often lead to an increase in the selfing rate, with the purging of inbreeding depression and the ultimate evolution of a selfing syndrome, where plants have smaller flowers with reduced pollen and nectar production. In this study, we used approximate Bayesian computation (ABC) to estimate the timing of divergence between populations of the plant Linaria cavanillesii that differ in SI status and in which SI is associated with low inbreeding depression but not with a transition to full selfing or a selfing syndrome. Our analysis suggests that the mixed‐mating self‐compatible (SC) population may have begun to diverge from the SI populations around 2810 generation ago, a period perhaps too short for the evolution of a selfing syndrome. We conjecture that the SC population of L. cavanillesii is at an intermediate stage of transition between outcrossing and selfing.     

THE ROLE OF INBREEDING DEPRESSION AND MATING SYSTEM IN THE EVOLUTION OF HETEROSTYLY

We investigated the role of morph‐based differences in the expression of inbreeding depression in loss of the mid‐styled morph from populations of tristylous Oxalis alpina. The extent of self‐compatibility (SC) of reproductive morphs, the degree of self‐fertilization, and the magnitude of inbreeding depression were investigated in three populations of O. alpina differing in their tristylous incompatibility relationships. All three populations exhibited significant inbreeding depression. In two populations with highly modified tristylous incompatibility, manifested as increased reciprocal compatibility between short‐ and long‐styled morphs, substantial SC and self‐fertilization of mid‐styled morphs were detected, and expected to result in expression of inbreeding depression in the progeny of mid‐styled morphs in the natural populations. In contrast, significant self‐fertility of the mid‐styled morph was absent from the population with typical tristylous incompatibility, and no self‐fertilization could be detected. Although self‐fertilization and expression of inbreeding depression should result in selection against the mid‐styled morph in the later stages of the transition from tristyly to distyly, in O. alpina selection against the mid‐styled morph in the early phases of the evolution of distyly is likely due to genic selection against mid‐alleles associated with modified tristylous incompatibility, rather than expression of inbreeding depression.     

Costs of selfing prevent the spread of a self‐compatibility mutation that causes reproductive assurance

In flowering plants, shifts from outcrossing to partial or complete self‐fertilization have occurred independently thousands of times, yet the underlying adaptive processes are difficult to discern. Selfing's ability to provide reproductive assurance when pollination is uncertain is an oft‐cited ecological explanation for its evolution, but this benefit may be outweighed by costs diminishing its selective advantage over outcrossing. We directly studied the fitness effects of a self‐compatibility mutation that was backcrossed into a self‐incompatible (SI) population of Leavenworthia alabamica, illuminating the direction and magnitude of selection on the mating‐system modifier. In array experiments conducted in two years, self‐compatible (SC) plants produced 17–26% more seed, but this advantage was counteracted by extensive seed discounting—the replacement of high‐quality outcrossed seeds by selfed seeds. Using a simple model and simulations, we demonstrate that SC mutations with these attributes rarely spread to high frequency in natural populations, unless inbreeding depression falls below a threshold value (0.57 ≤ δ_threshold ≤ 0.70) in SI populations. A combination of heavy seed discounting and inbreeding depression likely explains why outcrossing adaptations such as self‐incompatibility are maintained generally, despite persistent input of selfing mutations, and frequent limits on outcross seed production in nature.     

GENETIC ARCHITECTURE OF INBREEDING DEPRESSION AND THE MAINTENANCE OF GAMETOPHYTIC SELF‐INCOMPATIBILITY

Gametophytic self‐incompatibility (GSI) is a widespread genetic system, which enables hermaphroditic plants to avoid self‐fertilization and mating with close relatives. Inbreeding depression is thought to be the major force maintaining SI; however, inbreeding depression is a dynamical variable that depends in particular on the mating system. In this article we use multilocus, individual‐based simulations to examine the coevolution of SI and inbreeding depression within finite populations. We focus on the conditions for the maintenance of SI when self‐compatible (SC) mutants are introduced in the population by recurrent mutation, and compare simulation results with predictions from an analytical model treating inbreeding depression as a fixed parameter (thereby neglecting effects of purging within the SC subpopulation). In agreement with previous models, we observe that the maintenance of SI is associated with high inbreeding depression and is facilitated by high rates of self‐pollination. Purging of deleterious mutations by SC mutants has little effect on the spread of those mutants as long as most deleterious alleles have weak fitness effects: in this case, the genetic architecture of inbreeding depression has little effect on the maintenance of SI. By contrast, purging may greatly enhance the spread of SC mutants when deleterious alleles have strong fitness effects.     

The loss of self‐incompatibility in a range expansion

It is commonly observed that plant species' range margins are enriched for increased selfing rates and, in otherwise self‐incompatible species, for self‐compatibility (SC). This has often been attributed to a response to selection under mate and/or pollinator limitation. However, range expansion can also cause reduced inbreeding depression, and this could facilitate the evolution of selfing in the absence of mate or pollinator limitation. Here, we explore this idea using spatially explicit individual‐based simulations of a range expansion, in which inbreeding depression, variation in self‐incompatibility (SI), and mate availability evolve. Under a wide range of conditions, the simulated range expansion brought about the evolution of selfing after the loss of SI in range‐marginal populations. Under conditions of high recombination between the self‐incompatibility locus (S‐locus) and viability loci, SC remained marginal in the expanded metapopulation and could not invade the range core, which remained self‐incompatible. In contrast, under low recombination and migration rates, SC was frequently able to displace SI in the range core by maintaining its association with a genomic background with purged genetic load. We conclude that the evolution of inbreeding depression during a range expansion promotes the evolution of SC at range margins, especially under high rates of recombination.?     

Incest versus abstinence: reproductive trade‐offs between mate limitation and progeny fitness in a self‐incompatible invasive plant

Plant mating systems represent an evolutionary and ecological trade‐off between reproductive assurance through selfing and maximizing progeny fitness through outbreeding. However, many plants with sporophytic self‐incompatibility systems exhibit dominance interactions at the S‐locus that allow biparental inbreeding, thereby facilitating mating between individuals that share alleles at the S‐locus. We investigated this trade‐off by estimating mate availability and biparental inbreeding depression in wild radish from five different populations across Australia. We found dominance interactions among S‐alleles increased mate availability relative to estimates based on individuals that did not share S‐alleles. Twelve of the sixteen fitness variables were significantly reduced by inbreeding. For all the three life‐history phases evaluated, self‐fertilized offspring suffered a greater than 50% reduction in fitness, while full‐sib and half‐sib offspring suffered a less than 50% reduction in fitness. Theory indicates that fitness costs greater than 50% can result in an evolutionary trajectory toward a stable state of self‐incompatibility (SI). This study suggests that dominance interactions at the S‐locus provide a possible third stable state between SI and SC where biparental inbreeding increases mate availability with relatively minor fitness costs. This strategy allows weeds to establish in new environments while maintaining a functional SI system.     

TRANSMISSION ADVANTAGE FAVORS SELFING ALLELE IN EXPERIMENTAL POPULATIONS OF SELF‐INCOMPATIBLE WITHERINGIA SOLANACEA (SOLANACEAE)

The evolution of self‐fertilization is one of the most commonly traversed transitions in flowering plants, with profound implications for population genetic structure and evolutionary potential. We investigated factors influencing this transition using Witheringia solanacea, a predominantly self‐incompatible (SI) species within which self‐compatible (SC) genotypes have been identified. We showed that self‐compatibility in this species segregates with variation at the S‐locus as inherited by plants in F1 and F2 generations. To examine reproductive assurance and the transmission advantage of selfing, we placed SC and SI genotypes in genetically replicated gardens and monitored male and female reproductive success, as well as selfing rates of SC plants. Self‐compatibility did not lead to increased fruit or seed set, even under conditions of pollinator scarcity, and the realized selfing rate of SC plants was less than 10%. SC plants had higher fruit abortion rates, consistent with previous evidence showing strong inbreeding depression at the embryonic stage. Although the selfing allele did not provide reproductive assurance under observed conditions, it also did not cause pollen discounting, so the transmission advantage of selfing should promote its spread. Given observed numbers of S‐alleles and selfing rates, self‐compatibility should spread even under conditions of exceedingly high initial inbreeding depression.     

Significant differences in outcrossing rate,self‐incompatibility,and inbreeding depression between two widely hybridizing species of Geum

Geum urbanum and Geum rivale are two widely hybridizing perennial herbs. Estimation of the breeding systems of these taxa using nuclear microsatellite markers scored in mother–progeny arrays demonstrated that, in pure populations, G. urbanum is predominantly selfing (outcrossing rate, t = 0.058 to 0.177), whereas G. rivale is predominantly outcrossing (t = 0.686–0.775). Theory suggests that hybridization between inbreeding and outcrossing species can potentially generate novel inbreeding lineages. However, the establishment of such lineages may be restricted either by self‐incompatibility loci or deleterious recessive alleles derived from the outcrossing parent. To assess the likelihood that hybridization between G. urbanum and G. rivale will generate novel inbreeding lineages, self‐incompatibility and inbreeding depression were investigated in the two taxa. Seed set in the absence of pollinators, and after controlled self‐ and cross‐pollination, was measured to study self‐incompatibility. Inbreeding depression was measured by estimating the relative fitness of offspring from controlled self‐and cross‐pollinations. Geum urbanum was fully self‐compatible [self‐compatibility index (SCI) = 1] and bagged flowers showed full seed set. By contrast, only 3% of bagged flowers set seed in G. rivale and controlled self‐pollinations showed a 60–80% reduction in seed set compared to controlled outcross pollinations (SCI = 0.28). There was no evidence for inbreeding depression in G. urbanum, although significant, albeit low levels of inbreeding depression were detected in one of two G. rivale populations (δ = 0.33). The implication of these results is that if genetic material from G. rivale was incorporated into a hybrid with a selfing morphology, the establishment of this selfing lineage could be compromised by self‐incompatibility and inbreeding depression. The wider implications of these results for evolution in hybrid swarms between G. urbanum and G. rivale are discussed. © 2010 The Linnean Society of London, Biological Journal of the Linnean Society, 2010, 101 , 977–990.     

Relatively weak inbreeding depression in selfing but also in outcrossing populations of North American Arabidopsis lyrata

Hermaphroditic plants can potentially self‐fertilize, but most possess adaptations that promote outcrossing. However, evolutionary transitions to higher selfing rates are frequent. Selfing comes with a transmission advantage over outcrossing, but self‐progeny may suffer from inbreeding depression, which forms the main barrier to the evolution of higher selfing rates. Here, we assessed inbreeding depression in the North American herb Arabidopsis lyrata, which is normally self‐incompatible, with a low frequency of self‐compatible plants. However, a few populations have become fixed for self‐compatibility and have high selfing rates. Under greenhouse conditions, we estimated mean inbreeding depression per seed (based on cumulative vegetative performance calculated as the product of germination, survival and aboveground biomass) to be 0.34 for six outcrossing populations, and 0.26 for five selfing populations. Exposing plants to drought and inducing defences with jasmonic acid did not magnify these estimates. For outcrossing populations, however, inbreeding depression per seed may underestimate true levels of inbreeding depression, because self‐incompatible plants showed strong reductions in seed set after (enforced) selfing. Inbreeding‐depression estimates incorporating seed set averaged 0.63 for outcrossing populations (compared to 0.30 for selfing populations). However, this is likely an overestimate because exposing plants to 5% CO₂ to circumvent self‐incompatibility to produce selfed seed might leave residual effects of self‐incompatibility that contribute to reduced seed set. Nevertheless, our estimates of inbreeding depression were clearly lower than previous estimates based on the same performance traits in outcrossing European populations of A. lyrata, which may help explain why selfing could evolve in North American A. lyrata.     

Evolution towards self‐compatibility when mates are limited

Theory of plant mating system evolution predicts the spread of self‐compatibility (SC) in a predominantly self‐incompatible population when inbreeding depression (ID; the decline in fitness because of selfing) is small and when compatible mates are limited. I tested these two predictions by measuring the occurrence of SC in 13 natural populations of Ranunculus reptans L. that varied in ID and frequency of cross‐incompatible mates. Enforced selfing experiments were conducted in 2 years. In the first year, self‐pollination was applied at two flower ages to investigate the occurrence of delayed SC. I found that SC was not uncommon across all populations, but self‐compatible plants usually produced few seeds. There was no evidence for delayed SC. The occurrence of SC was not associated with population‐level ID, but populations with more limited availability of compatible mates had a significantly higher frequency of plants that were at least partially self‐compatible. The results indicate that, in R. reptans, a shortage of available mates in small populations may cause the evolution of partial SC and mixed mating.     

Recent selection for self‐compatibility in a population of Leavenworthia alabamica

The evolution of self‐compatibility (SC) is the first step in the evolutionary transition in plants from outcrossing enforced by self‐incompatibility (SI) to self‐fertilization. In the Brassicaceae, SI is controlled by alleles of two tightly linked genes at the S‐locus. Despite permitting inbreeding, mutations at the S‐locus leading to SC may be selected if they provide reproductive assurance and/or gain a transmission advantage in a population when SC plants self‐ and outcross. Positive selection can leave a genomic signature in the regions physically linked to the focus of selection when selection has occurred recently. From an SC population of Leavenworthia alabamica with a known nonfunctional mutation at the S‐locus, we collected sequence data from a ～690 Kb region surrounding the S‐locus, as well as from regions not linked to the S‐locus. To test for recent positive selection acting at the S‐locus, we examined polymorphism and the site‐frequency spectra. Using forward simulations, we demonstrate that recent selection of the strength expected for SC at a locus formerly under balancing selection can generate patterns similar to those seen in our empirical data.     

Effects of pollen availability and the mutation bias on the fixation of mutations disabling the male specificity of self‐incompatibility

The evolution of self‐compatibility (SC) by the loss of self‐incompatibility (SI) is regarded as one of the most frequent transitions in flowering plants. SI systems are generally characterized by specific interactions between the male and female specificity genes encoded at the S‐locus. Recent empirical studies have revealed that the evolution of SC is often driven by male SC‐conferring mutations at the S‐locus rather than by female mutations. In this study, using a forward simulation model, we compared the fixation probabilities of male vs. female SC‐conferring mutations at the S‐locus. We explicitly considered the effects of pollen availability in the population and bias in the occurrence of SC‐conferring mutations on the male and female specificity genes. We found that male SC‐conferring mutations were indeed more likely to be fixed than were female SC‐conferring mutations in a wide range of parameters. This pattern was particularly strong when pollen availability was relatively high. Under such a condition, even if the occurrence of mutations was biased strongly towards the female specificity gene, male SC‐conferring mutations were much more often fixed. Our study demonstrates that fixation probabilities of those two types of mutation vary strongly depending on ecological and genetic conditions, although both types result in the same evolutionary consequence—the loss of SI.     

Intrinsic and extrinsic factors acting on the reproductive process in alpine‐snowbed plants: roles of phenology,biological interaction,and breeding system

Pollinator activity and competition for pollinators lead to quantitative and qualitative pollen limitations on seed production and affect the reproductive success of plant species, depending on their breeding system (e.g., self‐compatibility and heterospecific compatibility) and genetic load (e.g., inbreeding depression and hybrid inviability). In alpine ecosystems, snowmelt regimes determine the distribution and phenology of plant communities. Plant species growing widely along a snowmelt gradient often grow with different species among local populations. Their pollinators also vary in their abundance, activity, and behavior during the season. These variations may modify plant–pollinator and plant–plant interactions. We integrated a series of our studies on the alpine dwarf shrub, Phyllodoce aleutica (Ericaceae), to elucidate the full set of intrinsic (species‐specific breeding system) and extrinsic factors (snow condition, pollinator activity, and interspecific competition) acting on their reproductive process. Seasonality of pollinator activity led to quantitative pollen limitation in the early‐blooming populations, whereas in the late‐blooming populations, high pollinator activity ensured pollination service, but interspecific competition for pollinators led to qualitative and quantitative pollen limitation in less competitive species. However, negative effects of illegitimate pollen receipt on seed‐set success might be reduced when cryptic incompatibility systems (i.e., outcross pollen grains took priority over self‐ and heterospecific pollen grains) could effectively prevent ovule and seed discounting. Our studies highlight the importance of species‐specific responses of plant reproduction to changing pollinator availability along environmental gradients to understand the general features of pollination networks in alpine ecosystems.     

Multiple losses of self‐incompatibility in North‐American Arabidopsis lyrata?: Phylogeographic context and population genetic consequences

Arabidopsis lyrata is mostly outcrossing due to a sporophytic self‐incompatibility (SI) system but around the Great Lakes of North America some populations have experienced a loss of SI. We researched the loss of SI in a phylogeographic context. We used cpDNA and microsatellite markers to test if populations of North‐American A. lyrata around the Great Lakes have experienced different (recent) histories, and linked this with individually established selfing phenotype and population level realized outcrossing rates calculated based on variation in progeny arrays at multi‐locus microsatellite markers. We found three chloroplast haplotypes, in two of which the loss of self‐incompatibility had occurred independently. Shifts to high rates of inbreeding were most apparent in one of these lineages but individuals showing loss of SI occurred in all three. Self‐compatible individuals usually showed a reduction of observed heterozygosity (H_O) compared to outcrossing individuals. In the lineage that included the populations with the highest levels of inbreeding, this reduction was more substantial. This may indicate that the loss of SI in this lineage did not occur as recently as in the other lineage. The geographic distribution of the haplotypes suggested that there had been at least two independent colonization routes to the north of the Great Lakes following the last glaciation. This is consistent with postglacial migration patterns that have been suggested for other organisms with limited dispersal, such as reptiles and amphibians.     

Reduced mate availability leads to evolution of self‐fertilization and purging of inbreeding depression in a hermaphrodite

Basic models of mating‐system evolution predict that hermaphroditic organisms should mostly either cross‐fertilize, or self‐fertilize, due to self‐reinforcing coevolution of inbreeding depression and outcrossing rates. However transitions between mating systems occur. A plausible scenario for such transitions assumes that a decrease in pollinator or mate availability temporarily constrains outcrossing populations to self‐fertilize as a reproductive assurance strategy. This should trigger a purge of inbreeding depression, which in turn encourages individuals to self‐fertilize more often and finally to reduce male allocation. We tested the predictions of this scenario using the freshwater snail Physa acuta, a self‐compatible hermaphrodite that preferentially outcrosses and exhibits high inbreeding depression in natural populations. From an outbred population, we built two types of experimental evolution lines, controls (outcrossing every generation) and constrained lines (in which mates were often unavailable, forcing individuals to self‐fertilize). After ca. 20 generations, individuals from constrained lines initiated self‐fertilization earlier in life and had purged most of their inbreeding depression compared to controls. However, their male allocation remained unchanged. Our study suggests that the mating system can rapidly evolve as a response to reduced mating opportunities, supporting the reproductive assurance scenario of transitions from outcrossing to selfing.     

STRONG INBREEDING DEPRESSION IN TWO SCANDINAVIAN POPULATIONS OF THE SELF‐INCOMPATIBLE PERENNIAL HERB ARABIDOPSIS LYRATA

Inbreeding depression is a key factor influencing mating system evolution in plants, but current understanding of its relationship with selfing rate is limited by a sampling bias with few estimates for self‐incompatible species. We quantified inbreeding depression (δ) over two growing seasons in two populations of the self‐incompatible perennial herb Arabidopsis lyrata ssp. petraea in Scandinavia. Inbreeding depression was strong and of similar magnitude in both populations. Inbreeding depression for overall fitness across two seasons (the product of number of seeds, offspring viability, and offspring biomass) was 81% and 78% in the two populations. Chlorophyll deficiency accounted for 81% of seedling mortality in the selfing treatment, and was not observed among offspring resulting from outcrossing. The strong reduction in both early viability and late quantitative traits suggests that inbreeding depression is due to deleterious alleles of both large and small effect, and that both populations experience strong selection against the loss of self‐incompatibility. A review of available estimates suggested that inbreeding depression tends to be stronger in self‐incompatible than in self‐compatible highly outcrossing species, implying that undersampling of self‐incompatible taxa may bias estimates of the relationship between mating system and inbreeding depression.     

RECONSTRUCTING ORIGINS OF LOSS OF SELF‐INCOMPATIBILITY AND SELFING IN NORTH AMERICAN ARABIDOPSIS LYRATA: A POPULATION GENETIC CONTEXT

Theoretical and empirical comparisons of molecular diversity in selfing and outcrossing plants have primarily focused on long‐term consequences of differences in mating system (between species). However, improving our understanding of the causes of mating system evolution requires ecological and genetic studies of the early stages of mating system transition. Here, we examine nuclear and chloroplast DNA sequences and microsatellite variation in a large sample of populations of Arabidopsis lyrata from the Great Lakes region of Eastern North American that show intra‐ and interpopulation variation in the degree of self‐incompatibility and realized outcrossing rates. Populations show strong geographic clustering irrespective of mating system, suggesting that selfing either evolved multiple times or has spread to multiple genetic backgrounds. Diversity is reduced in selfing populations, but not to the extent of the severe loss of variation expected if selfing evolved due to selection for reproductive assurance in connection with strong founder events. The spread of self‐compatibility in this region may have been favored as colonization bottlenecks following glaciation or migration from Europe reduced standing levels of inbreeding depression. However, our results do not suggest a single transition to selfing in this system, as has been suggested for some other species in the Brassicaceae.     

Variation in floral morphology and plant reproductive success in four Ipomoea species (Convolvulaceae) with contrasting breeding systems

• This study tested the hypothesis that self‐compatibility would be associated with floral traits that facilitate autonomous self‐pollination to ensure reproduction under low pollinator visitation. In a comparison of two pairs of Ipomoea species with contrasting breeding systems, we predicted that self‐compatible (SC) species would have smaller, less variable flowers, reduced herkogamy, lower pollinator visitation and higher reproductive success than their self‐incompatible (SI) congeners.
• We studied sympatric species pairs, I. hederacea (SC)– I. mitchellae (SI) and I. purpurea (SC)–I. indica (SI), in Mexico, over two years. We quantified variation in floral traits and nectar production, documented pollinator visitation, and determined natural fruit and seed set. Hand‐pollination and bagging experiments were conducted to determine potential for autonomous self‐pollination and apomixis.
• Self‐compatible Ipomoea species had smaller flowers and lower nectar production than SI species; however, floral variation and integration did not vary according to breeding system. Bees were primary pollinators of all species, but visitation rates were seven times lower in SC than SI species. SC species had a high capacity for autonomous self‐pollination due to reduced herkogamy at the highest anther levels. Self‐compatible species had two to six times higher fruit set than SI species.
• Results generally support the hypothesis that self‐compatibility and autonomous self‐pollination ensure reproduction under low pollinator visitation. However, high variation in morphological traits of SC Ipomoea species suggests they maintain variation through outcrossing. Furthermore, reduced herkogamy was associated with high potential for autonomous self‐pollination, providing a reproductive advantage that possibly underlies transitions to self‐compatibility in Ipomoea.

     

Patterns of self compatibility, inbreeding depression, outcrossing, and sex allocation in a marine bryozoan suggest the predominating influence of sperm competition

Sex allocation by simultaneous hermaphrodites is theoretically influenced by selfing rate, which is in turn influenced by the benefits of enhanced genomic transmission and reproductive assurance relative to the cost of inbreeding depression. The experimental investigation of these influences in seed plants has a rich pedigree, yet although such an approach is equally relevant to colonial invertebrates, which globally dominate subtidal communities on firm substrata, such studies have been scarce. We reared self‐compatible genets of the marine bryozoan Celleporella hyalina s.l. in the presence and absence of allosperm, and used molecular genetic markers for paternity analysis of progeny to test theoretical predictions that: (1) genets from focal populations with high selfing rates show less inbreeding depression than from focal populations with low selfing rates; (2) genets whose selfed progeny show inbreeding depression prefer outcross sperm (allosperm); and (3) genets bias sex allocation toward female function when reared in reproductive isolation. Offspring survivorship and paternity analysis were used to estimate levels of inbreeding depression and preference for outcrossing or selfing. Sex allocation was assessed by counting male and female zooids. As predicted, inbreeding depression was severe in selfed progeny of genets derived from the populations with low self‐compatibility rates, but, with one exception, was not detected in selfed progeny of genets derived from the populations with higher self‐compatibility rates. Also, as predicted, genets whose selfed progeny showed inbreeding depression preferred outcrossing, and a genet whose selfed progeny did not show inbreeding depression preferred selfing. Contrary to prediction, sex allocation in the majority of genets was not influenced by reproductive isolation. Lack of economy of male function may reflect the over‐riding influence of allosperm‐competition in typically dense breeding populations offering good opportunity for outcrossing. We suggest that hermaphroditism may be a plesiomorphic character of the crown group Bryozoa, prevented by phylogenetic constraint from being replaced by gonochorism and therefore not necessarily adaptive in all extant clades. © 2009 The Linnean Society of London, Biological Journal of the Linnean Society, 2009, 98 , 519–531.