Modeling Consonant-Vowel Coarticulation for Articulatory Speech Synthesis

A central challenge for articulatory speech synthesis is the simulation of realistic articulatory movements, which is critical for the generation of highly natural and intelligible speech. This includes modeling coarticulation, i.e., the context-dependent variation of the articulatory and acoustic realization of phonemes, especially of consonants. Here we propose a method to simulate the context-sensitive articulation of consonants in consonant-vowel syllables. To achieve this, the vocal tract target shape of a consonant in the context of a given vowel is derived as the weighted average of three measured and acoustically-optimized reference vocal tract shapes for that consonant in the context of the corner vowels /a/, /i/, and /u/. The weights are determined by mapping the target shape of the given context vowel into the vowel subspace spanned by the corner vowels. The model was applied for the synthesis of consonant-vowel syllables with the consonants /b/, /d/, /g/, /l/, /r/, /m/, /n/ in all combinations with the eight long German vowels. In a perception test, the mean recognition rate for the consonants in the isolated syllables was 82.4%. This demonstrates the potential of the approach for highly intelligible articulatory speech synthesis.     

Cleft palate repair by double opposing Z-plasty

In an attempt to improve speech results following palate repair while allowing adequate maxillary growth, a palatoplasty using two opposing Z-plasties of the soft palate, one of the oral and one of the nasal layers, has been used in 22 infants. Eight patients had unilateral cleft lip and palate, eight had bilateral cleft lip and palate, and six had cleft palate. The Z-plasties facilitate effective dissection and redirection of the palatal muscles to produce an overlapping muscle sling and lengthen the velum without using tissue from the hard palate, which permits hard palate closure without pushback or lateral relaxing incisions. Of the 20 children old enough for speech evaluation, 18 have no velopharyngeal insufficiency. Two have very mild velopharyngeal insufficiency. None has required a pharyngeal flap.     

Uvular transposition: a new method of cleft palate repair

This prospective study was done to determine whether a new cleft palate repair utilizing uvular transposition improved speech outcome as measured objectively by a speech pathologist. In the uvular transposition procedure, the palate was lengthened with tissue from the uvula by a double-opposing Z-plasty; an intravelar veloplasty was performed, and two-thirds of the mass of the uvula was transposed to the nasal surface of the soft palate. This procedure facilitates velopharyngeal closure by significantly lengthening the palate, anatomically reconstructing the muscles of the palate, and decreasing the palatal excursion necessary to achieve closure. Sixty-two children with a cleft palate were treated with this procedure performed by the senior surgeon between the years of 1988 and 1995. These children were then enrolled in cleft lip and palate clinic at age 2 to 3 years and blindly evaluated yearly by a single speech pathologist who specialized in pediatric speech pathology. Postoperative clinical follow-up ranged from 36 to 112 months (mean, 56.8 months). Perceptual nasal emission was found to be normal in 59 of the 62 patients (95 percent). Nasometry was performed in all 62 of these patients, and the mean score was 15.7 percent, well within the accepted normal range of 25 or less at our institution. Only two of these children (3 percent) required a pharyngeal flap for velopharyngeal insufficiency. These findings suggest that the uvula transposition cleft palate repair may result in good normalization of speech with negligible rates of velopharyngeal insufficiency.     

From birth to maturity: a group of patients who have completed their protocol management. Part I. Unilateral cleft lip and palate

The optimal management of the cleft lip and palate patient from birth to completion of treatment presents a formidable challenge to the plastic surgeon and the associated health care system. The multidisciplinary team approach for the management of these patients is widely accepted. However, a paucity of literature exists discussing specific protocol management, interventions, and the long-term outcomes of patients who have completed a strict treatment protocol with a consistent multidisciplinary team. The aim of this study was to present the details of the specific management protocol at the Australian Craniofacial Unit for cleft lip and palate patients and to present a group of patients who have completed this specific protocol and discuss the details of their long-term care. During a 28-year period from 1974 to 2002, the records of 337 patients treated for unilateral cleft lip and palate were evaluated. Of these 337 patients, 22 have completed the same specific protocol management. The same surgeon (David, the senior author) has been responsible for performing all operative interventions and for overseeing the care of each of the 22 patients, ensuring that the treatment protocol has been executed appropriately and without deviation. The interventions and outcomes were analyzed on the basis of speech, hearing, nasal airway, occlusion, psychosocial adjustment, and appearance. Because of the large volume of data and potential differences in outcomes, the authors' intention is to present this as part I of a four-part series beginning with unilateral cleft lip and palate. The results of isolated cleft palate, isolated cleft lip, and bilateral cleft lip and palate will be presented as parts II, III, and IV, respectively. Speech results were assessed as normal speech, mild abnormality, or severe abnormality by objective measures, and intervention for velopharyngeal insufficiency was noted. Seventeen patients were rated as having normal speech. Four patients were rated as having mild speech abnormality, one patient was rated as having severe speech abnormality, and seven patients required surgery for velopharyngeal insufficiency. Hearing results were measured objectively, and good hearing results were obtained in 18 cases. Five patients required tympanoplasty. All patients required alveolar bone grafting. The high Le Fort I osteotomy was performed in six cases. Bimaxillary surgery was performed in one case. Of all the patients assessed from birth to maturity, 13 required between three and five surgical interventions, and nine required six operations or more. Further details and photographs of preoperative and postoperative examples are provided.     

Velopharyngeal variations in relatives of cleft-affected individuals

Eighteen families each with two or more cleft lip and palate patients were studied by speech cephalometry for evidence of velopharyngeal inadequacy (VPI). With this total of 56 persons, three groups were recognized: 1) patients with cleft lip only (N = 7), 2) unaffected sibs of CL(P) probands and the unaffected parents with the positive clefting history on their side of the family (N = 33), and 3) unaffected parents with negative CL(P) history to their side of the family (N = 16). The latter served as controls. The velopharyngeal mechanism in function was evaluated by voicing the fricative/S/. The results showed no significant differences in the length of either the resting soft palate or pharyngeal depth among the three groups. Even though a significant (P less than 0.01) increase in soft palate length while voicing /S/ was found in group 2 relatives compared to group 3 controls, the failure to find differences in either resting palate length or pharyngeal depth coupled with a failure to demonstrate VPI in group 2 subjects by speech testing leaves the value of this observation uncertain.     

Cleft lip, cleft palate, and velopharyngeal insufficiency

This article provides an introduction to the anatomical and clinical features of the primary deformities associated with unilateral cleft lip-cleft palate, bilateral cleft lip-cleft palate, and cleft palate. The diagnosis and management of secondary velopharyngeal insufficiency are discussed. The accompanying videos demonstrate the features of the cleft lip nasal deformities and reliable surgical techniques for unilateral cleft lip repair, bilateral cleft lip repair, and radical intravelar veloplasty.     

Velopharyngeal incompetence: a guide for clinical evaluation

Various causes of velopharyngeal disorders and the myriad of diagnostic methods used by speech-language pathologists and plastic surgeons for assessment are described in this article. Velopharyngeal incompetence occurs when the velum and lateral and posterior pharyngeal walls fail to separate the oral cavity from the nasal cavity during speech and deglutination. The functional goals of cleft palate operations are to facilitate normal speech and hearing without interfering with the facial growth of a child. Basic and helpful techniques are presented to help the cleft palate team identify preoperative or postoperative velopharyngeal incompetence. This information will enable any member of the multidisciplinary cleft palate team to better assist in the differential diagnosis and management of patients with speech disorders.     

Long-term results following velopharyngoplasty with a cranially based pharyngeal flap.

Between 1980 and 1989, 82 velopharyngoplasties have been carried out in the Department of Oral and Maxillofacial Surgery at the Medical University of Hannover. Speech results of 51 of these patients, including 39 patients with cleft lip and palate, could be followed up in the context of a clinical follow-up examination. Besides evaluation of speech results by two senior speech pathologists and two untrained listeners, a frequency analysis of the speech results with a sonograph was obtained. Nasal air loss was documented with a fogged-mirror test and computer aerometry. Whereas in 37 of 51 patients a normal or almost normal colloquial speech could be demonstrated, 30 of 39 patients with cleft lip and palate showed a normal or almost normal realization of the test sentences. Thirty of the 37 patients (81.08 percent) with normal or almost normal colloquial speech showed extensive mobility of the lateral pharyngeal wall. Symmetry of the velopharyngeal flaps seemed to have no influence on the speech result. With a fogged-mirror test, an average reduction of mirror fogging from 2.0 rings preoperatively to 0.9 rings postoperatively could be shown. In 31 patients, there was no longer any air loss postoperatively. Besides one rupture of a flap, two flaps had to be diminished in their lateral dimensions because of excessive size. We regard the cranially pedicled pharyngeal flap as an important operative procedure for improving speech results, especially in cleft lip and palate patients.     

Single syllable tongue motion analysis using tagged cine MRI

The complicated muscle activity of the human tongue and the resultant surface shapes can give us important clues about speech motor control and pathological tongue motion. This study uses tagged magnetic resonance imaging to provide a 2D surface deformation analysis of the tongue, as well as a 4D compression–expansion analysis, during utterances of four different syllables (/ba/, /ta/, /sha/ and /ga/). All speech tasks were performed several times to confirm the repeatability of the motion analysis. The results showed that the tongue has unique motion patterns for utterances of different syllables, and these differences, which may not be observed by a simple surface analysis, can be examined thoroughly by a 4D motion model-based analysis of the tongue muscles.     

A prospective evaluation of the prevalence of submucous cleft palate in patients with isolated cleft lip versus controls.

Although there is an established relationship between cleft lip and overt cleft palate, the relationship between isolated cleft lip and submucous cleft palate has not been investigated. To test the hypothesis that patients with isolated cleft lip have a greater association with submucous cleft palate, a double-armed prospective trial was designed. A study group of 25 consecutive children presenting with an isolated cleft lip, with or without extension through the alveolus but not involving the secondary palate, was compared with a control group of 25 children with no known facial clefts. Eligible patients were examined for the presence of physical criteria associated with classic submucous cleft palate, namely, (1) bifid uvula, (2) absence of the posterior nasal spine, and (3) zona pellucida. Nasoendoscopy was subsequently performed just after induction of general anesthesia, and the findings were correlated with digital palpation of the palatal muscles. Patients who did not satisfy all three physical criteria and in whom nasoendoscopy was distinctly abnormal relative to the control group were classified as having occult submucous cleft palate. Classic submucous cleft palate was found in three study group patients (12 percent), all of whom had flattening or a midline depression of the posterior palate and musculus uvulae on nasoendoscopy and palpable diastasis of the palatal muscles under general anesthesia. An additional six study group patients (24 percent) had similar nasoendoscopic criteria and palpable diastasis of the palatal muscles; they were classified as having occult submucous cleft palate. No submucous cleft palate was identified in the control group. Seventeen patients in the study group had an alveolar cleft with a 53 percent (9 of 17) prevalence of submucous cleft palate. In the present study, classic submucous cleft palate in association with isolated cleft lip was 150 to 600 times the reported prevalence in the general population. All children with an isolated cleft lip should undergo peroral examination and speech/resonance assessment no later than the age of 3 years. Any child with an isolated cleft lip with velopharyngeal inadequacy or before an adenoidectomy should be assessed by flexible nasal endoscopy to avoid missing an occult submucous cleft palate.     

Speech results of the Zürich approach in the treatment of unilateral cleft lip and palate

The purpose of this study was to describe perceptually the speech articulation, voice quality, and velopharyngeal competency of subjects with complete unilateral cleft lip and palate treated by the Zürich approach. The mean age of the 37 subjects was 10.5 years. Although only one subject had had secondary palatal management, no subject was rated as exhibiting a severe articulation or nasality problem. Subjects were rated as exhibiting adequate to marginal velopharyngeal competency 94.5 percent of the time, and the incidence of compensatory articulation errors was low. In comparison with other studies that evaluated the two-stage palatal repair, the Zürich approach appears to give the better results. The type of initial soft palate repair is probably the significant factor which contributes to the better speech of these subjects.     

Sequence Complexity Effects on Speech Production in Healthy Speakers and Speakers with Hypokinetic or Ataxic Dysarthria

The present study investigated the effects of sequence complexity, defined in terms of phonemic similarity and phonotoactic probability, on the timing and accuracy of serial ordering for speech production in healthy speakers and speakers with either hypokinetic or ataxic dysarthria. Sequences were comprised of strings of consonant-vowel (CV) syllables with each syllable containing the same vowel, /a/, paired with a different consonant. High complexity sequences contained phonemically similar consonants, and sounds and syllables that had low phonotactic probabilities; low complexity sequences contained phonemically dissimilar consonants and high probability sounds and syllables. Sequence complexity effects were evaluated by analyzing speech error rates and within-syllable vowel and pause durations. This analysis revealed that speech error rates were significantly higher and speech duration measures were significantly longer during production of high complexity sequences than during production of low complexity sequences. Although speakers with dysarthria produced longer overall speech durations than healthy speakers, the effects of sequence complexity on error rates and speech durations were comparable across all groups. These findings indicate that the duration and accuracy of processes for selecting items in a speech sequence is influenced by their phonemic similarity and/or phonotactic probability. Moreover, this robust complexity effect is present even in speakers with damage to subcortical circuits involved in serial control for speech.     

A Geometric Morphometric Approach to the Analysis of Lip Shape during Speech: Development of a Clinical Outcome Measure

Objective assessments of lip movement can be beneficial in many disciplines including visual speech recognition, for surgical outcome assessment in patients with cleft lip and for the rehabilitation of patients with facial nerve impairments. The aim of this study was to develop an outcome measure for lip shape during speech using statistical shape analysis techniques. Lip movements during speech were captured from a sample of adult subjects considered as average using a three-dimensional motion capture system. Geometric Morphometrics was employed to extract three-dimensional coordinate data for lip shape during four spoken words decomposed into seven visemes (which included the resting lip shape). Canonical variate analysis was carried out in an attempt to statistically discriminate the seven visemes. The results showed that the second canonical variate discriminated the resting lip shape from articulation of the utterances and accounted for 17.2% of the total variance of the model. The first canonical variate was significant in discriminating between the utterances and accounted for 72.8% of the total variance of the model. The outcome measure was created using the 95% confidence intervals of the canonical variate scores for each subject plotted as ellipses for each viseme. The method and outcome model is proposed as reference to compare lip movement during speech in similar population groups.     

Risk factors for poor dental arch relationships in young children born with unilateral cleft lip and palate

To identify risk factors for poor dental arch relationships in children with unilateral cleft lip and palate in the United Kingdom, the authors performed a cross-sectional outcome study with retrospective data capture of treatment histories in children under the care of 44 cleft teams in the United Kingdom. The study sample comprised 238 children born with nonsyndromic complete unilateral cleft lip and palate between April 1, 1989, and March 31, 1991, who were between 5.0 and 7.7 years of age (mean age, 6.5 years) at the time of data collection. The Five-Year-Old Index was used to rank dental arch relationships from dental study models. Velopharyngeal insufficiency was assessed with the use of the Cleft Audit Protocol for Speech. An independent panel recorded surgical treatment histories from the clinical notes. There was no association between the technique and the timing of primary repair, the experience of the surgeon, or presurgical orthopedics and dental arch relationships. Secondary velopharyngeal surgery was independently associated with poor outcome (OR, 4.14; 95 percent CI, 1.6 to 10.7; p = 0.003). Primary nasal repair was protective (OR, 0.47; 95 percent CI, 0.23 to 0.93; p = 0.031) against poor dental arch relationships. Secondary velopharyngeal surgery and primary nasal repair were found to be independently associated with dental arch relationship outcomes in young children with unilateral cleft lip and palate in the United Kingdom.     

Surgical treatment of submucous cleft palate: a comparative trial of two modalities for palatal closure

Submucous cleft palate is a congenital malformation with specific clinical and anatomical features. It can be present with or without velopharyngeal insufficiency. Surgical treatment of this malformation is indicated only when velopharyngeal insufficiency has been demonstrated. This article compares two modalities of surgical treatment for submucous cleft palate. The first includes a minimal incision palatopharyngoplasty, as described in a previous report. The second combines the first technique with additional individualized velopharyngeal surgery (individualized pharyngeal flap or sphincter pharyngoplasty) performed simultaneously. The individualized part of the procedure was selected and performed according to the findings of videonasopharyngoscopy and multiview videofluoroscopy, as reported previously. Two hundred and three patients with submucous cleft palate were studied from 1990 to 1999. Videonasopharyngoscopy and multiview videofluoroscopy demonstrated velopharyngeal insufficiency in 72 patients, who were randomly divided into two groups. Those in group 1 (n = 37) underwent a minimal incision palatopharyngoplasty. Patients in group 2 (n = 35) also underwent that procedure but simultaneously received individualized pharyngeal flap or sphincter pharyngoplasty, according to the findings of videonasopharyngoscopy and multiview videofluoroscopy. The median age of the patients from both groups was not significantly different (p > 0.5). The frequency of residual velopharyngeal insufficiency after palatal closure was not significantly different in both groups of patients (14 percent versus 11 percent; p > 0.5). The mean size of the gap at the velopharyngeal sphincter during speech was not significantly different in both groups of patients before surgery (23 percent versus 22 percent; p > 0.5). After the surgical procedures, there was a nonsignificant difference between both groups of patients in mean residual size of the gap in cases of velopharyngeal insufficiency (7 percent versus 8 percent; p > 0.5). It seems that minimal incision palatopharyngoplasty is a safe and reliable procedure for palatal closure in patients with submucous cleft palate. The use of additional individualized velopharyngeal surgery performed simultaneously did not seem to decrease the frequency of residual velopharyngeal insufficiency. Moreover, the residual size of the gap at the velopharyngeal sphincter was not significantly reduced when an additional surgical procedure was performed simultaneously with palatal closure.     

No,There Is No 150 ms Lead of Visual Speech on Auditory Speech,but a Range of Audiovisual Asynchronies Varying from Small Audio Lead to Large Audio Lag

An increasing number of neuroscience papers capitalize on the assumption published in this journal that visual speech would be typically 150 ms ahead of auditory speech. It happens that the estimation of audiovisual asynchrony in the reference paper is valid only in very specific cases, for isolated consonant-vowel syllables or at the beginning of a speech utterance, in what we call “preparatory gestures”. However, when syllables are chained in sequences, as they are typically in most parts of a natural speech utterance, asynchrony should be defined in a different way. This is what we call “comodulatory gestures” providing auditory and visual events more or less in synchrony. We provide audiovisual data on sequences of plosive-vowel syllables (pa, ta, ka, ba, da, ga, ma, na) showing that audiovisual synchrony is actually rather precise, varying between 20 ms audio lead and 70 ms audio lag. We show how more complex speech material should result in a range typically varying between 40 ms audio lead and 200 ms audio lag, and we discuss how this natural coordination is reflected in the so-called temporal integration window for audiovisual speech perception. Finally we present a toy model of auditory and audiovisual predictive coding, showing that visual lead is actually not necessary for visual prediction.     

Late results of primary veloplasty: the Marburg Project

Forty-five randomly selected patients with unilateral cleft lip, alveolus, and palate, all operated upon by Dr. Wolfram Schweckendiek were evaluated by three American specialists to assess the validity of primary veloplasty. Examination revealed an unusually high incidence of short palate and poor mobility of the soft palate. Facial growth was found to be highly acceptable in the majority of the patients. Unusually high incidence of velopharyngeal incompetence was found in these patients.     

Predictors of velopharyngeal insufficiency in cleft palate orthognathic surgery

The purpose of this study was to appraise the value of preoperative speech assessments, nasopharyngoscopy, and surgical models as predictors of velopharyngeal deterioration after a Le Fort I maxillary advancement in cleft patients. This retrospective study involved a series of 26 cleft patients (16 unilateral complete and nine bilateral complete cleft lips and palates, and one isolated complete cleft palate) who had Le Fort I maxillary advancements between March 1, 1993, and February 7, 1996. The 13 male patients and 13 female patients ranged in age from 15.3 to 46 years (mean age, 19.5 years). Four of these patients had previously undergone pharyngeal flap surgery. Eleven patients had palatal fistulas and one had a bifid uvula that was repaired at the time of orthognathic surgery. Patients with perceived hypernasal speech preoperatively all had hypernasality after advancement (nine of nine). Velopharyngeal insufficiency was observed in two of the 16 whose resonance preoperatively was within normal limits. Speech assessment, therefore, predicted accurately the postoperative status in 23 of 26 patients. Twelve patients had preoperative nasopharyngoscopy that indicated a high risk for velopharyngeal insufficiency (borderline or inadequate closure). Nine of these patients had postoperative velopharyngeal insufficiency. Two of the 14 patients not judged at risk by nasopharyngoscopy developed velopharyngeal insufficiency. Therefore, 21 of the 26 patients were accurately predicted by nasopharyngoscopy. Scoping detected borderline velopharyngeal insufficiency in one patient who was not detected by speech alone. The combined predictive value of speech and scope identified all but one patient who would develop postoperative velopharyngeal insufficiency. The degree of anteroposterior movement determined from surgical models was not predictive of the outcome. Patients with hypernasal speech preoperatively continue to have hypernasal speech after Le Fort I advancement. Preoperative perceptual speech assessment by specially trained speech-language pathologists is an excellent test for predicting postoperative velopharyngeal insufficiency status. Nasopharyngoscopy is an invasive and resource-dependent test that should be assessed with respect to cost effectiveness. In this series, only one patient's risk was more accurately predicted using nasopharyngoscopy than by speech assessment alone.     

A fiberscopic study of velopharyngeal closure in patients with operated cleft palates.

We studied the differences in how velopharyngeal closure is learned and obtained by operated cleft palate patients during various activities. Sixty-eight operated cleft palate patients, who had complete closure during swallowing, were examined with the nasopharyngeal fiberscope to determine the extent of velopharyngeal closure while they were producing pressure consonants or vowels, and during blowing. We concluded that the complete closure when producing vowels was the most difficult to obtain, and closure when producing pressure consonants was a little more difficult than that during blowing.